Compare commits
12
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1e1ed5be02 | ||
|
|
4d21f48a8a | ||
|
|
a073dc613e | ||
|
|
2ff74fda25 | ||
|
|
d247e4c2e7 | ||
|
|
e3ee34bfd0 | ||
|
|
1b4b9e3af1 | ||
|
|
9ec57aaa86 | ||
|
|
1172366f83 | ||
|
|
818f2e22e1 | ||
|
|
6a2dae0f78 | ||
|
|
82f7fc8a6f |
+125
-1
@@ -2399,7 +2399,7 @@ the connection's `total_changes` is 0 at close and the data reads back unchanged
|
||||
`-shm` bytes are not compared: every WAL reader rewrites its read marks there, so no
|
||||
reader can keep it byte-identical (`test_health_check_never_writes`).
|
||||
|
||||
Recorded deviations (plan D60–D70, D79–D81 renumbered D92–D102 and D111–D113; D121–D131 assigned):
|
||||
Recorded deviations (plan D60–D70, D74, D75, D79–D82 renumbered D92–D102, D106, D107 and D111–D114; D121–D131 assigned):
|
||||
|
||||
- **D92** (shell): no "Sample configuration" chip. The Save status is truthful: "No
|
||||
unsaved changes.", "Unsaved changes.", "Saved in this browser." / "Saved to the server
|
||||
@@ -2615,6 +2615,69 @@ Recorded deviations (plan D60–D70, D79–D81 renumbered D92–D102 and D111–
|
||||
is D93's style. `maintenance matches the concept` (both themes and sizes) maps only this
|
||||
row; `maintenance comparison detects mutations` proves it is the only difference and
|
||||
catches a removed row, a renamed operation, a lost danger mark and a swapped token.
|
||||
- **D106** (Integrations, S10, Q8): the send bridge always asks for the ComfyUI window
|
||||
and node, and the nodes always follow ComfyUI's own renderer, so "Default send
|
||||
destination" offers only "Ask each time" and "Host compatibility" only "Follow host
|
||||
renderer"; the concept's other options stay listed but disabled, each row says why in a
|
||||
`p.small` under its description, and nothing is stored for them. "Return to canvas
|
||||
after send" is off and disabled with "Not available: sending applies the text in
|
||||
ComfyUI and keeps this page open." Host compatibility adds a `p.small[role=status]`
|
||||
with the renderer the open ComfyUI windows report over the bridge (`renderer`:
|
||||
`classic` or `nodes2` on each Prompt Manager destination,
|
||||
`web/js/prompt-bridge-adapter.js`; `#renderer-status`): "Detected: Nodes 2.0 (1 ComfyUI window).", "…
|
||||
Classic and Nodes 2.0 (2 ComfyUI windows).", "ComfyUI is open; add a Prompt Manager
|
||||
node to see which renderer it uses." or "No ComfyUI window is open in this browser, so
|
||||
its renderer is not known."; it looks again when the page becomes visible, and only the
|
||||
newest check paints. The two selects never shrink for the reason lines and, at phone
|
||||
width, may grow to 160px (the concept caps them at 120px and cuts "Follow host
|
||||
renderer"), so each shows its chosen option whole (`selectsFit` in the comparison). "LoraManager
|
||||
integration" replaces the concept's sample line with the detection ("Detected:
|
||||
custom_nodes/ComfyUI-Lora-Manager." / "Not detected in ComfyUI’s custom_nodes
|
||||
folders." / "Found at …, but outside ComfyUI’s custom_nodes folders, so it can’t be
|
||||
enabled." for a linked install the server would refuse, from `/lora/status`'s
|
||||
`detected_usable`); the legacy read-only "LoraManager path" row follows it; "Automatically
|
||||
include trigger words" adds "Trigger words are (not) loaded." (the trigger cache, a
|
||||
separate fact). The notes read as the row description (`.setting-row p`). Pinned by
|
||||
`integrations matches the concept` (both themes and sizes) and its mutation check
|
||||
(system-integrations-concept.spec), the D106 tests of
|
||||
workspace-system-integrations.spec, and `each destination reports its window
|
||||
renderer, and System › Integrations shows it (S10)` on the real host
|
||||
(tests/comfyui/prompt-bridge.spec).
|
||||
- **D107** (Integrations, S10): "LoRA reference data" reads "Review import" (the
|
||||
concept's button only simulated a job) and opens "Review LoRA import" in the concept's
|
||||
record-changing review (its `maintenancePlan` shell): `h3` "LoRA reference data", the
|
||||
`dl.meta-grid` Scope ("Replace N imported prompts with M LoRAs"), Found ("N metadata
|
||||
files · M readable"), Protected (the saved pairs that block it), Network ("Downloads
|
||||
up to N example images from CivitAI[ with your saved key]" / "No downloads"), Backup
|
||||
and Media on disk, a `details` "LoRAs found (N)", the hold line plus "Example images
|
||||
downloaded before a failure stay in the image cache." in the `p.small`, the D55 backup
|
||||
box and Cancel / "Start import" (`danger` when it would remove every import). A review
|
||||
that saved pairs block, or with nothing to import, shows its rows and only Close.
|
||||
Start runs the "LoRA reference import" Activity job (D111 descriptor: Imported, Linked
|
||||
to existing, Skipped, Images downloaded, Download errors, Errors; Cancel only while it
|
||||
downloads; a failure ends with the server's reason). Pinned by `LoRA import review
|
||||
matches the concept` and its mutation check (system-integrations-concept.spec) and
|
||||
the import tests of workspace-system-integrations.spec.
|
||||
- **D114** (Integrations, S10): the CivitAI key field is never filled in (the page never
|
||||
receives the key). Typing drafts a new key, a blank field keeps the saved one (no
|
||||
draft), and "Clear saved key" (shown only while a key is saved) drafts its removal.
|
||||
A `p.small#civitai-key-status[role=status]` (it also describes the field) says "A key
|
||||
is saved." / "No key saved." / "The new key replaces the saved one when you save." /
|
||||
"The saved key is removed when you save."; pressing "Clear saved key" keeps the
|
||||
keyboard on the key field, and the line is rewritten only when its text changes. A typed
|
||||
key is dropped when the section is left (the field comes back empty); leaving without one
|
||||
keeps the last Save's outcome, so Sources still marks a refused source. The field is `autocomplete=new-password`, so browsers never offer it
|
||||
as a login; the concept's
|
||||
prototype warning is dropped. The LoRA rows save through the one Save row:
|
||||
`POST /lora/enable` with `expected_revision` (a config.json changed elsewhere is
|
||||
`settings-changed`), then `/settings` for any other host change with the revision the
|
||||
first save returned (`writeHostInParts`). When only one part is written, the store keeps
|
||||
what saved, adopts its revision and highlights only the rest: the status is the
|
||||
server's reason (an unusable source) or "Some changes were saved to the server
|
||||
configuration; the highlighted ones were not.", and the next Save needs no conflict
|
||||
(`a save where only the LoRA part is written says so; the next save needs no conflict`). Pinned by `key never prefilled; blank keeps; Clear saved key
|
||||
clears; saved through Save preferences (D114)` and `a LoRA save over settings changed
|
||||
elsewhere is a conflict, never a blind write` (workspace-system-integrations.spec).
|
||||
- **D113** (Sources): the Worker threads range ends at the server's `cpu_count` (the
|
||||
concept's fixed `max=16`) and shows the server's real `worker_threads` (the concept's
|
||||
sample 4).
|
||||
@@ -3052,6 +3115,67 @@ Restoring a deduped backup on a version before this one is not supported: that
|
||||
version's restore accepts the file and then reads rewritten images without their
|
||||
workflow (README changelog).
|
||||
|
||||
### LoRA reference import contract (#221, #249 System S10)
|
||||
|
||||
The LoRA import is staged (`py/lora_import.py`): discover the LoraManager
|
||||
`*.metadata.json` files and fingerprint them (path, size, modification time); stage each
|
||||
record (CivitAI example images go into the image cache first, best effort, counted in
|
||||
`downloaded` / `download_errors`; each image is written to a temp file beside its cache
|
||||
entry and moved in place, so a half-written image is never seen as cached); the files are
|
||||
listed and fingerprinted again (a file changed, removed or added while staging aborts with
|
||||
nothing written); then `PromptDatabase.replace_category_prompts`
|
||||
(`database/category_import.py`) replaces the `lora-manager` category in one
|
||||
`BEGIN IMMEDIATE` transaction. Any failure leaves the previous prompts, tags and image
|
||||
links exactly as they were; images already downloaded stay in the cache and the job says
|
||||
so. A record whose preparation fails is skipped (legacy); a prompt a saved pair uses
|
||||
blocks the whole replacement (`PairDependencyError`, nothing deleted); a replacement with
|
||||
no record over a non-empty collection needs `allow_empty` (a reviewed import), so the
|
||||
legacy scan refuses it.
|
||||
|
||||
`POST /prompt_manager/lora/import/preview` (body `{}`) writes nothing and requests
|
||||
nothing (`test_preview_writes_nothing_and_makes_no_requests`: DB and `-wal` bytes, `-shm`
|
||||
size, rows, no cache folder, the downloader never called) and answers the maintenance
|
||||
`Plan` shape plus `lora_files`, `with_metadata`, `current_imports`, `blocked_by_pairs`,
|
||||
`planned_downloads` and `network`; a blocked plan has no token. `POST
|
||||
/prompt_manager/lora/import/apply {token}` starts the `lora_import` job (202): title "LoRA
|
||||
reference import", admission `records`, cancellable only while it stages, the safety
|
||||
backup the policy promises before the replace. Refusals: `plan-changed` (409, the files
|
||||
changed since the review, or the integration was turned off or moved to another install),
|
||||
`plan-expired`, `plan-used`, `storage-changed`, `busy`,
|
||||
`insufficient-space`, `apply-failed`. The job fails with `plan-changed` (the files or the
|
||||
imports changed meanwhile), `blocked-by-pairs`, `nothing-to-import`, `backup-failed` or
|
||||
`import-failed`, each with nothing written; the job keeps what staging skipped, by name,
|
||||
whatever happened after. The tests change real files behind a held download
|
||||
(`test_files_changed_while_staging_fail_the_job`, removed and added files too). The
|
||||
replace binds the category, not one parameter per prompt, so SQLite's variable limit
|
||||
never caps a collection. Cached CivitAI images are written to a `.pm-*.part` file in the
|
||||
same folder and renamed into place, with the umask's mode as a plain save gives; each import
|
||||
first removes `.pm-*.part` files older than an hour from the image cache and its LoRA
|
||||
folders (`TestStaleDownloadSweep`). The legacy `POST /lora/scan` runs the same stages under its SSE
|
||||
event shape; a page that closes its connection stops staging before its next LoRA, and the
|
||||
scan keeps its records hold until staging has stopped, then ends `cancelled` in Activity
|
||||
(`test_a_disconnected_legacy_scan_stops_and_holds_until_it_stopped`) (`test_legacy_scan_events_unchanged_on_success`), holds
|
||||
the registry as "LoRA import (admin page)" and takes no safety backup. Both imports are
|
||||
`records` jobs, so a restore (`exclusive`, #249 S8) is refused while one runs and they are
|
||||
refused while a restore runs; the review is a request lease ("LoRA import review"), refused
|
||||
(409 `busy`) during a restore, and a restore waits for it (`LoraImportRestoreTests`). `POST /lora/enable`
|
||||
accepts the optional `expected_revision`. LoraManager detection also searches every
|
||||
`custom_nodes` folder ComfyUI registers through `folder_paths` (#144), main folder first.
|
||||
|
||||
`tests/browser/workspace_integrations_fixture.py` (`/__test__/integrations/*`) makes a fake
|
||||
LoraManager install and LoRA folder under the server's temp root, holds or fails the
|
||||
import, saves a blocking pair and answers CivitAI downloads with a stub opener, so no
|
||||
test reaches the network.
|
||||
|
||||
```sh
|
||||
python3 -m unittest tests.test_lora_import tests.test_api_lora tests.test_lora_database \
|
||||
tests.test_lora_utils -v
|
||||
node --test tests/js/workspace-system-integrations.test.js tests/js/prompt-bridge-adapter.test.js
|
||||
npm run --prefix tools/vendor test:workflows -- workspace-system-integrations.spec.mjs \
|
||||
system-integrations-concept.spec.mjs \
|
||||
--project chromium --project touch --project firefox --project webkit
|
||||
```
|
||||
|
||||
### Browser cache ownership (#222)
|
||||
|
||||
The legacy gallery's Clear cache used to delete every localStorage key starting `pm_` or
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
"""Atomic replacement of one imported category (#221, #249 System S10).
|
||||
|
||||
The LoRA reference import used to delete its whole category and then save
|
||||
each record in its own transaction, so a failure part way left the previous
|
||||
collection deleted and the new one partial. ``replace_category_prompts``
|
||||
does both in one ``BEGIN IMMEDIATE`` transaction: the category's current
|
||||
prompts (with their tags and image links) are replaced by the staged
|
||||
records, or, on any error, nothing changes.
|
||||
|
||||
A staged record is ``{text, hash, tags, images}``. A record whose hash a
|
||||
prompt outside the category already has links its images to that prompt
|
||||
(the legacy import did the same); a second record with the hash of an
|
||||
earlier one links its images to that one. Neither is a new prompt.
|
||||
"""
|
||||
|
||||
from typing import Any, Dict, List, Optional, Sequence
|
||||
|
||||
try:
|
||||
from .maintenance import PlanChanged
|
||||
from .pairs import PairDependencyError
|
||||
from .workflow_documents import documents_of_rows, release_documents
|
||||
except ImportError: # pragma: no cover - flat test imports
|
||||
from database.maintenance import PlanChanged
|
||||
from database.pairs import PairDependencyError
|
||||
from database.workflow_documents import documents_of_rows, release_documents
|
||||
|
||||
|
||||
class EmptyReplacement(ValueError):
|
||||
"""The replacement has no record and would remove every current import."""
|
||||
|
||||
|
||||
# The category's prompt ids, as a subquery (one bound parameter).
|
||||
_IN_CATEGORY = "SELECT id FROM prompts WHERE category = ?"
|
||||
|
||||
# Each record's outcome and the result counter it adds to.
|
||||
_COUNTERS = {"imported": "imported", "linked": "linked", "duplicate": "duplicates"}
|
||||
|
||||
|
||||
def _placeholders(values: Sequence[Any]) -> str:
|
||||
return ",".join("?" * len(values))
|
||||
|
||||
|
||||
class CategoryImportOperations:
|
||||
"""Read the state of an imported category and replace it atomically."""
|
||||
|
||||
def category_import_state(self, category: str) -> Dict[str, Any]:
|
||||
"""``{ids, paired}``: the category's prompt ids (ascending) and how
|
||||
many of them a saved pair uses (those block a replacement)."""
|
||||
with self.model.get_connection() as conn:
|
||||
ids = [
|
||||
row[0]
|
||||
for row in conn.execute(
|
||||
"SELECT id FROM prompts WHERE category = ? ORDER BY id",
|
||||
(category,),
|
||||
).fetchall()
|
||||
]
|
||||
paired = conn.execute(
|
||||
"SELECT COUNT(*) FROM prompts WHERE category = ? AND id IN "
|
||||
"(SELECT positive_prompt_id FROM prompt_pairs UNION "
|
||||
"SELECT negative_prompt_id FROM prompt_pairs)",
|
||||
(category,),
|
||||
).fetchone()[0]
|
||||
return {"ids": ids, "paired": paired}
|
||||
|
||||
def existing_prompt_hashes(
|
||||
self, hashes: Sequence[str], exclude_category: str
|
||||
) -> set:
|
||||
"""The given hashes a prompt outside ``exclude_category`` already has."""
|
||||
found = set()
|
||||
wanted = list(dict.fromkeys(hashes))
|
||||
with self.model.get_connection() as conn:
|
||||
for start in range(0, len(wanted), 500):
|
||||
chunk = wanted[start : start + 500]
|
||||
rows = conn.execute(
|
||||
f"SELECT hash FROM prompts WHERE hash IN ({_placeholders(chunk)}) "
|
||||
"AND (category IS NULL OR category != ?)",
|
||||
[*chunk, exclude_category],
|
||||
).fetchall()
|
||||
found.update(row[0] for row in rows)
|
||||
return found
|
||||
|
||||
def replace_category_prompts(
|
||||
self,
|
||||
category: str,
|
||||
records: List[Dict[str, Any]],
|
||||
*,
|
||||
expected_ids: Optional[List[int]] = None,
|
||||
allow_empty: bool = False,
|
||||
) -> Dict[str, Any]:
|
||||
"""Replace every prompt of ``category`` with ``records`` in one transaction.
|
||||
|
||||
Raises (and changes nothing): ``PlanChanged`` when ``expected_ids`` is
|
||||
given and the category's prompt ids differ from it;
|
||||
``PairDependencyError`` when a saved pair uses one of its prompts;
|
||||
``EmptyReplacement`` when ``records`` is empty while the category has
|
||||
prompts, unless ``allow_empty`` (an empty replacement that was
|
||||
reviewed). Any other error rolls everything back too.
|
||||
|
||||
Returns ``{removed, imported, linked, duplicates, images, rows}``:
|
||||
prompts removed and created, records linked to an existing prompt or
|
||||
to an earlier record with the same text, image links added, and per
|
||||
record ``{name, outcome}`` (``imported``/``linked``/``duplicate``).
|
||||
"""
|
||||
with self._pair_transaction() as conn:
|
||||
ids = [
|
||||
row[0]
|
||||
for row in conn.execute(
|
||||
"SELECT id FROM prompts WHERE category = ? ORDER BY id",
|
||||
(category,),
|
||||
).fetchall()
|
||||
]
|
||||
if expected_ids is not None and ids != list(expected_ids):
|
||||
raise PlanChanged("The imported prompts changed since the review")
|
||||
if not records and ids and not allow_empty:
|
||||
raise EmptyReplacement(
|
||||
"Nothing to import; the previous imports were kept"
|
||||
)
|
||||
self._ensure_category_unpaired(conn, category)
|
||||
removed = self._delete_category_rows(conn, category, ids)
|
||||
result = {
|
||||
"removed": removed,
|
||||
"imported": 0,
|
||||
"linked": 0,
|
||||
"duplicates": 0,
|
||||
"images": 0,
|
||||
"rows": [],
|
||||
}
|
||||
created = {}
|
||||
for record in records:
|
||||
self._import_record(conn, category, record, created, result)
|
||||
return result
|
||||
|
||||
def _delete_category_rows(self, conn, category: str, ids: List[int]) -> int:
|
||||
"""Delete the category's prompts with their image links and tags.
|
||||
|
||||
No commit. The rows are selected by category (just verified against
|
||||
``ids`` under the write lock), so no statement binds one parameter per
|
||||
prompt and SQLite's variable limit never caps a collection.
|
||||
"""
|
||||
if not ids:
|
||||
return 0
|
||||
documents = documents_of_rows(conn, "prompt_id", ids) # chunked
|
||||
conn.execute(
|
||||
f"DELETE FROM generated_images WHERE prompt_id IN ({_IN_CATEGORY})",
|
||||
(category,),
|
||||
)
|
||||
release_documents(conn, documents)
|
||||
conn.execute(
|
||||
f"DELETE FROM prompt_tags WHERE prompt_id IN ({_IN_CATEGORY})",
|
||||
(category,),
|
||||
)
|
||||
return conn.execute(
|
||||
"DELETE FROM prompts WHERE category = ?", (category,)
|
||||
).rowcount
|
||||
|
||||
@staticmethod
|
||||
def _ensure_category_unpaired(conn, category: str) -> None:
|
||||
"""Raise PairDependencyError when a saved pair uses a category prompt."""
|
||||
row = conn.execute(
|
||||
f"SELECT id FROM prompt_pairs WHERE positive_prompt_id IN ({_IN_CATEGORY}) "
|
||||
f"OR negative_prompt_id IN ({_IN_CATEGORY}) LIMIT 1",
|
||||
(category, category),
|
||||
).fetchone()
|
||||
if row:
|
||||
raise PairDependencyError(
|
||||
f"Prompt belongs to saved pair {row[0]}; delete the pair first "
|
||||
"or save changed text as a new prompt"
|
||||
)
|
||||
|
||||
def _import_record(self, conn, category, record, created, result) -> None:
|
||||
"""Save one staged record on ``conn``, or link it to the prompt it repeats."""
|
||||
prompt_hash = record["hash"]
|
||||
name = record.get("name") or record["text"]
|
||||
if prompt_hash in created:
|
||||
prompt_id, outcome = created[prompt_hash], "duplicate"
|
||||
else:
|
||||
row = conn.execute(
|
||||
"SELECT id FROM prompts WHERE hash = ?", (prompt_hash,)
|
||||
).fetchone()
|
||||
if row is not None:
|
||||
prompt_id, outcome = row[0], "linked"
|
||||
else:
|
||||
prompt_id = self._insert_prompt(
|
||||
conn,
|
||||
record["text"],
|
||||
category,
|
||||
list(record.get("tags") or []),
|
||||
None,
|
||||
None,
|
||||
prompt_hash,
|
||||
)
|
||||
outcome = "imported"
|
||||
created[prompt_hash] = prompt_id
|
||||
result[_COUNTERS[outcome]] += 1
|
||||
for image in record.get("images") or []:
|
||||
if self._insert_image_link(conn, prompt_id, image):
|
||||
result["images"] += 1
|
||||
result["rows"].append({"name": name, "outcome": outcome})
|
||||
+100
-70
@@ -26,6 +26,7 @@ from .history_reads import HistoryReadOperations
|
||||
from .health import HealthOperations, iter_consistency_issues
|
||||
from .duplicate_copies import DuplicateCopyOperations
|
||||
from .workflow_storage import WorkflowStorageOperations
|
||||
from .category_import import CategoryImportOperations, EmptyReplacement # noqa: F401
|
||||
from .workflow_documents import (
|
||||
DocumentReader,
|
||||
documents_of_rows,
|
||||
@@ -366,6 +367,7 @@ class PromptDatabase(
|
||||
MaintenanceOperations,
|
||||
DuplicateCopyOperations,
|
||||
WorkflowStorageOperations,
|
||||
CategoryImportOperations,
|
||||
):
|
||||
"""Database operations class for managing prompts."""
|
||||
|
||||
@@ -455,26 +457,9 @@ class PromptDatabase(
|
||||
|
||||
try:
|
||||
with self.model.get_connection() as conn:
|
||||
cursor = conn.execute(
|
||||
"""
|
||||
INSERT INTO prompts (
|
||||
text, category, tags, rating, notes, hash, created_at,
|
||||
updated_at
|
||||
) VALUES (?, ?, NULL, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
text.strip(),
|
||||
category,
|
||||
rating,
|
||||
notes,
|
||||
prompt_hash,
|
||||
datetime.datetime.now(datetime.timezone.utc).isoformat(),
|
||||
datetime.datetime.now(datetime.timezone.utc).isoformat(),
|
||||
),
|
||||
prompt_id = self._insert_prompt(
|
||||
conn, text, category, tags, rating, notes, prompt_hash
|
||||
)
|
||||
prompt_id = cursor.lastrowid
|
||||
if tags:
|
||||
self._sync_prompt_tags(conn, prompt_id, tags)
|
||||
conn.commit()
|
||||
self.logger.debug(f"Successfully saved prompt with ID: {prompt_id}")
|
||||
return prompt_id
|
||||
@@ -490,6 +475,32 @@ class PromptDatabase(
|
||||
)
|
||||
return existing["id"]
|
||||
|
||||
def _insert_prompt(
|
||||
self,
|
||||
conn: sqlite3.Connection,
|
||||
text: str,
|
||||
category: Optional[str],
|
||||
tags: Optional[List[str]],
|
||||
rating: Optional[int],
|
||||
notes: Optional[str],
|
||||
prompt_hash: Optional[str],
|
||||
) -> int:
|
||||
"""Insert one prompt row and its tags on ``conn`` (no commit)."""
|
||||
now = datetime.datetime.now(datetime.timezone.utc).isoformat()
|
||||
cursor = conn.execute(
|
||||
"""
|
||||
INSERT INTO prompts (
|
||||
text, category, tags, rating, notes, hash, created_at,
|
||||
updated_at
|
||||
) VALUES (?, ?, NULL, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(text.strip(), category, rating, notes, prompt_hash, now, now),
|
||||
)
|
||||
prompt_id = cursor.lastrowid
|
||||
if tags:
|
||||
self._sync_prompt_tags(conn, prompt_id, tags)
|
||||
return prompt_id
|
||||
|
||||
def record_prompt_use(self, prompt_id: int, times: int = 1) -> bool:
|
||||
"""
|
||||
Count runs of a prompt and mark it as most recently used.
|
||||
@@ -1670,64 +1681,83 @@ class PromptDatabase(
|
||||
return 0
|
||||
|
||||
# Proceed with linking
|
||||
if not isinstance(metadata, dict):
|
||||
metadata = {}
|
||||
filename = os.path.basename(image_path)
|
||||
file_path = normalize_image_path(image_path)
|
||||
file_size, width, height, image_format = _coerce_file_info(metadata)
|
||||
# Each workflow and prompt graph is stored once (new form).
|
||||
documents = stored_values(
|
||||
conn,
|
||||
{
|
||||
"workflow_data": dumps_strict(metadata.get("workflow", {})),
|
||||
"prompt_metadata": dumps_strict(metadata.get("prompt", {})),
|
||||
},
|
||||
image_id = self._insert_image_link(
|
||||
conn, prompt_id_int, image_path, metadata
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
# INSERT OR IGNORE skips a file already linked to this prompt
|
||||
# (same prompt_id + normalised full path)
|
||||
cursor = conn.execute(
|
||||
"""
|
||||
INSERT OR IGNORE INTO generated_images
|
||||
(prompt_id, image_path, filename, file_path, file_size, width,
|
||||
height, format, workflow_data, prompt_metadata, parameters,
|
||||
workflow_document_id, prompt_document_id)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
if not image_id:
|
||||
self.logger.debug(
|
||||
f"Image {normalize_image_path(image_path)} already linked to "
|
||||
f"prompt {prompt_id_int}"
|
||||
)
|
||||
return 0
|
||||
|
||||
return image_id
|
||||
|
||||
def _insert_image_link(
|
||||
self,
|
||||
conn: sqlite3.Connection,
|
||||
prompt_id_int: int,
|
||||
image_path: str,
|
||||
metadata: Optional[Dict[str, Any]] = None,
|
||||
) -> int:
|
||||
"""Insert one image link on ``conn`` (no commit).
|
||||
|
||||
Returns the new image row's id, or 0 when the file is already linked
|
||||
to this prompt (``INSERT OR IGNORE``).
|
||||
"""
|
||||
if not isinstance(metadata, dict):
|
||||
metadata = {}
|
||||
filename = os.path.basename(image_path)
|
||||
file_path = normalize_image_path(image_path)
|
||||
file_size, width, height, image_format = _coerce_file_info(metadata)
|
||||
# Each workflow and prompt graph is stored once (new form).
|
||||
documents = stored_values(
|
||||
conn,
|
||||
{
|
||||
"workflow_data": dumps_strict(metadata.get("workflow", {})),
|
||||
"prompt_metadata": dumps_strict(metadata.get("prompt", {})),
|
||||
},
|
||||
)
|
||||
|
||||
# INSERT OR IGNORE skips a file already linked to this prompt
|
||||
# (same prompt_id + normalised full path)
|
||||
cursor = conn.execute(
|
||||
"""
|
||||
INSERT OR IGNORE INTO generated_images
|
||||
(prompt_id, image_path, filename, file_path, file_size, width,
|
||||
height, format, workflow_data, prompt_metadata, parameters,
|
||||
workflow_document_id, prompt_document_id)
|
||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
prompt_id_int,
|
||||
image_path,
|
||||
filename,
|
||||
file_path,
|
||||
file_size,
|
||||
width,
|
||||
height,
|
||||
image_format,
|
||||
documents["workflow_data"],
|
||||
documents["prompt_metadata"],
|
||||
dumps_strict(metadata.get("parameters", {})),
|
||||
documents["workflow_document_id"],
|
||||
documents["prompt_document_id"],
|
||||
),
|
||||
)
|
||||
if cursor.rowcount == 0:
|
||||
# Already linked: a document made only for this row goes.
|
||||
release_documents(
|
||||
conn,
|
||||
(
|
||||
prompt_id_int,
|
||||
image_path,
|
||||
filename,
|
||||
file_path,
|
||||
file_size,
|
||||
width,
|
||||
height,
|
||||
image_format,
|
||||
documents["workflow_data"],
|
||||
documents["prompt_metadata"],
|
||||
dumps_strict(metadata.get("parameters", {})),
|
||||
documents["workflow_document_id"],
|
||||
documents["prompt_document_id"],
|
||||
),
|
||||
)
|
||||
if cursor.rowcount == 0:
|
||||
# Already linked: a document made only for this row goes.
|
||||
release_documents(
|
||||
conn,
|
||||
(
|
||||
documents["workflow_document_id"],
|
||||
documents["prompt_document_id"],
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
if cursor.rowcount == 0:
|
||||
self.logger.debug(
|
||||
f"Image {file_path} already linked to prompt {prompt_id_int}"
|
||||
)
|
||||
return 0
|
||||
|
||||
return cursor.lastrowid
|
||||
return 0
|
||||
return cursor.lastrowid
|
||||
|
||||
def get_prompt_images(self, prompt_id: str) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
|
||||
@@ -19,6 +19,7 @@ Split into domain-specific mixins for maintainability:
|
||||
- MaintenanceRoutesMixin: reviewed maintenance plans (preview, apply)
|
||||
- WorkflowStorageRoutesMixin: store each workflow once, then Reclaim space
|
||||
- DuplicateFilesMixin: duplicate image files as find and removal jobs
|
||||
- LoraImportRoutesMixin: the reviewed, staged LoRA reference import job
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
@@ -64,6 +65,7 @@ from .jobs import JobRegistry
|
||||
from .logging_routes import LoggingRoutesMixin
|
||||
from .autotag_routes import AutotagRoutesMixin
|
||||
from .lora_integration import LoraIntegrationMixin
|
||||
from .lora_import_routes import LoraImportRoutesMixin
|
||||
|
||||
try:
|
||||
from ...database.operations import PromptDatabase
|
||||
@@ -309,6 +311,7 @@ class PromptManagerAPI(
|
||||
LoggingRoutesMixin,
|
||||
AutotagRoutesMixin,
|
||||
LoraIntegrationMixin,
|
||||
LoraImportRoutesMixin,
|
||||
):
|
||||
"""REST API handler for PromptManager operations and web interface.
|
||||
|
||||
@@ -557,6 +560,7 @@ class PromptManagerAPI(
|
||||
self._register_logging_routes(routes)
|
||||
self._register_autotag_routes(routes)
|
||||
self._register_lora_routes(routes)
|
||||
self._register_lora_import_routes(routes)
|
||||
|
||||
# Register gzip compression middleware (once)
|
||||
global _gzip_registered
|
||||
|
||||
@@ -0,0 +1,454 @@
|
||||
"""The staged LoRA reference import as a reviewed Activity job (#221, S10).
|
||||
|
||||
``POST /prompt_manager/lora/import/preview`` (body ``{}``) reviews an import:
|
||||
it lists the LoraManager metadata files and parses them, and counts the
|
||||
current imports, those a saved pair uses and the example images an import
|
||||
would download. It writes nothing and requests nothing. It answers a
|
||||
``Plan`` (the maintenance plan's shape, ``py/api/maintenance.py``) plus
|
||||
``{lora_files, with_metadata, current_imports, blocked_by_pairs,
|
||||
planned_downloads, network:{planned_downloads, api_key}}``. A plan that saved
|
||||
pairs block carries no token (nothing can be applied).
|
||||
|
||||
``POST /prompt_manager/lora/import/apply {token}`` starts the job (202
|
||||
``{job}``): kind ``lora_import``, titled "LoRA reference import", admission
|
||||
``records``, cancellable while it downloads only. Refused before it starts:
|
||||
``plan-changed`` (409: the metadata files changed since the review),
|
||||
``plan-expired``, ``plan-used``, ``storage-changed``, ``busy``,
|
||||
``insufficient-space`` (as maintenance). The job stages the records
|
||||
(downloads into the cache first, each counted in ``downloaded`` or
|
||||
``download_errors``), takes the safety backup the policy promises, then
|
||||
replaces the collection in one transaction bound to the reviewed imports. Any
|
||||
failure leaves the previous imports exactly as they were, and the job says so.
|
||||
|
||||
The legacy ``POST /lora/scan`` runs the same stages (``stage_lora_import``
|
||||
and ``replace_lora_imports``) under its SSE event shape.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sqlite3
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from .. import lora_import, lora_utils
|
||||
from .job_routes import busy_response
|
||||
from .jobs import LORA_IMPORT_JOB_KIND, JobBusy
|
||||
from .maintenance import (
|
||||
MAX_TOKEN_LENGTH,
|
||||
_failure,
|
||||
_fail_job,
|
||||
_invalid,
|
||||
_iso,
|
||||
disk_plan,
|
||||
short_space_message,
|
||||
)
|
||||
from .maintenance_plans import ScopeTooLarge
|
||||
|
||||
try:
|
||||
from ...database.category_import import EmptyReplacement
|
||||
from ...database.maintenance import PlanChanged
|
||||
from ...database.pairs import PairDependencyError
|
||||
except ImportError: # pragma: no cover - flat test imports
|
||||
from database.category_import import EmptyReplacement
|
||||
from database.maintenance import PlanChanged
|
||||
from database.pairs import PairDependencyError
|
||||
|
||||
LORA_OPERATION = "lora_import"
|
||||
# The review reads the library: a request lease, so it is refused while a
|
||||
# restore runs and a restore waits for it before switching the file (S8).
|
||||
LORA_PREVIEW_LEASE = "LoRA import review"
|
||||
CATEGORY = lora_import.LORA_CATEGORY
|
||||
SOURCE_CHANGED = (
|
||||
"The LoRA metadata files changed since the review; nothing was changed. "
|
||||
"Review the import again."
|
||||
)
|
||||
IMPORTS_CHANGED = (
|
||||
"The imported LoRA prompts changed since the review; the previous imports "
|
||||
"were kept. Review the import again."
|
||||
)
|
||||
BLOCKED = (
|
||||
"Saved pairs use some imported LoRA prompts, so the collection cannot be "
|
||||
"replaced; the previous imports were kept."
|
||||
)
|
||||
INTEGRATION_CHANGED = (
|
||||
"The LoraManager integration changed since the review; nothing was changed. "
|
||||
"Review the import again."
|
||||
)
|
||||
EMPTY = "No LoRA could be prepared for import; the previous imports were kept."
|
||||
FAILED = "The import failed; the previous LoRA imports were kept."
|
||||
CACHE_NOTE = "Images already downloaded stay in the image cache."
|
||||
CANCELLED = "Cancelled before anything was written; the previous imports were kept."
|
||||
DISABLED = "LoraManager integration is not enabled"
|
||||
NO_PATH = "LoraManager path not configured"
|
||||
|
||||
|
||||
def _noun(count, word, plural=None):
|
||||
return word if count == 1 else (plural or word + "s")
|
||||
|
||||
|
||||
def _count(count, word, plural=None):
|
||||
return f"{count:,} {_noun(count, word, plural)}"
|
||||
|
||||
|
||||
def _import_line(result, skipped):
|
||||
parts = [f"Imported {_count(result['imported'], 'LoRA')}"]
|
||||
if result["linked"] or result["duplicates"]:
|
||||
linked = result["linked"] + result["duplicates"]
|
||||
parts.append(f"{linked:,} linked to an existing prompt")
|
||||
if skipped:
|
||||
parts.append(f"{skipped:,} skipped")
|
||||
return " · ".join(parts)
|
||||
|
||||
|
||||
class LoraImportRoutesMixin:
|
||||
"""Mixin providing the reviewed LoRA import (preview, apply, job)."""
|
||||
|
||||
def _register_lora_import_routes(self, routes):
|
||||
@routes.post("/prompt_manager/lora/import/preview")
|
||||
async def lora_import_preview_route(request):
|
||||
return await self.preview_lora_import(request)
|
||||
|
||||
@routes.post("/prompt_manager/lora/import/apply")
|
||||
async def lora_import_apply_route(request):
|
||||
return await self.apply_lora_import(request)
|
||||
|
||||
# ── shared stages ─────────────────────────────────────────────────
|
||||
|
||||
async def _lora_source(self):
|
||||
"""``(path, None)``: the configured, allowed LoraManager path, or
|
||||
``(None, (code, message))`` when the import cannot run."""
|
||||
from ..config import IntegrationConfig
|
||||
|
||||
if not IntegrationConfig.LORA_MANAGER_ENABLED:
|
||||
return None, ("lora-disabled", DISABLED)
|
||||
lm_path = IntegrationConfig.LORA_MANAGER_PATH
|
||||
if not lm_path:
|
||||
return None, ("lora-path", NO_PATH)
|
||||
allowed = await self._run_in_executor(
|
||||
lora_utils.resolve_lora_manager_path, lm_path
|
||||
)
|
||||
if allowed is None:
|
||||
from .lora_integration import _LORA_PATH_ERROR
|
||||
|
||||
return None, ("lora-path", _LORA_PATH_ERROR)
|
||||
return allowed, None
|
||||
|
||||
async def stage_lora_import(self, files, *, on_record=None, cancelled=None):
|
||||
"""Stage ``files`` off the loop: records, skips and download counts."""
|
||||
from ..config import IntegrationConfig
|
||||
|
||||
cache_dir = await self._run_in_executor(lora_utils.get_lora_image_cache_dir)
|
||||
await self._run_in_executor(lora_utils.sweep_stale_downloads, cache_dir)
|
||||
loop = asyncio.get_running_loop()
|
||||
|
||||
def report(index, record, staged):
|
||||
if on_record is not None:
|
||||
loop.call_soon_threadsafe(on_record, index, record, staged)
|
||||
|
||||
return await self._run_in_executor(
|
||||
lambda: lora_import.stage(
|
||||
files,
|
||||
cache_dir,
|
||||
IntegrationConfig.CIVITAI_API_KEY,
|
||||
on_record=report,
|
||||
cancelled=cancelled or (lambda: False),
|
||||
logger=self.logger,
|
||||
)
|
||||
)
|
||||
|
||||
async def lora_files_unchanged(self, lm_path, expected):
|
||||
"""Whether the metadata files under ``lm_path`` still fingerprint as
|
||||
``expected`` (listed again, so an added file counts too)."""
|
||||
|
||||
def current():
|
||||
return lora_import.fingerprint(lora_import.discover(lm_path))
|
||||
|
||||
try:
|
||||
return await self._run_in_executor(current) == expected
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
async def replace_lora_imports(self, db, staged, *, expected_ids, allow_empty):
|
||||
"""Swap the collection for ``staged.records`` in one transaction."""
|
||||
return await self._run_in_executor(
|
||||
lambda: db.replace_category_prompts(
|
||||
CATEGORY,
|
||||
staged.records,
|
||||
expected_ids=expected_ids,
|
||||
allow_empty=allow_empty,
|
||||
)
|
||||
)
|
||||
|
||||
# ── preview ───────────────────────────────────────────────────────
|
||||
|
||||
async def preview_lora_import(self, request):
|
||||
try:
|
||||
async with self._jobs.lease(LORA_PREVIEW_LEASE):
|
||||
return await self._preview_lora_import(request)
|
||||
except JobBusy as busy:
|
||||
return busy_response(busy)
|
||||
|
||||
async def _preview_lora_import(self, request):
|
||||
data, error = await self._read_maintenance_body(request, ())
|
||||
if error is not None:
|
||||
return error
|
||||
lm_path, refused = await self._lora_source()
|
||||
if refused is not None:
|
||||
return _failure(refused[1], 400, refused[0])
|
||||
db = self.db
|
||||
identity = self._database_identity(db)
|
||||
try:
|
||||
cache_dir = await self._run_in_executor(
|
||||
lambda: lora_utils.get_lora_image_cache_dir(create=False)
|
||||
)
|
||||
found = await self._run_in_executor(lora_import.preview, lm_path, cache_dir)
|
||||
state = await self._run_in_executor(db.category_import_state, CATEGORY)
|
||||
except Exception:
|
||||
self.logger.exception("LoRA import preview failed")
|
||||
return _failure("The LoRA import could not be reviewed.", 500)
|
||||
if identity != self._database_identity(self.db):
|
||||
return _failure(
|
||||
"The database was replaced while it was reviewed. Review again.",
|
||||
409,
|
||||
"storage-changed",
|
||||
)
|
||||
return await self._lora_plan_response(db, identity, lm_path, found, state)
|
||||
|
||||
async def _lora_plan_response(self, db, identity, lm_path, found, state):
|
||||
from ..config import IntegrationConfig
|
||||
|
||||
current = len(state["ids"])
|
||||
blocked = state["paired"]
|
||||
policy = self._safety_backup_policy()
|
||||
try:
|
||||
disk, _short, size = await self._run_in_executor(
|
||||
disk_plan, db.model.db_path, backup=policy != "off", vacuum=False
|
||||
)
|
||||
except (OSError, sqlite3.Error):
|
||||
self.logger.exception("Could not measure the disk space for a plan")
|
||||
return _failure("The LoRA import could not be reviewed.", 500)
|
||||
token = None
|
||||
expires_at = None
|
||||
if not blocked:
|
||||
payload = {
|
||||
"path": lm_path,
|
||||
"fingerprint": found["fingerprint"],
|
||||
"expected_ids": state["ids"],
|
||||
"empty": found["with_metadata"] == 0,
|
||||
"backup": policy,
|
||||
}
|
||||
try:
|
||||
token = self._maintenance_plans.issue(
|
||||
LORA_OPERATION, found["fingerprint"], payload, identity
|
||||
)
|
||||
except ScopeTooLarge:
|
||||
return _failure(
|
||||
"This import is too large to review at once.", 413, "too-large"
|
||||
)
|
||||
expires_at = _iso(self._maintenance_plans.peek(token).expires_at)
|
||||
name = os.path.basename(db.model.db_path)
|
||||
downloads = found["planned_downloads"]
|
||||
reasons = (
|
||||
[f"{_count(blocked, 'imported prompt')} used by saved pairs"]
|
||||
if blocked
|
||||
else []
|
||||
)
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
"operation": LORA_OPERATION,
|
||||
"token": token,
|
||||
"expires_at": expires_at,
|
||||
"lora_files": found["lora_files"],
|
||||
"with_metadata": found["with_metadata"],
|
||||
"current_imports": current,
|
||||
"blocked_by_pairs": blocked,
|
||||
"planned_downloads": downloads,
|
||||
"scope": {
|
||||
"count": found["with_metadata"],
|
||||
"label": (
|
||||
f"Replace {_count(current, 'imported prompt')} with "
|
||||
f"{_count(found['with_metadata'], 'LoRA')}"
|
||||
),
|
||||
},
|
||||
"protected": {"count": blocked, "reasons": reasons},
|
||||
"backup": {
|
||||
"policy": policy,
|
||||
"destination_label": f"{name}.backup_<date and time>",
|
||||
"expected_bytes": size if policy != "off" else None,
|
||||
},
|
||||
"media": {
|
||||
"deletes_files": False,
|
||||
"label": "No files are removed; image links are replaced",
|
||||
},
|
||||
"network": {
|
||||
"planned_downloads": downloads,
|
||||
"api_key": bool(IntegrationConfig.CIVITAI_API_KEY),
|
||||
},
|
||||
"details": [{"label": n, "detail": ""} for n in found["names"]],
|
||||
"disk": disk,
|
||||
},
|
||||
headers={"Cache-Control": "no-store"},
|
||||
)
|
||||
|
||||
# ── apply ─────────────────────────────────────────────────────────
|
||||
|
||||
async def apply_lora_import(self, request):
|
||||
data, error = await self._read_maintenance_body(request, ("token",))
|
||||
if error is not None:
|
||||
return error
|
||||
token = data.get("token")
|
||||
if not isinstance(token, str) or not 0 < len(token) <= MAX_TOKEN_LENGTH:
|
||||
return _invalid("token must be the plan's token")
|
||||
plan, refused = self._peek_plan(token)
|
||||
if refused is not None:
|
||||
return refused
|
||||
if plan.operation != LORA_OPERATION:
|
||||
return _failure(
|
||||
"This review expired. Review the plan again.", 410, "plan-expired"
|
||||
)
|
||||
store = self._maintenance_plans
|
||||
db = self.db
|
||||
if plan.db_identity != self._database_identity(db):
|
||||
store.discard(token)
|
||||
return _failure(
|
||||
"The database was replaced since the review. Review again.",
|
||||
409,
|
||||
"storage-changed",
|
||||
)
|
||||
lm_path = plan.payload["path"]
|
||||
# M5: the integration must still be on and at the reviewed install.
|
||||
current, _refused = await self._lora_source()
|
||||
if current != lm_path:
|
||||
store.discard(token)
|
||||
return _failure(INTEGRATION_CHANGED, 409, "plan-changed")
|
||||
try:
|
||||
files = await self._run_in_executor(lora_import.discover, lm_path)
|
||||
now = await self._run_in_executor(lora_import.fingerprint, files)
|
||||
backup = self._backup_promised(plan)
|
||||
_disk, short, _size = await self._run_in_executor(
|
||||
disk_plan, db.model.db_path, backup=backup, vacuum=False
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception("LoRA import could not start")
|
||||
return _failure(
|
||||
"The import could not start. Nothing was changed.", 500, "apply-failed"
|
||||
)
|
||||
if now != plan.fingerprint:
|
||||
store.discard(token)
|
||||
return _failure(SOURCE_CHANGED, 409, "plan-changed")
|
||||
if short is not None:
|
||||
return _failure(short_space_message(short), 507, "insufficient-space")
|
||||
plan, refused = self._peek_plan(token)
|
||||
if refused is not None:
|
||||
return refused
|
||||
try:
|
||||
job = self._jobs.start(
|
||||
LORA_IMPORT_JOB_KIND,
|
||||
lora_import.JOB_TITLE,
|
||||
admission="records",
|
||||
cancellable=True,
|
||||
scope={"operation": LORA_OPERATION, "lora_files": len(files)},
|
||||
groups=lora_import.GROUPS,
|
||||
runner=lambda job: self._run_lora_import(job, db, plan, files, backup),
|
||||
)
|
||||
except JobBusy as busy:
|
||||
return busy_response(busy)
|
||||
store.consume(token, LORA_OPERATION, job_id=job.id)
|
||||
return web.json_response({"success": True, "job": job.summary()}, status=202)
|
||||
|
||||
# ── the job ───────────────────────────────────────────────────────
|
||||
|
||||
async def _run_lora_import(self, job, db, plan, files, backup_on):
|
||||
total = len(files)
|
||||
job.update(total=total, status_line="Preparing LoRA records")
|
||||
job.log_line(f"Found {_count(total, 'LoRA metadata file')}")
|
||||
|
||||
def on_record(index, record, staged):
|
||||
job.update(
|
||||
processed=index + 1,
|
||||
progress=int(90 * (index + 1) / max(total, 1)),
|
||||
status_line=f"Prepared {index + 1:,} of {total:,}",
|
||||
)
|
||||
job.publish({"type": "progress", "progress": job.progress})
|
||||
|
||||
try:
|
||||
staged = await self.stage_lora_import(
|
||||
files, on_record=on_record, cancelled=lambda: job.cancelled
|
||||
)
|
||||
except lora_import.ImportCancelled:
|
||||
job.log_line(f"{CANCELLED} {CACHE_NOTE}")
|
||||
job.update(status_line=CANCELLED)
|
||||
job.finish("cancelled")
|
||||
return
|
||||
except Exception:
|
||||
self.logger.exception("LoRA import staging failed")
|
||||
return _fail_job(job, "import-failed", f"{FAILED} {CACHE_NOTE}")
|
||||
self._count_downloads(job, staged)
|
||||
# M4: what staging skipped stays on the job whatever happens next.
|
||||
job.record_counted(
|
||||
"skipped",
|
||||
len(staged.skipped),
|
||||
[{"path": "", "detail": name} for name in staged.skipped],
|
||||
)
|
||||
# Past this point nothing stops halfway: the replace is one transaction.
|
||||
# Published, so a view that offered Cancel during the downloads drops it.
|
||||
job.cancellable = False
|
||||
job.publish({"type": "progress", "progress": job.progress})
|
||||
if job.cancelled:
|
||||
job.log_line(f"{CANCELLED} {CACHE_NOTE}")
|
||||
job.update(status_line=CANCELLED)
|
||||
job.finish("cancelled")
|
||||
return
|
||||
ok, _backup = await self._safety_backup_step(job, db, backup_on)
|
||||
if not ok:
|
||||
return
|
||||
# I1: the files are listed and fingerprinted again right before the
|
||||
# replace, so a file changed, removed or added while the downloads ran
|
||||
# (LoraManager refreshing its CivitAI info) aborts with nothing deleted.
|
||||
if not await self.lora_files_unchanged(plan.payload["path"], plan.fingerprint):
|
||||
return _fail_job(job, "plan-changed", f"{SOURCE_CHANGED} {CACHE_NOTE}")
|
||||
job.update(status_line="Replacing the LoRA collection", progress=95)
|
||||
await self._replace_in_job(job, db, plan, staged)
|
||||
|
||||
def _count_downloads(self, job, staged):
|
||||
downloaded = staged.stats.get("downloaded", 0)
|
||||
errors = staged.stats.get("download_errors", 0)
|
||||
job.record_counted("downloaded", downloaded)
|
||||
job.record_counted("download_errors", errors)
|
||||
if downloaded or errors:
|
||||
job.log_line(
|
||||
f"Downloaded {_count(downloaded, 'example image')}"
|
||||
+ (f"; {errors:,} could not be downloaded" if errors else "")
|
||||
)
|
||||
|
||||
async def _replace_in_job(self, job, db, plan, staged):
|
||||
try:
|
||||
result = await self.replace_lora_imports(
|
||||
db,
|
||||
staged,
|
||||
expected_ids=plan.payload["expected_ids"],
|
||||
allow_empty=plan.payload["empty"],
|
||||
)
|
||||
except PlanChanged:
|
||||
return _fail_job(job, "plan-changed", f"{IMPORTS_CHANGED} {CACHE_NOTE}")
|
||||
except PairDependencyError:
|
||||
return _fail_job(job, "blocked-by-pairs", f"{BLOCKED} {CACHE_NOTE}")
|
||||
except EmptyReplacement:
|
||||
return _fail_job(job, "nothing-to-import", f"{EMPTY} {CACHE_NOTE}")
|
||||
except Exception:
|
||||
self.logger.exception("LoRA import failed")
|
||||
return _fail_job(job, "import-failed", f"{FAILED} {CACHE_NOTE}")
|
||||
rows = {"imported": [], "linked": []}
|
||||
for row in result["rows"]:
|
||||
group = "imported" if row["outcome"] == "imported" else "linked"
|
||||
rows[group].append({"path": "", "detail": row["name"]})
|
||||
job.record_counted("imported", result["imported"], rows["imported"])
|
||||
job.record_counted(
|
||||
"linked", result["linked"] + result["duplicates"], rows["linked"]
|
||||
)
|
||||
line = _import_line(result, len(staged.skipped))
|
||||
if result["removed"]:
|
||||
job.log_line(f"Replaced {_count(result['removed'], 'previous import')}")
|
||||
job.log_line(line)
|
||||
job.update(progress=100, processed=job.total, status_line=line)
|
||||
+193
-136
@@ -1,7 +1,9 @@
|
||||
"""LoraManager integration API routes for PromptManager."""
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
from pathlib import Path
|
||||
import threading
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
@@ -37,6 +39,21 @@ def _lora_settings_mutation(values):
|
||||
return mutate
|
||||
|
||||
|
||||
# The admin page's own words for the refusals it can meet (it has no review).
|
||||
_FILES_CHANGED = (
|
||||
"The LoRA metadata files changed during the scan; the previous imports "
|
||||
"were kept. Run the scan again."
|
||||
)
|
||||
_IMPORTS_CHANGED = (
|
||||
"The imported LoRA prompts changed during the scan; the previous imports "
|
||||
"were kept. Run the scan again."
|
||||
)
|
||||
|
||||
|
||||
class LoraFilesChanged(Exception):
|
||||
"""A metadata file changed, appeared or went while the legacy scan staged."""
|
||||
|
||||
|
||||
def _lora_error_frame(message):
|
||||
"""An import stream's error event, as the admin page's modal reads it.
|
||||
|
||||
@@ -46,6 +63,23 @@ def _lora_error_frame(message):
|
||||
return {"type": "error", "status": message, "progress": 0, "message": message}
|
||||
|
||||
|
||||
def _closed(request):
|
||||
"""A check (safe off the loop) for whether ``request``'s client left."""
|
||||
|
||||
def closed():
|
||||
transport = request.transport
|
||||
return transport is None or transport.is_closing()
|
||||
|
||||
return closed
|
||||
|
||||
|
||||
def _stopped(error) -> bool:
|
||||
"""Whether a legacy scan ended because its page closed the connection."""
|
||||
from .. import lora_import
|
||||
|
||||
return isinstance(error, (ConnectionError, lora_import.ImportCancelled))
|
||||
|
||||
|
||||
class LoraIntegrationMixin:
|
||||
"""Mixin providing LoraManager detection, scanning, and trigger word endpoints."""
|
||||
|
||||
@@ -108,7 +142,11 @@ class LoraIntegrationMixin:
|
||||
"""Get current LoraManager integration status."""
|
||||
try:
|
||||
from ..config import IntegrationConfig
|
||||
from ..lora_utils import detect_lora_manager, get_trigger_cache
|
||||
from ..lora_utils import (
|
||||
detect_lora_manager,
|
||||
get_trigger_cache,
|
||||
resolve_lora_manager_path,
|
||||
)
|
||||
|
||||
config = IntegrationConfig.get_config()["lora_manager"]
|
||||
cache = get_trigger_cache()
|
||||
@@ -117,6 +155,12 @@ class LoraIntegrationMixin:
|
||||
detected_path = await self._run_in_executor(
|
||||
detect_lora_manager, config.get("path", "")
|
||||
)
|
||||
# A detected install reached through a link outside custom_nodes is
|
||||
# found but cannot be enabled (S10 review N5): say which.
|
||||
usable = bool(detected_path) and (
|
||||
await self._run_in_executor(resolve_lora_manager_path, detected_path)
|
||||
is not None
|
||||
)
|
||||
|
||||
return web.json_response(
|
||||
{
|
||||
@@ -126,6 +170,7 @@ class LoraIntegrationMixin:
|
||||
"trigger_words_enabled": config["trigger_words_enabled"],
|
||||
"has_civitai_api_key": bool(config.get("civitai_api_key")),
|
||||
"detected": detected_path is not None,
|
||||
"detected_usable": usable,
|
||||
"detected_path": (
|
||||
self._public_path(detected_path) if detected_path else ""
|
||||
),
|
||||
@@ -168,6 +213,9 @@ class LoraIntegrationMixin:
|
||||
async def lora_enable(self, request):
|
||||
"""Enable or disable LoraManager integration and save it to config.json.
|
||||
|
||||
``expected_revision`` (optional, GET /settings' ``revision``) refuses
|
||||
a save over a config.json changed meanwhile (``settings-changed``).
|
||||
|
||||
The values are committed through the config store first (#220) and
|
||||
published at runtime only once config.json holds them; a save that
|
||||
cannot be written reports ``persisted: false`` and changes nothing.
|
||||
@@ -180,6 +228,11 @@ class LoraIntegrationMixin:
|
||||
for flag in _LORA_FLAGS:
|
||||
if flag in data and not isinstance(data[flag], bool):
|
||||
return self._bad_lora_request(f"{flag} must be true or false")
|
||||
# The System Save row sends the revision it read (#220); the
|
||||
# legacy page sends none and merges into what is on disk.
|
||||
expected_revision = data.get("expected_revision")
|
||||
if expected_revision is not None and not isinstance(expected_revision, str):
|
||||
return self._bad_lora_request("expected_revision must be a string")
|
||||
enabled = data.get("enabled", False)
|
||||
path = data.get("path", "")
|
||||
trigger_words = data.get("trigger_words_enabled", False)
|
||||
@@ -226,7 +279,7 @@ class LoraIntegrationMixin:
|
||||
_saved, revision = await self._run_in_executor(
|
||||
get_config_store().commit,
|
||||
_lora_settings_mutation(values),
|
||||
expected_revision=None,
|
||||
expected_revision=expected_revision,
|
||||
publish=publish,
|
||||
)
|
||||
except (ConfigUnreadable, RevisionConflict, OSError) as exc:
|
||||
@@ -308,43 +361,69 @@ class LoraIntegrationMixin:
|
||||
) as job:
|
||||
await response.prepare(request)
|
||||
try:
|
||||
await self._lora_scan_stream(allowed, send_progress)
|
||||
await self._lora_scan_stream(
|
||||
allowed, send_progress, disconnected=_closed(request)
|
||||
)
|
||||
except asyncio.CancelledError:
|
||||
# The server cancelled the handler when the page left.
|
||||
job.finish("cancelled")
|
||||
raise
|
||||
except Exception as e:
|
||||
self.logger.exception("LoRA scan failed")
|
||||
job.finish("failed")
|
||||
await send_progress(_lora_error_frame(self._public_error(e)))
|
||||
|
||||
await response.write_eof()
|
||||
# A page that left stopped the scan; nothing failed (R2).
|
||||
job.finish("cancelled" if _stopped(e) else "failed")
|
||||
await self._send_scan_failure(e, send_progress)
|
||||
with contextlib.suppress(ConnectionError, RuntimeError):
|
||||
await response.write_eof()
|
||||
return response
|
||||
|
||||
except Exception as e:
|
||||
self.logger.error(f"LoRA scan failed: {e}")
|
||||
return self._error_response(e)
|
||||
|
||||
async def _lora_scan_stream(self, lm_path, send_progress):
|
||||
"""Import every LoRA under ``lm_path``, reporting through ``send_progress``."""
|
||||
from ..config import IntegrationConfig
|
||||
from ..lora_utils import (
|
||||
download_civitai_images,
|
||||
find_lora_directories,
|
||||
get_example_prompt_from_metadata,
|
||||
get_lora_image_cache_dir,
|
||||
get_preview_images_from_metadata,
|
||||
get_trigger_words_from_metadata,
|
||||
get_model_name_from_metadata,
|
||||
read_lora_metadata,
|
||||
async def _send_scan_failure(self, error, send_progress):
|
||||
"""The error frame for a failed legacy scan; quiet when the page left.
|
||||
|
||||
Expected refusals (files or imports changed, saved pairs, nothing to
|
||||
import) are warnings in the admin page's own words; anything else is
|
||||
logged with its traceback.
|
||||
"""
|
||||
from .lora_import_routes import (
|
||||
EmptyReplacement,
|
||||
PairDependencyError,
|
||||
PlanChanged,
|
||||
)
|
||||
|
||||
await send_progress(
|
||||
{
|
||||
"type": "progress",
|
||||
"status": "Clearing previous lora-manager imports...",
|
||||
"progress": 0,
|
||||
}
|
||||
)
|
||||
if _stopped(error):
|
||||
self.logger.info("LoRA scan stopped: the page closed its connection")
|
||||
return
|
||||
if isinstance(error, PlanChanged):
|
||||
message = _IMPORTS_CHANGED
|
||||
elif isinstance(error, LoraFilesChanged):
|
||||
message = str(error)
|
||||
else:
|
||||
message = self._public_error(error)
|
||||
if isinstance(
|
||||
error,
|
||||
(PlanChanged, LoraFilesChanged, EmptyReplacement, PairDependencyError),
|
||||
):
|
||||
self.logger.warning(f"LoRA scan refused: {message}")
|
||||
else:
|
||||
self.logger.exception("LoRA scan failed")
|
||||
with contextlib.suppress(ConnectionError, RuntimeError):
|
||||
await send_progress(_lora_error_frame(message))
|
||||
|
||||
# Clear previous imports so reimport is always clean
|
||||
await self._run_in_executor(self.db.delete_prompts_by_category, "lora-manager")
|
||||
async def _lora_scan_stream(
|
||||
self, lm_path, send_progress, disconnected=lambda: False
|
||||
):
|
||||
"""Import every LoRA under ``lm_path``, reporting through ``send_progress``.
|
||||
|
||||
The staged path of the System import (#221, ``lora_import_routes``):
|
||||
discover, stage every record (downloads into the cache first), then
|
||||
replace the collection in one transaction. A failure at any stage
|
||||
raises with the previous imports untouched; an empty replacement of a
|
||||
non-empty collection is refused (it needs the reviewed import).
|
||||
"""
|
||||
from .. import lora_import
|
||||
|
||||
await send_progress(
|
||||
{
|
||||
@@ -353,19 +432,8 @@ class LoraIntegrationMixin:
|
||||
"progress": 2,
|
||||
}
|
||||
)
|
||||
|
||||
lora_dirs = await self._run_in_executor(find_lora_directories, lm_path)
|
||||
|
||||
# Collect all metadata files
|
||||
meta_files = []
|
||||
for d in lora_dirs:
|
||||
dir_path = Path(d)
|
||||
meta_files.extend(dir_path.rglob("*.metadata.json"))
|
||||
|
||||
total = len(meta_files)
|
||||
imported = 0
|
||||
skipped = 0
|
||||
|
||||
files = await self._run_in_executor(lora_import.discover, lm_path)
|
||||
total = len(files)
|
||||
await send_progress(
|
||||
{
|
||||
"type": "progress",
|
||||
@@ -375,116 +443,105 @@ class LoraIntegrationMixin:
|
||||
}
|
||||
)
|
||||
|
||||
cache_dir = get_lora_image_cache_dir()
|
||||
reviewed = await self._run_in_executor(lora_import.fingerprint, files)
|
||||
events = asyncio.Queue()
|
||||
state = await self._run_in_executor(
|
||||
self.db.category_import_state, lora_import.LORA_CATEGORY
|
||||
)
|
||||
# Records whose prompt already exists outside the collection are
|
||||
# linked, not imported (the running counts; the end ones are exact).
|
||||
known = set()
|
||||
|
||||
for i, meta_file in enumerate(meta_files):
|
||||
metadata = await self._run_in_executor(read_lora_metadata, meta_file)
|
||||
if not metadata:
|
||||
skipped += 1
|
||||
continue
|
||||
def on_record(index, record, _staged):
|
||||
events.put_nowait((index, record))
|
||||
|
||||
model_name = get_model_name_from_metadata(metadata)
|
||||
trigger_words = get_trigger_words_from_metadata(metadata)
|
||||
|
||||
# Collect all images: local previews + downloaded civitai examples
|
||||
preview_paths = await self._run_in_executor(
|
||||
get_preview_images_from_metadata, metadata, meta_file
|
||||
)
|
||||
civitai_paths = await self._run_in_executor(
|
||||
download_civitai_images,
|
||||
metadata,
|
||||
meta_file,
|
||||
cache_dir,
|
||||
IntegrationConfig.CIVITAI_API_KEY,
|
||||
)
|
||||
|
||||
# Merge, local first, dedup
|
||||
seen = set(preview_paths)
|
||||
all_images = list(preview_paths)
|
||||
for cp in civitai_paths:
|
||||
if cp not in seen:
|
||||
all_images.append(cp)
|
||||
seen.add(cp)
|
||||
|
||||
# Build prompt text: prefer example prompt, then model name
|
||||
example_prompt = get_example_prompt_from_metadata(metadata)
|
||||
prompt_text = example_prompt or model_name
|
||||
|
||||
# Build tags
|
||||
tags = ["lora-manager", f"lora:{model_name}"]
|
||||
tags.extend(trigger_words)
|
||||
|
||||
# Save to database via existing mechanism
|
||||
try:
|
||||
import hashlib
|
||||
|
||||
prompt_hash = hashlib.sha256(
|
||||
prompt_text.strip().lower().encode("utf-8")
|
||||
).hexdigest()
|
||||
counts = {"imported": 0}
|
||||
|
||||
async def report(index, record):
|
||||
if record is not None and record["hash"] not in known:
|
||||
known.add(record["hash"])
|
||||
existing = await self._run_in_executor(
|
||||
self.db.get_prompt_by_hash, prompt_hash
|
||||
self.db.existing_prompt_hashes,
|
||||
[record["hash"]],
|
||||
lora_import.LORA_CATEGORY,
|
||||
)
|
||||
|
||||
if existing:
|
||||
# Link all images
|
||||
for pp in all_images:
|
||||
await self._run_in_executor(
|
||||
self.db.link_image_to_prompt,
|
||||
existing["id"],
|
||||
pp,
|
||||
)
|
||||
skipped += 1
|
||||
else:
|
||||
prompt_id = await self._run_in_executor(
|
||||
self.db.save_prompt,
|
||||
prompt_text,
|
||||
"lora-manager", # category
|
||||
tags,
|
||||
None, # rating
|
||||
None, # notes
|
||||
prompt_hash,
|
||||
)
|
||||
|
||||
if prompt_id:
|
||||
for pp in all_images:
|
||||
await self._run_in_executor(
|
||||
self.db.link_image_to_prompt,
|
||||
prompt_id,
|
||||
pp,
|
||||
)
|
||||
imported += 1
|
||||
else:
|
||||
skipped += 1
|
||||
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Failed to import LoRA {model_name}: {e}")
|
||||
skipped += 1
|
||||
|
||||
# Progress update for every LoRA
|
||||
progress = int(5 + (90 * (i + 1) / max(total, 1)))
|
||||
img_count = len(all_images)
|
||||
status = f"{model_name}"
|
||||
if img_count:
|
||||
status += f" ({img_count} images)"
|
||||
if not existing:
|
||||
counts["imported"] += 1
|
||||
status = record["name"] if record else files[index].name
|
||||
if record and record["images"]:
|
||||
status += f" ({len(record['images'])} images)"
|
||||
await send_progress(
|
||||
{
|
||||
"type": "progress",
|
||||
"status": status,
|
||||
"progress": progress,
|
||||
"processed": i + 1,
|
||||
"imported": imported,
|
||||
"skipped": skipped,
|
||||
"progress": int(5 + (90 * (index + 1) / max(total, 1))),
|
||||
"processed": index + 1,
|
||||
"imported": counts["imported"],
|
||||
"skipped": index + 1 - counts["imported"],
|
||||
}
|
||||
)
|
||||
|
||||
# Each record's event is queued (through the loop) before staging
|
||||
# ends, so reporting until staging is done and then draining the
|
||||
# queue reports every record in order. I2: when the page goes away
|
||||
# (its connection closes, or a report cannot be sent) staging stops
|
||||
# before its next LoRA, and this returns only once it has stopped, so
|
||||
# the caller's records hold covers every download.
|
||||
stop = threading.Event()
|
||||
staging = asyncio.ensure_future(
|
||||
self.stage_lora_import(
|
||||
files,
|
||||
on_record=on_record,
|
||||
cancelled=lambda: stop.is_set() or disconnected(),
|
||||
)
|
||||
)
|
||||
try:
|
||||
while not staging.done():
|
||||
getter = asyncio.ensure_future(events.get())
|
||||
await asyncio.wait(
|
||||
{getter, staging}, return_when=asyncio.FIRST_COMPLETED
|
||||
)
|
||||
if getter.done():
|
||||
await report(*getter.result())
|
||||
else:
|
||||
getter.cancel()
|
||||
while not events.empty():
|
||||
await report(*events.get_nowait())
|
||||
finally:
|
||||
stop.set()
|
||||
# Staging must have stopped before the records hold is released,
|
||||
# even when the handler is cancelled again while it waits (R3).
|
||||
interrupted = False
|
||||
while not staging.done():
|
||||
try:
|
||||
await asyncio.shield(staging)
|
||||
except asyncio.CancelledError:
|
||||
interrupted = True
|
||||
except Exception:
|
||||
pass
|
||||
if interrupted:
|
||||
raise asyncio.CancelledError
|
||||
staged = await staging
|
||||
if not await self.lora_files_unchanged(lm_path, reviewed):
|
||||
raise LoraFilesChanged(_FILES_CHANGED)
|
||||
|
||||
await send_progress(
|
||||
{
|
||||
"type": "progress",
|
||||
"status": "Replacing previous lora-manager imports...",
|
||||
"progress": 96,
|
||||
}
|
||||
)
|
||||
result = await self.replace_lora_imports(
|
||||
self.db, staged, expected_ids=state["ids"], allow_empty=False
|
||||
)
|
||||
await send_progress(
|
||||
{
|
||||
"type": "complete",
|
||||
"progress": 100,
|
||||
"total": total,
|
||||
"imported": imported,
|
||||
"skipped": skipped,
|
||||
"imported": result["imported"],
|
||||
"skipped": total - result["imported"],
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
"""The staged LoRA reference import (#221, #249 System S10).
|
||||
|
||||
The import reads LoraManager's ``*.metadata.json`` files and turns each LoRA
|
||||
into one prompt of the ``lora-manager`` category, tagged with its name and
|
||||
trigger words and linked to its preview images. It runs in three stages, and
|
||||
only the last writes to the database:
|
||||
|
||||
1. **discover**: list the metadata files (``discover``) and fingerprint them
|
||||
(path, size and modification time; ``fingerprint``). ``preview`` adds the
|
||||
counts a review shows. Nothing is written and nothing is requested.
|
||||
2. **stage**: read each file and build its record; CivitAI example images are
|
||||
downloaded into the image cache first (best effort, counted). A cancelled
|
||||
or failed stage leaves the database untouched; downloaded images stay in
|
||||
the cache.
|
||||
3. **replace**: the files are listed and fingerprinted once more (a file
|
||||
changed, removed or added while staging aborts the import), then
|
||||
``PromptDatabase.replace_category_prompts`` swaps the old collection for
|
||||
the staged records in one transaction (or changes nothing).
|
||||
|
||||
Both the System job (``POST /lora/import/apply``) and the legacy ``POST
|
||||
/lora/scan`` run these stages (``py/api/lora_integration.py``).
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Callable, Dict, List, Optional
|
||||
|
||||
from . import lora_utils
|
||||
|
||||
try:
|
||||
from ..utils.hashing import generate_prompt_hash
|
||||
except ImportError: # pragma: no cover - flat test imports
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
LORA_CATEGORY = "lora-manager"
|
||||
JOB_TITLE = "LoRA reference import"
|
||||
# The job's outcome groups, in its counter-tile order.
|
||||
GROUPS = ("imported", "linked", "skipped", "downloaded", "download_errors", "errors")
|
||||
METADATA_SUFFIX = "*.metadata.json"
|
||||
|
||||
|
||||
class ImportCancelled(Exception):
|
||||
"""The import was cancelled while staging; nothing was written."""
|
||||
|
||||
|
||||
def discover(lora_manager_path: str) -> List[Path]:
|
||||
"""Every LoRA metadata file, sorted and without repeats.
|
||||
|
||||
Raises whatever listing a folder raised: a discovery that cannot see
|
||||
every folder must not replace the collection with part of it.
|
||||
"""
|
||||
seen = set()
|
||||
files = []
|
||||
for folder in lora_utils.find_lora_directories(lora_manager_path):
|
||||
for path in Path(folder).rglob(METADATA_SUFFIX):
|
||||
key = str(path)
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
files.append(path)
|
||||
files.sort(key=str)
|
||||
return files
|
||||
|
||||
|
||||
def fingerprint(files: List[Path]) -> str:
|
||||
"""What a review saw of ``files``: each path, size and modification time."""
|
||||
digest = hashlib.sha256()
|
||||
for path in files:
|
||||
try:
|
||||
stat = path.stat()
|
||||
entry = f"{path}\0{stat.st_size}\0{stat.st_mtime_ns}\n"
|
||||
except OSError:
|
||||
entry = f"{path}\0missing\n"
|
||||
digest.update(entry.encode("utf-8", "surrogatepass"))
|
||||
return "sha256:" + digest.hexdigest()
|
||||
|
||||
|
||||
def preview(lora_manager_path: str, cache_dir: Path) -> Dict:
|
||||
"""The review's counts (blocking; reads only, requests nothing).
|
||||
|
||||
``{lora_files, with_metadata, planned_downloads, fingerprint, names}``:
|
||||
the metadata files found, those that parse, the example images an import
|
||||
would download (not yet in the cache) and the first LoRA names.
|
||||
"""
|
||||
files = discover(lora_manager_path)
|
||||
with_metadata = 0
|
||||
planned = 0
|
||||
names = []
|
||||
for path in files:
|
||||
metadata = lora_utils.read_lora_metadata(path)
|
||||
if not metadata:
|
||||
continue
|
||||
with_metadata += 1
|
||||
planned += lora_utils.planned_civitai_downloads(metadata, path, cache_dir)
|
||||
if len(names) < 50:
|
||||
names.append(lora_utils.get_model_name_from_metadata(metadata))
|
||||
return {
|
||||
"lora_files": len(files),
|
||||
"with_metadata": with_metadata,
|
||||
"planned_downloads": planned,
|
||||
"fingerprint": fingerprint(files),
|
||||
"names": names,
|
||||
}
|
||||
|
||||
|
||||
def build_record(metadata: Dict, path: Path, images: List[str]) -> Dict:
|
||||
"""The prompt a LoRA becomes: its example prompt (else its name), tags, images."""
|
||||
model_name = lora_utils.get_model_name_from_metadata(metadata)
|
||||
text = lora_utils.get_example_prompt_from_metadata(metadata) or model_name
|
||||
tags = ["lora-manager", f"lora:{model_name}"]
|
||||
tags.extend(lora_utils.get_trigger_words_from_metadata(metadata))
|
||||
return {
|
||||
"name": model_name,
|
||||
"text": text,
|
||||
"hash": generate_prompt_hash(text),
|
||||
"tags": list(dict.fromkeys(tags)),
|
||||
"images": images,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class StagedImport:
|
||||
"""The records an import will write, and what staging counted."""
|
||||
|
||||
total: int
|
||||
records: List[Dict] = field(default_factory=list)
|
||||
skipped: List[str] = field(default_factory=list)
|
||||
stats: Dict[str, int] = field(
|
||||
default_factory=lambda: {"downloaded": 0, "download_errors": 0}
|
||||
)
|
||||
|
||||
|
||||
def stage(
|
||||
files: List[Path],
|
||||
cache_dir: Path,
|
||||
api_key: str = "",
|
||||
*,
|
||||
on_record: Optional[Callable[[int, Dict, "StagedImport"], None]] = None,
|
||||
cancelled: Callable[[], bool] = lambda: False,
|
||||
logger=None,
|
||||
) -> StagedImport:
|
||||
"""Read each file and build its record, downloading its example images.
|
||||
|
||||
Blocking. A file that cannot be read, or whose record fails to build, is
|
||||
skipped by its name. ``cancelled()`` is asked before each file (raises
|
||||
ImportCancelled); ``on_record(index, record_or_None, staged)`` follows
|
||||
each one.
|
||||
"""
|
||||
staged = StagedImport(total=len(files))
|
||||
for index, path in enumerate(files):
|
||||
if cancelled():
|
||||
raise ImportCancelled()
|
||||
record = None
|
||||
metadata = lora_utils.read_lora_metadata(path)
|
||||
if not metadata:
|
||||
staged.skipped.append(path.name)
|
||||
else:
|
||||
try:
|
||||
local = lora_utils.get_preview_images_from_metadata(metadata, path)
|
||||
remote = lora_utils.download_civitai_images(
|
||||
metadata, path, cache_dir, api_key, stats=staged.stats
|
||||
)
|
||||
images = list(dict.fromkeys([*local, *remote]))
|
||||
record = build_record(metadata, path, images)
|
||||
staged.records.append(record)
|
||||
except Exception as error: # one LoRA never stops the import
|
||||
name = lora_utils.get_model_name_from_metadata(metadata)
|
||||
if logger is not None:
|
||||
logger.warning(f"Failed to prepare LoRA {name}: {error}")
|
||||
staged.skipped.append(name)
|
||||
if on_record is not None:
|
||||
on_record(index, record, staged)
|
||||
return staged
|
||||
+151
-37
@@ -82,23 +82,46 @@ def detect_lora_manager(custom_path: str = "") -> Optional[str]:
|
||||
if p.is_dir() and _looks_like_lora_manager(p):
|
||||
return str(p.resolve())
|
||||
|
||||
# 2. Auto-detect via custom_nodes (case-insensitive scan)
|
||||
# 2. Auto-detect in every custom_nodes folder (case-insensitive scan):
|
||||
# ComfyUI's own first, then the extra ones it registered (#144).
|
||||
folders = []
|
||||
root = find_comfyui_root()
|
||||
if root:
|
||||
custom_nodes = root / "custom_nodes"
|
||||
if custom_nodes.is_dir():
|
||||
for entry in custom_nodes.iterdir():
|
||||
if (
|
||||
entry.is_dir()
|
||||
and "lora" in entry.name.lower()
|
||||
and "manager" in entry.name.lower()
|
||||
and _looks_like_lora_manager(entry)
|
||||
):
|
||||
return str(entry.resolve())
|
||||
folders.append(root / "custom_nodes")
|
||||
folders.extend(custom_nodes_directories())
|
||||
seen = set()
|
||||
for custom_nodes in folders:
|
||||
key = os.path.normcase(os.path.abspath(str(custom_nodes)))
|
||||
if key in seen:
|
||||
continue
|
||||
seen.add(key)
|
||||
found = _lora_manager_in(Path(custom_nodes))
|
||||
if found:
|
||||
return found
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _lora_manager_in(custom_nodes: Path) -> Optional[str]:
|
||||
"""The LoraManager install directly under ``custom_nodes``, or None."""
|
||||
try:
|
||||
entries = sorted(custom_nodes.iterdir()) if custom_nodes.is_dir() else []
|
||||
except OSError:
|
||||
return None
|
||||
for entry in entries:
|
||||
try:
|
||||
if (
|
||||
entry.is_dir()
|
||||
and "lora" in entry.name.lower()
|
||||
and "manager" in entry.name.lower()
|
||||
and _looks_like_lora_manager(entry)
|
||||
):
|
||||
return str(entry.resolve())
|
||||
except OSError:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _looks_like_lora_manager(path: Path) -> bool:
|
||||
"""Heuristic: does this directory look like a LoraManager install?"""
|
||||
# Must have __init__.py (ComfyUI extension) or README.md
|
||||
@@ -491,47 +514,98 @@ def _download_one(url: str, local_path: Path, api_key: str) -> Optional[str]:
|
||||
|
||||
from PIL import Image
|
||||
|
||||
img = Image.open(BytesIO(raw))
|
||||
img.thumbnail((_THUMB_MAX_SIZE, _THUMB_MAX_SIZE), Image.LANCZOS)
|
||||
img = img.convert("RGB")
|
||||
img.save(str(local_path), "JPEG", quality=85)
|
||||
|
||||
with Image.open(BytesIO(raw)) as img:
|
||||
img.thumbnail((_THUMB_MAX_SIZE, _THUMB_MAX_SIZE), Image.LANCZOS)
|
||||
rgb = img.convert("RGB")
|
||||
_save_atomically(rgb, local_path)
|
||||
return str(local_path.resolve())
|
||||
except Exception as e:
|
||||
logger.debug(f"Failed to download {url}: {e}")
|
||||
return None
|
||||
|
||||
|
||||
def download_civitai_images(
|
||||
metadata: Dict, metadata_path: Path, cache_dir: Path, api_key: str = ""
|
||||
) -> List[str]:
|
||||
"""Download civitai example images to a local cache directory.
|
||||
_PART_PREFIX = ".pm-"
|
||||
_PART_SUFFIX = ".part"
|
||||
# A temp download older than this was left by a killed process, not a live one
|
||||
# (each download has a 10 s timeout).
|
||||
STALE_PART_SECONDS = 3600
|
||||
|
||||
Uses thumbnail URLs (512px) instead of full-size originals, and
|
||||
downloads in parallel (up to 8 concurrent) for speed.
|
||||
|
||||
Args:
|
||||
api_key: CivitAI API key for authenticated downloads (NSFW content).
|
||||
def _save_atomically(image, local_path: Path) -> None:
|
||||
"""Write ``image`` as JPEG so ``local_path`` only ever names a whole file.
|
||||
|
||||
Returns:
|
||||
List of absolute paths to downloaded image files.
|
||||
The image goes to a ``.pm-*.part`` file in the same folder, which is
|
||||
closed before ``os.replace`` puts it in place (Windows cannot replace an
|
||||
open file); a failure removes the temp file. A reader (another import's
|
||||
cache check) never sees a half-written image. The temp file is opened
|
||||
like ``img.save`` opens a file, so the image keeps the umask's mode.
|
||||
"""
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
import secrets
|
||||
|
||||
while True:
|
||||
temp = local_path.parent / (
|
||||
f"{_PART_PREFIX}{secrets.token_hex(8)}{_PART_SUFFIX}"
|
||||
)
|
||||
try:
|
||||
handle = open(temp, "xb")
|
||||
except FileExistsError:
|
||||
continue
|
||||
break
|
||||
try:
|
||||
with handle:
|
||||
image.save(handle, "JPEG", quality=85)
|
||||
os.replace(temp, local_path)
|
||||
except BaseException:
|
||||
try:
|
||||
os.remove(temp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
def sweep_stale_downloads(cache_dir: Path) -> None:
|
||||
"""Remove ``.pm-*.part`` files a killed download left in the image cache.
|
||||
|
||||
Only that exact pattern, only in ``cache_dir`` and its LoRA folders, and
|
||||
only once older than ``STALE_PART_SECONDS``; a file that cannot be read
|
||||
or removed is left for a later import.
|
||||
"""
|
||||
import time
|
||||
|
||||
cutoff = time.time() - STALE_PART_SECONDS
|
||||
try:
|
||||
folders = [cache_dir] + [p for p in cache_dir.iterdir() if p.is_dir()]
|
||||
except OSError:
|
||||
return
|
||||
for folder in folders:
|
||||
try:
|
||||
candidates = list(folder.glob(f"{_PART_PREFIX}*{_PART_SUFFIX}"))
|
||||
except OSError:
|
||||
continue
|
||||
for path in candidates:
|
||||
try:
|
||||
if path.is_file() and path.stat().st_mtime < cutoff:
|
||||
os.remove(path)
|
||||
except OSError:
|
||||
logger.debug(f"Could not remove stale download {path}")
|
||||
|
||||
|
||||
def _civitai_cache_plan(
|
||||
metadata: Dict, metadata_path: Path, cache_dir: Path
|
||||
) -> Tuple[Path, List[str], List[Tuple[str, Path]]]:
|
||||
"""``(lora_cache, cached, tasks)`` for a LoRA's CivitAI example images.
|
||||
|
||||
``cached`` are the images already in the cache, ``tasks`` the
|
||||
``(url, local_path)`` downloads still needed. Reads only; creates nothing.
|
||||
"""
|
||||
civitai = _get_civitai(metadata)
|
||||
images = civitai.get("images", []) or []
|
||||
if not images:
|
||||
return []
|
||||
|
||||
file_name = metadata.get("file_name", "")
|
||||
if not file_name:
|
||||
file_name = metadata_path.name.replace(".metadata.json", "")
|
||||
lora_stem = Path(file_name).stem
|
||||
lora_cache = cache_dir / Path(file_name).stem
|
||||
|
||||
lora_cache = cache_dir / lora_stem
|
||||
lora_cache.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Build download tasks
|
||||
cached = []
|
||||
tasks = [] # (url, local_path)
|
||||
for img in images:
|
||||
@@ -548,9 +622,42 @@ def download_civitai_images(
|
||||
cached.append(str(local_path.resolve()))
|
||||
else:
|
||||
tasks.append((url, local_path))
|
||||
return lora_cache, cached, tasks
|
||||
|
||||
|
||||
def planned_civitai_downloads(
|
||||
metadata: Dict, metadata_path: Path, cache_dir: Path
|
||||
) -> int:
|
||||
"""How many example images an import would download (no request, no write)."""
|
||||
return len(_civitai_cache_plan(metadata, metadata_path, cache_dir)[2])
|
||||
|
||||
|
||||
def download_civitai_images(
|
||||
metadata: Dict,
|
||||
metadata_path: Path,
|
||||
cache_dir: Path,
|
||||
api_key: str = "",
|
||||
stats: Optional[Dict[str, int]] = None,
|
||||
) -> List[str]:
|
||||
"""Download civitai example images to a local cache directory.
|
||||
|
||||
Uses thumbnail URLs (512px) instead of full-size originals, and
|
||||
downloads in parallel (up to 8 concurrent) for speed.
|
||||
|
||||
Args:
|
||||
api_key: CivitAI API key for authenticated downloads (NSFW content).
|
||||
stats: When given, ``downloaded`` and ``download_errors`` are
|
||||
increased by the downloads that succeeded and failed.
|
||||
|
||||
Returns:
|
||||
List of absolute paths to downloaded image files.
|
||||
"""
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
lora_cache, cached, tasks = _civitai_cache_plan(metadata, metadata_path, cache_dir)
|
||||
if not tasks:
|
||||
return cached
|
||||
lora_cache.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
# Download in parallel
|
||||
downloaded = []
|
||||
@@ -563,15 +670,21 @@ def download_civitai_images(
|
||||
if result:
|
||||
downloaded.append(result)
|
||||
|
||||
if stats is not None:
|
||||
stats["downloaded"] = stats.get("downloaded", 0) + len(downloaded)
|
||||
stats["download_errors"] = (
|
||||
stats.get("download_errors", 0) + len(tasks) - len(downloaded)
|
||||
)
|
||||
return cached + downloaded
|
||||
|
||||
|
||||
def get_lora_image_cache_dir() -> Path:
|
||||
def get_lora_image_cache_dir(create: bool = True) -> Path:
|
||||
"""Get the directory used to cache downloaded LoRA example images.
|
||||
|
||||
``<data folder>/data/lora_images`` (``utils.library_location``): the node
|
||||
folder for a library that was not moved, else ComfyUI's
|
||||
``user/__promptmanager``.
|
||||
``user/__promptmanager``. ``create=False`` only names it (a preview
|
||||
writes nothing).
|
||||
"""
|
||||
try:
|
||||
from ..utils import library_location
|
||||
@@ -582,7 +695,8 @@ def get_lora_image_cache_dir() -> Path:
|
||||
cache = Path(
|
||||
library_location.cache_dir("lora_images", extension_root=str(ext_root))
|
||||
)
|
||||
cache.mkdir(parents=True, exist_ok=True)
|
||||
if create:
|
||||
cache.mkdir(parents=True, exist_ok=True)
|
||||
return cache
|
||||
|
||||
|
||||
|
||||
@@ -23,6 +23,9 @@ from tests.browser.request_hold import register_request_holds
|
||||
from tests.browser import slow_reset_watch
|
||||
from tests.browser.workspace_activity_fixture import register_activity_commands
|
||||
from tests.browser.workspace_system_fixture import register_system_commands
|
||||
from tests.browser.workspace_integrations_fixture import (
|
||||
register_integrations_commands,
|
||||
)
|
||||
from tests.browser.workspace_concept_fixture import (
|
||||
MAX_EXTRA_IMAGES as CONCEPT_MAX_EXTRA_IMAGES,
|
||||
MAX_SEEDLESS_IMAGES as CONCEPT_MAX_SEEDLESS_IMAGES,
|
||||
@@ -203,6 +206,7 @@ def create_app(root: Path) -> web.Application:
|
||||
reveal_stub = register_reveal_stub(routes, api, root)
|
||||
register_activity_commands(app, routes, api)
|
||||
register_system_commands(app, routes, api, root)
|
||||
register_integrations_commands(app, routes, api, root)
|
||||
request_holds = register_request_holds(app, routes)
|
||||
slow_reset_watch.install(app, api)
|
||||
|
||||
|
||||
@@ -0,0 +1,343 @@
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { SIZES, THEMES, seedConcept, openConcept, openProduction, diffInventory, mutate } from './concept-inventory.mjs';
|
||||
|
||||
// #249 System S10 Task 28: System › Integrations (app.js renderSystem 682–684) and the LoRA import review against the
|
||||
// immutable concept, on the concept-shaped fixture with a fake LoraManager install. Each concept measurement is mapped
|
||||
// only by the recorded deviations (D106, D107, D114), each commented where it applies; after that the difference must
|
||||
// be empty, and a mutated label, option, row or token must be noticed. The concept starts its sample import without a
|
||||
// review, so the import review is compared with the concept's own record-changing review, `maintenancePlan`
|
||||
// (953–958), whose shell, grid, note and actions it shares.
|
||||
test.use({ locale: 'en-US', timezoneId: 'UTC' });
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
await seedConcept(request);
|
||||
await expect(await request.post('/__test__/system/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/integrations/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
|
||||
page.conceptErrors = [];
|
||||
page.on('pageerror', error => page.conceptErrors.push(error.message));
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, request, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
await request.post('/__test__/jobs/restart');
|
||||
await request.post('/__test__/integrations/reset');
|
||||
await request.post('/__test__/system/reset');
|
||||
expect(page.conceptErrors).toEqual([]);
|
||||
});
|
||||
|
||||
const PREVIEW_URL = '/prompt_manager/lora/import/preview';
|
||||
|
||||
/** Runs in the page: Integrations' surfaces under `mainSelector`, every row's text and control, geometry and tokens. */
|
||||
function measureIntegrations(mainSelector) {
|
||||
const layout = document.querySelector(`${mainSelector} .settings-layout`);
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const text = node => node?.textContent.trim() ?? null;
|
||||
const control = row => {
|
||||
const select = row.querySelector(':scope > select');
|
||||
if (select) return { kind: 'select', id: select.id, options: [...select.options].map(option => [option.textContent, option.disabled]),
|
||||
selected: select.selectedOptions[0]?.textContent ?? null, disabled: select.disabled };
|
||||
const input = row.querySelector(':scope > input');
|
||||
if (input) return { kind: `input.${input.type}`, id: input.id, value: input.value, readOnly: input.readOnly };
|
||||
const toggle = row.querySelector(':scope > * input[type="checkbox"]');
|
||||
return toggle ? { kind: 'switch', id: toggle.id, wrapper: name(toggle.parentElement), checked: toggle.checked,
|
||||
disabled: toggle.disabled, ...css(toggle.parentElement, 'width', 'height') } : null;
|
||||
};
|
||||
const surfaces = [...layout.querySelectorAll(':scope > section.surface')];
|
||||
const firstRow = surfaces[0].querySelector('.setting-row');
|
||||
const note = layout.querySelector('.setting-row p.small');
|
||||
return {
|
||||
layout: [...layout.children].map(name),
|
||||
surfaces: surfaces.map(surface => ({ tree: [...surface.children].map(name), heading: text(surface.querySelector(':scope > h2')),
|
||||
intro: text(surface.querySelector(':scope > p')),
|
||||
rows: [...surface.querySelectorAll(':scope > .setting-row')].map(row => ({ tree: [...row.children].map(name),
|
||||
text: [...row.firstElementChild.children].map(node => [name(node), text(node), node.localName === 'label' ? node.htmlFor : null]),
|
||||
control: control(row) })),
|
||||
operations: [...surface.querySelectorAll(':scope > .operation')].map(row => ({ tree: [...row.children].map(name),
|
||||
title: text(row.querySelector('h3')), description: text(row.querySelector('p')),
|
||||
actions: [...row.querySelectorAll('.row-actions > button')].map(item => [text(item), item.className]) })) })),
|
||||
surface: css(surfaces[1], 'paddingTop', 'paddingLeft', 'backgroundColor', 'borderTopWidth', 'borderTopColor', 'borderTopLeftRadius',
|
||||
'marginTop'),
|
||||
row: css(firstRow, 'display', 'alignItems', 'justifyContent', 'columnGap', 'paddingTop', 'paddingBottom', 'borderTopWidth',
|
||||
'borderTopColor', 'marginTop'),
|
||||
label: css(firstRow.querySelector('label'), 'fontSize', 'fontWeight', 'color'),
|
||||
description: css(firstRow.querySelector('p'), 'fontSize', 'color', 'marginTop', 'maxWidth'),
|
||||
select: css(firstRow.querySelector('select'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor', 'color', 'borderTopColor'),
|
||||
password: css(layout.querySelector('.setting-row input[type="password"]'), 'fontSize', 'minWidth', 'maxWidth', 'backgroundColor',
|
||||
'color', 'borderTopColor'),
|
||||
// The concept's `.small` text (its Save row status); production's row notes use the same class.
|
||||
small: css(layout.querySelector('#settings-status'), 'fontSize', 'color'),
|
||||
note: note && css(note, 'fontSize', 'color'),
|
||||
// Whether each select is as wide as the browser lays out a copy holding only its chosen option, i.e. it shows
|
||||
// that option whole (the browser's own select sizing, arrow included).
|
||||
selectsFit: [...layout.querySelectorAll('.setting-row > select')].map(select => {
|
||||
const copy = select.cloneNode(false);
|
||||
copy.removeAttribute('id');
|
||||
copy.append(new Option(select.selectedOptions[0]?.textContent ?? ''));
|
||||
Object.assign(copy.style, { position: 'absolute', visibility: 'hidden', width: 'max-content', minWidth: '0',
|
||||
maxWidth: 'none', flex: 'none' });
|
||||
select.parentElement.append(copy);
|
||||
const needed = copy.getBoundingClientRect().width;
|
||||
copy.remove();
|
||||
return select.getBoundingClientRect().width + 0.5 >= needed;
|
||||
}),
|
||||
pageScroll: document.documentElement.scrollWidth > document.documentElement.clientWidth,
|
||||
};
|
||||
}
|
||||
|
||||
const SEND_REASON = 'Sending always asks for the ComfyUI window and node, so “Ask each time” is the only option.';
|
||||
const RENDERER_REASON = 'Prompt Manager nodes always follow ComfyUI’s own renderer setting.';
|
||||
const RENDERER_NONE = 'No ComfyUI window is open in this browser, so its renderer is not known.';
|
||||
const RETURN_REASON = 'Not available: sending applies the text in ComfyUI and keeps this page open.';
|
||||
const PATH_DESCRIPTION = 'Found automatically in ComfyUI’s custom_nodes folders; read-only, as on the existing settings page.';
|
||||
|
||||
/**
|
||||
* The concept's Integrations with the recorded deviations applied, for the fixture's LoraManager (detected, disabled,
|
||||
* no key, trigger words off and not loaded).
|
||||
*/
|
||||
function mapIntegrationsConcept(reference) {
|
||||
const [comfy, lora] = reference.surfaces;
|
||||
const small = text => ['p.small', text, null];
|
||||
const firstOnly = row => ({ ...row, control: { ...row.control, options: row.control.options.map(([label], index) => [label, index > 0]) } });
|
||||
const [send, renderer, back] = comfy.rows;
|
||||
const [enable, triggers, key] = lora.rows;
|
||||
return {
|
||||
...reference,
|
||||
surfaces: [
|
||||
{ ...comfy, rows: [
|
||||
// D106: only "Ask each time" can be chosen, and the row says why.
|
||||
{ ...firstOnly(send), text: [...send.text, small(SEND_REASON)] },
|
||||
// D106: only "Follow host renderer" can be chosen; the reason, then the detected renderer.
|
||||
{ ...firstOnly(renderer), text: [...renderer.text, small(RENDERER_REASON), small(RENDERER_NONE)] },
|
||||
// D106: Return to canvas is not available: off, disabled and saying why.
|
||||
{ ...back, text: [...back.text, small(RETURN_REASON)], control: { ...back.control, checked: false, disabled: true } },
|
||||
] },
|
||||
{ ...lora,
|
||||
// D106: the LoraManager path row (legacy, read-only) follows the integration toggle.
|
||||
tree: [lora.tree[0], lora.tree[1], 'div.setting-row', 'div.setting-row', 'div.setting-row', 'div.setting-row', 'div.operation'],
|
||||
rows: [
|
||||
// The real detection replaces the concept's sample line; the integration is off in the fixture.
|
||||
{ ...enable, text: [enable.text[0], ['p', 'Detected: custom_nodes/ComfyUI-Lora-Manager.', null]],
|
||||
control: { ...enable.control, checked: false } },
|
||||
{ tree: ['div', 'input'],
|
||||
text: [['label', 'LoraManager path', 'lora-path'], ['p', PATH_DESCRIPTION, null]],
|
||||
control: { kind: 'input.text', id: 'lora-path', value: 'custom_nodes/ComfyUI-Lora-Manager', readOnly: true } },
|
||||
// The trigger cache is a separate fact from the setting: its own line.
|
||||
{ ...triggers, text: [...triggers.text, small('Trigger words are not loaded.')], control: { ...triggers.control, checked: false } },
|
||||
// D114: no prototype warning; the saved-key line under it.
|
||||
// (No key is saved in the fixture, so "Clear saved key" is not offered.)
|
||||
{ ...key, text: [key.text[0], ['p', 'Credentials stay in the server configuration.', null], small('No key saved.')],
|
||||
control: { ...key.control, value: '' } },
|
||||
],
|
||||
// D107: the import is reviewed first ("Review import"; the concept's button only simulated a job).
|
||||
operations: lora.operations.map(operation => ({ ...operation, actions: [['Review import', '']] })) },
|
||||
],
|
||||
// D106: both selects show their whole chosen option at every width (at 390 the concept cuts "Follow host
|
||||
// renderer"; production lets the two selects grow to 160px and never shrink for the reason lines).
|
||||
selectsFit: reference.selectsFit.map(() => true),
|
||||
select: reference.select.maxWidth === '120px' ? { ...reference.select, maxWidth: '160px' } : reference.select,
|
||||
// The D106/D114 row notes are `p.small` inside the row text, so they read as its description (`.setting-row p`).
|
||||
note: { fontSize: reference.description.fontSize, color: reference.description.color },
|
||||
};
|
||||
}
|
||||
|
||||
async function conceptIntegrations(page, theme) {
|
||||
await openConcept(page, 'system', theme);
|
||||
await page.locator('#main .section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
await expect(page.locator('#main .settings-layout > section.surface')).toHaveCount(2);
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureIntegrations, '#main');
|
||||
}
|
||||
|
||||
async function productionIntegrations(page, theme) {
|
||||
await openProduction(page, 'system/integrations', theme);
|
||||
// Compare once the LoRA state and the renderer check answered.
|
||||
await expect(page.locator('#workspace-main #lora-path')).toHaveValue('custom_nodes/ComfyUI-Lora-Manager');
|
||||
await expect(page.locator('#workspace-main #renderer-status')).toHaveText(RENDERER_NONE);
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureIntegrations, '#workspace-main');
|
||||
}
|
||||
|
||||
const fakeLoraManager = async (request, data = {}) =>
|
||||
expect(await request.post('/__test__/integrations/lora-manager', { data })).toBeOK();
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
test(`integrations matches the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
await fakeLoraManager(request);
|
||||
const concept = await conceptIntegrations(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-integrations-${theme}-${width}.png`), fullPage: true });
|
||||
const production = await productionIntegrations(page, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-integrations-${theme}-${width}.png`), fullPage: true });
|
||||
expect(diffInventory(mapIntegrationsConcept(concept), production)).toEqual([]);
|
||||
expect(production.pageScroll).toBe(false);
|
||||
});
|
||||
}
|
||||
|
||||
// D106 below the concept's smallest size: on a 320px phone both selects still show their whole option, without page
|
||||
// scroll (S10 re-review N9).
|
||||
for (const theme of THEMES) test(`integrations selects fit at 320 wide ${theme}`, async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 320, height: 700 });
|
||||
await fakeLoraManager(request);
|
||||
const production = await productionIntegrations(page, theme);
|
||||
expect(production.selectsFit).toEqual([true, true]);
|
||||
expect(production.pageScroll).toBe(false);
|
||||
});
|
||||
|
||||
test('integrations comparison detects mutations', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
await fakeLoraManager(request);
|
||||
const concept = await conceptIntegrations(page, 'dark');
|
||||
const expected = mapIntegrationsConcept(concept);
|
||||
const production = await productionIntegrations(page, 'dark');
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the section differs only where D106, D107 and D114 are recorded.
|
||||
expect(diffInventory(concept, production).map(item => item.path.split('.').slice(0, 4).join('.'))
|
||||
.filter((path, index, all) => all.indexOf(path) === index).sort()).toEqual([
|
||||
'note', 'note.color', 'note.fontSize', 'surfaces.0.rows.0', 'surfaces.0.rows.1', 'surfaces.0.rows.2', 'surfaces.1.operations.0',
|
||||
'surfaces.1.rows.0', 'surfaces.1.rows.1', 'surfaces.1.rows.2', 'surfaces.1.rows.3', 'surfaces.1.rows.length',
|
||||
'surfaces.1.tree.5', 'surfaces.1.tree.6', 'surfaces.1.tree.length'].sort());
|
||||
const measure = () => page.evaluate(measureIntegrations, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
// A removed row.
|
||||
const undoRow = await mutate(page, { remove: ['#workspace-main section.surface:nth-of-type(2) > .setting-row:nth-of-type(3)'] });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['surfaces.1.rows.length', 'surfaces.1.tree.length']));
|
||||
await undoRow();
|
||||
// A changed label and a changed option.
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="renderer"]').textContent = 'Renderer'; });
|
||||
expect(await paths()).toEqual(['surfaces.0.rows.1.text.0.1']);
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main label[for="renderer"]').textContent = 'Host compatibility'; });
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main #send-destination option').textContent = 'Always ask'; });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['surfaces.0.rows.0.control.options.0.0']));
|
||||
await page.evaluate(() => { document.querySelector('#workspace-main #send-destination option').textContent = 'Ask each time'; });
|
||||
// An unsupported option that became selectable.
|
||||
await page.evaluate(() => { document.querySelectorAll('#workspace-main #renderer option')[1].disabled = false; });
|
||||
expect(await paths()).toEqual(['surfaces.0.rows.1.control.options.1.1']);
|
||||
await page.evaluate(() => { document.querySelectorAll('#workspace-main #renderer option')[1].disabled = true; });
|
||||
// A select squeezed until its chosen option is cut (S10 review M3).
|
||||
const undoSqueeze = await mutate(page, { css: '#workspace-main #renderer { max-width: 70px !important; }' });
|
||||
expect(await paths()).toEqual(['selectsFit.1']);
|
||||
await undoSqueeze();
|
||||
// A swapped token.
|
||||
const undoToken = await mutate(page, { css: '#workspace-main .settings-layout { --line: rgb(255, 0, 0); }' });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['row.borderTopColor']));
|
||||
await undoToken();
|
||||
expect(diffInventory(expected, await measure())).toEqual([]);
|
||||
});
|
||||
|
||||
/** Runs in the page: the open review dialog: its title, body and the concept's tokens. */
|
||||
function measureReview() {
|
||||
const dialog = document.querySelector('dialog#modal[open]');
|
||||
const css = (node, ...keys) => node ? Object.fromEntries(keys.map(key => [key, getComputedStyle(node)[key]])) : null;
|
||||
const name = node => [node.localName, ...[...node.classList].sort()].join('.');
|
||||
const text = node => node?.textContent.trim() ?? null;
|
||||
const body = dialog.querySelector('#modal-body');
|
||||
const grid = body.querySelector(':scope > dl.meta-grid');
|
||||
const note = [...body.querySelectorAll(':scope > p.small')].at(-1);
|
||||
const actions = body.querySelector(':scope > .dialog-actions');
|
||||
return {
|
||||
title: text(dialog.querySelector('#modal-title')),
|
||||
tree: [...body.children].map(name),
|
||||
heading: { text: text(body.querySelector(':scope > h3')),
|
||||
...css(body.querySelector(':scope > h3'), 'fontSize', 'fontWeight', 'color', 'marginBottom') },
|
||||
grid: { rows: [...grid.children].map(row => [text(row.querySelector('dt')), text(row.querySelector('dd'))]),
|
||||
...css(grid, 'display', 'gridTemplateColumns', 'columnGap', 'rowGap'),
|
||||
dt: css(grid.querySelector('dt'), 'fontSize', 'color', 'textTransform'), dd: css(grid.querySelector('dd'), 'fontSize', 'color') },
|
||||
details: body.querySelector(':scope > details') && { summary: text(body.querySelector(':scope > details > summary')),
|
||||
tree: [...body.querySelector(':scope > details').children].map(name) },
|
||||
backup: body.querySelector(':scope > .notice') && { tree: [...body.querySelector(':scope > .notice').children].map(name),
|
||||
text: [...body.querySelectorAll(':scope > .notice > div:first-child > *')].map(node => [node.localName, text(node)]),
|
||||
buttons: [...body.querySelectorAll(':scope > .notice button')].map(text) },
|
||||
note: { text: text(note), ...css(note, 'fontSize', 'color', 'marginTop') },
|
||||
actions: { tree: [...actions.children].map(name), ...css(actions, 'display', 'justifyContent', 'columnGap', 'marginTop'),
|
||||
buttons: [...actions.children].map(item => ({ text: text(item), className: item.className,
|
||||
...css(item, 'backgroundColor', 'color', 'fontSize', 'paddingTop', 'paddingLeft', 'borderTopColor') })) },
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* D107: the server's import plan in the concept's review: "Review LoRA import" with "LoRA reference data" as its `h3`,
|
||||
* the concept's four rows with the plan's values plus Found and Network, the LoRAs it found (`details`), the hold
|
||||
* line with the cache sentence in the concept's `p.small`, the D55 backup box, and Cancel / "Start import".
|
||||
*/
|
||||
function mapImportConcept(reference, plan) {
|
||||
const [cancel, start] = reference.actions.buttons;
|
||||
return {
|
||||
...reference,
|
||||
title: 'Review LoRA import',
|
||||
tree: ['h3', 'dl.meta-grid', 'details', 'p.small', 'div.notice', 'div.dialog-actions'],
|
||||
heading: { ...reference.heading, text: 'LoRA reference data' },
|
||||
grid: { ...reference.grid, rows: [['Scope', plan.scope.label], ['Found', '2 metadata files · 2 readable'],
|
||||
['Protected', 'Nothing is protected'], ['Network', 'Downloads up to 2 example images from CivitAI'],
|
||||
['Backup', `Create and verify before changes (${plan.backup.destination_label})`],
|
||||
['Media on disk', 'No files are removed; image links are replaced']] },
|
||||
details: { summary: 'LoRAs found (2)', tree: ['summary', 'pre'] },
|
||||
backup: { tree: ['div', 'div.row-actions'], text: [['h3', 'Back up the database first (recommended)'],
|
||||
['p', 'Downloads a copy of the prompts database before this operation changes any records.']], buttons: ['Back up now'] },
|
||||
note: { ...reference.note, text: 'Nothing changes until you apply. This review holds for 15 minutes. '
|
||||
+ 'Example images downloaded before a failure stay in the image cache.' },
|
||||
actions: { ...reference.actions, buttons: [cancel, { ...start, text: 'Start import' }] },
|
||||
};
|
||||
}
|
||||
|
||||
async function conceptReview(page, theme) {
|
||||
await openConcept(page, 'system', theme);
|
||||
await page.locator('#main .section-nav').getByRole('button', { name: 'Maintenance', exact: true }).click();
|
||||
await page.locator('#main [data-plan="duplicate prompts"]').click();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Review maintenance plan');
|
||||
await page.mouse.move(0, 0);
|
||||
return page.evaluate(measureReview);
|
||||
}
|
||||
|
||||
async function productionReview(page, request, theme) {
|
||||
await openProduction(page, 'system/integrations', theme);
|
||||
await expect(page.locator('#workspace-main #lora-path')).toHaveValue('custom_nodes/ComfyUI-Lora-Manager');
|
||||
// The renderer line above settles first: its new text moves the rows below (a click must not land mid-shift).
|
||||
await expect(page.locator('#workspace-main #renderer-status')).toHaveText(RENDERER_NONE);
|
||||
// The import reviews what is enabled: save the integration on first.
|
||||
await page.locator('#workspace-main #lora-enable').check();
|
||||
await page.getByRole('button', { name: 'Save preferences', exact: true }).click();
|
||||
await expect(page.locator('#settings-status')).toHaveText('Saved to the server configuration.');
|
||||
const answer = page.waitForResponse(response => response.url().endsWith(PREVIEW_URL));
|
||||
await page.locator('#workspace-main').getByRole('button', { name: 'Review import', exact: true }).click();
|
||||
const plan = await (await answer).json();
|
||||
await expect(page.locator('dialog#modal[open] #modal-title')).toHaveText('Review LoRA import');
|
||||
await page.mouse.move(0, 0);
|
||||
return { plan, production: await page.evaluate(measureReview) };
|
||||
}
|
||||
|
||||
for (const theme of THEMES) for (const [width, height] of SIZES) {
|
||||
test(`LoRA import review matches the concept ${theme} ${width}x${height}`, async ({ page, request }, testInfo) => {
|
||||
await page.setViewportSize({ width, height });
|
||||
await fakeLoraManager(request, { loras: 2, civitai: true });
|
||||
const concept = await conceptReview(page, theme);
|
||||
expect(concept.tree).toEqual(['p.demo-note', 'h3', 'dl.meta-grid', 'p.small', 'div.dialog-actions']);
|
||||
await page.screenshot({ path: testInfo.outputPath(`concept-plan-review-${theme}-${width}.png`) });
|
||||
const { plan, production } = await productionReview(page, request, theme);
|
||||
await page.screenshot({ path: testInfo.outputPath(`production-lora-review-${theme}-${width}.png`) });
|
||||
expect(diffInventory(mapImportConcept(concept, plan), production)).toEqual([]);
|
||||
});
|
||||
}
|
||||
|
||||
test('LoRA import review comparison detects mutations', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
await fakeLoraManager(request, { loras: 2, civitai: true });
|
||||
const concept = await conceptReview(page, 'dark');
|
||||
const { plan } = await productionReview(page, request, 'dark');
|
||||
const expected = mapImportConcept(concept, plan);
|
||||
const paths = async () => diffInventory(expected, await page.evaluate(measureReview)).map(item => item.path);
|
||||
expect(await paths()).toEqual([]);
|
||||
await page.evaluate(() => { document.querySelectorAll('dialog#modal[open] dl.meta-grid dt')[3].textContent = 'Downloads'; });
|
||||
expect(await paths()).toEqual(['grid.rows.3.0']);
|
||||
await page.evaluate(() => { document.querySelectorAll('dialog#modal[open] dl.meta-grid dt')[3].textContent = 'Network'; });
|
||||
const undoGrid = await mutate(page, { css: 'dialog#modal dl.meta-grid { column-gap: 3px; }' });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['grid.columnGap']));
|
||||
await undoGrid();
|
||||
const undoBox = await mutate(page, { remove: ['dialog#modal[open] #modal-body > .notice'] });
|
||||
expect(await paths()).toEqual(expect.arrayContaining(['backup', 'tree.length']));
|
||||
await undoBox();
|
||||
expect(await paths()).toEqual([]);
|
||||
});
|
||||
@@ -0,0 +1,432 @@
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
import { seedConcept } from './concept-inventory.mjs';
|
||||
|
||||
// System › Integrations (#249 System S10 Task 27) on the concept-shaped fixture with a fake LoraManager install
|
||||
// (workspace_integrations_fixture.py): the D106 ComfyUI rows and renderer status, the LoRA rows and the D114 key, saved
|
||||
// through the one Save row, and the D107 import review run as an Activity job. No test reaches the network: CivitAI
|
||||
// downloads are answered by the fixture's stub opener.
|
||||
const workspace = '/prompt_manager/workspace';
|
||||
const PREVIEW_URL = '/prompt_manager/lora/import/preview';
|
||||
const APPLY_URL = '/prompt_manager/lora/import/apply';
|
||||
const ENABLE_URL = '/prompt_manager/lora/enable';
|
||||
const KEPT = 'The import failed; the previous LoRA imports were kept. Images already downloaded stay in the image cache.';
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
await seedConcept(request);
|
||||
await expect(await request.post('/__test__/system/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/integrations/reset')).toBeOK();
|
||||
await expect(await request.post('/__test__/jobs/restart')).toBeOK();
|
||||
page.systemErrors = [];
|
||||
page.on('pageerror', error => page.systemErrors.push(error.message));
|
||||
});
|
||||
|
||||
test.afterEach(async ({ page, request, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
await request.post('/__test__/integrations/gate', { data: { hold: false } });
|
||||
await request.post('/__test__/release');
|
||||
await request.post('/__test__/jobs/restart');
|
||||
await request.post('/__test__/integrations/reset');
|
||||
await request.post('/__test__/system/reset');
|
||||
expect(page.systemErrors).toEqual([]);
|
||||
});
|
||||
|
||||
const toast = page => page.locator('#workspace-toast');
|
||||
const layoutOf = page => page.locator('#workspace-main .settings-layout');
|
||||
const dialog = page => page.locator('dialog#modal[open]');
|
||||
const status = page => page.locator('#settings-status');
|
||||
const save = page => page.getByRole('button', { name: 'Save preferences', exact: true });
|
||||
const settingOf = (page, label) => layoutOf(page).locator('.setting-row').filter({ has: page.locator(`label:text-is("${label}")`) });
|
||||
|
||||
async function serverJson(request, url) {
|
||||
const response = await request.get(url);
|
||||
await expect(response).toBeOK();
|
||||
return response.json();
|
||||
}
|
||||
|
||||
async function lora(request, data = {}) {
|
||||
await expect(await request.post('/__test__/integrations/lora-manager', { data })).toBeOK();
|
||||
}
|
||||
|
||||
async function openIntegrations(page) {
|
||||
// A fresh page each time: the drafts live for the page session (Q10), and a hash-only goto would not reload.
|
||||
if (page.url().endsWith('#/system/integrations')) await page.reload();
|
||||
else await page.goto(`${workspace}#/system/integrations`);
|
||||
await expect(layoutOf(page).locator(':scope > section.surface')).toHaveCount(2);
|
||||
// The LoRA rows are painted once the server's LoRA state was read.
|
||||
await expect(settingOf(page, 'LoraManager integration').locator('input')).toBeEnabled();
|
||||
// The renderer line settles first: its new text moves the rows below, so no click lands mid-shift.
|
||||
await expect(settingOf(page, 'Host compatibility').getByRole('status')).not.toHaveText('Looking for open ComfyUI windows…');
|
||||
return layoutOf(page);
|
||||
}
|
||||
|
||||
/** Enable the integration (and optionally its trigger words) through the Save row. */
|
||||
async function enable(page, { triggers = false } = {}) {
|
||||
await settingOf(page, 'LoraManager integration').locator('input').check();
|
||||
if (triggers) await settingOf(page, 'Automatically include trigger words').locator('input').check();
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved to the server configuration.');
|
||||
}
|
||||
|
||||
/** The open review's `[dt, dd]` rows. */
|
||||
const gridOf = page => dialog(page).locator('dl.meta-grid > div').evaluateAll(items => items.map(item =>
|
||||
[item.querySelector('dt').textContent, item.querySelector('dd').textContent]));
|
||||
|
||||
async function ended(request, id) {
|
||||
await expect.poll(async () => (await serverJson(request, `/prompt_manager/jobs/${id}`)).job.state).not.toBe('running');
|
||||
return (await serverJson(request, `/prompt_manager/jobs/${id}`)).job;
|
||||
}
|
||||
|
||||
/** A second page of this origin that serves a fake ComfyUI window on the send bridge. */
|
||||
async function fakeComfyWindow(context, renderer) {
|
||||
const host = await context.newPage();
|
||||
await host.goto(`${workspace}#/overview`);
|
||||
await host.evaluate(async drawn => {
|
||||
const { createBridgeHost } = await import('/prompt_manager/js/shared/toolkit/prompt-bridge.js');
|
||||
window.fakeHost = createBridgeHost({ namespace: '/', origin: location.origin, label: 'PromptManager — ComfyUI',
|
||||
adapter: { listTargets: () => (drawn ? [{ id: '7', label: 'Prompt Manager', type: 'PromptManager', roles: ['positive'],
|
||||
renderer: drawn }] : []), preview() { throw new Error('unused'); }, apply() { throw new Error('unused'); } } });
|
||||
}, renderer);
|
||||
return host;
|
||||
}
|
||||
|
||||
test('send destination offers Ask each time; unsupported options disabled with the reason (D106)', async ({ page }) => {
|
||||
const layout = await openIntegrations(page);
|
||||
expect(await layout.locator(':scope > section.surface > h2').allTextContents()).toEqual(['ComfyUI', 'LoRA & resources']);
|
||||
const send = settingOf(page, 'Default send destination');
|
||||
expect(await send.locator('option').evaluateAll(options => options.map(option => [option.textContent, option.disabled])))
|
||||
.toEqual([['Ask each time', false], ['Selected Prompt Manager node', true], ['Create a Prompt Manager (+/-) node', true]]);
|
||||
await expect(send.locator('select')).toHaveValue('Ask each time');
|
||||
await expect(send.locator('p.small')).toHaveText(
|
||||
'Sending always asks for the ComfyUI window and node, so “Ask each time” is the only option.');
|
||||
const renderer = settingOf(page, 'Host compatibility');
|
||||
expect(await renderer.locator('option').evaluateAll(options => options.map(option => [option.textContent, option.disabled])))
|
||||
.toEqual([['Follow host renderer', false], ['Classic', true], ['Nodes 2.0', true]]);
|
||||
await expect(renderer.locator('p.small').first()).toHaveText('Prompt Manager nodes always follow ComfyUI’s own renderer setting.');
|
||||
const back = settingOf(page, 'Return to canvas after send');
|
||||
await expect(back.locator('input')).toBeDisabled();
|
||||
await expect(back.locator('input')).not.toBeChecked();
|
||||
await expect(back.locator('p.small')).toHaveText('Not available: sending applies the text in ComfyUI and keeps this page open.');
|
||||
// Nothing on this surface is ever drafted: the Save row stays clean.
|
||||
await expect(status(page)).toHaveText('No unsaved changes.');
|
||||
await expect(save(page)).toHaveAttribute('aria-disabled', 'true');
|
||||
});
|
||||
|
||||
test('host compatibility shows the detected renderer as status (D106)', async ({ page, context }) => {
|
||||
await openIntegrations(page);
|
||||
const line = settingOf(page, 'Host compatibility').getByRole('status');
|
||||
await expect(line).toHaveText('No ComfyUI window is open in this browser, so its renderer is not known.');
|
||||
const host = await fakeComfyWindow(context, 'nodes2');
|
||||
try {
|
||||
await page.reload();
|
||||
await expect(line).toHaveText('Detected: Nodes 2.0 (1 ComfyUI window).');
|
||||
await host.evaluate(() => window.fakeHost.dispose());
|
||||
await host.evaluate(async () => {
|
||||
const { createBridgeHost } = await import('/prompt_manager/js/shared/toolkit/prompt-bridge.js');
|
||||
window.fakeHost = createBridgeHost({ namespace: '/', origin: location.origin, label: 'PromptManager — ComfyUI',
|
||||
adapter: { listTargets: () => [], preview() {}, apply() {} } });
|
||||
});
|
||||
await page.reload();
|
||||
await expect(line).toHaveText('ComfyUI is open; add a Prompt Manager node to see which renderer it uses.');
|
||||
} finally {
|
||||
await host.close();
|
||||
}
|
||||
});
|
||||
|
||||
test('LoRA rows show detected / enabled / cache-loaded separately', async ({ page, request }) => {
|
||||
await openIntegrations(page);
|
||||
const integration = settingOf(page, 'LoraManager integration');
|
||||
const triggers = settingOf(page, 'Automatically include trigger words');
|
||||
await expect(integration.locator('p').first()).toHaveText('Not detected in ComfyUI’s custom_nodes folders.');
|
||||
await expect(integration.locator('input')).not.toBeChecked();
|
||||
await expect(triggers.locator('p.small')).toHaveText('Trigger words are not loaded.');
|
||||
// Enabling what is not installed is refused by the server: the toggle stays highlighted, nothing was saved.
|
||||
await integration.locator('input').check();
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Some changes were not saved. Review the highlighted settings.');
|
||||
await expect(integration.locator('input')).toHaveAttribute('aria-invalid', 'true');
|
||||
expect((await serverJson(request, '/__test__/integrations/state')).enabled).toBe(false);
|
||||
|
||||
await lora(request);
|
||||
await openIntegrations(page);
|
||||
await expect(integration.locator('p').first()).toHaveText('Detected: custom_nodes/ComfyUI-Lora-Manager.');
|
||||
await expect(integration.locator('input')).not.toBeChecked();
|
||||
await enable(page, { triggers: true });
|
||||
await expect(integration.locator('input')).toBeChecked();
|
||||
await expect(triggers.locator('p.small')).toHaveText('Trigger words are loaded.');
|
||||
expect(await serverJson(request, '/__test__/integrations/state')).toMatchObject({ enabled: true, trigger_words: true });
|
||||
// A reload reads the saved state back.
|
||||
await openIntegrations(page);
|
||||
await expect(integration.locator('input')).toBeChecked();
|
||||
await expect(triggers.locator('input')).toBeChecked();
|
||||
await expect(status(page)).toHaveText('No unsaved changes.');
|
||||
});
|
||||
|
||||
test('LoraManager path row shows the detected path read-only, as legacy (D106)', async ({ page, request }) => {
|
||||
await openIntegrations(page);
|
||||
const row = settingOf(page, 'LoraManager path');
|
||||
const input = row.locator('input');
|
||||
await expect(input).toHaveAttribute('readonly', '');
|
||||
await expect(input).toHaveValue('');
|
||||
await expect(input).toHaveAttribute('placeholder', 'Not detected');
|
||||
await lora(request);
|
||||
await openIntegrations(page);
|
||||
await expect(input).toHaveValue('custom_nodes/ComfyUI-Lora-Manager');
|
||||
await input.focus();
|
||||
await page.keyboard.type('elsewhere');
|
||||
await expect(input).toHaveValue('custom_nodes/ComfyUI-Lora-Manager');
|
||||
await expect(status(page)).toHaveText('No unsaved changes.');
|
||||
});
|
||||
|
||||
test('key never prefilled; blank keeps; Clear saved key clears; saved through Save preferences (D114)', async ({ page, request }) => {
|
||||
const enables = [];
|
||||
page.on('request', item => { if (item.url().endsWith(ENABLE_URL)) enables.push(item.postDataJSON()); });
|
||||
await lora(request);
|
||||
await openIntegrations(page);
|
||||
const key = settingOf(page, 'CivitAI API key');
|
||||
const field = key.locator('input[type="password"]');
|
||||
const clear = key.getByRole('button', { name: 'Clear saved key', exact: true });
|
||||
await expect(field).toHaveValue('');
|
||||
await expect(key.locator('p.small')).toHaveText('No key saved.');
|
||||
await expect(clear).toHaveCount(0);
|
||||
// A typed key is dropped when the section is left: the field comes back empty with nothing hidden to save (N4).
|
||||
await field.fill('sk-dropped');
|
||||
await expect(status(page)).toHaveText('Unsaved changes.');
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Appearance', exact: true }).click();
|
||||
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
await expect(status(page)).toHaveText('No unsaved changes.');
|
||||
await expect(key.locator('p.small')).toHaveText('No key saved.');
|
||||
// Typing a key drafts it; Save sends it once through /lora/enable with the settings revision.
|
||||
await field.fill('sk-test-key');
|
||||
await expect(key.locator('p.small')).toHaveText('The new key replaces the saved one when you save.');
|
||||
await expect(status(page)).toHaveText('Unsaved changes.');
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved to the server configuration.');
|
||||
expect(enables).toHaveLength(1);
|
||||
expect(enables[0]).toMatchObject({ civitai_api_key: 'sk-test-key', enabled: false });
|
||||
expect(enables[0].expected_revision).toMatch(/^(sha256:|absent)/);
|
||||
await expect(field).toHaveValue('');
|
||||
await expect(key.locator('p.small')).toHaveText('A key is saved.');
|
||||
expect((await serverJson(request, '/__test__/integrations/state')).key_saved).toBe(true);
|
||||
// Never prefilled, even after a reload; the page never receives the key.
|
||||
const statusBodies = [];
|
||||
page.on('response', async response => {
|
||||
if (response.url().endsWith('/prompt_manager/lora/status')) statusBodies.push(await response.text());
|
||||
});
|
||||
await openIntegrations(page);
|
||||
await expect(field).toHaveValue('');
|
||||
await expect(key.locator('p.small')).toHaveText('A key is saved.');
|
||||
expect(statusBodies.join('')).not.toContain('sk-test-key');
|
||||
// The announced line is rewritten only when its text changes, not on every edit elsewhere (S10 re-review N8).
|
||||
await page.evaluate(() => {
|
||||
window.__keyNoteWrites = 0;
|
||||
new MutationObserver(records => { window.__keyNoteWrites += records.length; })
|
||||
.observe(document.querySelector('#civitai-key-status'), { childList: true, characterData: true, subtree: true });
|
||||
});
|
||||
const triggers = settingOf(page, 'Automatically include trigger words').locator('input');
|
||||
await triggers.check();
|
||||
await expect(status(page)).toHaveText('Unsaved changes.');
|
||||
await triggers.uncheck();
|
||||
await expect(status(page)).toHaveText('No unsaved changes.');
|
||||
expect(await page.evaluate(() => window.__keyNoteWrites)).toBe(0);
|
||||
// A blank field keeps the saved key: nothing to save.
|
||||
await field.fill(' ');
|
||||
await expect(status(page)).toHaveText('No unsaved changes.');
|
||||
await field.fill('');
|
||||
// Clear saved key drafts the removal; Save removes it. The keyboard stays on the key field, and the line that
|
||||
// says so is announced (S10 review M2).
|
||||
await expect(key.locator('p.small')).toHaveAttribute('role', 'status');
|
||||
await expect(field).toHaveAttribute('aria-describedby', 'civitai-key-status');
|
||||
await clear.focus();
|
||||
await page.keyboard.press('Enter');
|
||||
await expect(key.locator('p.small')).toHaveText('The saved key is removed when you save.');
|
||||
await expect(clear).toHaveCount(0);
|
||||
await expect(field).toBeFocused();
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved to the server configuration.');
|
||||
expect(enables.at(-1)).toMatchObject({ clear_civitai_api_key: true });
|
||||
expect(enables.at(-1)).not.toHaveProperty('civitai_api_key');
|
||||
await expect(key.locator('p.small')).toHaveText('No key saved.');
|
||||
expect((await serverJson(request, '/__test__/integrations/state')).key_saved).toBe(false);
|
||||
});
|
||||
|
||||
test('a LoRA save over settings changed elsewhere is a conflict, never a blind write', async ({ page, request }) => {
|
||||
await lora(request);
|
||||
await openIntegrations(page);
|
||||
await settingOf(page, 'Automatically include trigger words').locator('input').check();
|
||||
// Another page saves the settings meanwhile: the revision this page read is stale.
|
||||
await expect(await request.post('/prompt_manager/settings', { data: { result_timeout: 42 } })).toBeOK();
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Settings changed elsewhere. Reload this section to see them.');
|
||||
expect((await serverJson(request, '/__test__/integrations/state')).trigger_words).toBe(false);
|
||||
// The draft stays; the next Save carries the reread revision and succeeds.
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved to the server configuration.');
|
||||
expect((await serverJson(request, '/__test__/integrations/state')).trigger_words).toBe(true);
|
||||
});
|
||||
|
||||
test('a save where only the LoRA part is written says so; the next save needs no conflict (S10 review I3)',
|
||||
async ({ page, request }) => {
|
||||
await lora(request);
|
||||
await expect(await request.post('/__test__/system/missing-source', { data: { inside: true } })).toBeOK();
|
||||
await openIntegrations(page);
|
||||
const triggers = settingOf(page, 'Automatically include trigger words').locator('input');
|
||||
await triggers.check();
|
||||
// In Sources, the drive is unplugged after the page read the list, and another source is removed.
|
||||
const nav = page.locator('#workspace-main .section-nav');
|
||||
await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click();
|
||||
const sourceRow = title => layoutOf(page).locator('#source-list .operation').filter({ has: page.locator('h3', { hasText: title }) });
|
||||
await expect(sourceRow('unplugged')).toBeVisible();
|
||||
await expect(await request.post('/__test__/system/unplug')).toBeOK();
|
||||
await sourceRow('ComfyUI input').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
await save(page).click();
|
||||
// The LoRA part was saved, the sources were refused: the status gives the server's reason for the source, and
|
||||
// only the source row is highlighted.
|
||||
await expect(status(page)).toHaveText("output/unplugged can't be used: Gallery root does not exist. Remove it or reconnect it, then save.");
|
||||
await expect(sourceRow('unplugged')).toHaveAttribute('aria-invalid', 'true');
|
||||
expect((await serverJson(request, '/__test__/integrations/state')).trigger_words).toBe(true);
|
||||
await nav.getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
await expect(triggers).toBeChecked();
|
||||
await expect(triggers).not.toHaveAttribute('aria-invalid', 'true');
|
||||
// The next save carries the revision the saved part left: it saves, with no "changed elsewhere".
|
||||
await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click();
|
||||
await sourceRow('unplugged').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
await save(page).click();
|
||||
await expect(status(page)).toHaveText('Saved to the server configuration.');
|
||||
expect((await serverJson(request, '/prompt_manager/settings')).sources.configured.map(source => source.path)).toEqual(['output']);
|
||||
});
|
||||
|
||||
test('a partial save made from Integrations keeps its outcome when Sources opens (S10 re-review R1)',
|
||||
async ({ page, request }) => {
|
||||
await lora(request);
|
||||
await expect(await request.post('/__test__/system/missing-source', { data: { inside: true } })).toBeOK();
|
||||
await openIntegrations(page);
|
||||
const nav = page.locator('#workspace-main .section-nav');
|
||||
const sourceRow = title => layoutOf(page).locator('#source-list .operation').filter({ has: page.locator('h3', { hasText: title }) });
|
||||
await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click();
|
||||
await expect(sourceRow('unplugged')).toBeVisible();
|
||||
await expect(await request.post('/__test__/system/unplug')).toBeOK();
|
||||
await sourceRow('ComfyUI input').getByRole('button', { name: 'Remove', exact: true }).click();
|
||||
// Back in Integrations, a LoRA change, then Save from there.
|
||||
await nav.getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
await settingOf(page, 'Automatically include trigger words').locator('input').check();
|
||||
await save(page).click();
|
||||
const reason = "output/unplugged can't be used: Gallery root does not exist. Remove it or reconnect it, then save.";
|
||||
await expect(status(page)).toHaveText(reason);
|
||||
// Leaving Integrations (no key was typed) keeps the outcome: Sources shows the reason on the refused row.
|
||||
await nav.getByRole('button', { name: 'Sources & scans', exact: true }).click();
|
||||
await expect(sourceRow('unplugged')).toHaveAttribute('aria-invalid', 'true');
|
||||
await expect(status(page)).toHaveText(reason);
|
||||
});
|
||||
|
||||
test('Review import shows replacement, blocked items and network effects with the backup box; Start opens the job (D107)',
|
||||
async ({ page, request }) => {
|
||||
await lora(request, { loras: 3, civitai: true, previous: 2 });
|
||||
const previews = [];
|
||||
const applies = [];
|
||||
page.on('request', item => {
|
||||
if (item.url().endsWith(PREVIEW_URL)) previews.push(item.postDataJSON());
|
||||
if (item.url().endsWith(APPLY_URL)) applies.push(item.postDataJSON());
|
||||
});
|
||||
await openIntegrations(page);
|
||||
await enable(page);
|
||||
const before = await serverJson(request, '/__test__/integrations/state');
|
||||
const review = layoutOf(page).locator('.operation').filter({ hasText: 'LoRA reference data' })
|
||||
.getByRole('button', { name: 'Review import', exact: true });
|
||||
await review.click();
|
||||
await expect(dialog(page).locator('#modal-title')).toHaveText('Review LoRA import');
|
||||
await expect(dialog(page).locator('#modal-body > h3')).toHaveText('LoRA reference data');
|
||||
expect(previews).toEqual([{}]);
|
||||
expect(await gridOf(page)).toEqual([
|
||||
['Scope', 'Replace 2 imported prompts with 3 LoRAs'],
|
||||
['Found', '3 metadata files · 3 readable'],
|
||||
['Protected', 'Nothing is protected'],
|
||||
['Network', 'Downloads up to 3 example images from CivitAI'],
|
||||
['Backup', 'Create and verify before changes (browser.db.backup_<date and time>)'],
|
||||
['Media on disk', 'No files are removed; image links are replaced'],
|
||||
]);
|
||||
await expect(dialog(page).locator('details > summary')).toHaveText('LoRAs found (3)');
|
||||
await expect(dialog(page).locator('.notice h3')).toHaveText('Back up the database first (recommended)');
|
||||
await expect(dialog(page).locator('p.small').last()).toContainText('stay in the image cache');
|
||||
// The review wrote and downloaded nothing; Cancel neither.
|
||||
await dialog(page).getByRole('button', { name: 'Cancel', exact: true }).click();
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect(review).toBeFocused();
|
||||
expect(await serverJson(request, '/__test__/integrations/state')).toEqual(before);
|
||||
|
||||
await review.click();
|
||||
await dialog(page).getByRole('button', { name: 'Start import', exact: true }).click();
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
|
||||
await expect(toast(page)).toHaveText(/LoRA reference import started\.|LoRA import finished · 3 imported · 0 errors/);
|
||||
expect(applies).toHaveLength(1);
|
||||
expect(Object.keys(applies[0])).toEqual(['token']);
|
||||
const job = await ended(request, page.url().split('/').pop());
|
||||
expect([job.kind, job.title, job.state]).toEqual(['lora_import', 'LoRA reference import', 'finished']);
|
||||
expect(job.counters).toMatchObject({ imported: 3, downloaded: 3, download_errors: 0, errors: 0 });
|
||||
const main = page.locator('#workspace-main');
|
||||
await expect(main.locator('h1')).toHaveText('LoRA reference import');
|
||||
await expect(main.locator('.stat-row .stat dt')).toHaveText(['Imported', 'Linked to existing', 'Skipped', 'Images downloaded',
|
||||
'Download errors', 'Errors']);
|
||||
await expect(main.getByRole('button', { name: 'Cancel import', exact: true })).toHaveCount(0);
|
||||
const after = await serverJson(request, '/__test__/integrations/state');
|
||||
expect(after.imports).toEqual(['style_01 example prompt', 'style_02 example prompt', 'style_03 example prompt']);
|
||||
expect(after.downloads).toBe(3);
|
||||
});
|
||||
|
||||
test('saved pairs block the import: the review explains it and offers only Close', async ({ page, request }) => {
|
||||
await lora(request, { previous: 1 });
|
||||
await expect(await request.post('/__test__/integrations/pair')).toBeOK();
|
||||
await openIntegrations(page);
|
||||
await enable(page);
|
||||
await layoutOf(page).getByRole('button', { name: 'Review import', exact: true }).click();
|
||||
expect((await gridOf(page))[2]).toEqual(['Protected', '1 imported prompt used by saved pairs']);
|
||||
await expect(dialog(page).locator('p.small')).toHaveText(
|
||||
'Delete or change the saved pairs that use these prompts, then review the import again. Nothing was changed.');
|
||||
expect(await dialog(page).locator('.dialog-actions button').allTextContents()).toEqual(['Close']);
|
||||
await dialog(page).getByRole('button', { name: 'Close', exact: true }).click();
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
});
|
||||
|
||||
test('a failed import keeps old imports and its job says so', async ({ page, request }) => {
|
||||
await lora(request, { loras: 2, previous: 2 });
|
||||
await openIntegrations(page);
|
||||
await enable(page);
|
||||
const before = await serverJson(request, '/__test__/integrations/state');
|
||||
await expect(await request.post('/__test__/integrations/fail-import')).toBeOK();
|
||||
await layoutOf(page).getByRole('button', { name: 'Review import', exact: true }).click();
|
||||
await dialog(page).getByRole('button', { name: 'Start import', exact: true }).click();
|
||||
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
|
||||
await expect(toast(page)).toHaveText(`LoRA import failed · ${KEPT}`);
|
||||
const job = await ended(request, page.url().split('/').pop());
|
||||
expect([job.state, job.scope.failure, job.counters.errors]).toEqual(['failed', 'import-failed', 1]);
|
||||
await expect(page.locator('#workspace-main')).toContainText(KEPT);
|
||||
expect((await serverJson(request, '/__test__/integrations/state')).imports).toEqual(before.imports);
|
||||
expect(before.imports).toEqual(['previous lora import 1', 'previous lora import 2']);
|
||||
});
|
||||
|
||||
test('the import can be cancelled while it downloads; then nothing is written', async ({ page, request }) => {
|
||||
await lora(request, { loras: 2, civitai: true, previous: 1 });
|
||||
await openIntegrations(page);
|
||||
await enable(page);
|
||||
const before = await serverJson(request, '/__test__/integrations/state');
|
||||
await expect(await request.post('/__test__/integrations/gate', { data: { hold: true } })).toBeOK();
|
||||
await layoutOf(page).getByRole('button', { name: 'Review import', exact: true }).click();
|
||||
await dialog(page).getByRole('button', { name: 'Start import', exact: true }).click();
|
||||
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
|
||||
const id = page.url().split('/').pop();
|
||||
await expect.poll(async () => (await serverJson(request, '/__test__/integrations/state')).downloads).toBeGreaterThan(0);
|
||||
const cancel = page.locator('#workspace-main').getByRole('button', { name: 'Cancel import', exact: true });
|
||||
await expect(cancel).toBeVisible();
|
||||
// The cancel must have reached the server before the held download goes on (S10 review M1).
|
||||
await Promise.all([
|
||||
page.waitForResponse(response => /\/prompt_manager\/jobs\/[0-9a-f]+\/cancel$/.test(response.url()) && response.ok()),
|
||||
cancel.click(),
|
||||
]);
|
||||
await expect(await request.post('/__test__/integrations/gate', { data: { hold: false } })).toBeOK();
|
||||
const job = await ended(request, id);
|
||||
expect(job.state).toBe('cancelled');
|
||||
expect((await serverJson(request, '/__test__/integrations/state')).imports).toEqual(before.imports);
|
||||
await expect(cancel).toHaveCount(0);
|
||||
});
|
||||
@@ -409,11 +409,11 @@ test('out-of-order overview responses never repaint a newer section', async ({ p
|
||||
await nav.getByRole('button', { name: 'Appearance', exact: true }).click();
|
||||
await nav.getByRole('button', { name: 'Overview', exact: true }).click();
|
||||
await expect.poll(held).toBe(2);
|
||||
// Integrations still hands off (S10), so its body is one known panel.
|
||||
await nav.getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
// Auto-tagging still hands off (S11), so its body is one known panel.
|
||||
await nav.getByRole('button', { name: 'Auto-tagging', exact: true }).click();
|
||||
await expect(await request.post('/__test__/release')).toBeOK();
|
||||
await expect.poll(held).toBe(0);
|
||||
// Both late answers reached a section that had left: Integrations keeps its own body.
|
||||
// Both late answers reached a section that had left: Auto-tagging keeps its own body.
|
||||
await expect(layout.locator(':scope > section.handoff')).toBeVisible();
|
||||
await page.evaluate(() => new Promise(resolve => requestAnimationFrame(() => requestAnimationFrame(resolve))));
|
||||
await expect(layout.locator('.summary-metrics')).toHaveCount(0);
|
||||
|
||||
@@ -413,7 +413,7 @@ for (const width of [1600, 390]) test(`overview and handoff render with concept
|
||||
|
||||
await sidebarLink(page, 'System').click();
|
||||
// System's Overview is native (S2 Task 6); a section not delivered yet still hands off (D94).
|
||||
await main.locator('.section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
|
||||
await main.locator('.section-nav').getByRole('button', { name: 'Auto-tagging', exact: true }).click();
|
||||
const handoff = main.locator('section.handoff');
|
||||
await expect(handoff).toHaveClass(/\bsurface\b/);
|
||||
expect(await handoff.evaluate(surfaceStyle)).toMatchObject(SURFACE);
|
||||
@@ -463,16 +463,18 @@ test('System is native: the concept shell, section navigation, sections, hand-of
|
||||
// The section changes in place: focus stays on the pressed section, not the page title.
|
||||
await expect(button).toBeFocused();
|
||||
// S2 Task 6: Overview is native (workspace-system.spec.mjs), and so are Appearance (S3 Task 8) and Sources & scans
|
||||
// (S4 Task 11, workspace-system-sources.spec.mjs, its update notice shown in a fresh browser) and Maintenance (S7
|
||||
// Task 19, workspace-system-maintenance.spec.mjs); every other section hands off.
|
||||
// (S4 Task 11, workspace-system-sources.spec.mjs, its update notice shown in a fresh browser), Integrations (S10
|
||||
// Task 27, workspace-system-integrations.spec.mjs) and Maintenance (S7 Task 19,
|
||||
// workspace-system-maintenance.spec.mjs); every other section hands off.
|
||||
if (hash === 'overview') await expect(layout.locator(':scope > .summary-metrics')).toBeVisible();
|
||||
if (hash === 'sources') await expect(layout.locator('#source-list .operation')).toHaveCount(2);
|
||||
expect(await layout.evaluate(node => [...node.children].map(child => `${child.localName}.${child.className}`)))
|
||||
.toEqual({ overview: ['div.summary-metrics', 'div.two-col', 'div.save-row'], appearance: ['section.surface', 'section.surface', 'div.save-row'],
|
||||
sources: ['div.notice', 'section.surface', 'div.save-row'],
|
||||
integrations: ['section.surface', 'section.surface', 'div.save-row'],
|
||||
maintenance: ['div.notice', 'section.surface', 'section.surface', 'div.save-row'] }[hash]
|
||||
?? ['section.surface handoff', 'div.save-row']);
|
||||
if (['overview', 'appearance', 'sources', 'maintenance'].includes(hash)) continue;
|
||||
if (['overview', 'appearance', 'sources', 'integrations', 'maintenance'].includes(hash)) continue;
|
||||
const handoff = layout.locator('section.handoff');
|
||||
await expect(handoff.locator('h2')).toHaveText('Your current tools are ready');
|
||||
expect(await handoff.locator('a.workspace-link').evaluateAll(links => links.map(link => link.getAttribute('href'))))
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
"""System › Integrations test hooks for the browser tests (#249 System S10).
|
||||
|
||||
A fake LoraManager install lives under this server's temp root; nothing ever
|
||||
reaches a real LoraManager, ComfyUI install or the network:
|
||||
|
||||
- ``POST /__test__/integrations/reset`` turns the integration off (no path,
|
||||
no key, no trigger words), removes the fake install, its LoRA folder and
|
||||
the image cache, forgets every LoRA import and injected failure, and opens
|
||||
the download gate;
|
||||
- ``POST /__test__/integrations/lora-manager`` ``{loras, civitai, previous}``
|
||||
writes ``custom_nodes/ComfyUI-Lora-Manager`` (detected through the
|
||||
``folder_paths`` stub's base path) and ``loras`` metadata files under
|
||||
``models/loras``, each with a local preview PNG; ``civitai`` adds one
|
||||
CivitAI example image per LoRA (served by the stub opener below);
|
||||
``previous`` saves that many earlier ``lora-manager`` imports with tags and
|
||||
image links;
|
||||
- ``POST /__test__/integrations/gate`` ``{hold}`` holds every CivitAI
|
||||
download until it is released (``{"hold": false}``), so a test sees the job
|
||||
while it downloads;
|
||||
- ``POST /__test__/integrations/fail-import`` makes the next collection
|
||||
replacement fail (a database error after staging);
|
||||
- ``POST /__test__/integrations/pair`` saves a pair whose positive is the first
|
||||
imported prompt (a saved pair that blocks the replacement);
|
||||
- ``GET /__test__/integrations/state`` answers the imported prompt texts,
|
||||
whether a CivitAI key is saved and the downloads served.
|
||||
|
||||
CivitAI downloads are always answered by a stub opener (a small PNG), so no
|
||||
test ever opens a network connection.
|
||||
"""
|
||||
|
||||
import io
|
||||
import json
|
||||
import shutil
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
from aiohttp import web
|
||||
from PIL import Image
|
||||
|
||||
MAX_LORAS = 20
|
||||
CATEGORY = "lora-manager"
|
||||
|
||||
|
||||
def _png():
|
||||
buffer = io.BytesIO()
|
||||
Image.new("RGB", (8, 8), "#3a6ea5").save(buffer, "PNG")
|
||||
return buffer.getvalue()
|
||||
|
||||
|
||||
class _Response:
|
||||
def __init__(self, body):
|
||||
self._body = io.BytesIO(body)
|
||||
self.headers = {"Content-Length": str(len(body))}
|
||||
|
||||
def read(self, amount=-1):
|
||||
return self._body.read(amount)
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc):
|
||||
return False
|
||||
|
||||
|
||||
def register_integrations_commands(app, routes, api, root: Path):
|
||||
from py import lora_utils
|
||||
from py.config import IntegrationConfig
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
lora_manager = root / "custom_nodes" / "ComfyUI-Lora-Manager"
|
||||
loras = root / "models" / "loras"
|
||||
gate = threading.Event()
|
||||
gate.set()
|
||||
served = []
|
||||
failures = {"replace": 0}
|
||||
pairs = [] # the blocking pairs this fixture saved
|
||||
real_replace = type(api.db).replace_category_prompts
|
||||
|
||||
def stub_open(request, timeout=10):
|
||||
served.append(request.full_url)
|
||||
gate.wait(30)
|
||||
return _Response(_png())
|
||||
|
||||
lora_utils._open_url = stub_open
|
||||
|
||||
def replace(db, *args, **kwargs):
|
||||
if failures["replace"] > 0:
|
||||
failures["replace"] -= 1
|
||||
raise RuntimeError("Injected replacement failure")
|
||||
return real_replace(db, *args, **kwargs)
|
||||
|
||||
type(api.db).replace_category_prompts = replace
|
||||
|
||||
def reset():
|
||||
IntegrationConfig.LORA_MANAGER_ENABLED = False
|
||||
IntegrationConfig.LORA_MANAGER_PATH = ""
|
||||
IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED = False
|
||||
IntegrationConfig.CIVITAI_API_KEY = ""
|
||||
lora_utils.get_trigger_cache().clear()
|
||||
gate.set()
|
||||
served.clear()
|
||||
failures["replace"] = 0
|
||||
cache = Path(lora_utils.get_lora_image_cache_dir(create=False))
|
||||
for folder in (lora_manager, loras, cache):
|
||||
shutil.rmtree(folder, ignore_errors=True)
|
||||
for pair_id in pairs:
|
||||
api.db.delete_prompt_pair(pair_id)
|
||||
pairs.clear()
|
||||
real_replace(api.db, CATEGORY, [], allow_empty=True)
|
||||
|
||||
@routes.post("/__test__/integrations/reset")
|
||||
async def integrations_reset(request):
|
||||
reset()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/integrations/lora-manager")
|
||||
async def make_lora_manager(request):
|
||||
data = await request.json() if request.can_read_body else {}
|
||||
count = data.get("loras", 2)
|
||||
previous = data.get("previous", 0)
|
||||
for value in (count, previous):
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise web.HTTPBadRequest()
|
||||
if not 0 <= count <= MAX_LORAS or not 0 <= previous <= MAX_LORAS:
|
||||
raise web.HTTPBadRequest()
|
||||
(lora_manager / "py").mkdir(parents=True, exist_ok=True)
|
||||
(lora_manager / "__init__.py").write_text("")
|
||||
loras.mkdir(parents=True, exist_ok=True)
|
||||
for number in range(count):
|
||||
stem = f"style_{number + 1:02}"
|
||||
civitai = {"trainedWords": [f"trigger{number + 1}"]}
|
||||
if data.get("civitai") is True:
|
||||
civitai["images"] = [
|
||||
{
|
||||
"url": f"https://image.civitai.com/x/{stem}.jpeg",
|
||||
"meta": {"prompt": f"{stem} example prompt"},
|
||||
}
|
||||
]
|
||||
meta = {
|
||||
"file_name": f"{stem}.safetensors",
|
||||
"model_name": f"Style {number + 1:02}",
|
||||
"civitai": civitai,
|
||||
}
|
||||
(loras / f"{stem}.metadata.json").write_text(json.dumps(meta))
|
||||
(loras / f"{stem}.png").write_bytes(_png())
|
||||
for number in range(previous):
|
||||
text = f"previous lora import {number + 1}"
|
||||
prompt_id = api.db.save_prompt(
|
||||
text=text,
|
||||
category=CATEGORY,
|
||||
tags=[CATEGORY, f"lora:previous-{number + 1}"],
|
||||
prompt_hash=generate_prompt_hash(text),
|
||||
)
|
||||
api.db.link_image_to_prompt(str(prompt_id), str(loras / "old.png"))
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/integrations/gate")
|
||||
async def download_gate(request):
|
||||
data = await request.json()
|
||||
if not isinstance(data.get("hold"), bool):
|
||||
raise web.HTTPBadRequest()
|
||||
if data["hold"]:
|
||||
gate.clear()
|
||||
else:
|
||||
gate.set()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/integrations/fail-import")
|
||||
async def fail_import(request):
|
||||
failures["replace"] = 1
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/integrations/pair")
|
||||
async def pair(request):
|
||||
with api.db.model.get_connection() as conn:
|
||||
row = conn.execute(
|
||||
"SELECT text FROM prompts WHERE category = ? ORDER BY id LIMIT 1",
|
||||
(CATEGORY,),
|
||||
).fetchone()
|
||||
if row is None:
|
||||
raise web.HTTPConflict()
|
||||
pairs.append(api.db.save_prompt_pair(row[0], "blurry"))
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.get("/__test__/integrations/state")
|
||||
async def state(request):
|
||||
imports = api.db.search_prompts(category=CATEGORY, limit=MAX_LORAS * 2)
|
||||
return web.json_response(
|
||||
{
|
||||
"imports": sorted(prompt["text"] for prompt in imports),
|
||||
"key_saved": bool(IntegrationConfig.CIVITAI_API_KEY),
|
||||
"enabled": IntegrationConfig.LORA_MANAGER_ENABLED,
|
||||
"trigger_words": IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED,
|
||||
"downloads": len(served),
|
||||
}
|
||||
)
|
||||
|
||||
return reset
|
||||
@@ -190,3 +190,21 @@ test('bridge remains live through both renderer switches and workflow reload wit
|
||||
await expect(page.locator('.pm-focused').getByRole('textbox',{name:'Main prompt',exact:true})).toHaveValue('After reload');
|
||||
});
|
||||
});
|
||||
|
||||
test('each destination reports its window renderer, and System › Integrations shows it (S10)',async({page,context},testInfo)=>{
|
||||
const initial=testInfo.project.metadata.initialRenderer;
|
||||
const id=await addNode(page,'PromptManager',{text:'Renderer check'});
|
||||
await withClient(page,context,async clientPage=>{
|
||||
const name=renderer=>renderer?'Nodes 2.0':'Classic';
|
||||
const status=clientPage.locator('#workspace-main #renderer-status');
|
||||
expect((await destination(clientPage,id)).target.renderer).toBe(initial?'nodes2':'classic');
|
||||
await clientPage.evaluate(()=>{location.hash='#/system/integrations';});
|
||||
await expect(status).toHaveText(`Detected: ${name(initial)} (1 ComfyUI window).`);
|
||||
// Switching the renderer in ComfyUI is what the next check reports; nothing is sent or queued.
|
||||
await page.evaluate(async renderer=>{const {app}=await import('/scripts/app.js');await app.ui.settings.setSettingValue('Comfy.VueNodes.Enabled',renderer);await app.loadGraphData(app.graph.serialize());},!initial);
|
||||
await expect.poll(()=>page.evaluate(()=>LiteGraph.vueNodesMode)).toBe(!initial);
|
||||
expect((await destination(clientPage,id)).target.renderer).toBe(initial?'classic':'nodes2');
|
||||
await clientPage.reload();
|
||||
await expect(status).toHaveText(`Detected: ${name(!initial)} (1 ComfyUI window).`);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -17,6 +17,17 @@ test('lists only explicit PromptManager destinations and roles for all five sche
|
||||
assert.equal(targets[0].id,'0'); assert.equal(targets[0].label,'My prompts');
|
||||
assert.deepEqual(createPromptAdapter({}).listTargets(),[]);
|
||||
});
|
||||
test('each destination says which renderer its window draws nodes with (System › Integrations, S10)',async()=>{
|
||||
const {createPromptAdapter}=await source; const {app}=fixture();
|
||||
assert.equal(createPromptAdapter(app,{runtime:{LiteGraph:{vueNodesMode:true}}}).listTargets()[0].renderer,'nodes2');
|
||||
assert.equal(createPromptAdapter(app,{runtime:{LiteGraph:{vueNodesMode:false}}}).listTargets()[0].renderer,'classic');
|
||||
assert.equal(createPromptAdapter(app,{runtime:{LiteGraph:{}}}).listTargets()[0].renderer,'classic');
|
||||
assert.equal(Object.hasOwn(createPromptAdapter(app,{runtime:{}}).listTargets()[0],'renderer'),false);
|
||||
// Read at every listing: a window that switches renderer reports the new one.
|
||||
const runtime={LiteGraph:{vueNodesMode:false}}; const adapter=createPromptAdapter(app,{runtime});
|
||||
runtime.LiteGraph.vueNodesMode=true;
|
||||
assert.equal(adapter.listTargets()[0].renderer,'nodes2');
|
||||
});
|
||||
test('previews exact composition, updates requested visible/serialized values, preserves other fields',async()=>{
|
||||
const {createPromptAdapter}=await source; const {app,node,widget}=fixture(); const adapter=createPromptAdapter(app);
|
||||
const {preview,snapshot}=adapter.preview('7',{positive:'new'});
|
||||
|
||||
@@ -0,0 +1,197 @@
|
||||
// System › Integrations (#249 System S10 Task 27): the host values and saves behind the LoRA rows (D114), the D106
|
||||
// renderer and detection lines, and the D107 import review built from the server's plan.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
|
||||
const integrations = import('../../web/js/workspace/system-integrations.mjs');
|
||||
const kinds = import('../../web/js/workspace/activity-kinds.mjs');
|
||||
|
||||
const CONCEPT = fs.readFileSync(path.join(__dirname, '../../docs/concepts/workspace/app.js'), 'utf8');
|
||||
const SOURCE = fs.readFileSync(path.join(__dirname, '../../web/js/workspace/system-integrations.mjs'), 'utf8');
|
||||
|
||||
test('the section keeps the concept rows, labels and options in order', () => {
|
||||
const start = CONCEPT.indexOf('case "integrations":');
|
||||
const body = CONCEPT.slice(start, CONCEPT.indexOf('break;', start));
|
||||
const rows = [...body.matchAll(/setting\("([^"]+)", "([^"]+)", "([^"]+)"/g)].map(([, id, label]) => [id, label]);
|
||||
assert.deepEqual(rows.map(([id]) => id), ['send-destination', 'renderer', 'return-canvas', 'lora-enable', 'lora-triggers',
|
||||
'civitai-key']);
|
||||
for (const [id, label] of rows) {
|
||||
assert.ok(SOURCE.includes(`id: '${id}', label: '${label}'`), `${id} keeps the concept label "${label}"`);
|
||||
}
|
||||
for (const option of ['Ask each time', 'Selected Prompt Manager node', 'Create a Prompt Manager (+/-) node',
|
||||
'Follow host renderer', 'Classic', 'Nodes 2.0']) assert.ok(body.includes(option) && SOURCE.includes(`'${option}'`), option);
|
||||
for (const surface of ['ComfyUI', 'LoRA & resources']) assert.ok(SOURCE.includes(`'h2', '${surface}'`), surface);
|
||||
assert.ok(body.includes('"LoRA reference data", "Review and import resource metadata."'));
|
||||
// Prototype copy is never shipped (Global constraints): the sample detection line and the mock warning.
|
||||
assert.ok(!SOURCE.includes('Sample detection'));
|
||||
assert.ok(!SOURCE.includes('Do not enter a real key'));
|
||||
assert.ok(!SOURCE.includes('Simulate import'));
|
||||
});
|
||||
|
||||
test('LoRA status becomes host values: drafted flags and read-only facts kept apart', async () => {
|
||||
const { loraValues } = await integrations;
|
||||
assert.deepEqual(loraValues({ enabled: true, trigger_words_enabled: false, path: 'custom_nodes/ComfyUI-Lora-Manager',
|
||||
has_civitai_api_key: true, detected: true, detected_path: 'custom_nodes/ComfyUI-Lora-Manager', trigger_cache_loaded: true }), {
|
||||
lora_enabled: true, lora_trigger_words_enabled: false, clear_civitai_api_key: false,
|
||||
lora_path: 'custom_nodes/ComfyUI-Lora-Manager', lora_detected: true, lora_detected_path: 'custom_nodes/ComfyUI-Lora-Manager',
|
||||
lora_detected_usable: true, lora_has_key: true, lora_cache_loaded: true,
|
||||
});
|
||||
const empty = loraValues({});
|
||||
assert.equal(empty.lora_enabled, false);
|
||||
assert.equal(empty.lora_path, '');
|
||||
assert.equal(empty.lora_has_key, false);
|
||||
// The key itself is never part of what the page holds from the server.
|
||||
assert.ok(!Object.keys(loraValues({ civitai_api_key: 'secret' })).includes('civitai_api_key'));
|
||||
});
|
||||
|
||||
test('host changes split between /lora/enable and /settings', async () => {
|
||||
const { splitHostChanges } = await integrations;
|
||||
assert.deepEqual(splitHostChanges({ lora_enabled: true, result_timeout: 5 }),
|
||||
{ lora: { lora_enabled: true }, settings: { result_timeout: 5 } });
|
||||
assert.deepEqual(splitHostChanges({ result_timeout: 5 }), { lora: null, settings: { result_timeout: 5 } });
|
||||
assert.deepEqual(splitHostChanges({ civitai_api_key: 'k', clear_civitai_api_key: false }),
|
||||
{ lora: { civitai_api_key: 'k', clear_civitai_api_key: false }, settings: null });
|
||||
assert.deepEqual(splitHostChanges(undefined), { lora: null, settings: null });
|
||||
});
|
||||
|
||||
test('key never prefilled; blank keeps; Clear saved key clears; saved through Save preferences (D114)', async () => {
|
||||
const { loraPayload, keyText, KEY_COPY } = await integrations;
|
||||
const saved = { lora_enabled: true, lora_trigger_words_enabled: false, lora_path: 'custom_nodes/LM', lora_has_key: true };
|
||||
// The whole LoRA state goes with every save; the saved path is kept; no key unless one was typed.
|
||||
assert.deepEqual(loraPayload({ lora_trigger_words_enabled: true }, saved),
|
||||
{ enabled: true, trigger_words_enabled: true, path: 'custom_nodes/LM' });
|
||||
assert.deepEqual(loraPayload({ civitai_api_key: ' ' }, saved), { enabled: true, trigger_words_enabled: false, path: 'custom_nodes/LM' });
|
||||
assert.deepEqual(loraPayload({ civitai_api_key: ' new-key ' }, saved),
|
||||
{ enabled: true, trigger_words_enabled: false, path: 'custom_nodes/LM', civitai_api_key: 'new-key' });
|
||||
assert.deepEqual(loraPayload({ clear_civitai_api_key: true, civitai_api_key: 'ignored' }, saved),
|
||||
{ enabled: true, trigger_words_enabled: false, path: 'custom_nodes/LM', clear_civitai_api_key: true });
|
||||
assert.deepEqual(loraPayload({ lora_enabled: true }, {}), { enabled: true, trigger_words_enabled: false, path: '' });
|
||||
assert.equal(keyText({ lora_has_key: true }, {}), KEY_COPY.saved);
|
||||
assert.equal(keyText({ lora_has_key: false }, {}), KEY_COPY.none);
|
||||
assert.equal(keyText({ lora_has_key: true }, { clear_civitai_api_key: true }), KEY_COPY.clearing);
|
||||
assert.equal(keyText({ lora_has_key: false }, { civitai_api_key: 'k' }), KEY_COPY.replacing);
|
||||
assert.equal(keyText({}, {}), null);
|
||||
assert.deepEqual(KEY_COPY, { saved: 'A key is saved.', none: 'No key saved.', clearing: 'The saved key is removed when you save.',
|
||||
replacing: 'The new key replaces the saved one when you save.' });
|
||||
});
|
||||
|
||||
test('LoRA rows show detected / enabled / cache-loaded separately', async () => {
|
||||
const { detectionText, cacheText } = await integrations;
|
||||
assert.equal(detectionText({ lora_detected: true, lora_detected_path: 'custom_nodes/ComfyUI-Lora-Manager' }),
|
||||
'Detected: custom_nodes/ComfyUI-Lora-Manager.');
|
||||
assert.equal(detectionText({ lora_detected: true, lora_detected_path: '' }), 'Detected.');
|
||||
assert.equal(detectionText({ lora_detected: false }), 'Not detected in ComfyUI’s custom_nodes folders.');
|
||||
assert.equal(detectionText({}), null);
|
||||
// S10 review N5: a linked install outside custom_nodes is found but can't be enabled.
|
||||
assert.equal(detectionText({ lora_detected: true, lora_detected_path: 'LoRA-Manager', lora_detected_usable: false }),
|
||||
'Found at LoRA-Manager, but outside ComfyUI’s custom_nodes folders, so it can’t be enabled.');
|
||||
assert.equal((await integrations).loraValues({ detected: true, detected_usable: false }).lora_detected_usable, false);
|
||||
assert.equal(cacheText({ lora_cache_loaded: true }), 'Trigger words are loaded.');
|
||||
assert.equal(cacheText({ lora_cache_loaded: false }), 'Trigger words are not loaded.');
|
||||
assert.equal(cacheText({}), null);
|
||||
});
|
||||
|
||||
test('host compatibility shows the detected renderer as status (D106)', async () => {
|
||||
const { rendererText, RENDERER_COPY } = await integrations;
|
||||
const host = (...renderers) => ({ hostId: 'h', label: 'PromptManager — ComfyUI', targets: renderers.map((renderer, id) =>
|
||||
({ id: String(id), roles: ['positive'], renderer })) });
|
||||
assert.equal(rendererText([]), RENDERER_COPY.none);
|
||||
assert.equal(rendererText(null), RENDERER_COPY.none);
|
||||
assert.equal(rendererText([host()]), RENDERER_COPY.noNodes);
|
||||
assert.equal(rendererText([host('vue')]), RENDERER_COPY.noNodes);
|
||||
assert.equal(rendererText([host('classic')]), 'Detected: Classic (1 ComfyUI window).');
|
||||
assert.equal(rendererText([host('nodes2', 'nodes2')]), 'Detected: Nodes 2.0 (1 ComfyUI window).');
|
||||
assert.equal(rendererText([host('nodes2'), host('classic')]), 'Detected: Classic and Nodes 2.0 (2 ComfyUI windows).');
|
||||
});
|
||||
|
||||
test('the import review shows replacement, blocked items and network effects (D107)', async () => {
|
||||
const { importGrid } = await integrations;
|
||||
const plan = { lora_files: 3, with_metadata: 2, current_imports: 5, blocked_by_pairs: 0, planned_downloads: 4,
|
||||
scope: { count: 2, label: 'Replace 5 imported prompts with 2 LoRAs' }, protected: { count: 0, reasons: [] },
|
||||
backup: { policy: 'on', destination_label: 'prompts.db.backup_<date and time>' }, network: { planned_downloads: 4, api_key: true },
|
||||
media: { deletes_files: false, label: 'No files are removed; image links are replaced' } };
|
||||
assert.deepEqual(importGrid(plan), [
|
||||
['Scope', 'Replace 5 imported prompts with 2 LoRAs'],
|
||||
['Found', '3 metadata files · 2 readable'],
|
||||
['Protected', 'Nothing is protected'],
|
||||
['Network', 'Downloads up to 4 example images from CivitAI with your saved key'],
|
||||
['Backup', 'Create and verify before changes (prompts.db.backup_<date and time>)'],
|
||||
['Media on disk', 'No files are removed; image links are replaced'],
|
||||
]);
|
||||
const blocked = importGrid({ ...plan, planned_downloads: 1, network: { api_key: false }, protected: { count: 2,
|
||||
reasons: ['2 imported prompts used by saved pairs'] } });
|
||||
assert.deepEqual(blocked[2], ['Protected', '2 imported prompts used by saved pairs']);
|
||||
assert.deepEqual(blocked[3], ['Network', 'Downloads up to 1 example image from CivitAI']);
|
||||
assert.deepEqual(importGrid({})[0], ['Scope', '0 LoRAs']);
|
||||
assert.deepEqual(importGrid({})[3], ['Network', 'No downloads']);
|
||||
assert.deepEqual(importGrid({})[5], ['Media on disk', 'No files are removed']);
|
||||
});
|
||||
|
||||
test('a LoRA import job reads in Activity: tiles, outcomes and the kept collection on failure', async () => {
|
||||
const { kindOf, finishToast, outcomeLine, viewOf } = await kinds;
|
||||
const view = kindOf('lora_import');
|
||||
assert.equal(view.pillVerb, 'Importing LoRAs');
|
||||
assert.deepEqual(view.tiles.map(([group]) => group), ['imported', 'linked', 'skipped', 'downloaded', 'download_errors', 'errors']);
|
||||
const counters = { imported: 3, linked: 1, skipped: 0, downloaded: 2, download_errors: 0, errors: 0 };
|
||||
const job = { kind: 'lora_import', title: 'LoRA reference import', state: 'finished', cancellable: false, counters };
|
||||
assert.equal(outcomeLine(job), '3 imported · 0 errors');
|
||||
assert.equal(finishToast(job), 'LoRA import finished · 3 imported · 0 errors');
|
||||
assert.equal(finishToast({ ...job, state: 'cancelled' }), 'LoRA import cancelled · 3 imported');
|
||||
const failed = { ...job, state: 'failed', status_line: 'The import failed; the previous LoRA imports were kept.' };
|
||||
assert.equal(finishToast(failed), 'LoRA import failed · The import failed; the previous LoRA imports were kept.');
|
||||
assert.equal(finishToast({ ...failed, status_line: '' }), 'LoRA import failed · 3 imported · 0 errors');
|
||||
assert.equal(viewOf(job).tiles.length, 6);
|
||||
// The legacy page's held import (no counters) reads as the generic job.
|
||||
assert.equal(viewOf({ kind: 'lora_import', title: 'LoRA import (admin page)', cancellable: false, counters: {} }).tiles.length, 0);
|
||||
});
|
||||
|
||||
test('a failed import the files or saved pairs overtook is reviewed again in Integrations', async () => {
|
||||
const { reviewAgainOf } = await import('../../web/js/workspace/activity-job-view.mjs');
|
||||
const failed = failure => ({ kind: 'lora_import', state: 'failed', scope: { operation: 'lora_import', failure } });
|
||||
for (const failure of ['plan-changed', 'storage-changed', 'plan-expired', 'blocked-by-pairs']) {
|
||||
assert.deepEqual(reviewAgainOf(failed(failure)), { section: '#/system/integrations' }, failure);
|
||||
}
|
||||
assert.equal(reviewAgainOf(failed('import-failed')), null);
|
||||
assert.equal(reviewAgainOf({ ...failed('plan-changed'), state: 'cancelled' }), null);
|
||||
});
|
||||
|
||||
test('a host save in two parts reports exactly which part saved (S10 review I3)', async () => {
|
||||
const { writeHostInParts } = await integrations;
|
||||
const saved = { lora_enabled: true, lora_trigger_words_enabled: false, lora_path: 'custom_nodes/LM' };
|
||||
const calls = [];
|
||||
const ok = revision => async body => { calls.push(body); return { success: true, persisted: true, revision }; };
|
||||
const refuse = (code, message) => async body => { calls.push(body); throw Object.assign(new Error(message), { code }); };
|
||||
const run = (changes, postLora, postSettings) => writeHostInParts({ changes, saved, expected_revision: 'r0', postLora, postSettings });
|
||||
|
||||
// Both parts: the second carries the revision the first returned.
|
||||
assert.equal((await run({ lora_trigger_words_enabled: true, result_timeout: 5 }, ok('r1'), ok('r2'))).revision, 'r2');
|
||||
assert.deepEqual(calls.map(body => body.expected_revision), ['r0', 'r1']);
|
||||
// LoRA saved, settings refused: the LoRA keys and the new revision ride on the settings error.
|
||||
calls.length = 0;
|
||||
const later = await run({ lora_trigger_words_enabled: true, gallery_root_paths: ['x'] }, ok('r1'),
|
||||
refuse('invalid-source', 'x cannot be used')).catch(error => error);
|
||||
assert.equal(later.code, 'invalid-source');
|
||||
assert.deepEqual(later.partial, { saved: ['lora_trigger_words_enabled'], revision: 'r1' });
|
||||
// LoRA refused (not detected), settings still sent with the unchanged revision and saved.
|
||||
calls.length = 0;
|
||||
const first = await run({ lora_enabled: true, result_timeout: 5 }, refuse(null, 'LoraManager not found'), ok('r2'))
|
||||
.catch(error => error);
|
||||
assert.equal(first.message, 'LoraManager not found');
|
||||
assert.deepEqual(first.partial, { saved: ['result_timeout'], revision: 'r2' });
|
||||
assert.deepEqual(calls.map(body => body.expected_revision), ['r0', 'r0']);
|
||||
// A conflict on the first write stops there; a lone part fails as itself.
|
||||
calls.length = 0;
|
||||
const conflict = await run({ lora_enabled: false, result_timeout: 5 }, refuse('settings-changed', 'changed'), ok('r2'))
|
||||
.catch(error => error);
|
||||
assert.equal(conflict.code, 'settings-changed');
|
||||
assert.equal(conflict.partial, undefined);
|
||||
assert.equal(calls.length, 1);
|
||||
const alone = await run({ result_timeout: 5 }, ok('r1'), refuse('busy', 'scan running')).catch(error => error);
|
||||
assert.equal(alone.code, 'busy');
|
||||
assert.equal(alone.partial, undefined);
|
||||
const both = await run({ lora_enabled: true, result_timeout: 5 }, refuse(null, 'a'), refuse(null, 'b')).catch(error => error);
|
||||
assert.equal(both.message, 'b');
|
||||
assert.equal(both.partial, undefined);
|
||||
});
|
||||
@@ -190,6 +190,7 @@ test('status text per outcome', async () => {
|
||||
host: 'Saved to the server configuration.',
|
||||
runtime: 'Applied until the server restarts.',
|
||||
partial: 'Some changes were not saved. Review the highlighted settings.',
|
||||
partialSaved: 'Some changes were saved to the server configuration; the highlighted ones were not.',
|
||||
tabOnly: 'Saved for this tab only; browser storage is unavailable.',
|
||||
conflict: 'Settings changed elsewhere. Reload this section to see them.',
|
||||
restart: 'Restart ComfyUI to apply:',
|
||||
@@ -364,3 +365,42 @@ test('a host save refused for one source says the server reason and names it unt
|
||||
assert.equal(drafts.invalidSource(), null);
|
||||
assert.equal(drafts.statusText(), 'Unsaved changes.');
|
||||
});
|
||||
|
||||
test('a partial host save keeps what saved, adopts its revision and fails only the rest (S10 review I3)', async () => {
|
||||
const { SETTINGS_STATUS } = await source;
|
||||
const domains = adapters({ host: { lora_trigger_words_enabled: false, gallery_root_paths: ['a'] } });
|
||||
const drafts = await loaded(domains);
|
||||
// The LoRA part is written (config.json moves to sha256:lora), then /settings refuses one source.
|
||||
domains.outcome.host = () => {
|
||||
throw Object.assign(new Error('missing can\'t be used: not found. Remove it or reconnect it, then save.'), {
|
||||
code: 'invalid-source', source: 'missing', partial: { saved: ['lora_trigger_words_enabled'], revision: 'sha256:lora' } });
|
||||
};
|
||||
drafts.set('host', 'lora_trigger_words_enabled', true);
|
||||
drafts.set('host', 'gallery_root_paths', ['a', 'missing']);
|
||||
const first = await drafts.save();
|
||||
assert.equal(first.host, 'partial');
|
||||
assert.deepEqual([...drafts.failed('host')], ['gallery_root_paths']);
|
||||
assert.equal(drafts.saved('host').lora_trigger_words_enabled, true);
|
||||
assert.deepEqual(drafts.changes('host'), { gallery_root_paths: ['a', 'missing'] });
|
||||
assert.equal(drafts.invalidSource(), 'missing');
|
||||
assert.equal(drafts.statusText(), 'missing can\'t be used: not found. Remove it or reconnect it, then save.');
|
||||
// The next save carries the revision the saved part produced: no false "changed elsewhere".
|
||||
domains.outcome.host = null;
|
||||
drafts.set('host', 'gallery_root_paths', ['a']);
|
||||
const second = await drafts.save();
|
||||
assert.equal(second.host, 'saved');
|
||||
assert.equal(domains.calls.at(-1)[2].expected_revision, 'sha256:lora');
|
||||
assert.deepEqual(domains.calls.at(-1)[1], { gallery_root_paths: ['a'] });
|
||||
// Without a server reason, the status says some were saved and the highlighted ones were not.
|
||||
domains.outcome.host = () => { throw Object.assign(new Error('boom'), { partial: { saved: ['x'], revision: 'sha256:three' } }); };
|
||||
drafts.set('host', 'x', 1);
|
||||
drafts.set('host', 'y', 2);
|
||||
assert.equal((await drafts.save()).host, 'partial');
|
||||
assert.equal(drafts.statusText(), SETTINGS_STATUS.partialSaved);
|
||||
assert.deepEqual([...drafts.failed('host')], ['y']);
|
||||
// A partial result that saved nothing is a plain failure.
|
||||
domains.outcome.host = () => { throw Object.assign(new Error('boom'), { partial: { saved: [] } }); };
|
||||
drafts.set('host', 'z', 3);
|
||||
assert.equal((await drafts.save()).host, 'failed');
|
||||
assert.equal(drafts.statusText(), SETTINGS_STATUS.partial);
|
||||
});
|
||||
|
||||
+64
-4
@@ -36,7 +36,7 @@ from aiohttp.test_utils import AioHTTPTestCase
|
||||
|
||||
from database.operations import PromptDatabase
|
||||
from py.api import PromptManagerAPI
|
||||
from py import config_store, lora_utils
|
||||
from py import config_store, lora_import, lora_utils
|
||||
from py.config import IntegrationConfig, PromptManagerConfig
|
||||
from py.api import settings_routes as settings_module
|
||||
from py.api import lora_integration as lora_module
|
||||
@@ -252,6 +252,21 @@ class TestLoraStatusRoute(LoraAPITestCase):
|
||||
self.assertFalse(os.path.isabs(public))
|
||||
self.assertNotIn(str(self.tmp_root), body)
|
||||
self.assertFalse(data["trigger_cache_loaded"])
|
||||
self.assertTrue(data["detected_usable"])
|
||||
|
||||
async def test_status_says_a_detected_install_outside_custom_nodes_is_unusable(
|
||||
self,
|
||||
):
|
||||
# S10 review N5: found (a linked install), but enabling it is refused.
|
||||
outside = _make_lora_manager(self.tmp_root)
|
||||
self.detect_mock.return_value = str(outside)
|
||||
resp = await self.client.request("GET", "/prompt_manager/lora/status")
|
||||
data = await resp.json()
|
||||
self.assertEqual((data["detected"], data["detected_usable"]), (True, False))
|
||||
|
||||
self.detect_mock.return_value = None
|
||||
resp = await self.client.request("GET", "/prompt_manager/lora/status")
|
||||
self.assertFalse((await resp.json())["detected_usable"])
|
||||
|
||||
async def test_status_error_is_500(self):
|
||||
with patch("py.lora_utils.get_trigger_cache", side_effect=RuntimeError("x")):
|
||||
@@ -528,6 +543,48 @@ class TestLoraEnableRoute(LoraAPITestCase):
|
||||
self.assertEqual(self.config_file.read_bytes(), b"{not json")
|
||||
self.assertFalse(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED)
|
||||
|
||||
async def test_enable_with_a_stale_revision_changes_nothing(self):
|
||||
# System S10: the Save row sends the revision it read; a file changed
|
||||
# since is a settings-changed conflict and nothing is written.
|
||||
self.config_file.write_text(json.dumps({"web_ui": {"result_timeout": 9}}))
|
||||
resp = await self.client.request("GET", "/prompt_manager/settings")
|
||||
revision = (await resp.json())["revision"]
|
||||
self.config_file.write_text(json.dumps({"web_ui": {"result_timeout": 10}}))
|
||||
before = self.config_file.read_bytes()
|
||||
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{
|
||||
"enabled": False,
|
||||
"trigger_words_enabled": True,
|
||||
"expected_revision": revision,
|
||||
},
|
||||
)
|
||||
body = await resp.json()
|
||||
|
||||
self.assertEqual((resp.status, body["code"]), (409, "settings-changed"))
|
||||
self.assertEqual(self.config_file.read_bytes(), before)
|
||||
self.assertFalse(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED)
|
||||
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{
|
||||
"enabled": False,
|
||||
"trigger_words_enabled": True,
|
||||
"expected_revision": body["revision"],
|
||||
},
|
||||
)
|
||||
saved = await resp.json()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertTrue(saved["revision"].startswith("sha256:"))
|
||||
self.assertTrue(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED)
|
||||
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": False, "expected_revision": 7},
|
||||
)
|
||||
self.assertEqual(resp.status, 400)
|
||||
|
||||
async def test_enable_requires_boolean_flags_and_body_cap(self):
|
||||
for body in (
|
||||
{"enabled": "yes"},
|
||||
@@ -762,9 +819,9 @@ class TestLoraScanRoute(LoraAPITestCase):
|
||||
async def test_scan_error_hides_the_server_path(self):
|
||||
self._enable()
|
||||
err = PermissionError(13, "Permission denied", str(self.tmp_root / "x.db"))
|
||||
with patch.object(self.api.db, "delete_prompts_by_category", side_effect=err):
|
||||
with patch.object(self.api.db, "replace_category_prompts", side_effect=err):
|
||||
resp = await self.client.request("POST", "/prompt_manager/lora/scan")
|
||||
body = await resp.text()
|
||||
body = await resp.text() # the stream ends inside the patch
|
||||
self.assertNotIn(str(self.tmp_root), body)
|
||||
self.assertIn("x.db", body)
|
||||
# The admin page's modal shows the failure as its status and closes.
|
||||
@@ -862,7 +919,10 @@ class TestLoraScanRoute(LoraAPITestCase):
|
||||
loras.mkdir()
|
||||
_write_metadata(loras, "neon", words=["glow"])
|
||||
|
||||
with patch.object(self.api.db, "save_prompt", side_effect=RuntimeError("db")):
|
||||
# Preparing one LoRA's record fails: it is skipped, the import goes on.
|
||||
with patch.object(
|
||||
lora_import, "build_record", side_effect=RuntimeError("bad record")
|
||||
):
|
||||
resp, events, _ = await self._scan(loras)
|
||||
|
||||
self.assertEqual(events[-1]["imported"], 0)
|
||||
|
||||
@@ -0,0 +1,912 @@
|
||||
"""The staged LoRA reference import (#221, #249 System S10).
|
||||
|
||||
A failed import must leave the previous ``lora-manager`` collection exactly as
|
||||
it was (prompts, tags and image links); a preview writes nothing and requests
|
||||
nothing; an apply does what was reviewed or nothing. Discovery, metadata and
|
||||
CivitAI downloads are disposable fixtures: no network, no LoraManager install.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
try:
|
||||
from tests.test_api_lora import (
|
||||
SECRET_KEY,
|
||||
LoraAPITestCase,
|
||||
_FakeResponse,
|
||||
_make_lora_manager,
|
||||
_png_bytes,
|
||||
_write_metadata,
|
||||
)
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_api_lora import (
|
||||
SECRET_KEY,
|
||||
LoraAPITestCase,
|
||||
_FakeResponse,
|
||||
_make_lora_manager,
|
||||
_png_bytes,
|
||||
_write_metadata,
|
||||
)
|
||||
|
||||
from database.category_import import EmptyReplacement
|
||||
from database.maintenance import PlanChanged
|
||||
from database.operations import PairDependencyError, PromptDatabase
|
||||
from py import lora_import, lora_utils
|
||||
from py.api.jobs import RESTORE_JOB_KIND
|
||||
from py.api.lora_import_routes import LORA_PREVIEW_LEASE
|
||||
from py.config import IntegrationConfig, PromptManagerConfig
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
CATEGORY = "lora-manager"
|
||||
PREVIEW = "/prompt_manager/lora/import/preview"
|
||||
APPLY = "/prompt_manager/lora/import/apply"
|
||||
SCAN = "/prompt_manager/lora/scan"
|
||||
|
||||
|
||||
def _snapshot(db):
|
||||
"""Every row of the tables an import touches, in a fixed order."""
|
||||
tables = {
|
||||
"prompts": "SELECT id, text, category, hash FROM prompts ORDER BY id",
|
||||
"prompt_tags": "SELECT prompt_id, tag_id FROM prompt_tags ORDER BY 1, 2",
|
||||
"tags": "SELECT id, name FROM tags ORDER BY id",
|
||||
"generated_images": (
|
||||
"SELECT id, prompt_id, image_path, file_path FROM generated_images "
|
||||
"ORDER BY id"
|
||||
),
|
||||
}
|
||||
with db.model.get_connection() as conn:
|
||||
return {
|
||||
name: [tuple(row) for row in conn.execute(sql).fetchall()]
|
||||
for name, sql in tables.items()
|
||||
}
|
||||
|
||||
|
||||
def _seed_previous_imports(db):
|
||||
"""Two earlier imports with tags and image links, and one other prompt."""
|
||||
ids = []
|
||||
for name in ("old-one", "old-two"):
|
||||
prompt_id = db.save_prompt(
|
||||
text=f"{name} prompt",
|
||||
category=CATEGORY,
|
||||
tags=[CATEGORY, f"lora:{name}"],
|
||||
prompt_hash=generate_prompt_hash(f"{name} prompt"),
|
||||
)
|
||||
db.link_image_to_prompt(str(prompt_id), f"/cache/{name}/a.jpg")
|
||||
db.link_image_to_prompt(str(prompt_id), f"/cache/{name}/b.jpg")
|
||||
ids.append(prompt_id)
|
||||
db.save_prompt(
|
||||
text="mine",
|
||||
category="portraits",
|
||||
tags=["keep"],
|
||||
prompt_hash=generate_prompt_hash("mine"),
|
||||
)
|
||||
return ids
|
||||
|
||||
|
||||
def _record(text, images=(), name=None):
|
||||
return {
|
||||
"name": name or text,
|
||||
"text": text,
|
||||
"hash": generate_prompt_hash(text),
|
||||
"tags": [CATEGORY, f"lora:{name or text}"],
|
||||
"images": list(images),
|
||||
}
|
||||
|
||||
|
||||
class CategoryReplacementTests(unittest.TestCase):
|
||||
"""``replace_category_prompts``: one transaction, all or nothing."""
|
||||
|
||||
def setUp(self):
|
||||
handle = tempfile.NamedTemporaryFile(delete=False, suffix=".db")
|
||||
handle.close()
|
||||
self.path = handle.name
|
||||
self.db = PromptDatabase(self.path)
|
||||
self.ids = _seed_previous_imports(self.db)
|
||||
|
||||
def tearDown(self):
|
||||
self.db.close_all()
|
||||
for suffix in ("", "-wal", "-shm"):
|
||||
if os.path.exists(self.path + suffix):
|
||||
os.unlink(self.path + suffix)
|
||||
|
||||
def test_transaction_failure_rolls_back_everything(self):
|
||||
before = _snapshot(self.db)
|
||||
records = [_record("new one", ["/c/n1.jpg"]), _record("new two", ["/c/n2.jpg"])]
|
||||
real = self.db._insert_image_link
|
||||
calls = []
|
||||
|
||||
def failing(conn, prompt_id, image, metadata=None):
|
||||
calls.append(image)
|
||||
if len(calls) == 2: # after the old rows went and one record landed
|
||||
raise sqlite3.OperationalError("disk I/O error")
|
||||
return real(conn, prompt_id, image, metadata)
|
||||
|
||||
with patch.object(self.db, "_insert_image_link", side_effect=failing):
|
||||
with self.assertRaises(sqlite3.OperationalError):
|
||||
self.db.replace_category_prompts(CATEGORY, records)
|
||||
|
||||
self.assertEqual(len(calls), 2)
|
||||
self.assertEqual(_snapshot(self.db), before)
|
||||
|
||||
def test_pair_dependency_blocks_without_partial_delete(self):
|
||||
# A saved pair reuses an imported prompt as its positive component.
|
||||
self.db.save_prompt_pair("old-two prompt", "blurry")
|
||||
before = _snapshot(self.db)
|
||||
self.assertEqual(
|
||||
self.db.category_import_state(CATEGORY), {"ids": self.ids, "paired": 1}
|
||||
)
|
||||
|
||||
with self.assertRaises(PairDependencyError):
|
||||
self.db.replace_category_prompts(CATEGORY, [_record("new one")])
|
||||
|
||||
self.assertEqual(_snapshot(self.db), before)
|
||||
|
||||
def test_explicit_empty_replacement_requires_review_flag(self):
|
||||
before = _snapshot(self.db)
|
||||
with self.assertRaises(EmptyReplacement):
|
||||
self.db.replace_category_prompts(CATEGORY, [])
|
||||
self.assertEqual(_snapshot(self.db), before)
|
||||
|
||||
result = self.db.replace_category_prompts(CATEGORY, [], allow_empty=True)
|
||||
|
||||
self.assertEqual(result["removed"], 2)
|
||||
self.assertEqual(self.db.category_import_state(CATEGORY)["ids"], [])
|
||||
self.assertEqual(len(self.db.search_prompts(category="portraits")), 1)
|
||||
|
||||
def test_changed_imports_since_review_are_plan_changed(self):
|
||||
before = _snapshot(self.db)
|
||||
with self.assertRaises(PlanChanged):
|
||||
self.db.replace_category_prompts(
|
||||
CATEGORY, [_record("new one")], expected_ids=self.ids[:1]
|
||||
)
|
||||
self.assertEqual(_snapshot(self.db), before)
|
||||
|
||||
def test_replacement_links_repeats_and_existing_prompts(self):
|
||||
records = [
|
||||
_record("fresh", ["/c/f.jpg"]),
|
||||
_record("Fresh ", ["/c/f2.jpg"], name="fresh-again"), # same hash
|
||||
_record("mine", ["/c/m.jpg"], name="mine-lora"), # exists elsewhere
|
||||
]
|
||||
|
||||
result = self.db.replace_category_prompts(
|
||||
CATEGORY, records, expected_ids=self.ids
|
||||
)
|
||||
|
||||
self.assertEqual(
|
||||
(result["removed"], result["imported"], result["linked"]), (2, 1, 1)
|
||||
)
|
||||
self.assertEqual(result["duplicates"], 1)
|
||||
self.assertEqual(result["images"], 3)
|
||||
imported = self.db.search_prompts(category=CATEGORY)
|
||||
self.assertEqual([p["text"] for p in imported], ["fresh"])
|
||||
self.assertEqual(len(self.db.get_prompt_images(imported[0]["id"])), 2)
|
||||
mine = self.db.search_prompts(category="portraits")[0]
|
||||
self.assertEqual(len(self.db.get_prompt_images(mine["id"])), 1)
|
||||
self.assertEqual(
|
||||
[row["outcome"] for row in result["rows"]],
|
||||
["imported", "duplicate", "linked"],
|
||||
)
|
||||
|
||||
# N1: a collection larger than SQLite's bound-variable limit is replaced.
|
||||
def test_more_imports_than_sqlite_variables(self):
|
||||
conn = self.db.model.get_connection()
|
||||
if not hasattr(conn, "setlimit"):
|
||||
self.skipTest("sqlite3.Connection.setlimit needs Python 3.11")
|
||||
rows = [
|
||||
(f"bulk lora {n}", CATEGORY, generate_prompt_hash(f"bulk lora {n}"))
|
||||
for n in range(1200)
|
||||
]
|
||||
with conn:
|
||||
conn.executemany(
|
||||
"INSERT INTO prompts(text, category, hash, created_at, updated_at) "
|
||||
"VALUES(?, ?, ?, '2026-01-01', '2026-01-01')",
|
||||
rows,
|
||||
)
|
||||
conn.setlimit(sqlite3.SQLITE_LIMIT_VARIABLE_NUMBER, 999)
|
||||
state = self.db.category_import_state(CATEGORY)
|
||||
self.assertEqual((len(state["ids"]), state["paired"]), (1202, 0))
|
||||
|
||||
result = self.db.replace_category_prompts(
|
||||
CATEGORY, [_record("only one")], expected_ids=state["ids"]
|
||||
)
|
||||
|
||||
self.assertEqual((result["removed"], result["imported"]), (1202, 1))
|
||||
|
||||
|
||||
class LoraImportAPITestCase(LoraAPITestCase):
|
||||
"""The routes with LoraManager enabled over a disposable LoRA folder."""
|
||||
|
||||
async def get_application(self):
|
||||
app = await super().get_application()
|
||||
self.lm = _make_lora_manager(self.custom_nodes)
|
||||
self.loras = self.tmp_root / "loras"
|
||||
self.loras.mkdir()
|
||||
self.cache_dir = self.tmp_root / "img_cache"
|
||||
IntegrationConfig.LORA_MANAGER_ENABLED = True
|
||||
IntegrationConfig.LORA_MANAGER_PATH = str(self.lm)
|
||||
IntegrationConfig.CIVITAI_API_KEY = SECRET_KEY
|
||||
self.requests = []
|
||||
self.gate = None # a threading.Event a download waits on
|
||||
self.downloading = threading.Event()
|
||||
|
||||
def fake_open(req, timeout=None):
|
||||
self.requests.append(req)
|
||||
self.downloading.set()
|
||||
if self.gate is not None:
|
||||
self.gate.wait(10)
|
||||
return _FakeResponse(_png_bytes())
|
||||
|
||||
self._patches = [
|
||||
patch.object(
|
||||
lora_utils, "find_lora_directories", return_value=[str(self.loras)]
|
||||
),
|
||||
patch.object(
|
||||
lora_utils, "get_lora_image_cache_dir", return_value=self.cache_dir
|
||||
),
|
||||
patch.object(lora_utils, "_open_url", fake_open),
|
||||
patch.object(PromptManagerConfig, "SAFETY_BACKUP", False),
|
||||
]
|
||||
for item in self._patches:
|
||||
item.start()
|
||||
self.old_ids = _seed_previous_imports(self.api.db)
|
||||
return app
|
||||
|
||||
async def tearDownAsync(self):
|
||||
if self.gate is not None:
|
||||
self.gate.set()
|
||||
for job in self.api._jobs.list():
|
||||
if job.task is not None and not job.task.done():
|
||||
await job.task
|
||||
for item in reversed(self._patches):
|
||||
item.stop()
|
||||
await super().tearDownAsync()
|
||||
|
||||
def _two_loras(self):
|
||||
_write_metadata(
|
||||
self.loras,
|
||||
"neon",
|
||||
words=["glow"],
|
||||
images=["https://civitai.com/a.png"],
|
||||
prompt="neon street",
|
||||
)
|
||||
(self.loras / "neon.png").write_bytes(_png_bytes())
|
||||
_write_metadata(self.loras, "plain")
|
||||
|
||||
async def _post(self, url, body=None):
|
||||
resp = await self._post_json(url, {} if body is None else body)
|
||||
return resp, json.loads(await resp.text())
|
||||
|
||||
async def _apply_reviewed(self):
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
self.assertEqual(resp.status, 200, plan)
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual(resp.status, 202, body)
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
return plan, job
|
||||
|
||||
@staticmethod
|
||||
def _sse(text):
|
||||
return [
|
||||
json.loads(line[len("data: ") :])
|
||||
for line in text.splitlines()
|
||||
if line.startswith("data: ")
|
||||
]
|
||||
|
||||
|
||||
class LoraImportRouteTests(LoraImportAPITestCase):
|
||||
|
||||
async def test_preview_writes_nothing_and_makes_no_requests(self):
|
||||
self._two_loras()
|
||||
(self.loras / "broken.metadata.json").write_text("{{{")
|
||||
db_path = self.api.db.model.db_path
|
||||
before = {s: _read(db_path + s) for s in ("", "-wal")}
|
||||
shm = _size(db_path + "-shm")
|
||||
rows = _snapshot(self.api.db)
|
||||
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
|
||||
self.assertEqual(resp.status, 200, plan)
|
||||
self.assertEqual(
|
||||
{k: plan[k] for k in ("lora_files", "with_metadata", "current_imports")},
|
||||
{"lora_files": 3, "with_metadata": 2, "current_imports": 2},
|
||||
)
|
||||
self.assertEqual(plan["blocked_by_pairs"], 0)
|
||||
self.assertEqual(plan["planned_downloads"], 1)
|
||||
self.assertEqual(plan["network"], {"planned_downloads": 1, "api_key": True})
|
||||
self.assertTrue(plan["token"])
|
||||
self.assertEqual(plan["scope"]["count"], 2)
|
||||
self.assertFalse(plan["media"]["deletes_files"])
|
||||
self.assertEqual(self.requests, [])
|
||||
self.assertFalse(self.cache_dir.exists())
|
||||
self.assertEqual({s: _read(db_path + s) for s in ("", "-wal")}, before)
|
||||
self.assertEqual(_size(db_path + "-shm"), shm)
|
||||
self.assertEqual(_snapshot(self.api.db), rows)
|
||||
self.assertNotIn(SECRET_KEY, json.dumps(plan))
|
||||
self.assertNotIn(str(self.tmp_root), json.dumps(plan))
|
||||
|
||||
async def test_preview_requires_the_integration(self):
|
||||
IntegrationConfig.LORA_MANAGER_ENABLED = False
|
||||
resp, body = await self._post(PREVIEW)
|
||||
self.assertEqual((resp.status, body["code"]), (400, "lora-disabled"))
|
||||
|
||||
async def test_blocked_preview_offers_no_token(self):
|
||||
self._two_loras()
|
||||
self.api.db.save_prompt_pair("old-one prompt", "blurry")
|
||||
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(plan["blocked_by_pairs"], 1)
|
||||
self.assertIsNone(plan["token"])
|
||||
self.assertEqual(plan["protected"]["count"], 1)
|
||||
|
||||
async def test_changed_source_after_preview_is_409(self):
|
||||
self._two_loras()
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
rows = _snapshot(self.api.db)
|
||||
meta = self.loras / "plain.safetensors.metadata.json"
|
||||
meta.write_text(meta.read_text() + " ")
|
||||
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
|
||||
self.assertEqual((resp.status, body["code"]), (409, "plan-changed"))
|
||||
self.assertEqual(self.api._jobs.list(), [])
|
||||
self.assertEqual(self.requests, [])
|
||||
self.assertEqual(_snapshot(self.api.db), rows)
|
||||
|
||||
async def test_apply_imports_through_the_job(self):
|
||||
self._two_loras()
|
||||
|
||||
plan, job = await self._apply_reviewed()
|
||||
self.assertEqual(
|
||||
(job.kind, job.title, job.admission),
|
||||
("lora_import", plan_title(), "records"),
|
||||
)
|
||||
await job.task
|
||||
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(
|
||||
{k: job.counters[k] for k in lora_import.GROUPS},
|
||||
{
|
||||
"imported": 2,
|
||||
"linked": 0,
|
||||
"skipped": 0,
|
||||
"downloaded": 1,
|
||||
"download_errors": 0,
|
||||
"errors": 0,
|
||||
},
|
||||
)
|
||||
self.assertFalse(job.cancellable)
|
||||
self.assertEqual(
|
||||
self.requests[0].get_header("Authorization"), f"Bearer {SECRET_KEY}"
|
||||
)
|
||||
imported = self.api.db.search_prompts(category=CATEGORY)
|
||||
self.assertEqual(sorted(p["text"] for p in imported), ["neon street", "plain"])
|
||||
neon = next(p for p in imported if p["text"] == "neon street")
|
||||
self.assertEqual(len(self.api.db.get_prompt_images(neon["id"])), 2)
|
||||
self.assertEqual(len(self.api.db.search_prompts(category="portraits")), 1)
|
||||
# The token is single-use.
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual((resp.status, body["code"]), (409, "plan-used"))
|
||||
|
||||
async def test_discovery_failure_keeps_previous_imports(self):
|
||||
self._two_loras()
|
||||
before = _snapshot(self.api.db)
|
||||
|
||||
# Legacy scan: discovery raises once the stream is open.
|
||||
with patch.object(
|
||||
lora_utils, "find_lora_directories", side_effect=OSError("share gone")
|
||||
):
|
||||
resp = await self.client.request("POST", SCAN)
|
||||
events = self._sse(await resp.text())
|
||||
self.assertEqual(events[-1]["type"], "error")
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
# Job: staging fails after the review.
|
||||
with patch.object(lora_import, "stage", side_effect=OSError("share gone")):
|
||||
_plan, job = await self._apply_reviewed()
|
||||
await job.task
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertIn("previous LoRA imports were kept", job.status_line)
|
||||
self.assertEqual(job.scope["failure"], "import-failed")
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def test_a_failed_replacement_keeps_previous_imports_and_says_so(self):
|
||||
self._two_loras()
|
||||
before = _snapshot(self.api.db)
|
||||
with patch.object(
|
||||
self.api.db,
|
||||
"replace_category_prompts",
|
||||
side_effect=sqlite3.OperationalError("database is locked"),
|
||||
):
|
||||
_plan, job = await self._apply_reviewed()
|
||||
await job.task
|
||||
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertEqual(job.counters["errors"], 1)
|
||||
self.assertEqual(job.counters["downloaded"], 1)
|
||||
self.assertIn("previous LoRA imports were kept", job.status_line)
|
||||
self.assertIn("stay in the image cache", job.status_line)
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def test_cancel_during_downloads_writes_nothing(self):
|
||||
self._two_loras()
|
||||
before = _snapshot(self.api.db)
|
||||
self.gate = threading.Event()
|
||||
|
||||
_plan, job = await self._apply_reviewed()
|
||||
loop = asyncio.get_running_loop()
|
||||
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
|
||||
self.assertTrue(job.summary()["cancellable"])
|
||||
self.assertTrue(self.api._jobs.cancel(job.id))
|
||||
self.gate.set()
|
||||
await job.task
|
||||
|
||||
self.assertEqual(job.state, "cancelled")
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
self.assertTrue(any("stay in the image cache" in line for line in job.log))
|
||||
|
||||
async def test_empty_reviewed_import_removes_the_collection(self):
|
||||
# No metadata at all: the review says so, and its apply is the flag.
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
self.assertEqual((plan["with_metadata"], plan["current_imports"]), (0, 2))
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
await job.task
|
||||
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(self.api.db.search_prompts(category=CATEGORY), [])
|
||||
|
||||
# The legacy scan has no review: an empty replacement is refused.
|
||||
_seed_previous_imports(self.api.db)
|
||||
before = _snapshot(self.api.db)
|
||||
resp = await self.client.request("POST", SCAN)
|
||||
events = self._sse(await resp.text())
|
||||
self.assertEqual(events[-1]["type"], "error")
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def test_apply_refused_while_a_records_job_holds(self):
|
||||
self._two_loras()
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
async with self.api._jobs.hold("scan", "Metadata scan", admission="records"):
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual((resp.status, body["code"]), (409, "busy"))
|
||||
# The token was not used: it still applies.
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual(resp.status, 202)
|
||||
await self.api._jobs.get(body["job"]["id"]).task
|
||||
|
||||
async def test_safety_backup_policy_runs_before_the_replace(self):
|
||||
self._two_loras()
|
||||
order = []
|
||||
real = self.api.db.replace_category_prompts
|
||||
|
||||
async def step(job, db, backup_on):
|
||||
order.append(("backup", backup_on))
|
||||
return True, "prompts.db.backup_x"
|
||||
|
||||
def replace(*args, **kwargs):
|
||||
order.append(("replace", None))
|
||||
return real(*args, **kwargs)
|
||||
|
||||
with (
|
||||
patch.object(PromptManagerConfig, "SAFETY_BACKUP", True),
|
||||
patch.object(self.api, "_safety_backup_step", side_effect=step),
|
||||
patch.object(self.api.db, "replace_category_prompts", side_effect=replace),
|
||||
):
|
||||
plan, job = await self._apply_reviewed()
|
||||
await job.task
|
||||
self.assertEqual(plan["backup"]["policy"], "on")
|
||||
self.assertEqual(order, [("backup", True), ("replace", None)])
|
||||
|
||||
async def test_a_disconnected_legacy_scan_stops_and_holds_until_it_stopped(self):
|
||||
# I2: the admin page closes its modal while the downloads run.
|
||||
for number in range(4):
|
||||
_write_metadata(
|
||||
self.loras,
|
||||
f"lora{number}",
|
||||
images=[f"https://civitai.com/{number}.png"],
|
||||
)
|
||||
before = _snapshot(self.api.db)
|
||||
self.gate = threading.Event()
|
||||
resp = await self.client.request("POST", SCAN)
|
||||
await resp.content.readline() # the stream is open
|
||||
loop = asyncio.get_running_loop()
|
||||
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
|
||||
resp.close()
|
||||
await asyncio.sleep(0.05) # the server sees the closed connection
|
||||
# Its records hold stays while the in-flight download runs.
|
||||
self.assertIsNotNone(self.api._jobs.active("lora_import"))
|
||||
self.gate.set()
|
||||
for _ in range(500):
|
||||
if self.api._jobs.active("lora_import") is None:
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
self.assertIsNone(self.api._jobs.active("lora_import"))
|
||||
# R2 (re-review): the user left and nothing failed, so Activity says so.
|
||||
job = next(j for j in self.api._jobs.list() if j.kind == "lora_import")
|
||||
self.assertEqual(job.state, "cancelled")
|
||||
# Staging stopped at the next LoRA: no download runs after the hold.
|
||||
served = len(self.requests)
|
||||
self.assertLess(served, 4)
|
||||
await asyncio.sleep(0.1)
|
||||
self.assertEqual(len(self.requests), served)
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def test_an_import_sweeps_stale_temp_downloads_from_the_cache(self):
|
||||
# N7 (re-review): a download killed mid-write leaves a .pm-*.part file;
|
||||
# the next import removes it once it is an hour old.
|
||||
self._two_loras()
|
||||
stale = self.cache_dir / "old_lora" / ".pm-dead.part"
|
||||
stale.parent.mkdir(parents=True)
|
||||
stale.write_bytes(b"x")
|
||||
stamp = time.time() - 2 * 3600
|
||||
os.utime(stale, (stamp, stamp))
|
||||
|
||||
resp = await self.client.request("POST", SCAN)
|
||||
await resp.text()
|
||||
|
||||
self.assertFalse(stale.exists())
|
||||
|
||||
async def test_legacy_scan_events_unchanged_on_success(self):
|
||||
self._two_loras()
|
||||
(self.loras / "broken.metadata.json").write_text("{{{")
|
||||
|
||||
resp = await self.client.request("POST", SCAN)
|
||||
events = self._sse(await resp.text())
|
||||
|
||||
self.assertEqual(resp.status, 200)
|
||||
progress_keys = {
|
||||
"type",
|
||||
"status",
|
||||
"progress",
|
||||
"total",
|
||||
"processed",
|
||||
"imported",
|
||||
"skipped",
|
||||
}
|
||||
for event in events[:-1]:
|
||||
self.assertEqual(event["type"], "progress")
|
||||
self.assertLessEqual(set(event), progress_keys)
|
||||
self.assertIsInstance(event["status"], str)
|
||||
self.assertIsInstance(event["progress"], int)
|
||||
per_record = [e for e in events if "processed" in e]
|
||||
self.assertEqual([e["processed"] for e in per_record], [1, 2, 3])
|
||||
self.assertEqual(
|
||||
events[-1],
|
||||
{
|
||||
"type": "complete",
|
||||
"progress": 100,
|
||||
"total": 3,
|
||||
"imported": 2,
|
||||
"skipped": 1,
|
||||
},
|
||||
)
|
||||
imported = self.api.db.search_prompts(category=CATEGORY)
|
||||
self.assertEqual(sorted(p["text"] for p in imported), ["neon street", "plain"])
|
||||
|
||||
|
||||
class LoraImportRefusalTests(LoraImportAPITestCase):
|
||||
"""Every refusal leaves the previous imports as they were."""
|
||||
|
||||
async def test_preview_refusals(self):
|
||||
IntegrationConfig.LORA_MANAGER_PATH = ""
|
||||
resp, body = await self._post(PREVIEW)
|
||||
self.assertEqual((resp.status, body["code"]), (400, "lora-path"))
|
||||
|
||||
IntegrationConfig.LORA_MANAGER_PATH = str(_make_lora_manager(self.tmp_root))
|
||||
resp, body = await self._post(PREVIEW)
|
||||
self.assertEqual((resp.status, body["code"]), (400, "lora-path"))
|
||||
self.assertNotIn(str(self.tmp_root), json.dumps(body))
|
||||
|
||||
IntegrationConfig.LORA_MANAGER_PATH = str(self.lm)
|
||||
resp, body = await self._post(PREVIEW, {"extra": 1})
|
||||
self.assertEqual((resp.status, body["code"]), (400, "invalid-request"))
|
||||
|
||||
with patch.object(lora_import, "preview", side_effect=OSError("gone")):
|
||||
resp, body = await self._post(PREVIEW)
|
||||
self.assertEqual(resp.status, 500)
|
||||
self.assertFalse(body["success"])
|
||||
|
||||
generations = iter([(0, "a"), (1, "a")])
|
||||
with patch.object(
|
||||
self.api, "_database_identity", side_effect=lambda db: next(generations)
|
||||
):
|
||||
resp, body = await self._post(PREVIEW)
|
||||
self.assertEqual((resp.status, body["code"]), (409, "storage-changed"))
|
||||
|
||||
async def test_apply_refusals(self):
|
||||
self._two_loras()
|
||||
before = _snapshot(self.api.db)
|
||||
resp, body = await self._post(APPLY, {"token": ""})
|
||||
self.assertEqual((resp.status, body["code"]), (400, "invalid-request"))
|
||||
resp, body = await self._post(APPLY, {"token": "unknown"})
|
||||
self.assertEqual((resp.status, body["code"]), (410, "plan-expired"))
|
||||
|
||||
other = self.api._maintenance_plans.issue(
|
||||
"optimize", "f", {"backup": "off"}, self.api._database_identity(self.api.db)
|
||||
)
|
||||
resp, body = await self._post(APPLY, {"token": other})
|
||||
self.assertEqual((resp.status, body["code"]), (410, "plan-expired"))
|
||||
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
with patch.object(lora_import, "discover", side_effect=OSError("gone")):
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual((resp.status, body["code"]), (500, "apply-failed"))
|
||||
|
||||
short = ("database folder", "the safety backup", 10, 1)
|
||||
with patch("py.api.lora_import_routes.disk_plan", return_value=({}, short, 0)):
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual((resp.status, body["code"]), (507, "insufficient-space"))
|
||||
|
||||
self.api._db_generation += 1
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual((resp.status, body["code"]), (409, "storage-changed"))
|
||||
self.assertEqual(self.api._jobs.list(), [])
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def _failed_job(self, **patches):
|
||||
before = _snapshot(self.api.db)
|
||||
_plan, job = await self._apply_reviewed()
|
||||
await job.task
|
||||
self.assertEqual(job.state, "failed", job.log)
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
return job
|
||||
|
||||
async def test_start_refused_when_the_integration_changed_since_the_review(self):
|
||||
# M5: disabled, or moved to another install, after the review.
|
||||
self._two_loras()
|
||||
before = _snapshot(self.api.db)
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
IntegrationConfig.LORA_MANAGER_ENABLED = False
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual((resp.status, body["code"]), (409, "plan-changed"))
|
||||
self.assertIn("integration changed", body["error"])
|
||||
|
||||
IntegrationConfig.LORA_MANAGER_ENABLED = True
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
other = _make_lora_manager(self.custom_nodes / "other")
|
||||
target = self.custom_nodes / "Other-Lora-Manager"
|
||||
other.rename(target)
|
||||
IntegrationConfig.LORA_MANAGER_PATH = str(target)
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual((resp.status, body["code"]), (409, "plan-changed"))
|
||||
self.assertEqual(self.api._jobs.list(), [])
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def test_imports_changed_after_the_review_fail_the_job(self):
|
||||
self._two_loras()
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
self.api.db.save_prompt(
|
||||
text="late", category=CATEGORY, prompt_hash=generate_prompt_hash("late")
|
||||
)
|
||||
before = _snapshot(self.api.db)
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
await job.task
|
||||
self.assertEqual((job.state, job.scope["failure"]), ("failed", "plan-changed"))
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def test_a_pair_saved_after_the_review_blocks_the_job(self):
|
||||
self._two_loras()
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
self.api.db.save_prompt_pair("old-one prompt", "blurry")
|
||||
before = _snapshot(self.api.db)
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
await job.task
|
||||
self.assertEqual(
|
||||
(job.state, job.scope["failure"]), ("failed", "blocked-by-pairs")
|
||||
)
|
||||
self.assertIn("Saved pairs", job.status_line)
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def test_nothing_prepared_keeps_the_collection(self):
|
||||
self._two_loras()
|
||||
with patch.object(lora_import, "build_record", side_effect=ValueError("x")):
|
||||
job = await self._failed_job()
|
||||
self.assertEqual(job.scope["failure"], "nothing-to-import")
|
||||
# M4: the job keeps what staging skipped, by name, though it failed.
|
||||
self.assertEqual(job.counters["skipped"], 2)
|
||||
self.assertEqual(
|
||||
sorted(row["detail"] for row in job.outcomes["skipped"]), ["neon", "plain"]
|
||||
)
|
||||
|
||||
async def _change_during_download(self, change):
|
||||
"""Start the reviewed import, run ``change()`` while a download is held."""
|
||||
self._two_loras()
|
||||
before = _snapshot(self.api.db)
|
||||
self.gate = threading.Event()
|
||||
_plan, job = await self._apply_reviewed()
|
||||
loop = asyncio.get_running_loop()
|
||||
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
|
||||
change()
|
||||
self.gate.set()
|
||||
await job.task
|
||||
self.assertEqual(job.state, "failed", job.log)
|
||||
self.assertEqual(job.scope["failure"], "plan-changed")
|
||||
self.assertIn("Review the import again", job.status_line)
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
return job
|
||||
|
||||
async def test_files_changed_while_staging_fail_the_job(self):
|
||||
# I1: LoraManager rewrites a metadata file while the downloads run.
|
||||
meta = self.loras / "plain.safetensors.metadata.json"
|
||||
await self._change_during_download(
|
||||
lambda: meta.write_text(meta.read_text() + " ")
|
||||
)
|
||||
|
||||
async def test_a_file_removed_while_staging_fails_the_job(self):
|
||||
meta = self.loras / "plain.safetensors.metadata.json"
|
||||
await self._change_during_download(meta.unlink)
|
||||
|
||||
async def test_a_file_added_while_staging_fails_the_job(self):
|
||||
await self._change_during_download(lambda: _write_metadata(self.loras, "late"))
|
||||
|
||||
async def test_legacy_scan_refuses_files_changed_while_it_stages(self):
|
||||
self._two_loras()
|
||||
before = _snapshot(self.api.db)
|
||||
self.gate = threading.Event()
|
||||
meta = self.loras / "plain.safetensors.metadata.json"
|
||||
resp = await self.client.request("POST", SCAN)
|
||||
loop = asyncio.get_running_loop()
|
||||
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
|
||||
meta.write_text(meta.read_text() + " ")
|
||||
self.gate.set()
|
||||
events = self._sse(await resp.text())
|
||||
self.assertEqual(events[-1]["type"], "error")
|
||||
self.assertIn("changed during the scan", events[-1]["message"])
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
async def test_a_failed_safety_backup_changes_nothing(self):
|
||||
self._two_loras()
|
||||
from py.api import maintenance
|
||||
|
||||
with (
|
||||
patch.object(PromptManagerConfig, "SAFETY_BACKUP", True),
|
||||
patch.object(
|
||||
maintenance,
|
||||
"take_safety_backup",
|
||||
side_effect=maintenance.SafetyBackupFailed("no room"),
|
||||
),
|
||||
):
|
||||
job = await self._failed_job()
|
||||
self.assertEqual(job.scope["failure"], "backup-failed")
|
||||
|
||||
async def test_cancel_right_after_the_start_writes_nothing(self):
|
||||
self._two_loras()
|
||||
before = _snapshot(self.api.db)
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
self.api._jobs.cancel(job.id) # at once, while it stages
|
||||
await job.task
|
||||
self.assertEqual(job.state, "cancelled")
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
|
||||
|
||||
def plan_title():
|
||||
return lora_import.JOB_TITLE
|
||||
|
||||
|
||||
def _read(path):
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
return handle.read()
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
|
||||
|
||||
def _size(path):
|
||||
return os.path.getsize(path) if os.path.exists(path) else None
|
||||
|
||||
|
||||
class LoraImportRestoreTests(LoraImportAPITestCase):
|
||||
"""The import and a restore (#249 S8) never run beside each other."""
|
||||
|
||||
async def test_a_restore_is_refused_while_the_import_runs(self):
|
||||
self._two_loras()
|
||||
self.gate = threading.Event()
|
||||
_plan, job = await self._apply_reviewed()
|
||||
loop = asyncio.get_running_loop()
|
||||
self.assertTrue(await loop.run_in_executor(None, self.downloading.wait, 10))
|
||||
|
||||
resp = await self.client.request("POST", "/prompt_manager/restore")
|
||||
body = json.loads(await resp.text())
|
||||
self.assertEqual(resp.status, 409)
|
||||
self.assertIn(lora_import.JOB_TITLE, body["error"])
|
||||
# The workspace restore job is refused for the same running import.
|
||||
blocker = self.api._jobs.blocker(RESTORE_JOB_KIND, admission="exclusive")
|
||||
self.assertIs(blocker, job)
|
||||
|
||||
self.gate.set()
|
||||
await job.task
|
||||
self.assertEqual(job.state, "finished")
|
||||
|
||||
async def test_the_import_is_refused_while_a_restore_runs(self):
|
||||
self._two_loras()
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
self.assertEqual(resp.status, 200, plan)
|
||||
before = _snapshot(self.api.db)
|
||||
async with self.api._jobs.hold(
|
||||
RESTORE_JOB_KIND, "Restore library", admission="exclusive"
|
||||
):
|
||||
# A review reads the library: refused, not run beside the switch.
|
||||
resp, body = await self._post(PREVIEW)
|
||||
self.assertEqual(resp.status, 409, body)
|
||||
self.assertEqual(body["code"], "busy")
|
||||
self.assertEqual(body["running"]["title"], "Restore library")
|
||||
resp, body = await self._post(APPLY, {"token": plan["token"]})
|
||||
self.assertEqual(resp.status, 409, body)
|
||||
self.assertEqual(body["code"], "busy")
|
||||
# The admin page's scan says so in its own error event.
|
||||
resp = await self.client.request("POST", SCAN)
|
||||
events = self._sse(await resp.text())
|
||||
self.assertEqual(events[-1]["type"], "error")
|
||||
self.assertIn("Restore library", events[-1]["message"])
|
||||
self.assertEqual(_snapshot(self.api.db), before)
|
||||
self.assertIsNone(self.api._jobs.active("lora_import"))
|
||||
|
||||
async def test_a_running_review_holds_a_lease_a_restore_waits_for(self):
|
||||
self._two_loras()
|
||||
seen = []
|
||||
real_preview = lora_import.preview
|
||||
|
||||
def preview(*args):
|
||||
seen.append(self.api._jobs.leases)
|
||||
return real_preview(*args)
|
||||
|
||||
with patch.object(lora_import, "preview", preview):
|
||||
resp, plan = await self._post(PREVIEW)
|
||||
self.assertEqual(resp.status, 200, plan)
|
||||
self.assertEqual(seen, [{LORA_PREVIEW_LEASE: 1}])
|
||||
self.assertEqual(self.api._jobs.leases, {})
|
||||
|
||||
|
||||
class LoraImportStageTests(unittest.TestCase):
|
||||
|
||||
def setUp(self):
|
||||
self.tmp = Path(tempfile.mkdtemp())
|
||||
self.addCleanup(lambda: __import__("shutil").rmtree(self.tmp, True))
|
||||
|
||||
def test_discover_sorts_and_drops_repeats(self):
|
||||
a = self.tmp / "a"
|
||||
b = self.tmp / "a" / "b"
|
||||
b.mkdir(parents=True)
|
||||
_write_metadata(b, "zeta")
|
||||
_write_metadata(a, "alpha")
|
||||
with patch.object(
|
||||
lora_utils, "find_lora_directories", return_value=[str(a), str(b)]
|
||||
):
|
||||
files = lora_import.discover(str(self.tmp))
|
||||
self.assertEqual([f.name for f in files][0], "alpha.safetensors.metadata.json")
|
||||
self.assertEqual(len(files), 2)
|
||||
|
||||
def test_fingerprint_follows_size_and_mtime(self):
|
||||
meta = _write_metadata(self.tmp, "one")
|
||||
first = lora_import.fingerprint([meta])
|
||||
self.assertEqual(lora_import.fingerprint([meta]), first)
|
||||
meta.write_text(meta.read_text() + " ")
|
||||
self.assertNotEqual(lora_import.fingerprint([meta]), first)
|
||||
meta.unlink()
|
||||
self.assertNotEqual(lora_import.fingerprint([meta]), first)
|
||||
|
||||
def test_stage_cancel_before_a_file_raises(self):
|
||||
meta = _write_metadata(self.tmp, "one")
|
||||
with self.assertRaises(lora_import.ImportCancelled):
|
||||
lora_import.stage([meta], self.tmp, cancelled=lambda: True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+196
-1
@@ -12,6 +12,7 @@ import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import types
|
||||
import unittest
|
||||
import urllib.parse
|
||||
@@ -524,6 +525,114 @@ class TestDownloadOne(unittest.TestCase):
|
||||
self.assertFalse(self.target.exists())
|
||||
|
||||
|
||||
class TestDownloadIsAtomic(unittest.TestCase):
|
||||
"""I2 (S10 review): a cached image is never seen half written."""
|
||||
|
||||
def setUp(self):
|
||||
self.tmpdir = Path(tempfile.mkdtemp())
|
||||
self.addCleanup(shutil.rmtree, self.tmpdir, True)
|
||||
self.target = self.tmpdir / "out.jpg"
|
||||
|
||||
def test_the_image_appears_only_once_complete(self):
|
||||
seen = []
|
||||
real_replace = os.replace
|
||||
|
||||
def replace(source, destination):
|
||||
# At the moment the name appears, the temp copy is whole and closed.
|
||||
seen.append((Path(source).parent, self.target.exists()))
|
||||
with open(source, "rb") as handle:
|
||||
self.assertEqual(handle.read(2), b"\xff\xd8")
|
||||
return real_replace(source, destination)
|
||||
|
||||
opener = _FakeOpener(_FakeResponse(_png_bytes()))
|
||||
with (
|
||||
patch.object(lora_utils, "_open_url", opener),
|
||||
patch.object(lora_utils.os, "replace", side_effect=replace),
|
||||
):
|
||||
result = lora_utils._download_one(
|
||||
"https://civitai.com/api/download/1", self.target, ""
|
||||
)
|
||||
self.assertEqual(seen, [(self.tmpdir, False)])
|
||||
self.assertEqual(os.path.realpath(result), os.path.realpath(self.target))
|
||||
self.assertEqual(sorted(p.name for p in self.tmpdir.iterdir()), ["out.jpg"])
|
||||
|
||||
def test_a_failed_write_leaves_neither_the_image_nor_a_temp_file(self):
|
||||
opener = _FakeOpener(_FakeResponse(_png_bytes()))
|
||||
with (
|
||||
patch.object(lora_utils, "_open_url", opener),
|
||||
patch.object(lora_utils.os, "replace", side_effect=OSError("disk full")),
|
||||
):
|
||||
result = lora_utils._download_one(
|
||||
"https://civitai.com/api/download/1", self.target, ""
|
||||
)
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(list(self.tmpdir.iterdir()), [])
|
||||
|
||||
def test_an_unreadable_image_leaves_no_temp_file(self):
|
||||
opener = _FakeOpener(_FakeResponse(b"not an image"))
|
||||
with patch.object(lora_utils, "_open_url", opener):
|
||||
result = lora_utils._download_one(
|
||||
"https://civitai.com/api/download/1", self.target, ""
|
||||
)
|
||||
self.assertIsNone(result)
|
||||
self.assertEqual(list(self.tmpdir.iterdir()), [])
|
||||
|
||||
@unittest.skipIf(os.name == "nt", "POSIX file modes")
|
||||
def test_the_image_gets_the_permissions_a_plain_save_gives(self):
|
||||
# S10 re-review N7: the cached JPEG honours the umask like img.save did.
|
||||
previous = os.umask(0o022)
|
||||
self.addCleanup(os.umask, previous)
|
||||
opener = _FakeOpener(_FakeResponse(_png_bytes()))
|
||||
with patch.object(lora_utils, "_open_url", opener):
|
||||
lora_utils._download_one(
|
||||
"https://civitai.com/api/download/1", self.target, ""
|
||||
)
|
||||
self.assertEqual(self.target.stat().st_mode & 0o777, 0o644)
|
||||
|
||||
|
||||
class TestStaleDownloadSweep(unittest.TestCase):
|
||||
"""S10 re-review N7: a killed download's ``.pm-*.part`` is swept later."""
|
||||
|
||||
def setUp(self):
|
||||
self.cache = Path(tempfile.mkdtemp())
|
||||
self.addCleanup(shutil.rmtree, self.cache, True)
|
||||
self.lora = self.cache / "my_lora"
|
||||
self.lora.mkdir()
|
||||
|
||||
def _file(self, path, age_seconds):
|
||||
path.write_bytes(b"x")
|
||||
stamp = time.time() - age_seconds
|
||||
os.utime(path, (stamp, stamp))
|
||||
return path
|
||||
|
||||
def test_only_old_temp_downloads_are_removed(self):
|
||||
old = self._file(self.lora / ".pm-abc123.part", 2 * 3600)
|
||||
old_top = self._file(self.cache / ".pm-top.part", 2 * 3600)
|
||||
fresh = self._file(self.lora / ".pm-new.part", 60)
|
||||
image = self._file(self.lora / "abc.jpg", 2 * 3600)
|
||||
other = self._file(self.lora / "keep.part", 2 * 3600)
|
||||
prefixed = self._file(self.lora / ".pm-keep.jpg", 2 * 3600)
|
||||
deep = self.lora / "nested"
|
||||
deep.mkdir()
|
||||
nested = self._file(deep / ".pm-deep.part", 2 * 3600)
|
||||
|
||||
lora_utils.sweep_stale_downloads(self.cache)
|
||||
|
||||
self.assertFalse(old.exists())
|
||||
self.assertFalse(old_top.exists())
|
||||
for kept in (fresh, image, other, prefixed, nested):
|
||||
self.assertTrue(kept.exists(), kept.name)
|
||||
|
||||
def test_a_missing_cache_is_fine(self):
|
||||
lora_utils.sweep_stale_downloads(self.cache / "absent")
|
||||
|
||||
def test_a_file_that_cannot_be_removed_is_left(self):
|
||||
old = self._file(self.lora / ".pm-abc.part", 2 * 3600)
|
||||
with patch.object(lora_utils.os, "remove", side_effect=PermissionError):
|
||||
lora_utils.sweep_stale_downloads(self.cache)
|
||||
self.assertTrue(old.exists())
|
||||
|
||||
|
||||
class _FakeHTTPSHandler(urllib.request.HTTPSHandler):
|
||||
"""Serves canned responses through a real urllib opener, recording requests.
|
||||
|
||||
@@ -817,9 +926,95 @@ class TestDetectLoraManager(unittest.TestCase):
|
||||
self.assertTrue(lora_utils._looks_like_lora_manager(lm))
|
||||
|
||||
def test_nothing_found(self):
|
||||
with patch.object(lora_utils, "find_comfyui_root", return_value=None):
|
||||
with (
|
||||
patch.object(lora_utils, "find_comfyui_root", return_value=None),
|
||||
patch.object(lora_utils, "custom_nodes_directories", return_value=[]),
|
||||
):
|
||||
self.assertIsNone(lora_utils.detect_lora_manager())
|
||||
|
||||
def test_auto_detects_in_an_extra_custom_nodes_folder(self):
|
||||
# #144: ComfyUI's main custom_nodes has no LoraManager; an extra
|
||||
# custom_nodes path registered through folder_paths (extra model
|
||||
# paths, --base-directory) has it.
|
||||
root = _make_comfy_root(self.tmp)
|
||||
extra = self.tmp / "shared_nodes"
|
||||
extra.mkdir()
|
||||
lm = _make_lora_manager(extra, "ComfyUI-Lora-Manager")
|
||||
fake = types.SimpleNamespace(
|
||||
base_path=str(root),
|
||||
get_folder_paths=lambda name: (
|
||||
[str(root / "custom_nodes"), str(extra)]
|
||||
if name == "custom_nodes"
|
||||
else []
|
||||
),
|
||||
)
|
||||
with (
|
||||
patch.dict(sys.modules, {"folder_paths": fake}),
|
||||
patch.object(lora_utils, "find_comfyui_root", return_value=root),
|
||||
):
|
||||
self.assertEqual(lora_utils.detect_lora_manager(), str(lm.resolve()))
|
||||
|
||||
def test_main_custom_nodes_wins_over_an_extra_folder(self):
|
||||
root = _make_comfy_root(self.tmp)
|
||||
main = _make_lora_manager(root / "custom_nodes")
|
||||
extra = self.tmp / "shared_nodes"
|
||||
extra.mkdir()
|
||||
_make_lora_manager(extra)
|
||||
with (
|
||||
patch.object(lora_utils, "find_comfyui_root", return_value=root),
|
||||
patch.object(
|
||||
lora_utils,
|
||||
"custom_nodes_directories",
|
||||
return_value=[(root / "custom_nodes").resolve(), extra.resolve()],
|
||||
),
|
||||
):
|
||||
self.assertEqual(lora_utils.detect_lora_manager(), str(main.resolve()))
|
||||
|
||||
def test_an_entry_that_cannot_be_checked_is_skipped(self):
|
||||
# S10 review N5: a permission error on one entry, not a missing folder.
|
||||
root = _make_comfy_root(self.tmp)
|
||||
blocked = root / "custom_nodes" / "a-lora-manager"
|
||||
blocked.mkdir()
|
||||
lm = _make_lora_manager(root / "custom_nodes", "b-Lora-Manager")
|
||||
real_is_dir = Path.is_dir
|
||||
|
||||
def is_dir(path):
|
||||
if path == blocked:
|
||||
raise PermissionError(13, "Permission denied", str(path))
|
||||
return real_is_dir(path)
|
||||
|
||||
with (
|
||||
patch.object(lora_utils, "find_comfyui_root", return_value=root),
|
||||
patch.object(lora_utils, "custom_nodes_directories", return_value=[]),
|
||||
patch.object(Path, "is_dir", is_dir),
|
||||
):
|
||||
self.assertEqual(lora_utils.detect_lora_manager(), str(lm.resolve()))
|
||||
|
||||
def test_a_folder_that_cannot_be_listed_is_skipped(self):
|
||||
root = _make_comfy_root(self.tmp)
|
||||
with (
|
||||
patch.object(lora_utils, "find_comfyui_root", return_value=root),
|
||||
patch.object(lora_utils, "custom_nodes_directories", return_value=[]),
|
||||
patch.object(Path, "iterdir", side_effect=PermissionError(13, "denied")),
|
||||
):
|
||||
self.assertIsNone(lora_utils.detect_lora_manager())
|
||||
|
||||
def test_an_unreadable_extra_folder_is_skipped(self):
|
||||
root = _make_comfy_root(self.tmp)
|
||||
missing = self.tmp / "gone"
|
||||
extra = self.tmp / "shared_nodes"
|
||||
extra.mkdir()
|
||||
lm = _make_lora_manager(extra)
|
||||
with (
|
||||
patch.object(lora_utils, "find_comfyui_root", return_value=root),
|
||||
patch.object(
|
||||
lora_utils,
|
||||
"custom_nodes_directories",
|
||||
return_value=[missing, extra.resolve()],
|
||||
),
|
||||
):
|
||||
self.assertEqual(lora_utils.detect_lora_manager(), str(lm.resolve()))
|
||||
|
||||
|
||||
class TestFindLoraDirectories(unittest.TestCase):
|
||||
|
||||
|
||||
Vendored
+3
-2
@@ -15,14 +15,15 @@ test('real-ComfyUI suite registers Classic and Nodes 2.0 startup projects', () =
|
||||
assert.match(output, /\[nodes2-start\]/);
|
||||
});
|
||||
|
||||
test('execution script discovers only the new matrix and preserves all 78 original stock cases plus the native editor host case', () => {
|
||||
test('execution script discovers only the new matrix and preserves all 78 original stock cases plus the native editor and renderer host cases', () => {
|
||||
const manifest = JSON.parse(readFileSync(new URL('../../tools/vendor/package.json', import.meta.url), 'utf8'));
|
||||
assert.equal(manifest.scripts['test:executions'], 'playwright test --config playwright.execution.config.mjs');
|
||||
const list = config => execFileSync(process.execPath, ['tools/vendor/node_modules/@playwright/test/cli.js',
|
||||
'test', '--config', config, '--list'], { cwd: root, encoding: 'utf8',
|
||||
env: { ...process.env, PM_COMFY_ROOT: '/unused-pinned-checkout' } });
|
||||
const stock = list('tools/vendor/comfyui.config.mjs');
|
||||
assert.match(stock, /Total: 82 tests in 8 files/); // + local Inspect host case (inspect-sender.spec.mjs, 2 projects)
|
||||
// + local Inspect host case (inspect-sender.spec.mjs, 2 projects), + the S10 renderer case (prompt-bridge.spec.mjs, 2 projects)
|
||||
assert.match(stock, /Total: 84 tests in 8 files/);
|
||||
assert.doesNotMatch(stock, /execution-history/);
|
||||
assert.doesNotMatch(stock, /execution-outputs/);
|
||||
assert.doesNotMatch(stock, /library-location/); // it starts its own fresh host (#249 S6b)
|
||||
|
||||
+10
-4
@@ -163,7 +163,7 @@
|
||||
},
|
||||
"workspace/activity-job-view.mjs": {
|
||||
"lines": 97,
|
||||
"branches": 86,
|
||||
"branches": 87,
|
||||
"functions": 98,
|
||||
"statements": 97
|
||||
},
|
||||
@@ -415,7 +415,7 @@
|
||||
},
|
||||
"workspace/maintenance-review.mjs": {
|
||||
"lines": 94,
|
||||
"branches": 81,
|
||||
"branches": 85,
|
||||
"functions": 90,
|
||||
"statements": 94
|
||||
},
|
||||
@@ -503,11 +503,17 @@
|
||||
"functions": 89,
|
||||
"statements": 98
|
||||
},
|
||||
"workspace/system-integrations.mjs": {
|
||||
"lines": 96,
|
||||
"branches": 88,
|
||||
"functions": 91,
|
||||
"statements": 96
|
||||
},
|
||||
"workspace/system-maintenance.mjs": {
|
||||
"lines": 92,
|
||||
"lines": 93,
|
||||
"branches": 80,
|
||||
"functions": 92,
|
||||
"statements": 92
|
||||
"statements": 93
|
||||
},
|
||||
"workspace/system-overview.mjs": {
|
||||
"lines": 96,
|
||||
|
||||
@@ -47,10 +47,19 @@ function redraw(app,node) {
|
||||
app.graph?.setDirtyCanvas?.(true,true);
|
||||
}
|
||||
|
||||
export function createPromptAdapter(app) {
|
||||
// The renderer this window draws nodes with (System › Integrations, S10): Nodes 2.0 while LiteGraph's Vue node mode is
|
||||
// on, else Classic; unknown (omitted) without LiteGraph.
|
||||
function renderer(runtime) {
|
||||
const graph = runtime?.LiteGraph;
|
||||
if (!graph) return {};
|
||||
return {renderer:graph.vueNodesMode === true ? 'nodes2' : 'classic'};
|
||||
}
|
||||
|
||||
export function createPromptAdapter(app, {runtime = globalThis} = {}) {
|
||||
const snapshots = new WeakMap();
|
||||
function listTargets() {
|
||||
return nodes(app.graph).filter(node => schemas.has(node.type)).map(node => ({...target(node),roles:Object.keys(schemas.get(node.type))}));
|
||||
const drawn = renderer(runtime);
|
||||
return nodes(app.graph).filter(node => schemas.has(node.type)).map(node => ({...target(node),roles:Object.keys(schemas.get(node.type)),...drawn}));
|
||||
}
|
||||
function preview(targetId,rawRoles) {
|
||||
const roles = validateRoles(rawRoles);
|
||||
|
||||
@@ -82,8 +82,8 @@ const OVERTAKEN = new Set(['plan-changed', 'storage-changed', 'plan-expired']);
|
||||
|
||||
/**
|
||||
* Where a failed apply's Review again goes (S7 review M2): a maintenance plan the library overtook is reviewed again
|
||||
* (`{operation}`); a duplicate cleanup whose sources or plan changed needs a new search, in Maintenance (`{section}`);
|
||||
* otherwise null.
|
||||
* (`{operation}`); a duplicate cleanup whose sources or plan changed needs a new search, in Maintenance (`{section}`); a
|
||||
* LoRA import the files or saved pairs overtook is reviewed again in Integrations (`{section}`, S10); otherwise null.
|
||||
*/
|
||||
export function reviewAgainOf(job) {
|
||||
if (job.state !== 'failed') return null;
|
||||
@@ -94,6 +94,10 @@ export function reviewAgainOf(job) {
|
||||
if (job.kind === 'duplicate_cleanup' && (OVERTAKEN.has(failure) || failure === 'sources-changed')) {
|
||||
return { section: '#/system/maintenance' };
|
||||
}
|
||||
// System S10: a LoRA import whose files or imports changed, or that saved pairs block, is reviewed in Integrations.
|
||||
if (job.kind === 'lora_import' && (OVERTAKEN.has(failure) || failure === 'blocked-by-pairs')) {
|
||||
return { section: '#/system/integrations' };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
|
||||
@@ -13,7 +13,8 @@
|
||||
* `scan` is exactly the D56 view and `image-tags` the D89 one. `thumbnails` counts Generated (green), Skipped and
|
||||
* Errors (red). System S7: `maintenance` (a reviewed plan's apply) counts Changed, Protected and Errors and, when it
|
||||
* failed, ends with the server's reason; `duplicates` (the search) counts Groups found; `duplicate_cleanup` counts
|
||||
* Removed, Kept and Errors. Any other kind (the health check, a held legacy-page operation, a kind this page does not know) gets
|
||||
* Removed, Kept and Errors; S10: `lora_import` counts Imported, Linked to existing, Skipped, Images downloaded, Download
|
||||
* errors and Errors and, when it failed, ends with the server's reason (the previous imports were kept). Any other kind (the health check, a held legacy-page operation, a kind this page does not know) gets
|
||||
* the generic descriptor: title-based copy and no tiles or outcomes, so it never shows another kind's zeros (S1 review
|
||||
* M6, S2 review M3). `viewOf(job)` keeps only the groups the job itself counts, so a hold (no groups) has none either.
|
||||
*
|
||||
@@ -46,6 +47,8 @@ export const groupsOf = job => {
|
||||
};
|
||||
/** "3 removed" (System S7: a duplicate cleanup). */
|
||||
export const removedOf = job => counted(job, 'removed', 'removed');
|
||||
/** "12 imported" (System S10: the LoRA reference import). */
|
||||
export const importedOf = job => counted(job, 'imported', 'imported');
|
||||
|
||||
/** "2 output folders": what a scan or a thumbnail job walks. */
|
||||
function foldersOf(job) {
|
||||
@@ -85,6 +88,11 @@ const tagLine = job => `${taggedOf(job)} · ${errorsOf(job)}`;
|
||||
const thumbnailLine = job => `${generatedOf(job)} · ${errorsOf(job)}`;
|
||||
const changedLine = job => `${changedOf(job)} · ${errorsOf(job)}`;
|
||||
const cleanupLine = job => `${removedOf(job)} · ${errorsOf(job)}`;
|
||||
const importLine = job => `${importedOf(job)} · ${errorsOf(job)}`;
|
||||
|
||||
/** A job that ends with the server's reason when it failed (the LoRA import keeps the old collection and says so). */
|
||||
const reasonFinish = (name, kept, line) => job => (job.state === 'failed' && job.status_line
|
||||
? `${name} failed · ${job.status_line}` : namedFinish(() => name, kept, line)(job));
|
||||
|
||||
/** A maintenance apply ends with its counters, or, when it failed, the server's reason (plan-changed: review again). */
|
||||
function maintenanceFinish(job) {
|
||||
@@ -182,6 +190,20 @@ export const KINDS = Object.freeze({
|
||||
openable: groups('kept'),
|
||||
log: 'Cleanup log',
|
||||
}),
|
||||
// System S10: the reviewed LoRA reference import; cancellable only while it downloads (the server says when).
|
||||
lora_import: Object.freeze({
|
||||
...GENERIC,
|
||||
pillVerb: 'Importing LoRAs',
|
||||
finishToast: reasonFinish('LoRA import', importedOf, importLine),
|
||||
line: importLine,
|
||||
kept: importedOf,
|
||||
tiles: tiles([['imported', 'Imported', 'added'], ['linked', 'Linked to existing', ''], ['skipped', 'Skipped', 'warn'],
|
||||
['downloaded', 'Images downloaded', ''], ['download_errors', 'Download errors', 'warn'], ['errors', 'Errors', 'bad']]),
|
||||
outcomes: tiles([['imported', 'Imported'], ['errors', 'Errors'], ['skipped', 'Skipped'], ['linked', 'Linked to existing']]),
|
||||
log: 'Import log',
|
||||
cancelLabel: 'Cancel import',
|
||||
cancel: 'Cancel import',
|
||||
}),
|
||||
});
|
||||
|
||||
/** The descriptor of a job `kind`; any kind without one gets the generic descriptor. */
|
||||
|
||||
@@ -55,10 +55,10 @@ export const OPERATION_TITLES = Object.freeze({
|
||||
workflow_storage: 'Store each workflow once',
|
||||
});
|
||||
// The review is gone or no longer matches the database: review it again.
|
||||
const REVIEW_AGAIN = new Set(['plan-expired', 'storage-changed']);
|
||||
const REVIEW_AGAIN = new Set(['plan-expired', 'storage-changed', 'plan-changed']);
|
||||
// Refusals whose server message is the whole truth.
|
||||
const SERVER_SAYS = new Set(['insufficient-space', 'nothing-to-do', 'too-large', 'invalid-request', 'not-found',
|
||||
'job-running', 'no-groups', 'apply-failed', 'sources-changed']);
|
||||
'job-running', 'no-groups', 'apply-failed', 'sources-changed', 'lora-disabled', 'lora-path']);
|
||||
|
||||
const number = value => value.toLocaleString('en-US');
|
||||
|
||||
@@ -169,13 +169,15 @@ async function holding(trigger, busyText, work) {
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply a reviewed plan's `token`; the job opens in Activity. `reviewAgain()` reviews the operation once more when the
|
||||
* server says the review expired or the database was replaced.
|
||||
* Apply a reviewed plan's `token` (at `url`: maintenance's apply, or the LoRA import's, S10); the job opens in Activity.
|
||||
* `reviewAgain()` reviews the operation once more when the server says the review expired, the database was replaced
|
||||
* or (the LoRA import) its files changed.
|
||||
*/
|
||||
export async function applyPlan({ api, store, scope, token, reviewAgain, notify = showWorkspaceToast, navigate = navigateTo }) {
|
||||
export async function applyPlan({ api, store, scope, token, reviewAgain, notify = showWorkspaceToast, navigate = navigateTo,
|
||||
url = APPLY_URL }) {
|
||||
let result;
|
||||
try {
|
||||
result = await api.post(APPLY_URL, { token });
|
||||
result = await api.post(url, { token });
|
||||
} catch (error) {
|
||||
if (scope.signal.aborted) return;
|
||||
const refusal = applyRefusal(error);
|
||||
|
||||
@@ -0,0 +1,446 @@
|
||||
/**
|
||||
* System › Integrations (#249 System S10 Task 27): the concept's `renderSystem` "integrations" body (app.js 682–684).
|
||||
*
|
||||
* - "ComfyUI": Default send destination, Host compatibility and Return to canvas after send. D106 (Q8): the send
|
||||
* bridge always asks for the window and node, and the nodes always follow ComfyUI's own renderer, so only "Ask each
|
||||
* time" and "Follow host renderer" can be chosen; the other options and the return toggle are disabled, with the
|
||||
* reason under the row, and nothing is stored for them. Host compatibility also says which renderer the open
|
||||
* ComfyUI windows use, as the send bridge reports it (`renderer` on each Prompt Manager node it lists).
|
||||
* - "LoRA & resources": "LoraManager integration" (enabled, a host draft) says where LoraManager was detected;
|
||||
* D106 adds the read-only "LoraManager path" row, as on the legacy page. "Automatically include trigger words"
|
||||
* (a host draft) says whether the trigger words are loaded. The three are separate facts (`/lora/status`).
|
||||
* D114: the CivitAI key field is never filled in; typing drafts a new key, a blank field keeps the saved one, and
|
||||
* "Clear saved key" (shown while a key is saved) drafts its removal; the line under it says "A key is saved." or "No key
|
||||
* saved." All of them are
|
||||
* saved by the one Save row (`/lora/enable` with the settings revision, system.mjs).
|
||||
* - "LoRA reference data" → "Review import" (D107; the concept's button only simulated a job): the server
|
||||
* reviews the import (`POST /lora/import/preview`, read-only) and the shared review dialog shows what it replaces,
|
||||
* what saved pairs block, what it downloads and the backup box; Start runs the "LoRA reference import" Activity job.
|
||||
*/
|
||||
import { createButton } from '../shared/toolkit/core.js';
|
||||
import { createBridgeClient } from '../shared/toolkit/prompt-bridge.js';
|
||||
import { settingRow, operationRow } from './system-controls.mjs';
|
||||
import { HOST_UNAVAILABLE } from './system-appearance.mjs';
|
||||
import { closeWorkspaceDialog, openContentDialog, openReviewDialog } from './workspace-dialogs.mjs';
|
||||
import { BACKUP_COPY, applyPlan, backupText, planNote, protectedText, reviewRefusal } from './maintenance-review.mjs';
|
||||
import { downloadDatabaseBackup } from './activity-scan.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
import { navigateTo } from './state.mjs';
|
||||
import { noun } from './plural.mjs';
|
||||
|
||||
export const LORA_STATUS_URL = '/prompt_manager/lora/status';
|
||||
export const LORA_ENABLE_URL = '/prompt_manager/lora/enable';
|
||||
export const IMPORT_PREVIEW_URL = '/prompt_manager/lora/import/preview';
|
||||
export const IMPORT_APPLY_URL = '/prompt_manager/lora/import/apply';
|
||||
export const IMPORT_TITLE = 'Review LoRA import';
|
||||
export const IMPORT_START = 'Start import';
|
||||
/** The host draft keys this section saves through `/lora/enable`; every other host key goes to `/settings`. */
|
||||
export const LORA_KEYS = Object.freeze(['lora_enabled', 'lora_trigger_words_enabled', 'civitai_api_key', 'clear_civitai_api_key']);
|
||||
|
||||
/** D106: why the options the bridge cannot honour are disabled. */
|
||||
export const REASONS = Object.freeze({
|
||||
send: 'Sending always asks for the ComfyUI window and node, so “Ask each time” is the only option.',
|
||||
renderer: 'Prompt Manager nodes always follow ComfyUI’s own renderer setting.',
|
||||
returnCanvas: 'Not available: sending applies the text in ComfyUI and keeps this page open.',
|
||||
});
|
||||
export const RENDERER_COPY = Object.freeze({
|
||||
looking: 'Looking for open ComfyUI windows…',
|
||||
none: 'No ComfyUI window is open in this browser, so its renderer is not known.',
|
||||
noNodes: 'ComfyUI is open; add a Prompt Manager node to see which renderer it uses.',
|
||||
unavailable: 'The renderer could not be checked in this browser.',
|
||||
});
|
||||
const RENDERER_NAMES = Object.freeze({ classic: 'Classic', nodes2: 'Nodes 2.0' });
|
||||
export const KEY_COPY = Object.freeze({ saved: 'A key is saved.', none: 'No key saved.',
|
||||
clearing: 'The saved key is removed when you save.', replacing: 'The new key replaces the saved one when you save.' });
|
||||
export const NOTHING_TO_IMPORT = 'There is nothing to import; nothing was changed.';
|
||||
export const BLOCKED_NOTE = 'Delete or change the saved pairs that use these prompts, then review the import again. Nothing was changed.';
|
||||
export const CACHE_NOTE = 'Example images downloaded before a failure stay in the image cache.';
|
||||
|
||||
const number = value => value.toLocaleString('en-US');
|
||||
|
||||
function el(tag, text, className) {
|
||||
const node = document.createElement(tag);
|
||||
if (text !== undefined) node.textContent = text;
|
||||
if (className) node.className = className;
|
||||
return node;
|
||||
}
|
||||
|
||||
/** The host values a `/lora/status` answer gives: drafted ones (enabled, trigger words, clear) and read-only facts. */
|
||||
export function loraValues(status) {
|
||||
return {
|
||||
lora_enabled: status?.enabled === true,
|
||||
lora_trigger_words_enabled: status?.trigger_words_enabled === true,
|
||||
clear_civitai_api_key: false,
|
||||
lora_path: typeof status?.path === 'string' ? status.path : '',
|
||||
lora_detected: status?.detected === true,
|
||||
lora_detected_path: typeof status?.detected_path === 'string' ? status.detected_path : '',
|
||||
lora_detected_usable: status?.detected_usable !== false,
|
||||
lora_has_key: status?.has_civitai_api_key === true,
|
||||
lora_cache_loaded: status?.trigger_cache_loaded === true,
|
||||
};
|
||||
}
|
||||
|
||||
/** Host draft changes split by route: `{lora, settings}`, each null when it has none. */
|
||||
export function splitHostChanges(changes) {
|
||||
const lora = {};
|
||||
const settings = {};
|
||||
for (const [key, value] of Object.entries(changes ?? {})) (LORA_KEYS.includes(key) ? lora : settings)[key] = value;
|
||||
return { lora: Object.keys(lora).length ? lora : null, settings: Object.keys(settings).length ? settings : null };
|
||||
}
|
||||
|
||||
/**
|
||||
* The `/lora/enable` body for the drafted LoRA `changes` over the `saved` host values: the whole LoRA state (the route
|
||||
* saves all of it), the saved path, and the key only when one was typed (a blank or absent key keeps the saved one).
|
||||
*/
|
||||
export function loraPayload(changes, saved) {
|
||||
const value = key => (Object.hasOwn(changes, key) ? changes[key] : saved?.[key]);
|
||||
const body = { enabled: value('lora_enabled') === true, trigger_words_enabled: value('lora_trigger_words_enabled') === true,
|
||||
path: saved?.lora_path || '' };
|
||||
if (changes.clear_civitai_api_key === true) body.clear_civitai_api_key = true;
|
||||
else if (typeof changes.civitai_api_key === 'string' && changes.civitai_api_key.trim()) body.civitai_api_key = changes.civitai_api_key.trim();
|
||||
return body;
|
||||
}
|
||||
|
||||
/**
|
||||
* Write host `changes` in their two parts (S10 review I3): the LoRA keys through `postLora` (`/lora/enable`), then the
|
||||
* other keys through `postSettings` (`/settings`), each with the revision the previous write left. Resolves with the
|
||||
* last answer when everything saved. When only one part saved, rejects with the other part's error carrying
|
||||
* `partial: {saved: [keys], revision}`, so the store keeps what saved and adopts the new revision; a
|
||||
* conflict on the first write (another page changed config.json) stops before the second.
|
||||
*/
|
||||
export async function writeHostInParts({ changes, saved, expected_revision, postLora, postSettings }) {
|
||||
const { lora, settings } = splitHostChanges(changes);
|
||||
let revision = expected_revision;
|
||||
let result = null;
|
||||
const written = [];
|
||||
let refused = null;
|
||||
if (lora) {
|
||||
try {
|
||||
result = await postLora({ ...loraPayload(lora, saved), expected_revision: revision });
|
||||
if (typeof result?.revision === 'string') revision = result.revision;
|
||||
written.push(...Object.keys(lora));
|
||||
} catch (error) {
|
||||
if (!settings || error?.code === 'settings-changed') throw error;
|
||||
refused = error;
|
||||
}
|
||||
}
|
||||
if (settings) {
|
||||
try {
|
||||
result = await postSettings({ ...settings, expected_revision: revision });
|
||||
if (typeof result?.revision === 'string') revision = result.revision;
|
||||
written.push(...Object.keys(settings));
|
||||
} catch (error) {
|
||||
if (!written.length) throw error;
|
||||
throw Object.assign(error, { partial: { saved: written, revision } });
|
||||
}
|
||||
}
|
||||
if (refused) {
|
||||
throw Object.assign(refused, { partial: { saved: written, revision } });
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/** The renderer line for discovered bridge hosts (D106). */
|
||||
export function rendererText(hosts) {
|
||||
if (!Array.isArray(hosts) || hosts.length === 0) return RENDERER_COPY.none;
|
||||
const seen = new Set();
|
||||
for (const host of hosts) for (const target of host?.targets ?? []) if (RENDERER_NAMES[target?.renderer]) seen.add(target.renderer);
|
||||
if (!seen.size) return RENDERER_COPY.noNodes;
|
||||
const names = ['classic', 'nodes2'].filter(name => seen.has(name)).map(name => RENDERER_NAMES[name]);
|
||||
return `Detected: ${names.join(' and ')} (${number(hosts.length)} ComfyUI ${noun(hosts.length, 'window')}).`;
|
||||
}
|
||||
|
||||
/** The LoraManager row's line: where it was detected, or that it was not. */
|
||||
export function detectionText(values) {
|
||||
if (values.lora_detected === undefined) return null;
|
||||
if (!values.lora_detected) return 'Not detected in ComfyUI’s custom_nodes folders.';
|
||||
if (values.lora_detected_usable === false) {
|
||||
// A linked install outside custom_nodes is found but cannot be enabled (S10 review N5).
|
||||
return `Found at ${values.lora_detected_path || 'a linked folder'}, but outside ComfyUI’s custom_nodes folders, so it can’t be enabled.`;
|
||||
}
|
||||
return values.lora_detected_path ? `Detected: ${values.lora_detected_path}.` : 'Detected.';
|
||||
}
|
||||
|
||||
/** The trigger-words row's line (the cache, separate from the setting). */
|
||||
export function cacheText(values) {
|
||||
if (values.lora_cache_loaded === undefined) return null;
|
||||
return values.lora_cache_loaded ? 'Trigger words are loaded.' : 'Trigger words are not loaded.';
|
||||
}
|
||||
|
||||
/** D114: the key line for the saved state and the drafts. */
|
||||
export function keyText(values, changes) {
|
||||
if (changes?.clear_civitai_api_key === true) return KEY_COPY.clearing;
|
||||
if (typeof changes?.civitai_api_key === 'string' && changes.civitai_api_key.trim()) return KEY_COPY.replacing;
|
||||
if (values.lora_has_key === undefined) return null;
|
||||
return values.lora_has_key ? KEY_COPY.saved : KEY_COPY.none;
|
||||
}
|
||||
|
||||
/** D107: the import review's rows from the server's plan. */
|
||||
export function importGrid(plan) {
|
||||
const downloads = Number.isSafeInteger(plan.planned_downloads) ? plan.planned_downloads : 0;
|
||||
const network = downloads
|
||||
? `Downloads up to ${number(downloads)} example ${noun(downloads, 'image')} from CivitAI${plan.network?.api_key ? ' with your saved key' : ''}`
|
||||
: 'No downloads';
|
||||
const found = Number.isSafeInteger(plan.lora_files) ? plan.lora_files : 0;
|
||||
const readable = Number.isSafeInteger(plan.with_metadata) ? plan.with_metadata : 0;
|
||||
return [
|
||||
['Scope', plan.scope?.label || `${number(readable)} ${noun(readable, 'LoRA')}`],
|
||||
['Found', `${number(found)} metadata ${noun(found, 'file')} · ${number(readable)} readable`],
|
||||
['Protected', protectedText(plan.protected)],
|
||||
['Network', network],
|
||||
['Backup', backupText(plan.backup)],
|
||||
['Media on disk', plan.media?.label || 'No files are removed'],
|
||||
];
|
||||
}
|
||||
|
||||
function namesList(plan) {
|
||||
const names = (Array.isArray(plan.details) ? plan.details : []).map(row => row?.label).filter(label => typeof label === 'string');
|
||||
if (!names.length) return [];
|
||||
const list = el('details');
|
||||
const count = Number.isSafeInteger(plan.with_metadata) ? plan.with_metadata : names.length;
|
||||
list.append(el('summary', names.length < count ? `LoRAs found (${number(names.length)} of ${number(count)})` : `LoRAs found (${number(names.length)})`));
|
||||
const body = el('pre', names.join('\n'));
|
||||
body.tabIndex = 0;
|
||||
list.append(body);
|
||||
return [list];
|
||||
}
|
||||
|
||||
function closeOnly({ scope, trigger, plan, note }) {
|
||||
const grid = el('dl', undefined, 'meta-grid');
|
||||
for (const [term, value] of importGrid(plan)) {
|
||||
const entry = el('div');
|
||||
entry.append(el('dt', term), el('dd', value));
|
||||
grid.append(entry);
|
||||
}
|
||||
const close = createButton('Close');
|
||||
close.addEventListener('click', () => closeWorkspaceDialog());
|
||||
const actions = el('div', undefined, 'dialog-actions');
|
||||
actions.append(close);
|
||||
openContentDialog({ scope, trigger, title: IMPORT_TITLE, nodes: [el('h3', 'LoRA reference data'), grid, ...namesList(plan),
|
||||
el('p', note, 'small'), actions] });
|
||||
}
|
||||
|
||||
/** Show the reviewed import; Start applies exactly this token (D107). */
|
||||
export function showImportPlan(plan, { api, store, scope, trigger = null, reviewAgain, notify = showWorkspaceToast,
|
||||
navigate = navigateTo, backup = () => downloadDatabaseBackup() }) {
|
||||
if (!plan.token) {
|
||||
closeOnly({ scope, trigger, plan, note: plan.blocked_by_pairs ? BLOCKED_NOTE : NOTHING_TO_IMPORT });
|
||||
return;
|
||||
}
|
||||
if (!plan.with_metadata && !plan.current_imports) {
|
||||
closeOnly({ scope, trigger, plan, note: NOTHING_TO_IMPORT });
|
||||
return;
|
||||
}
|
||||
const removesAll = !plan.with_metadata;
|
||||
const sentence = [planNote(plan), removesAll ? 'No LoRA metadata was found: this removes every imported LoRA prompt.' : '', CACHE_NOTE]
|
||||
.filter(Boolean).join(' ');
|
||||
openReviewDialog({ scope, trigger, title: IMPORT_TITLE, heading: 'LoRA reference data', grid: importGrid(plan),
|
||||
options: namesList(plan), sentence, backup: { copy: BACKUP_COPY, onBackup: backup }, cancelLabel: 'Cancel',
|
||||
primary: { label: IMPORT_START, danger: removesAll },
|
||||
onStart: () => applyPlan({ api, store, scope, token: plan.token, reviewAgain, notify, navigate, url: IMPORT_APPLY_URL }) });
|
||||
}
|
||||
|
||||
/** Review the import (read-only) and show its plan; the pressed button reads "Reviewing…" until it answered. */
|
||||
export async function reviewImport({ api, store, scope, trigger = null, notify = showWorkspaceToast, navigate = navigateTo }) {
|
||||
if (trigger?.getAttribute('aria-disabled') === 'true') return;
|
||||
const again = () => reviewImport({ api, store, scope, trigger, notify, navigate });
|
||||
const label = trigger?.textContent;
|
||||
trigger?.setAttribute('aria-disabled', 'true');
|
||||
if (trigger) trigger.textContent = 'Reviewing…';
|
||||
try {
|
||||
let plan;
|
||||
try {
|
||||
plan = await api.post(IMPORT_PREVIEW_URL, {}, { signal: scope.signal });
|
||||
} catch (error) {
|
||||
if (!scope.signal.aborted) notify(reviewRefusal(error));
|
||||
return;
|
||||
}
|
||||
if (!scope.signal.aborted) showImportPlan(plan, { api, store, scope, trigger, notify, navigate, reviewAgain: again });
|
||||
} finally {
|
||||
trigger?.removeAttribute('aria-disabled');
|
||||
if (trigger) trigger.textContent = label;
|
||||
}
|
||||
}
|
||||
|
||||
/** The row's control (`data-setting`), wherever the Toolkit switch nests it. */
|
||||
const controlOf = (row, id) => row.querySelector(`[data-setting="${id}"]`);
|
||||
|
||||
/** A `p.small` status line under a row's description, created on first use. */
|
||||
function noteOf(row) {
|
||||
let note = row.children[0].querySelector(':scope > p.small');
|
||||
if (!note) {
|
||||
note = el('p', undefined, 'small');
|
||||
row.children[0].append(note);
|
||||
}
|
||||
return note;
|
||||
}
|
||||
|
||||
function setNote(row, text) {
|
||||
if (text) noteOf(row).textContent = text;
|
||||
else row.children[0].querySelector(':scope > p.small')?.remove();
|
||||
}
|
||||
|
||||
/** A select row whose options after the first are disabled (D106). */
|
||||
function onlyFirst(spec) {
|
||||
const row = settingRow({ ...spec, control: { type: 'select', options: spec.options } });
|
||||
for (const option of controlOf(row, spec.id).options) option.disabled = option.index > 0;
|
||||
return row;
|
||||
}
|
||||
|
||||
/** The "ComfyUI" surface (D106) and its renderer detection over the send bridge. */
|
||||
function comfySurface({ scope, clientFactory }) {
|
||||
const surface = el('section', undefined, 'surface');
|
||||
surface.append(el('h2', 'ComfyUI'), el('p', 'One send contract across cards, prompt history and every metadata view.'));
|
||||
const send = onlyFirst({ id: 'send-destination', label: 'Default send destination',
|
||||
description: 'A destination picker previews both roles before applying.',
|
||||
options: ['Ask each time', 'Selected Prompt Manager node', 'Create a Prompt Manager (+/-) node'] });
|
||||
const renderer = onlyFirst({ id: 'renderer', label: 'Host compatibility',
|
||||
description: 'Use the shared compatibility layer for Classic and Nodes 2.0.',
|
||||
options: ['Follow host renderer', 'Classic', 'Nodes 2.0'] });
|
||||
const returnRow = settingRow({ id: 'return-canvas', label: 'Return to canvas after send',
|
||||
description: 'Apply text without queuing generation.', control: { type: 'toggle', checked: false } });
|
||||
controlOf(returnRow, 'return-canvas').disabled = true;
|
||||
setNote(send, REASONS.send);
|
||||
setNote(returnRow, REASONS.returnCanvas);
|
||||
const status = el('p', RENDERER_COPY.looking, 'small');
|
||||
status.setAttribute('role', 'status');
|
||||
status.id = 'renderer-status';
|
||||
renderer.children[0].append(el('p', REASONS.renderer, 'small'), status);
|
||||
surface.append(send, renderer, returnRow);
|
||||
|
||||
let client = null;
|
||||
// Overlapping checks (two quick visibility changes) paint only the newest answer (S10 review N2).
|
||||
let latest = 0;
|
||||
const detect = async () => {
|
||||
const check = ++latest;
|
||||
let text;
|
||||
try {
|
||||
client ??= clientFactory();
|
||||
text = rendererText(await client.discover({ waitMs: 300 }));
|
||||
} catch {
|
||||
text = RENDERER_COPY.unavailable;
|
||||
}
|
||||
if (check === latest && !scope.signal.aborted) status.textContent = text;
|
||||
};
|
||||
scope.cleanup(() => client?.dispose());
|
||||
// Another tab may open or switch ComfyUI meanwhile: look again when this page is shown again.
|
||||
scope.listen(document, 'visibilitychange', () => { if (document.visibilityState === 'visible') detect(); });
|
||||
detect();
|
||||
return surface;
|
||||
}
|
||||
|
||||
function bridgeFactory() {
|
||||
const url = new URL(window.ApiClient.resolveUrl('/prompt_manager/'), window.location.href);
|
||||
return () => createBridgeClient({ namespace: url.pathname.slice(0, -'prompt_manager/'.length), origin: window.location.origin });
|
||||
}
|
||||
|
||||
/**
|
||||
* Mount Integrations in `container` (System's `.settings-layout`); returns the section's two surfaces. `drafts` is the
|
||||
* System draft store (its host domain carries the LoRA values, system.mjs), `activity` the page's Activity store.
|
||||
*/
|
||||
export function mountIntegrations(container, { api, activity, scope, drafts, clientFactory = bridgeFactory(),
|
||||
notify = showWorkspaceToast, navigate = navigateTo }) {
|
||||
const comfy = comfySurface({ scope, clientFactory });
|
||||
const surface = el('section', undefined, 'surface');
|
||||
surface.append(el('h2', 'LoRA & resources'), el('p', 'Preserves LoraManager detection, data import and positive-only trigger injection.'));
|
||||
const enable = settingRow({ id: 'lora-enable', label: 'LoraManager integration', description: 'Checking for LoraManager…',
|
||||
control: { type: 'toggle', checked: false }, scope, onChange: value => drafts.set('host', 'lora_enabled', value) });
|
||||
const pathRow = settingRow({ id: 'lora-path', label: 'LoraManager path',
|
||||
description: 'Found automatically in ComfyUI’s custom_nodes folders; read-only, as on the existing settings page.',
|
||||
control: { type: 'text' } });
|
||||
const pathInput = controlOf(pathRow, 'lora-path');
|
||||
pathInput.readOnly = true;
|
||||
pathInput.placeholder = 'Not detected';
|
||||
// Wide enough for the usual install path within the concept's 230px control limit.
|
||||
pathInput.size = 34;
|
||||
const triggers = settingRow({ id: 'lora-triggers', label: 'Automatically include trigger words',
|
||||
description: 'Trigger words apply only to positive text; connected node inputs remain authoritative.',
|
||||
control: { type: 'toggle', checked: false }, scope, onChange: value => drafts.set('host', 'lora_trigger_words_enabled', value) });
|
||||
const key = settingRow({ id: 'civitai-key', label: 'CivitAI API key', description: 'Credentials stay in the server configuration.',
|
||||
control: { type: 'password' } });
|
||||
const keyInput = controlOf(key, 'civitai-key');
|
||||
// Never a login the browser offers to save or fill (S10 review N3).
|
||||
keyInput.autocomplete = 'new-password';
|
||||
keyInput.spellcheck = false;
|
||||
keyInput.placeholder = 'Optional';
|
||||
// D114: the saved-key line is announced (a polite live region) and describes the field (S10 review M2).
|
||||
const keyNote = el('p', '', 'small');
|
||||
keyNote.id = 'civitai-key-status';
|
||||
keyNote.setAttribute('role', 'status');
|
||||
key.children[0].append(keyNote);
|
||||
keyInput.setAttribute('aria-describedby', keyNote.id);
|
||||
const clear = createButton('Clear saved key');
|
||||
clear.dataset.setting = 'clear_civitai_api_key';
|
||||
clear.dataset.key = 'clear_civitai_api_key';
|
||||
const keyActions = el('div', undefined, 'row-actions');
|
||||
keyActions.append(clear);
|
||||
key.children[0].append(keyActions);
|
||||
scope.listen(keyInput, 'input', () => {
|
||||
// D114: a blank field keeps the saved key: it is no draft at all.
|
||||
drafts.set('host', 'civitai_api_key', keyInput.value.trim() ? keyInput.value : undefined);
|
||||
if (keyInput.value.trim()) drafts.set('host', 'clear_civitai_api_key', false);
|
||||
});
|
||||
scope.listen(clear, 'click', () => {
|
||||
keyInput.value = '';
|
||||
// The button leaves with the saved key: keep the keyboard on the field it belongs to (S10 review M2).
|
||||
keyInput.focus();
|
||||
drafts.set('host', 'civitai_api_key', undefined);
|
||||
drafts.set('host', 'clear_civitai_api_key', true);
|
||||
});
|
||||
const importRow = operationRow({ scope, title: 'LoRA reference data', description: 'Review and import resource metadata.',
|
||||
label: 'Review import', onClick: trigger => reviewImport({ api, store: activity, scope, trigger, notify, navigate }) });
|
||||
surface.append(enable, pathRow, triggers, key, importRow);
|
||||
|
||||
const controls = [[enable, 'lora-enable', 'lora_enabled'], [triggers, 'lora-triggers', 'lora_trigger_words_enabled']]
|
||||
.map(([row, id, name]) => ({ row, name, input: controlOf(row, id) }));
|
||||
for (const control of controls) control.input.dataset.key = control.name;
|
||||
keyInput.dataset.key = 'civitai_api_key';
|
||||
|
||||
// The saved LoRA state as last painted: a save (or another tab's change read back) that changes it reads
|
||||
// `/lora/status` again, so detection, the key line and the trigger cache follow what the server now holds.
|
||||
let shown;
|
||||
const repaint = () => {
|
||||
const values = drafts.saved('host');
|
||||
const changes = drafts.changes('host');
|
||||
const unread = values.lora_enabled === undefined;
|
||||
const failed = drafts.hostState?.() === 'failed' || (drafts.hostState?.() === 'loaded' && unread);
|
||||
for (const { input, name } of controls) {
|
||||
const value = drafts.get('host', name);
|
||||
input.disabled = value === undefined;
|
||||
if (value !== undefined) input.checked = value === true;
|
||||
}
|
||||
enable.children[0].children[1].textContent = failed && unread ? HOST_UNAVAILABLE
|
||||
: detectionText(values) ?? 'Checking for LoraManager…';
|
||||
pathInput.value = values.lora_path || values.lora_detected_path || '';
|
||||
setNote(triggers, cacheText(values));
|
||||
keyInput.disabled = unread;
|
||||
// A live region: rewritten only when it says something new, so it is not re-announced on every edit.
|
||||
const note = keyText(values, changes) ?? '';
|
||||
if (keyNote.textContent !== note) keyNote.textContent = note;
|
||||
// D114: "Clear saved key" is offered only while a saved key would stay.
|
||||
if (values.lora_has_key === true && changes.clear_civitai_api_key !== true) key.children[0].append(keyActions);
|
||||
else {
|
||||
if (keyActions.contains(document.activeElement)) keyInput.focus();
|
||||
keyActions.remove();
|
||||
}
|
||||
const state = JSON.stringify([values.lora_enabled, values.lora_trigger_words_enabled, values.civitai_api_key,
|
||||
values.clear_civitai_api_key]);
|
||||
const saved = values.civitai_api_key !== undefined || values.clear_civitai_api_key === true;
|
||||
if (!unread && shown !== undefined && (state !== shown || saved)) {
|
||||
if (saved) keyInput.value = '';
|
||||
shown = undefined;
|
||||
drafts.reloadHost();
|
||||
return;
|
||||
}
|
||||
if (!unread) shown = state;
|
||||
};
|
||||
scope.cleanup(drafts.subscribe(repaint));
|
||||
// A typed key is held only while its field shows it: leaving the section drops it, since the field comes back
|
||||
// empty and a hidden key draft could not be seen or discarded (S10 review N4). Only a typed key: any set clears
|
||||
// the last Save's outcome, which Sources still needs (S10 re-review R1).
|
||||
scope.cleanup(() => {
|
||||
if (drafts.changes('host').civitai_api_key !== undefined) drafts.set('host', 'civitai_api_key', undefined);
|
||||
});
|
||||
repaint();
|
||||
return [comfy, surface];
|
||||
}
|
||||
@@ -19,6 +19,8 @@
|
||||
* 'settings-changed'` when another page changed a drafted value (a conflict: the browser values are read again),
|
||||
* `writeHost(changes, {expected_revision})` the settings response (`{success, persisted, revision}`, and the
|
||||
* `restart_required_fields` the status names), `writeRuntime(changes)` anything; a write that throws failed. A host
|
||||
* write that saved part of its changes throws with `partial: {saved: [keys], revision}`: those keys are saved, the
|
||||
* revision is adopted and only the rest fail (`partial`, S10). A host
|
||||
* write refused as `invalid-source` (its error's `source` and server message) makes that message the status and
|
||||
* `invalidSource()` that source, so Sources marks its row (S4).
|
||||
*/
|
||||
@@ -32,6 +34,8 @@ export const SETTINGS_STATUS = Object.freeze({
|
||||
host: 'Saved to the server configuration.',
|
||||
runtime: 'Applied until the server restarts.',
|
||||
partial: 'Some changes were not saved. Review the highlighted settings.',
|
||||
// S10 review I3: a host save written in two parts where only one was saved.
|
||||
partialSaved: 'Some changes were saved to the server configuration; the highlighted ones were not.',
|
||||
tabOnly: 'Saved for this tab only; browser storage is unavailable.',
|
||||
conflict: 'Settings changed elsewhere. Reload this section to see them.',
|
||||
// D128: a saved setting the running server applies only after a restart.
|
||||
@@ -55,6 +59,7 @@ export function statusText(outcome) {
|
||||
const results = DRAFT_DOMAINS.map(domain => outcome?.[domain]);
|
||||
if (results.includes('conflict')) return SETTINGS_STATUS.conflict;
|
||||
if (results.includes('failed')) return outcome?.invalid?.message || SETTINGS_STATUS.partial;
|
||||
if (results.includes('partial')) return outcome?.invalid?.message || SETTINGS_STATUS.partialSaved;
|
||||
const saved = [];
|
||||
if (outcome?.browser === 'saved') saved.push(SETTINGS_STATUS.browser);
|
||||
if (outcome?.browser === 'tab-only') saved.push(SETTINGS_STATUS.tabOnly);
|
||||
@@ -177,10 +182,21 @@ export function createSystemDrafts(adapters) {
|
||||
commit(domain, written);
|
||||
return 'applied';
|
||||
} catch (error) {
|
||||
fail(domain, written);
|
||||
if (domain === 'host' && error?.code === INVALID_SOURCE_CODE && typeof error.message === 'string') {
|
||||
hostDetail.invalid = { source: typeof error.source === 'string' ? error.source : null, message: error.message };
|
||||
}
|
||||
const partial = domain === 'host' ? error?.partial : null;
|
||||
if (partial && Array.isArray(partial.saved)) {
|
||||
// S10 review I3: part of the host save was written (config.json moved on): keep what saved, adopt the
|
||||
// revision it produced, and fail only the rest, so the next save is not a false conflict.
|
||||
if (typeof partial.revision === 'string') revision = partial.revision;
|
||||
const saved = new Map([...written].filter(([key]) => partial.saved.includes(key)));
|
||||
commit(domain, saved);
|
||||
fail(domain, new Map([...written].filter(([key]) => !saved.has(key))));
|
||||
if (error?.code === CONFLICT_CODE) return 'conflict';
|
||||
return saved.size ? 'partial' : 'failed';
|
||||
}
|
||||
fail(domain, written);
|
||||
return error?.code === CONFLICT_CODE ? 'conflict' : 'failed';
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,7 +5,8 @@
|
||||
* (`span.small#settings-status` + "Save preferences"). D92: no "Sample configuration" chip, the status copy is
|
||||
* truthful and "Save preferences" is aria-disabled while nothing is unsaved. A section not delivered yet shows the
|
||||
* per-section hand-off to its legacy tool (D94), never mock placeholders. Overview (system-overview.mjs),
|
||||
* Appearance (system-appearance.mjs), Sources & scans (system-sources.mjs) and Maintenance (system-maintenance.mjs) are native.
|
||||
* Appearance (system-appearance.mjs), Sources & scans (system-sources.mjs), Integrations (system-integrations.mjs) and
|
||||
* Maintenance (system-maintenance.mjs) are native.
|
||||
*
|
||||
* Switching sections keeps the shell (and focus on the pressed button) and disposes the old section's scope, so its
|
||||
* listeners and timers never outlive it. The drafts are module-level (Q10): they live for the page session.
|
||||
@@ -18,6 +19,7 @@ import { mountOverview } from './system-overview.mjs';
|
||||
import { mountAppearance } from './system-appearance.mjs';
|
||||
import { mountSources } from './system-sources.mjs';
|
||||
import { mountMaintenance } from './system-maintenance.mjs';
|
||||
import { LORA_ENABLE_URL, LORA_STATUS_URL, loraValues, mountIntegrations, writeHostInParts } from './system-integrations.mjs';
|
||||
import { busyToast } from './activity-scan.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
import { displayPreferences, readPreferences, reconcilePreferences, subscribePreferences, writePreferences } from './preferences.mjs';
|
||||
@@ -30,7 +32,6 @@ const ADMIN = Object.freeze({ href: '/prompt_manager/admin', label: 'Open existi
|
||||
* settings, thumbnails, duplicates and cache tools.
|
||||
*/
|
||||
export const SECTION_HANDOFFS = Object.freeze({
|
||||
integrations: [ADMIN],
|
||||
tagging: [ADMIN],
|
||||
backups: [ADMIN],
|
||||
diagnostics: [ADMIN],
|
||||
@@ -77,14 +78,21 @@ export function systemDrafts(api) {
|
||||
},
|
||||
writeRuntime: undelivered,
|
||||
async readHost() {
|
||||
const result = await api.get(SETTINGS_URL, { cache: 'no-store' });
|
||||
// S10: the LoRA state rides along; when it cannot be read its values stay missing (Integrations says so)
|
||||
// and every other host setting still loads.
|
||||
const [result, lora] = await Promise.all([api.get(SETTINGS_URL, { cache: 'no-store' }),
|
||||
api.get(LORA_STATUS_URL, { cache: 'no-store' }).catch(() => null)]);
|
||||
// The store reads a missing value as none and a missing revision as unreadable (no host save). #317: which
|
||||
// thumbnail formats the server can write rides along as a read-only value (never drafted or sent).
|
||||
return { values: { ...result.settings, thumbnail_encoders: result.thumbnail_encoders }, revision: result.revision };
|
||||
return { values: { ...result.settings, thumbnail_encoders: result.thumbnail_encoders, ...(lora ? loraValues(lora) : {}) },
|
||||
revision: result.revision };
|
||||
},
|
||||
async writeHost(changes, { expected_revision }) {
|
||||
// S10: the LoRA keys save through /lora/enable, the rest through /settings, each with the current revision;
|
||||
// a save where only one part was written reports which (writeHostInParts, review I3).
|
||||
try {
|
||||
return await api.post(SETTINGS_URL, { ...changes, expected_revision });
|
||||
return await writeHostInParts({ changes, saved: drafts.saved('host'), expected_revision,
|
||||
postLora: body => api.post(LORA_ENABLE_URL, body), postSettings: body => api.post(SETTINGS_URL, body) });
|
||||
} catch (error) {
|
||||
// A sources change waits for a running record-changing job (S4): name it, as a refused start does (D57).
|
||||
if (error?.code === 'busy') showWorkspaceToast(busyToast(await runningJob(error)));
|
||||
@@ -119,13 +127,14 @@ function markFailed(layout, store) {
|
||||
}
|
||||
|
||||
/**
|
||||
* The sections delivered so far: Overview (S2 Task 6), Appearance (S3 Task 8), Sources & scans (S4 Task 11) and
|
||||
* Maintenance (S7 Task 19).
|
||||
* The sections delivered so far: Overview (S2 Task 6), Appearance (S3 Task 8), Sources & scans (S4 Task 11),
|
||||
* Integrations (S10 Task 27) and Maintenance (S7 Task 19).
|
||||
*/
|
||||
export const DELIVERED_SECTIONS = Object.freeze({
|
||||
overview: mountOverview,
|
||||
appearance: mountAppearance,
|
||||
sources: mountSources,
|
||||
integrations: mountIntegrations,
|
||||
maintenance: mountMaintenance,
|
||||
});
|
||||
|
||||
|
||||
@@ -423,3 +423,9 @@ pre.prompt-text { font: inherit; font-size: 12px; line-height: 1.75; margin-bloc
|
||||
/* D101: a duplicate copy's Keep label is its source and path, often long names with no space to break at; it wraps
|
||||
inside its `.compare-item` so the review never scrolls sideways. */
|
||||
.duplicate-groups .compare-item label { overflow-wrap: anywhere; /* D101 */ }
|
||||
/* D106: Integrations' two selects keep their chosen option readable: they never shrink for the row's reason lines, and
|
||||
at phone width they may grow past the concept's 120px cap (which cuts "Follow host renderer") up to 160px. */
|
||||
.setting-row :is(#send-destination, #renderer) { flex-shrink: 0; /* D106 */ }
|
||||
@media (max-width: 700px) {
|
||||
#workspace-main .setting-row :is(#send-destination, #renderer) { max-width: 160px !important; /* D106 */ }
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user