Files
ComfyAssets-ComfyUI_PromptM…/tests/browser/workspace_integrations_fixture.py
T
Vito 8252d9d226 feat(system): Integrations and a staged LoRA import (System S10) (#336)
* fix(lora): detect LoraManager in ComfyUI's extra custom_nodes folders

Auto-detection looked only in ComfyUI's main custom_nodes folder (#144).
It now also searches every custom_nodes folder ComfyUI registers through
folder_paths, main folder first, skipping folders that cannot be read.

* fix(lora): stage imports before replacing the collection (#221)

The LoRA import deleted the whole lora-manager category first and then
saved each record in its own transaction, so a discovery, download or
database failure left the previous collection gone or partial.

The import now runs in three stages: discover and fingerprint the
metadata files, stage every record (CivitAI downloads go into the image
cache first, counted), then replace the category in one BEGIN IMMEDIATE
transaction (replace_category_prompts). Any failure leaves the previous
prompts, tags and image links exactly as they were. Prompts a saved pair
uses block the replacement; an empty replacement of a non-empty
collection needs a reviewed import.

POST /prompt_manager/lora/import/preview reviews an import without
writing or requesting anything; POST /lora/import/apply {token} runs it
as the Activity job "LoRA reference import" (records admission,
cancellable while it downloads, the safety backup policy before the
replace, refused with plan-changed when the files changed since the
review). The legacy POST /lora/scan runs the same stages with its SSE
event shape.

* feat(lora): the LoRA save takes the settings revision

POST /prompt_manager/lora/enable accepts the optional expected_revision
that GET /settings returns, so the System Save row refuses to write over
a config.json changed elsewhere (settings-changed, nothing written). The
legacy page sends none and still merges into what is on disk.

* feat(system): configure ComfyUI and LoRA integrations natively

System › Integrations replaces its hand-off with the concept's two
surfaces. ComfyUI: only the send destination and renderer the bridge
supports can be chosen; the other options and Return to canvas are
disabled with the reason, and Host compatibility shows the renderer the
open ComfyUI windows report (each bridge destination now carries
renderer: classic or nodes2) (D106). LoRA & resources: detection, the
enabled setting and the loaded trigger words read as separate facts,
the read-only LoraManager path row follows the toggle (D106), and the
CivitAI key is never filled in: typing drafts a new key, blank keeps the
saved one and Clear saved key drafts its removal (D114). They save
through the one Save row (/lora/enable with the settings revision).

LoRA reference data → Review import shows the server's read-only review
(replacement, what saved pairs block, downloads, backup box) and Start
runs the LoRA reference import Activity job (D107), with its own
Activity descriptor and Review again into Integrations when the files or
pairs overtook it.

* test(system): compare Integrations with the concept

system-integrations-concept.spec compares System › Integrations (both
themes, 1600 and 390 wide) with the concept, mapping only D106 and D114,
and the LoRA import review with the concept's record-changing review,
mapping D107; each comparison catches a removed row, a renamed label or
option, a re-enabled unsupported option and a swapped token.

On the real ComfyUI host, each bridge destination reports the renderer
its window draws nodes with, before and after switching it, and System ›
Integrations shows it.

* test(coverage): record the Integrations frontend floors

coverage:update after the full frontend run: system-integrations.mjs at
its measured floor, and the raised maintenance review, job view and
maintenance floors.

* test(system): wait for the renderer line before clicking Integrations rows

CI Firefox (390 wide) clicked the LoraManager toggle while the renderer
check's line above it changed text and moved the rows, so the click did
not reach the switch. The specs now wait for that line to settle (a
visible state) before interacting.

* fix(lora): recheck the files before the replace and stop an abandoned scan

Review fixes for the staged LoRA import (#221):

- The metadata files are listed and fingerprinted again right before the
  replace transaction, so a file changed, removed or added while the
  downloads ran aborts the import with nothing deleted (System job:
  plan-changed, Review again; legacy scan: an error frame in its own
  words). The tests now change real files behind a held download.
- A legacy scan whose page closes stops staging before its next LoRA and
  keeps its records hold until staging has stopped; CivitAI images are
  written to a temp file beside the cache entry and moved in place, so a
  half-written image is never seen as cached.
- Start refuses with plan-changed when the integration was turned off or
  moved to another install since the review.
- A failed job keeps the skipped count and the names of the LoRAs that
  could not be prepared.
- The replace binds the category instead of one parameter per prompt
  (SQLite's variable limit), /lora/status says whether a detected install
  can be used, and the legacy scan logs expected refusals as warnings.

* fix(system): partial saves, key focus and readable selects in Integrations

A save that writes the LoRA part but not the settings part now keeps the
saved part, adopts the new revision and highlights only what failed, so
the next save needs no conflict. The CivitAI key status is a persistent
live region, Clear saved key returns focus to the key field, and the key
draft is dropped when the section is left. The send-destination and
renderer selects keep their width on narrow screens. A detected
LoraManager outside custom_nodes says it cannot be enabled.

* fix(lora): keep the save outcome, honour the umask and sweep stale downloads

Leaving Integrations drops the CivitAI key draft only when a key was
typed, so the last Save's outcome survives and Sources still marks a
refused source. The key status line is rewritten only when it changes.

Cached CivitAI images are opened like a plain save, so they keep the
umask's mode, and each import removes .pm-*.part files older than an
hour from the image cache. A legacy scan stopped because its page left
ends cancelled in Activity, and staging is awaited to the end even if
the handler is cancelled again.

* fix(lora): the import review takes a lease beside a restore

The LoRA import review reads the library, so like the maintenance and
restore previews it now runs under a request lease: it is refused with
409 busy while a restore runs, and a restore waits for it before it
switches the file. Tests pin that a restore is refused while an import
runs and that the review, the reviewed import and the admin page's scan
are refused while a restore runs.
2026-10-25 10:50:48 -07:00

199 lines
7.3 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""System › Integrations test hooks for the browser tests (#249 System S10).
A fake LoraManager install lives under this server's temp root; nothing ever
reaches a real LoraManager, ComfyUI install or the network:
- ``POST /__test__/integrations/reset`` turns the integration off (no path,
no key, no trigger words), removes the fake install, its LoRA folder and
the image cache, forgets every LoRA import and injected failure, and opens
the download gate;
- ``POST /__test__/integrations/lora-manager`` ``{loras, civitai, previous}``
writes ``custom_nodes/ComfyUI-Lora-Manager`` (detected through the
``folder_paths`` stub's base path) and ``loras`` metadata files under
``models/loras``, each with a local preview PNG; ``civitai`` adds one
CivitAI example image per LoRA (served by the stub opener below);
``previous`` saves that many earlier ``lora-manager`` imports with tags and
image links;
- ``POST /__test__/integrations/gate`` ``{hold}`` holds every CivitAI
download until it is released (``{"hold": false}``), so a test sees the job
while it downloads;
- ``POST /__test__/integrations/fail-import`` makes the next collection
replacement fail (a database error after staging);
- ``POST /__test__/integrations/pair`` saves a pair whose positive is the first
imported prompt (a saved pair that blocks the replacement);
- ``GET /__test__/integrations/state`` answers the imported prompt texts,
whether a CivitAI key is saved and the downloads served.
CivitAI downloads are always answered by a stub opener (a small PNG), so no
test ever opens a network connection.
"""
import io
import json
import shutil
import threading
from pathlib import Path
from aiohttp import web
from PIL import Image
MAX_LORAS = 20
CATEGORY = "lora-manager"
def _png():
buffer = io.BytesIO()
Image.new("RGB", (8, 8), "#3a6ea5").save(buffer, "PNG")
return buffer.getvalue()
class _Response:
def __init__(self, body):
self._body = io.BytesIO(body)
self.headers = {"Content-Length": str(len(body))}
def read(self, amount=-1):
return self._body.read(amount)
def __enter__(self):
return self
def __exit__(self, *exc):
return False
def register_integrations_commands(app, routes, api, root: Path):
from py import lora_utils
from py.config import IntegrationConfig
from utils.hashing import generate_prompt_hash
lora_manager = root / "custom_nodes" / "ComfyUI-Lora-Manager"
loras = root / "models" / "loras"
gate = threading.Event()
gate.set()
served = []
failures = {"replace": 0}
pairs = [] # the blocking pairs this fixture saved
real_replace = type(api.db).replace_category_prompts
def stub_open(request, timeout=10):
served.append(request.full_url)
gate.wait(30)
return _Response(_png())
lora_utils._open_url = stub_open
def replace(db, *args, **kwargs):
if failures["replace"] > 0:
failures["replace"] -= 1
raise RuntimeError("Injected replacement failure")
return real_replace(db, *args, **kwargs)
type(api.db).replace_category_prompts = replace
def reset():
IntegrationConfig.LORA_MANAGER_ENABLED = False
IntegrationConfig.LORA_MANAGER_PATH = ""
IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED = False
IntegrationConfig.CIVITAI_API_KEY = ""
lora_utils.get_trigger_cache().clear()
gate.set()
served.clear()
failures["replace"] = 0
cache = Path(lora_utils.get_lora_image_cache_dir(create=False))
for folder in (lora_manager, loras, cache):
shutil.rmtree(folder, ignore_errors=True)
for pair_id in pairs:
api.db.delete_prompt_pair(pair_id)
pairs.clear()
real_replace(api.db, CATEGORY, [], allow_empty=True)
@routes.post("/__test__/integrations/reset")
async def integrations_reset(request):
reset()
return web.json_response({"success": True})
@routes.post("/__test__/integrations/lora-manager")
async def make_lora_manager(request):
data = await request.json() if request.can_read_body else {}
count = data.get("loras", 2)
previous = data.get("previous", 0)
for value in (count, previous):
if isinstance(value, bool) or not isinstance(value, int):
raise web.HTTPBadRequest()
if not 0 <= count <= MAX_LORAS or not 0 <= previous <= MAX_LORAS:
raise web.HTTPBadRequest()
(lora_manager / "py").mkdir(parents=True, exist_ok=True)
(lora_manager / "__init__.py").write_text("")
loras.mkdir(parents=True, exist_ok=True)
for number in range(count):
stem = f"style_{number + 1:02}"
civitai = {"trainedWords": [f"trigger{number + 1}"]}
if data.get("civitai") is True:
civitai["images"] = [
{
"url": f"https://image.civitai.com/x/{stem}.jpeg",
"meta": {"prompt": f"{stem} example prompt"},
}
]
meta = {
"file_name": f"{stem}.safetensors",
"model_name": f"Style {number + 1:02}",
"civitai": civitai,
}
(loras / f"{stem}.metadata.json").write_text(json.dumps(meta))
(loras / f"{stem}.png").write_bytes(_png())
for number in range(previous):
text = f"previous lora import {number + 1}"
prompt_id = api.db.save_prompt(
text=text,
category=CATEGORY,
tags=[CATEGORY, f"lora:previous-{number + 1}"],
prompt_hash=generate_prompt_hash(text),
)
api.db.link_image_to_prompt(str(prompt_id), str(loras / "old.png"))
return web.json_response({"success": True})
@routes.post("/__test__/integrations/gate")
async def download_gate(request):
data = await request.json()
if not isinstance(data.get("hold"), bool):
raise web.HTTPBadRequest()
if data["hold"]:
gate.clear()
else:
gate.set()
return web.json_response({"success": True})
@routes.post("/__test__/integrations/fail-import")
async def fail_import(request):
failures["replace"] = 1
return web.json_response({"success": True})
@routes.post("/__test__/integrations/pair")
async def pair(request):
with api.db.model.get_connection() as conn:
row = conn.execute(
"SELECT text FROM prompts WHERE category = ? ORDER BY id LIMIT 1",
(CATEGORY,),
).fetchone()
if row is None:
raise web.HTTPConflict()
pairs.append(api.db.save_prompt_pair(row[0], "blurry"))
return web.json_response({"success": True})
@routes.get("/__test__/integrations/state")
async def state(request):
imports = api.db.search_prompts(category=CATEGORY, limit=MAX_LORAS * 2)
return web.json_response(
{
"imports": sorted(prompt["text"] for prompt in imports),
"key_saved": bool(IntegrationConfig.CIVITAI_API_KEY),
"enabled": IntegrationConfig.LORA_MANAGER_ENABLED,
"trigger_words": IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED,
"downloads": len(served),
}
)
return reset