Files
ComfyAssets-ComfyUI_PromptM…/tests/test_pair_scan.py
Vito a6f6697c50 feat(activity): shared job registry, Activity page and Scan for metadata (#269)
* feat(activity): shared session job registry with the scan as its first kind

The output scan now runs as the first kind of an in-memory JobRegistry
(py/api/jobs.py, replacing py/api/scan_job.py). Each job keeps exact
counters, at most 200 outcome rows per group and a log capped at 2,000
lines, with root-relative paths and public root labels only. New routes
list, inspect, follow (SSE) and cancel jobs and start the scan
(POST /prompt_manager/jobs/scan, 409 busy while another record-changing
job runs). The legacy /prompt_manager/scan routes attach to the same job
and stream unchanged payloads.

* feat(activity): concept Activity page, top-bar button and progress pill

The workspace gains the concept's Activity page (renderActivity), its
top-bar button and sidebar row, fed by one session job store: a /jobs
read, one SSE stream per running job, a 1-30 s capped backoff after a
dropped stream and a 15 s list poll while visible. A job the server no
longer lists shows as ended, so no pill stays stuck after a restart.

D52: the counts show running jobs and hide at 0; the phone top-bar row
may wrap. D53: the mockup's progress pill links the running job. D54:
real job items with state chips, detail lines and Cancel job. Start and
finish toasts follow the spec copy. Fake browser jobs come from
tests/browser/workspace_activity_fixture.py.

* feat(activity): Scan for metadata with its review dialog and job view

Images -> Import keeps Rescan and adds the concept's "Review scan" row as
"Scan for metadata". It opens the concept's "Review metadata scan" with the
real source count, the fixed "All files" scope, a backup box that downloads
the existing database backup, and Cancel / Start scan. Start scan starts the
session scan job and opens its view; a start refused by another
record-changing job toasts its title as a link.

#/activity/<id> is now the approved job-view mockup: the heading with
Download report (disabled until Activity-2) and Cancel scan, the Progress
surface with the five counter tiles, the Results surface with the outcome
list, path filter and capped rows (Added and Already known open the viewer
through the listing identity), and the collapsed Scan log. Activity item
titles link their views. The scan's progress line groups thousands.

Deviations D55-D57 are recorded in TESTING.md.

* fix(activity): stop the job store on pagehide and let proxied routes outlive teardown

CI's touch coverage run failed "a resolver failure for the ComfyUI link never
stops the shell from mounting" with "route.fulfill: Fetch response has been
disposed": its trace shows two Library thumbnail requests started by the last
route still in the spec's proxy-mount handler while the page fixture was torn
down.

- The proxy-mount handler ignores only that teardown error, and the workspace
  and Activity specs unroute with ignoreErrors before teardown.
- The Activity store now ends its work on pagehide: the poll and backoff timers
  stop, the job streams close and a list read in flight is aborted (an older
  read is also aborted when a newer one starts). A pageshow from the
  back-forward cache reads the list again.

* test(activity): compare the job view log's line height to the hundredth of a pixel

WebKit reports the mockup's 11px/1.7 line height as 18.700001px, so the job view comparison failed on WebKit only. The measurement now rounds it, as other engine-dependent values are normalized.

* fix(activity): final review round for the job store, job view and scan rows

- Store: one failed list read no longer marks running jobs as ended; two in
  a row, or a good list without them, do (M1). A good read resets the
  backoff, except for a stream that keeps dropping while the list reads fine
  (M2). Finished jobs the server pruned are dropped (M3). Numbers group as
  en-US, like the server's lines (M13).
- Job view: a job the store's list no longer has shows the not-found state
  (M3). Focus moves to the job heading when Cancel scan goes away, and to the
  item's title link in the list (M4). The detail is read only when its listed
  rows change or the open log grows, using the summary's new listed and
  log_length (M12).
- Viewer rows open by exact identity: each scan row records its root's
  listing index, the source must still carry the row's label, and the
  workspace listing gains an exact `path` filter (M8, M9).
- Scan rows name the prompt by its Library title, stored title first (M10).
- A toast with a link stays 20 s, and any toast stays while hovered or
  focused (M14).
- A real two-batch scan cancelled after batch 1 keeps partial counts (M7).
- TESTING.md: D54, D56 and D57 updated (M5, M6).

* test(activity): cover the held and linked toast and a job view pruned by the server

Unit tests for the toast's 6.5 s and 20 s timeouts and its hover and focus hold, and a browser test for a finished job's open view after the server stops listing it. Coverage floors are only raised.
2026-10-21 14:22:30 -07:00

177 lines
7.5 KiB
Python

"""Metadata scanning enriches known pairs and conservatively leaves conflicts."""
import json
from pathlib import Path
from tests.test_api_admin import AdminAPITestCase
def graph(positive="cat", negative="blur"):
return {
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": positive}},
"2": {"class_type": "CLIPTextEncode", "inputs": {"text": negative}},
"3": {
"class_type": "KSampler",
"inputs": {"positive": ["1", 0], "negative": ["2", 0]},
},
}
def sampled_graph(positive="cat", negative="blur"):
"""A KSampler with a model link, as real workflows have."""
g = graph(positive, negative)
g["4"] = {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}}
g["3"]["inputs"]["model"] = ["4", 0]
return g
class TestPairScan(AdminAPITestCase):
def test_pair_scan_repeated_is_unchanged_and_empty_negative_is_supported(self):
path = self.output_dir / "pair.png"
meta = {"prompt": json.dumps(graph(negative=""))}
self.assertEqual(self.api._ingest_scanned_file(path, meta), "added")
self.assertEqual(self.api._ingest_scanned_file(path, meta), "unchanged")
pairs = self.api.db.list_prompt_pairs()["pairs"]
self.assertEqual(len(pairs), 1)
self.assertEqual(pairs[0]["negative_text"], "")
self.assertEqual(pairs[0]["run_count"], 0)
def _scanned(self, name, prompt_graph):
meta = {"prompt": json.dumps(prompt_graph)}
return self.api._ingest_scanned_file(self.output_dir / name, meta)
def _texts(self):
return [p["text"] for p in self.api.db.get_recent_prompts()["prompts"]]
def test_samplers_sharing_one_prompt_link_that_prompt(self):
# #249: every branch resolves to one exact pair, so the pair is ingested.
g = sampled_graph()
g["5"] = {**g["3"]}
self.assertEqual(self._scanned("multi.png", g), "added")
self.assertEqual(sorted(self._texts()), ["blur", "cat"])
pairs = self.api.db.list_prompt_pairs()["pairs"]
self.assertEqual(
[(p["positive_text"], p["negative_text"]) for p in pairs], [("cat", "blur")]
)
def test_samplers_with_different_prompts_link_the_first_sampler_prompt(self):
# The prompt the inspector and the legacy panels show for this file.
g = sampled_graph()
g["6"] = {"class_type": "CLIPTextEncode", "inputs": {"text": "dog"}}
g["12"] = {**g["3"], "inputs": {**g["3"]["inputs"], "positive": ["6", 0]}}
self.assertEqual(self._scanned("different.png", g), "added")
self.assertEqual(self._texts(), ["cat"])
def test_an_unsupported_pass_through_still_links_the_traced_prompt(self):
g = sampled_graph()
g["7"] = {
"class_type": "MyConditioningTweak",
"inputs": {"conditioning": ["1", 0]},
}
g["3"]["inputs"]["positive"] = ["7", 0]
g["5"] = {**g["3"]}
self.assertEqual(self._scanned("tweak.png", g), "added")
self.assertEqual(self._texts(), ["cat"])
def test_a_graph_without_a_traceable_prompt_saves_nothing(self):
g = sampled_graph()
g["5"] = {**g["3"]}
for sampler in ("3", "5"):
g[sampler]["inputs"]["positive"] = ["9", 0]
g["9"] = {
"class_type": "BasicGuider",
"inputs": {"width": 2048, "height": 2048},
}
self.assertEqual(self._scanned("untraceable.png", g), "ambiguous")
self.assertEqual(self.api.db.list_prompt_pairs()["total"], 0)
self.assertEqual(self._texts(), [])
def test_numeric_widgets_are_never_stored_as_a_prompt(self):
# The pre-redesign scan stored "2048" from a resolution widget here.
g = {
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": ["8", 0]}},
"8": {"class_type": "DisplayText", "inputs": {"text_size": 2048}},
"3": {
"class_type": "SamplerCustomAdvanced",
"inputs": {"guider": ["4", 0], "sigmas": ["5", 0]},
},
"4": {"class_type": "BasicGuider", "inputs": {"conditioning": ["1", 0]}},
}
self.assertIn(
self._scanned("numbers.png", g), ("ambiguous", "unresolved", "found")
)
self.assertEqual(self._texts(), [])
def test_batch_exposes_pair_counts_and_actual_unchanged_status(self):
meta = {"parameters": "cat\nNegative prompt: blur\nSteps: 20, Sampler: Euler"}
batch = [(Path(self.output_dir / "a.png"), meta)]
first = self.api._ingest_scan_batch_sync(batch)
self.assertEqual(first["pairs_added"], 1)
self.assertEqual(first["pair_links_added"], 1)
second = self.api._ingest_scan_batch_sync(batch)
self.assertEqual(second["unchanged"], 1)
self.assertEqual(second["linked"], 0)
self.assertEqual(second["pairs_added"], 0)
def test_legacy_single_prompt_second_pass_not_reported_as_new_link(self):
meta = {"prompt": json.dumps({"1": graph()["1"]})}
path = self.output_dir / "old.png"
self.assertEqual(self.api._ingest_scanned_file(path, meta), "added")
self.assertEqual(self.api._ingest_scanned_file(path, meta), "unchanged")
async def test_cancel_finishes_at_safe_boundary_and_restart_is_idempotent(self):
from py.api.jobs import Job
job = Job("scan", "Scan")
self.api._scan_job = job
response = await self.client.post("/prompt_manager/scan/cancel")
self.assertEqual(response.status, 200)
self.assertTrue((await response.json())["cancellation_requested"])
counts = self.api._ingest_scan_batch_sync(
[(self.output_dir / "cancel.png", {"prompt": json.dumps(graph())})]
)
self.assertEqual(counts["processed"], 0)
self.assertEqual(self.api.db.list_prompt_pairs()["total"], 0)
job.finish()
self.api._scan_job = Job("scan", "Scan")
first = self.api._ingest_scan_batch_sync(
[(self.output_dir / "cancel.png", {"prompt": json.dumps(graph())})]
)
self.assertEqual(first["pairs_added"], 1)
self.assertEqual(
self.api._ingest_scan_batch_sync(
[(self.output_dir / "cancel.png", {"prompt": json.dumps(graph())})]
)["unchanged"],
1,
)
def test_legacy_found_outcome_is_not_counted_twice(self):
result = self.api._ingest_scan_batch_sync(
[(self.output_dir / "blank.png", {"parameters": "Negative prompt: blur"})]
)
self.assertEqual(result["processed"], 1)
self.assertEqual(result["found"], 1)
def test_ambiguous_a1111_does_not_fall_back_to_legacy_guess(self):
meta = {
"parameters": (
"cat\nNegative prompt: blur\nNegative prompt: noise"
"\nSteps: 20, Sampler: Euler"
)
}
self.assertEqual(
self.api._ingest_scanned_file(self.output_dir / "ambiguous.png", meta),
"ambiguous",
)
self.assertEqual(self.api.db.get_recent_prompts()["prompts"], [])
def test_raw_png_workflow_is_stored_as_object_and_repeat_is_unchanged(self):
path = self.output_dir / "workflow.png"
workflow = {"nodes": [{"id": 1, "type": "PromptManagerPair"}]}
metadata = {"prompt": json.dumps(graph()), "workflow": json.dumps(workflow)}
self.assertEqual(self.api._ingest_scanned_file(path, metadata), "added")
self.assertEqual(
self.api.db.get_image_prompt_info(str(path))["workflow_data"], workflow
)
self.assertEqual(self.api._ingest_scanned_file(path, metadata), "unchanged")