* feat(activity): shared session job registry with the scan as its first kind The output scan now runs as the first kind of an in-memory JobRegistry (py/api/jobs.py, replacing py/api/scan_job.py). Each job keeps exact counters, at most 200 outcome rows per group and a log capped at 2,000 lines, with root-relative paths and public root labels only. New routes list, inspect, follow (SSE) and cancel jobs and start the scan (POST /prompt_manager/jobs/scan, 409 busy while another record-changing job runs). The legacy /prompt_manager/scan routes attach to the same job and stream unchanged payloads. * feat(activity): concept Activity page, top-bar button and progress pill The workspace gains the concept's Activity page (renderActivity), its top-bar button and sidebar row, fed by one session job store: a /jobs read, one SSE stream per running job, a 1-30 s capped backoff after a dropped stream and a 15 s list poll while visible. A job the server no longer lists shows as ended, so no pill stays stuck after a restart. D52: the counts show running jobs and hide at 0; the phone top-bar row may wrap. D53: the mockup's progress pill links the running job. D54: real job items with state chips, detail lines and Cancel job. Start and finish toasts follow the spec copy. Fake browser jobs come from tests/browser/workspace_activity_fixture.py. * feat(activity): Scan for metadata with its review dialog and job view Images -> Import keeps Rescan and adds the concept's "Review scan" row as "Scan for metadata". It opens the concept's "Review metadata scan" with the real source count, the fixed "All files" scope, a backup box that downloads the existing database backup, and Cancel / Start scan. Start scan starts the session scan job and opens its view; a start refused by another record-changing job toasts its title as a link. #/activity/<id> is now the approved job-view mockup: the heading with Download report (disabled until Activity-2) and Cancel scan, the Progress surface with the five counter tiles, the Results surface with the outcome list, path filter and capped rows (Added and Already known open the viewer through the listing identity), and the collapsed Scan log. Activity item titles link their views. The scan's progress line groups thousands. Deviations D55-D57 are recorded in TESTING.md. * fix(activity): stop the job store on pagehide and let proxied routes outlive teardown CI's touch coverage run failed "a resolver failure for the ComfyUI link never stops the shell from mounting" with "route.fulfill: Fetch response has been disposed": its trace shows two Library thumbnail requests started by the last route still in the spec's proxy-mount handler while the page fixture was torn down. - The proxy-mount handler ignores only that teardown error, and the workspace and Activity specs unroute with ignoreErrors before teardown. - The Activity store now ends its work on pagehide: the poll and backoff timers stop, the job streams close and a list read in flight is aborted (an older read is also aborted when a newer one starts). A pageshow from the back-forward cache reads the list again. * test(activity): compare the job view log's line height to the hundredth of a pixel WebKit reports the mockup's 11px/1.7 line height as 18.700001px, so the job view comparison failed on WebKit only. The measurement now rounds it, as other engine-dependent values are normalized. * fix(activity): final review round for the job store, job view and scan rows - Store: one failed list read no longer marks running jobs as ended; two in a row, or a good list without them, do (M1). A good read resets the backoff, except for a stream that keeps dropping while the list reads fine (M2). Finished jobs the server pruned are dropped (M3). Numbers group as en-US, like the server's lines (M13). - Job view: a job the store's list no longer has shows the not-found state (M3). Focus moves to the job heading when Cancel scan goes away, and to the item's title link in the list (M4). The detail is read only when its listed rows change or the open log grows, using the summary's new listed and log_length (M12). - Viewer rows open by exact identity: each scan row records its root's listing index, the source must still carry the row's label, and the workspace listing gains an exact `path` filter (M8, M9). - Scan rows name the prompt by its Library title, stored title first (M10). - A toast with a link stays 20 s, and any toast stays while hovered or focused (M14). - A real two-batch scan cancelled after batch 1 keeps partial counts (M7). - TESTING.md: D54, D56 and D57 updated (M5, M6). * test(activity): cover the held and linked toast and a job view pruned by the server Unit tests for the toast's 6.5 s and 20 s timeouts and its hover and focus hold, and a browser test for a finished job's open view after the server stops listing it. Coverage floors are only raised.
177 lines
7.5 KiB
Python
177 lines
7.5 KiB
Python
"""Metadata scanning enriches known pairs and conservatively leaves conflicts."""
|
|
|
|
import json
|
|
from pathlib import Path
|
|
|
|
from tests.test_api_admin import AdminAPITestCase
|
|
|
|
|
|
def graph(positive="cat", negative="blur"):
|
|
return {
|
|
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": positive}},
|
|
"2": {"class_type": "CLIPTextEncode", "inputs": {"text": negative}},
|
|
"3": {
|
|
"class_type": "KSampler",
|
|
"inputs": {"positive": ["1", 0], "negative": ["2", 0]},
|
|
},
|
|
}
|
|
|
|
|
|
def sampled_graph(positive="cat", negative="blur"):
|
|
"""A KSampler with a model link, as real workflows have."""
|
|
g = graph(positive, negative)
|
|
g["4"] = {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}}
|
|
g["3"]["inputs"]["model"] = ["4", 0]
|
|
return g
|
|
|
|
|
|
class TestPairScan(AdminAPITestCase):
|
|
def test_pair_scan_repeated_is_unchanged_and_empty_negative_is_supported(self):
|
|
path = self.output_dir / "pair.png"
|
|
meta = {"prompt": json.dumps(graph(negative=""))}
|
|
self.assertEqual(self.api._ingest_scanned_file(path, meta), "added")
|
|
self.assertEqual(self.api._ingest_scanned_file(path, meta), "unchanged")
|
|
pairs = self.api.db.list_prompt_pairs()["pairs"]
|
|
self.assertEqual(len(pairs), 1)
|
|
self.assertEqual(pairs[0]["negative_text"], "")
|
|
self.assertEqual(pairs[0]["run_count"], 0)
|
|
|
|
def _scanned(self, name, prompt_graph):
|
|
meta = {"prompt": json.dumps(prompt_graph)}
|
|
return self.api._ingest_scanned_file(self.output_dir / name, meta)
|
|
|
|
def _texts(self):
|
|
return [p["text"] for p in self.api.db.get_recent_prompts()["prompts"]]
|
|
|
|
def test_samplers_sharing_one_prompt_link_that_prompt(self):
|
|
# #249: every branch resolves to one exact pair, so the pair is ingested.
|
|
g = sampled_graph()
|
|
g["5"] = {**g["3"]}
|
|
self.assertEqual(self._scanned("multi.png", g), "added")
|
|
self.assertEqual(sorted(self._texts()), ["blur", "cat"])
|
|
pairs = self.api.db.list_prompt_pairs()["pairs"]
|
|
self.assertEqual(
|
|
[(p["positive_text"], p["negative_text"]) for p in pairs], [("cat", "blur")]
|
|
)
|
|
|
|
def test_samplers_with_different_prompts_link_the_first_sampler_prompt(self):
|
|
# The prompt the inspector and the legacy panels show for this file.
|
|
g = sampled_graph()
|
|
g["6"] = {"class_type": "CLIPTextEncode", "inputs": {"text": "dog"}}
|
|
g["12"] = {**g["3"], "inputs": {**g["3"]["inputs"], "positive": ["6", 0]}}
|
|
self.assertEqual(self._scanned("different.png", g), "added")
|
|
self.assertEqual(self._texts(), ["cat"])
|
|
|
|
def test_an_unsupported_pass_through_still_links_the_traced_prompt(self):
|
|
g = sampled_graph()
|
|
g["7"] = {
|
|
"class_type": "MyConditioningTweak",
|
|
"inputs": {"conditioning": ["1", 0]},
|
|
}
|
|
g["3"]["inputs"]["positive"] = ["7", 0]
|
|
g["5"] = {**g["3"]}
|
|
self.assertEqual(self._scanned("tweak.png", g), "added")
|
|
self.assertEqual(self._texts(), ["cat"])
|
|
|
|
def test_a_graph_without_a_traceable_prompt_saves_nothing(self):
|
|
g = sampled_graph()
|
|
g["5"] = {**g["3"]}
|
|
for sampler in ("3", "5"):
|
|
g[sampler]["inputs"]["positive"] = ["9", 0]
|
|
g["9"] = {
|
|
"class_type": "BasicGuider",
|
|
"inputs": {"width": 2048, "height": 2048},
|
|
}
|
|
self.assertEqual(self._scanned("untraceable.png", g), "ambiguous")
|
|
self.assertEqual(self.api.db.list_prompt_pairs()["total"], 0)
|
|
self.assertEqual(self._texts(), [])
|
|
|
|
def test_numeric_widgets_are_never_stored_as_a_prompt(self):
|
|
# The pre-redesign scan stored "2048" from a resolution widget here.
|
|
g = {
|
|
"1": {"class_type": "CLIPTextEncode", "inputs": {"text": ["8", 0]}},
|
|
"8": {"class_type": "DisplayText", "inputs": {"text_size": 2048}},
|
|
"3": {
|
|
"class_type": "SamplerCustomAdvanced",
|
|
"inputs": {"guider": ["4", 0], "sigmas": ["5", 0]},
|
|
},
|
|
"4": {"class_type": "BasicGuider", "inputs": {"conditioning": ["1", 0]}},
|
|
}
|
|
self.assertIn(
|
|
self._scanned("numbers.png", g), ("ambiguous", "unresolved", "found")
|
|
)
|
|
self.assertEqual(self._texts(), [])
|
|
|
|
def test_batch_exposes_pair_counts_and_actual_unchanged_status(self):
|
|
meta = {"parameters": "cat\nNegative prompt: blur\nSteps: 20, Sampler: Euler"}
|
|
batch = [(Path(self.output_dir / "a.png"), meta)]
|
|
first = self.api._ingest_scan_batch_sync(batch)
|
|
self.assertEqual(first["pairs_added"], 1)
|
|
self.assertEqual(first["pair_links_added"], 1)
|
|
second = self.api._ingest_scan_batch_sync(batch)
|
|
self.assertEqual(second["unchanged"], 1)
|
|
self.assertEqual(second["linked"], 0)
|
|
self.assertEqual(second["pairs_added"], 0)
|
|
|
|
def test_legacy_single_prompt_second_pass_not_reported_as_new_link(self):
|
|
meta = {"prompt": json.dumps({"1": graph()["1"]})}
|
|
path = self.output_dir / "old.png"
|
|
self.assertEqual(self.api._ingest_scanned_file(path, meta), "added")
|
|
self.assertEqual(self.api._ingest_scanned_file(path, meta), "unchanged")
|
|
|
|
async def test_cancel_finishes_at_safe_boundary_and_restart_is_idempotent(self):
|
|
from py.api.jobs import Job
|
|
|
|
job = Job("scan", "Scan")
|
|
self.api._scan_job = job
|
|
response = await self.client.post("/prompt_manager/scan/cancel")
|
|
self.assertEqual(response.status, 200)
|
|
self.assertTrue((await response.json())["cancellation_requested"])
|
|
counts = self.api._ingest_scan_batch_sync(
|
|
[(self.output_dir / "cancel.png", {"prompt": json.dumps(graph())})]
|
|
)
|
|
self.assertEqual(counts["processed"], 0)
|
|
self.assertEqual(self.api.db.list_prompt_pairs()["total"], 0)
|
|
job.finish()
|
|
self.api._scan_job = Job("scan", "Scan")
|
|
first = self.api._ingest_scan_batch_sync(
|
|
[(self.output_dir / "cancel.png", {"prompt": json.dumps(graph())})]
|
|
)
|
|
self.assertEqual(first["pairs_added"], 1)
|
|
self.assertEqual(
|
|
self.api._ingest_scan_batch_sync(
|
|
[(self.output_dir / "cancel.png", {"prompt": json.dumps(graph())})]
|
|
)["unchanged"],
|
|
1,
|
|
)
|
|
|
|
def test_legacy_found_outcome_is_not_counted_twice(self):
|
|
result = self.api._ingest_scan_batch_sync(
|
|
[(self.output_dir / "blank.png", {"parameters": "Negative prompt: blur"})]
|
|
)
|
|
self.assertEqual(result["processed"], 1)
|
|
self.assertEqual(result["found"], 1)
|
|
|
|
def test_ambiguous_a1111_does_not_fall_back_to_legacy_guess(self):
|
|
meta = {
|
|
"parameters": (
|
|
"cat\nNegative prompt: blur\nNegative prompt: noise"
|
|
"\nSteps: 20, Sampler: Euler"
|
|
)
|
|
}
|
|
self.assertEqual(
|
|
self.api._ingest_scanned_file(self.output_dir / "ambiguous.png", meta),
|
|
"ambiguous",
|
|
)
|
|
self.assertEqual(self.api.db.get_recent_prompts()["prompts"], [])
|
|
|
|
def test_raw_png_workflow_is_stored_as_object_and_repeat_is_unchanged(self):
|
|
path = self.output_dir / "workflow.png"
|
|
workflow = {"nodes": [{"id": 1, "type": "PromptManagerPair"}]}
|
|
metadata = {"prompt": json.dumps(graph()), "workflow": json.dumps(workflow)}
|
|
self.assertEqual(self.api._ingest_scanned_file(path, metadata), "added")
|
|
self.assertEqual(
|
|
self.api.db.get_image_prompt_info(str(path))["workflow_data"], workflow
|
|
)
|
|
self.assertEqual(self.api._ingest_scanned_file(path, metadata), "unchanged")
|