feat(maintenance): repair old Pair prompts' empty category and tags (#550)
Squashed from PR #579. Closes #550.
This commit is contained in:
Vito Sansevero
+49
-1
@@ -2978,7 +2978,8 @@ change: ComfyUI needs a restart.
|
||||
A Pair save, by a run or the node's Save pair button, writes the category and tags to the positive prompt only when that
|
||||
save creates the prompt row (maintainer ruling). An existing prompt never changes on a run: empty, cleared or set by a person, including one first seen as
|
||||
another pair's negative. The negative stays empty, the saved pair keeps its own metadata, and `save_prompt_pair` writes
|
||||
no prompt metadata unless called with `fill_new_positive`. No backfill; optional repair is #550.
|
||||
no prompt metadata unless called with `fill_new_positive`. No backfill on a run; the opt-in repair for prompts older
|
||||
Pair runs saved is System › Maintenance › "Pair category and tags" (#550, "Pair category and tags repair" below).
|
||||
Tests: `tests/test_prompt_pair_node.py`, `tests/test_api_pairs.py`, `tests/comfyui/pair-node.spec.mjs`. A Python change: ComfyUI needs a restart.
|
||||
|
||||
### Delete… on the archived shelf (#492)
|
||||
@@ -4307,6 +4308,12 @@ the per-section hand-off panel, was retired by S12 (#329): every section is nati
|
||||
primary stays the concept's plain button; the plan it opens has the primary.
|
||||
`duplicates dialog matches the concept` and `duplicates comparison detects mutations`
|
||||
(system-maintenance-concept.spec) pin the mapping.
|
||||
- **D192** (Maintenance, #550, beyond the concept): Database adds "Pair category and
|
||||
tags" after "Unused prompt components", a plain Review plan row (no danger mark: it
|
||||
deletes nothing) with the line "Fill an empty category or tags on prompts older Pair
|
||||
node runs saved, from their saved pairs. Only empty fields of never-edited prompts."
|
||||
`system-maintenance-concept.spec.mjs` maps it (`mapPairRow`) as a copy of Optimize
|
||||
database's row.
|
||||
- **D102** (Maintenance, S7, Q5, #222): Metadata cache clears only the browser entries
|
||||
Prompt Manager owns as a cache (`web/js/cache-ownership.js`). The inventory of every
|
||||
storage writer has none (see "Browser cache ownership" below), so the row's line adds
|
||||
@@ -4881,6 +4888,47 @@ replaced the database since the review; the plan is dropped). The job:
|
||||
prompt ids only), logs the backup by its file name, and prunes safety backups to the
|
||||
newest 10. With no backup the log and the status line say "No safety backup".
|
||||
|
||||
#### Pair category and tags repair (#550)
|
||||
|
||||
`pair_metadata` ("Pair category and tags", D192; `database/pair_metadata_repair.py`) fills
|
||||
the empty category and the empty tag set of prompts that Pair node runs before #546
|
||||
saved, from their saved pairs. It is reviewed and applied like the operations above and
|
||||
follows the safety backup policy like them (no forced backup). Rules
|
||||
(`docs/egm/post-3.4.1/550-design.md`):
|
||||
|
||||
- Considered: prompts the Pair node created as a positive: their earliest pair (lowest
|
||||
id, either role, archived included) has them as the positive and its `created_at` is
|
||||
within 2 s of theirs. A prompt first saved by the PromptManager node, or first as a
|
||||
negative, is out of scope and not counted.
|
||||
- Sources: the prompt's non-archived pairs where it is the positive; pairs with a blank
|
||||
category or no tags are ignored for that field. Category values are trimmed and
|
||||
case-sensitive, tag sets compared by tag id; a field fills when exactly one value is
|
||||
left. A whitespace-only category counts as empty.
|
||||
- Kept, per field, in this order: archived prompt, edited prompt (`updated_at` not within
|
||||
2 s of `created_at`; NULL or unreadable counts as edited), pairs that disagree, a tag
|
||||
set over 20 tags. A prompt can fill one field and keep the other.
|
||||
- The plan lists protected rows first, then fills, then a "Record" row, within the 50
|
||||
detail rows; at most 20,000 prompts per plan ("review again for the rest").
|
||||
- The fingerprint covers each considered prompt's computed result (values, tag ids and
|
||||
names, source pair ids), never a pair's title, notes, rating, run count or use
|
||||
stamps, so a Pair run between review and apply still applies.
|
||||
- The apply's writes are guarded (`category` still blank, no `prompt_tags` row) and a
|
||||
guard that hits nothing rolls the whole transaction back as `plan-changed`. It never
|
||||
writes `prompt_pairs`, `prompt_pair_tags`, `updated_at`, a tag or Library state.
|
||||
- The job lists 200 changed rows ("Prompt 12: category “night”; tags a, b"); its
|
||||
Download report lists every one (`truncated: false`, `Job.record_counted(complete=True)`).
|
||||
Protected lists one row per kept prompt in the review's wording and order ("Prompt 7:
|
||||
kept, pairs disagree on category: Adult, asian"), also complete in the report.
|
||||
- Activity shows a row with no path as a record: its detail as plain text, never an open
|
||||
or locate action, and the filter reads the detail. A maintenance job of records (every
|
||||
operation but Missing-file links) says "No records in this group.", "No listed record
|
||||
matches this filter." and "Filter records…" (`groupCopy`); file jobs keep "files".
|
||||
|
||||
Tests: `tests/test_maintenance_pair_metadata.py`, `tests/test_api_maintenance_pair_metadata.py`,
|
||||
`tests/test_jobs.py`, `tests/js/workspace-system-maintenance.test.js`,
|
||||
`tests/browser/workspace-system-maintenance.spec.mjs` (hook `/__test__/system/pair-metadata`).
|
||||
A Python change: ComfyUI needs a restart.
|
||||
|
||||
The legacy restore (`POST /prompt_manager/restore`) takes a backup of any size the disks
|
||||
have room for, so a real library's safety backup restores: the upload, its staged copy
|
||||
and the swap (the upload twice in the temporary folder, once more plus the current
|
||||
|
||||
+15
-2
@@ -2,7 +2,8 @@
|
||||
|
||||
Four operations, each the native owner of a legacy ``/maintenance`` one and
|
||||
built on the same queries, so a preview counts what the legacy cleanup would
|
||||
change:
|
||||
change (``workflow_storage`` and #550's ``pair_metadata``, which fills old
|
||||
Pair prompts' empty category and tags, live in their own modules):
|
||||
|
||||
- ``duplicate_prompts``: prompts whose text is equal ignoring case and outer
|
||||
spaces (``DUPLICATE_PROMPTS_SQL``, legacy ``cleanup_duplicates``); each
|
||||
@@ -60,6 +61,9 @@ MAINTENANCE_OPERATIONS = (
|
||||
"missing_links",
|
||||
# Applied as its own batched job (database/workflow_storage.py).
|
||||
"workflow_storage",
|
||||
# #550: fill old Pair prompts' empty category and tags
|
||||
# (database/pair_metadata_repair.py).
|
||||
"pair_metadata",
|
||||
)
|
||||
MAX_DETAILS = 50 # detail rows a plan lists
|
||||
MAX_RESULT_ROWS = 200 # changed records an apply describes (the job lists 200)
|
||||
@@ -533,6 +537,10 @@ class MaintenanceOperations:
|
||||
lambda conn: (self._link_rows(conn), _mark_rows(conn)),
|
||||
)
|
||||
return self._plan_missing_links(rows, roots, _stale_marks(marks, roots))
|
||||
if operation == "pair_metadata":
|
||||
from .pair_metadata_repair import plan_pair_metadata
|
||||
|
||||
return _read_snapshot(self.model.db_path, plan_pair_metadata)
|
||||
reader = (
|
||||
self._plan_duplicate_prompts
|
||||
if operation == "duplicate_prompts"
|
||||
@@ -800,7 +808,8 @@ class MaintenanceOperations:
|
||||
records changed and up to ``MAX_RESULT_ROWS`` descriptions of them
|
||||
(``{prompt_id[, kept_id]}`` or ``{image_path}``). ``marks`` are a
|
||||
missing-links plan's stale removal marks; that apply also returns
|
||||
``marks``, how many it forgot.
|
||||
``marks``, how many it forgot. ``pair_metadata`` returns every filled
|
||||
prompt, uncapped (its job keeps the complete record, #550).
|
||||
"""
|
||||
if operation not in MAINTENANCE_OPERATIONS or operation == "workflow_storage":
|
||||
raise ValueError(f"Unknown maintenance operation {operation!r}")
|
||||
@@ -815,6 +824,10 @@ class MaintenanceOperations:
|
||||
"unused_components": self._apply_unused_components,
|
||||
}
|
||||
with self._pair_transaction() as conn:
|
||||
if operation == "pair_metadata":
|
||||
from .pair_metadata_repair import apply_pair_metadata
|
||||
|
||||
return apply_pair_metadata(conn, fingerprint, targets)
|
||||
if operation == "missing_links":
|
||||
return self._apply_missing_links(
|
||||
conn, fingerprint, targets, roots, list(marks)
|
||||
|
||||
@@ -0,0 +1,452 @@
|
||||
"""Pair category and tags repair (#550): the ``pair_metadata`` maintenance plan.
|
||||
|
||||
Before #546 a Pair node run saved its category and tags on the saved pair
|
||||
only; the positive prompt it created stayed Uncategorized and untagged. This
|
||||
plan fills the **empty** category and the **empty** tag set of those prompts
|
||||
from their saved pairs (``docs/egm/post-3.4.1/550-design.md``):
|
||||
|
||||
- Considered: prompts the Pair node created as a positive (their earliest
|
||||
pair, in either role, has them as the positive and its ``created_at`` is
|
||||
within ``PAIR_CREATED_TOLERANCE`` of theirs) whose category is blank
|
||||
and/or that have no tags.
|
||||
- Sources: the prompt's non-archived pairs where it is the positive. Pairs
|
||||
with a blank category or no tags are ignored for that field. A field fills
|
||||
when exactly one value remains (category: trimmed, case-sensitive; tags:
|
||||
the tag-id set).
|
||||
- Kept (protected), in this order: archived prompt, edited prompt
|
||||
(``updated_at`` not within the tolerance of ``created_at``), pairs that
|
||||
disagree, a tag set over ``MAX_LIBRARY_TAGS``.
|
||||
|
||||
The apply runs in the caller's ``BEGIN IMMEDIATE`` transaction, recomputes
|
||||
the fingerprint, then writes guarded statements (the field must still be
|
||||
empty) that never touch a pair, a negative or ``updated_at``. A guard that
|
||||
hits nothing raises ``PlanChanged`` so the whole transaction rolls back.
|
||||
"""
|
||||
|
||||
import datetime
|
||||
|
||||
from . import maintenance as _maintenance
|
||||
from .library_commands import MAX_LIBRARY_TAGS
|
||||
from .maintenance import (
|
||||
MAX_DETAILS,
|
||||
MaintenanceDraft,
|
||||
PlanChanged,
|
||||
_excerpt,
|
||||
_Fingerprint,
|
||||
_marks,
|
||||
_plural,
|
||||
)
|
||||
|
||||
OPERATION = "pair_metadata"
|
||||
# Seconds: the Pair save stamps its new prompt and its pair with separate
|
||||
# utc_now_iso() calls in one transaction (microseconds apart).
|
||||
PAIR_CREATED_TOLERANCE = 2.0
|
||||
# Fill prompts one plan applies; the rest are applied by the next review.
|
||||
# Bounds the complete record a job keeps in memory (about 100 bytes a row).
|
||||
MAX_PLAN_FILLS = 20000
|
||||
DETAIL_VALUES = 3
|
||||
DETAIL_TAGS = 5
|
||||
MEDIA_LABEL = "No image file or saved pair is changed."
|
||||
EMPTY_LABEL = "No Pair prompts are missing a category or tags"
|
||||
RECORD_NOTE = (
|
||||
"The job's report lists every filled field. Download it to keep it: "
|
||||
"Activity forgets jobs after a restart."
|
||||
)
|
||||
# Protected classes in precedence order: (key, detail, reason one, many).
|
||||
EDITED_DETAIL = (
|
||||
"Kept: edited after it was saved; its empty category or tags may be intentional"
|
||||
)
|
||||
PROTECTED = (
|
||||
(
|
||||
"archived",
|
||||
"Kept: archived in the Library",
|
||||
"archived in the Library",
|
||||
"archived in the Library",
|
||||
),
|
||||
(
|
||||
"edited",
|
||||
EDITED_DETAIL,
|
||||
"edited after it was saved",
|
||||
"edited after they were saved",
|
||||
),
|
||||
("conflict", None, "where pairs disagree", "where pairs disagree"),
|
||||
(
|
||||
"too-many-tags",
|
||||
f"Kept: its pair has more than {MAX_LIBRARY_TAGS} tags",
|
||||
f"whose pair has more than {MAX_LIBRARY_TAGS} tags",
|
||||
f"whose pair has more than {MAX_LIBRARY_TAGS} tags",
|
||||
),
|
||||
)
|
||||
_PROTECTED_ORDER = [key for key, *_rest in PROTECTED]
|
||||
# Detail rows: what the user can act on (a conflict) first.
|
||||
DETAIL_ORDER = ("conflict", "edited", "archived", "too-many-tags")
|
||||
|
||||
_CANDIDATES_SQL = """
|
||||
SELECT p.id, p.hash, p.text, p.created_at, p.updated_at,
|
||||
COALESCE(TRIM(p.category), '') = '' AS blank_category,
|
||||
NOT EXISTS (SELECT 1 FROM prompt_tags pt WHERE pt.prompt_id = p.id)
|
||||
AS untagged,
|
||||
EXISTS (SELECT 1 FROM prompt_archive pa WHERE pa.prompt_id = p.id)
|
||||
AS archived,
|
||||
(SELECT CASE WHEN pp.positive_prompt_id = p.id
|
||||
THEN pp.created_at END
|
||||
FROM prompt_pairs pp
|
||||
WHERE pp.positive_prompt_id = p.id OR pp.negative_prompt_id = p.id
|
||||
ORDER BY pp.id LIMIT 1) AS first_pair_at
|
||||
FROM prompts p
|
||||
WHERE EXISTS (SELECT 1 FROM prompt_pairs pp WHERE pp.positive_prompt_id = p.id)
|
||||
AND (COALESCE(TRIM(p.category), '') = ''
|
||||
OR NOT EXISTS (SELECT 1 FROM prompt_tags pt WHERE pt.prompt_id = p.id))
|
||||
ORDER BY p.id
|
||||
"""
|
||||
_SOURCES_SQL = (
|
||||
"SELECT id, positive_prompt_id, TRIM(COALESCE(category, '')) FROM prompt_pairs "
|
||||
"WHERE archived = 0 AND positive_prompt_id IN ({marks}) ORDER BY id"
|
||||
)
|
||||
_PAIR_TAGS_SQL = (
|
||||
"SELECT ppt.pair_id, t.id, t.name FROM prompt_pair_tags ppt "
|
||||
"JOIN tags t ON t.id = ppt.tag_id WHERE ppt.pair_id IN ({marks}) "
|
||||
"ORDER BY ppt.pair_id, t.id"
|
||||
)
|
||||
|
||||
|
||||
def parse_stamp(value):
|
||||
"""A stored timestamp as an aware UTC datetime, or None when unreadable."""
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
return None
|
||||
try:
|
||||
stamp = datetime.datetime.fromisoformat(value.strip())
|
||||
except ValueError:
|
||||
return None
|
||||
if stamp.tzinfo is None:
|
||||
stamp = stamp.replace(tzinfo=datetime.timezone.utc)
|
||||
return stamp
|
||||
|
||||
|
||||
def _close(first, second):
|
||||
"""Whether two stored stamps parse and lie within the tolerance."""
|
||||
a, b = parse_stamp(first), parse_stamp(second)
|
||||
if a is None or b is None:
|
||||
return False
|
||||
return abs((a - b).total_seconds()) <= PAIR_CREATED_TOLERANCE
|
||||
|
||||
|
||||
def _batches(ids):
|
||||
size = _maintenance.ID_BATCH
|
||||
for start in range(0, len(ids), size):
|
||||
yield ids[start : start + size]
|
||||
|
||||
|
||||
def _sources(conn, ids):
|
||||
"""``{prompt_id: [(pair_id, category, ((tag_id, name), ...)), ...]}``."""
|
||||
pairs = {}
|
||||
for batch in _batches(ids):
|
||||
for pair_id, prompt_id, category in conn.execute(
|
||||
_SOURCES_SQL.format(marks=_marks(batch)), batch
|
||||
):
|
||||
pairs.setdefault(prompt_id, []).append([pair_id, category, []])
|
||||
by_pair = {pair[0]: pair for group in pairs.values() for pair in group}
|
||||
pair_ids = sorted(by_pair)
|
||||
for batch in _batches(pair_ids):
|
||||
for pair_id, tag_id, name in conn.execute(
|
||||
_PAIR_TAGS_SQL.format(marks=_marks(batch)), batch
|
||||
):
|
||||
by_pair[pair_id][2].append((tag_id, name))
|
||||
return {
|
||||
prompt_id: [(pid, category, tuple(tags)) for pid, category, tags in group]
|
||||
for prompt_id, group in pairs.items()
|
||||
}
|
||||
|
||||
|
||||
def _field(values, guard):
|
||||
"""``(class, value)`` of one field from its distinct values, or None."""
|
||||
if not values:
|
||||
return None
|
||||
if guard:
|
||||
return (guard, None)
|
||||
if len(values) > 1:
|
||||
return ("conflict", sorted(values))
|
||||
return ("fill", next(iter(values)))
|
||||
|
||||
|
||||
def _classify(row, sources):
|
||||
"""The result of one considered prompt, or None when nothing applies."""
|
||||
guard = None
|
||||
if row["archived"]:
|
||||
guard = "archived"
|
||||
elif not _close(row["updated_at"], row["created_at"]):
|
||||
guard = "edited"
|
||||
category = tags = None
|
||||
tag_pair = None
|
||||
if row["blank_category"]:
|
||||
category = _field({c for _p, c, _t in sources if c}, guard)
|
||||
if row["untagged"]:
|
||||
sets = {}
|
||||
for pair_id, _category, pair_tags in sources:
|
||||
if pair_tags:
|
||||
sets.setdefault(frozenset(pair_tags), pair_id)
|
||||
tags = _field(set(sets), guard)
|
||||
if tags and tags[0] == "fill":
|
||||
tag_pair = sets[tags[1]]
|
||||
if len(tags[1]) > MAX_LIBRARY_TAGS:
|
||||
tags = ("too-many-tags", None)
|
||||
else:
|
||||
tags = ("fill", tuple(sorted(tags[1])))
|
||||
elif tags and tags[0] == "conflict":
|
||||
tags = ("conflict", [sorted(s) for s in sorted(sets, key=sorted)])
|
||||
if category is None and tags is None:
|
||||
return None
|
||||
return {
|
||||
"id": row["id"],
|
||||
"hash": row["hash"],
|
||||
"text": row["text"],
|
||||
"category": category,
|
||||
"tags": tags,
|
||||
"tag_pair": tag_pair if tags and tags[0] == "fill" else None,
|
||||
"pairs": [pair_id for pair_id, _c, _t in sources],
|
||||
}
|
||||
|
||||
|
||||
def _state(conn):
|
||||
"""``(results, fingerprint)``: every considered prompt's result, by id."""
|
||||
considered = []
|
||||
for row in conn.execute(_CANDIDATES_SQL):
|
||||
row = dict(
|
||||
zip(
|
||||
(
|
||||
"id",
|
||||
"hash",
|
||||
"text",
|
||||
"created_at",
|
||||
"updated_at",
|
||||
"blank_category",
|
||||
"untagged",
|
||||
"archived",
|
||||
"first_pair_at",
|
||||
),
|
||||
tuple(row),
|
||||
)
|
||||
)
|
||||
if _close(row["created_at"], row["first_pair_at"]):
|
||||
considered.append(row)
|
||||
sources = _sources(conn, [row["id"] for row in considered])
|
||||
fingerprint = _Fingerprint(OPERATION)
|
||||
results = []
|
||||
for row in considered:
|
||||
result = _classify(row, sources.get(row["id"], []))
|
||||
if result is None:
|
||||
continue
|
||||
results.append(result)
|
||||
fingerprint.add(
|
||||
[
|
||||
result["id"],
|
||||
result["hash"],
|
||||
result["category"],
|
||||
result["tags"],
|
||||
result["tag_pair"],
|
||||
result["pairs"],
|
||||
]
|
||||
)
|
||||
return results, fingerprint.hexdigest()
|
||||
|
||||
|
||||
def _fills(result):
|
||||
return (
|
||||
bool(result["category"] and result["category"][0] == "fill"),
|
||||
bool(result["tags"] and result["tags"][0] == "fill"),
|
||||
)
|
||||
|
||||
|
||||
def _targets(results):
|
||||
targets = []
|
||||
for result in results:
|
||||
category, tags = _fills(result)
|
||||
if category or tags:
|
||||
targets.append([result["id"], category, tags])
|
||||
return targets[:MAX_PLAN_FILLS]
|
||||
|
||||
|
||||
def _protected_classes(result):
|
||||
"""The protected classes of a result's fields, in precedence order."""
|
||||
found = {
|
||||
field[0]
|
||||
for field in (result["category"], result["tags"])
|
||||
if field and field[0] != "fill"
|
||||
}
|
||||
return [key for key in _PROTECTED_ORDER if key in found]
|
||||
|
||||
|
||||
def _more(values, limit):
|
||||
shown = ", ".join(values[:limit])
|
||||
return shown + (", …" if len(values) > limit else "")
|
||||
|
||||
|
||||
def _fill_detail(result):
|
||||
parts = []
|
||||
category, tags = _fills(result)
|
||||
if category:
|
||||
parts.append(f"category “{result['category'][1]}”")
|
||||
if tags:
|
||||
names = sorted(name for _id, name in result["tags"][1])
|
||||
noun = "tag" if len(names) == 1 else "tags"
|
||||
parts.append(f"{len(names)} {noun}: {_more(names, DETAIL_TAGS)}")
|
||||
pairs = [str(pair_id) for pair_id in result["pairs"]]
|
||||
label = "pair" if len(pairs) == 1 else "pairs"
|
||||
return f"Fills {' and '.join(parts)} ({label} {_more(pairs, DETAIL_VALUES)})"
|
||||
|
||||
|
||||
def _protected_detail(result, key):
|
||||
if key != "conflict":
|
||||
return next(detail for k, detail, *_r in PROTECTED if k == key)
|
||||
if result["category"] and result["category"][0] == "conflict":
|
||||
values = result["category"][1]
|
||||
return f"Kept: pairs disagree on category: {_more(values, DETAIL_VALUES)}"
|
||||
return "Kept: pairs disagree on tags"
|
||||
|
||||
|
||||
def _kept(results):
|
||||
"""One row per kept prompt, its review reasons without "Kept: " (#550)."""
|
||||
rows = []
|
||||
for result in results:
|
||||
classes = set(_protected_classes(result))
|
||||
if classes:
|
||||
rows.append(
|
||||
{
|
||||
"prompt_id": result["id"],
|
||||
"reasons": [
|
||||
_protected_detail(result, key).removeprefix("Kept: ")
|
||||
for key in DETAIL_ORDER
|
||||
if key in classes
|
||||
],
|
||||
}
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
def _details(results, targets):
|
||||
rows = []
|
||||
for key in DETAIL_ORDER:
|
||||
for result in results:
|
||||
if key in _protected_classes(result):
|
||||
rows.append(
|
||||
{
|
||||
"label": _excerpt(result["text"]),
|
||||
"detail": _protected_detail(result, key),
|
||||
}
|
||||
)
|
||||
planned = {target[0] for target in targets}
|
||||
for result in results:
|
||||
if result["id"] in planned:
|
||||
rows.append(
|
||||
{"label": _excerpt(result["text"]), "detail": _fill_detail(result)}
|
||||
)
|
||||
if not targets:
|
||||
return rows[:MAX_DETAILS]
|
||||
return rows[: MAX_DETAILS - 1] + [{"label": "Record", "detail": RECORD_NOTE}]
|
||||
|
||||
|
||||
def _scope_label(targets, total):
|
||||
if not targets:
|
||||
return EMPTY_LABEL
|
||||
categories = sum(1 for _id, category, _tags in targets if category)
|
||||
tag_sets = sum(1 for _id, _category, tags in targets if tags)
|
||||
label = (
|
||||
f"{_plural(len(targets), 'prompt')} to fill: "
|
||||
f"{_plural(categories, 'category', 'categories')}, "
|
||||
f"{_plural(tag_sets, 'tag set')}"
|
||||
)
|
||||
if total > len(targets):
|
||||
label += (
|
||||
f" (the first {len(targets):,} of {total:,}; review again for the rest)"
|
||||
)
|
||||
return label
|
||||
|
||||
|
||||
def plan_pair_metadata(conn):
|
||||
"""The reviewed ``pair_metadata`` plan on ``conn``'s snapshot."""
|
||||
results, fingerprint = _state(conn)
|
||||
targets = _targets(results)
|
||||
total = sum(1 for result in results if any(_fills(result)))
|
||||
counts = {key: 0 for key in _PROTECTED_ORDER}
|
||||
held = 0
|
||||
for result in results:
|
||||
classes = _protected_classes(result)
|
||||
held += bool(classes)
|
||||
for key in classes:
|
||||
counts[key] += 1
|
||||
reasons = [
|
||||
f"{counts[key]:,} {one if counts[key] == 1 else many}"
|
||||
for key, _detail, one, many in PROTECTED
|
||||
if counts[key]
|
||||
]
|
||||
return MaintenanceDraft(
|
||||
operation=OPERATION,
|
||||
fingerprint=fingerprint,
|
||||
targets=targets,
|
||||
scope={"count": len(targets), "label": _scope_label(targets, total)},
|
||||
protected={"count": held, "reasons": reasons},
|
||||
media={"deletes_files": False, "label": MEDIA_LABEL},
|
||||
details=_details(results, targets),
|
||||
)
|
||||
|
||||
|
||||
def _fill_categories(conn, fills):
|
||||
"""Set each blank category; raises PlanChanged when one is no longer blank."""
|
||||
cursor = conn.executemany(
|
||||
"UPDATE prompts SET category = ? WHERE id = ? "
|
||||
"AND COALESCE(TRIM(category), '') = ''",
|
||||
fills,
|
||||
)
|
||||
if cursor.rowcount != len(fills):
|
||||
raise PlanChanged(OPERATION)
|
||||
|
||||
|
||||
def _fill_tags(conn, fills, expected):
|
||||
"""Copy each source pair's tags onto its untagged prompt."""
|
||||
cursor = conn.executemany(
|
||||
"INSERT INTO prompt_tags(prompt_id, tag_id) "
|
||||
"SELECT ?, ppt.tag_id FROM prompt_pair_tags ppt "
|
||||
"JOIN tags t ON t.id = ppt.tag_id WHERE ppt.pair_id = ? "
|
||||
"AND NOT EXISTS (SELECT 1 FROM prompt_tags WHERE prompt_id = ?)",
|
||||
fills,
|
||||
)
|
||||
if cursor.rowcount != expected:
|
||||
raise PlanChanged(OPERATION)
|
||||
|
||||
|
||||
def apply_pair_metadata(conn, fingerprint, targets):
|
||||
"""Apply the reviewed fills on ``conn`` (inside BEGIN IMMEDIATE).
|
||||
|
||||
Returns ``{changed, categories, tag_sets, rows, kept}``; ``rows`` lists
|
||||
every filled prompt (``{prompt_id, category?, tags?}``) and ``kept`` every
|
||||
prompt the plan keeps (``{prompt_id, reasons}``, in the review's wording
|
||||
and order), both never capped.
|
||||
"""
|
||||
results, current = _state(conn)
|
||||
if current != fingerprint or _targets(results) != targets:
|
||||
raise PlanChanged(OPERATION)
|
||||
by_id = {result["id"]: result for result in results}
|
||||
categories, tags, rows = [], [], []
|
||||
tag_count = 0
|
||||
for prompt_id, fill_category, fill_tags in targets:
|
||||
result = by_id[prompt_id]
|
||||
row = {"prompt_id": prompt_id}
|
||||
if fill_category:
|
||||
row["category"] = result["category"][1]
|
||||
categories.append((row["category"], prompt_id))
|
||||
if fill_tags:
|
||||
row["tags"] = sorted(name for _id, name in result["tags"][1])
|
||||
tags.append((prompt_id, result["tag_pair"], prompt_id))
|
||||
tag_count += len(result["tags"][1])
|
||||
rows.append(row)
|
||||
if categories:
|
||||
_fill_categories(conn, categories)
|
||||
if tags:
|
||||
_fill_tags(conn, tags, tag_count)
|
||||
return {
|
||||
"changed": len(rows),
|
||||
"categories": len(categories),
|
||||
"tag_sets": len(tags),
|
||||
"rows": rows,
|
||||
"kept": _kept(results),
|
||||
}
|
||||
+30
-17
@@ -204,6 +204,8 @@ class Job:
|
||||
self.total = 0
|
||||
self.counters = dict.fromkeys(groups, 0)
|
||||
self.outcomes = {group: [] for group in groups}
|
||||
# Groups whose report lists every row, beyond MAX_OUTCOME_ROWS (#550).
|
||||
self.complete = {}
|
||||
self.log = []
|
||||
self._cancel_requested = False
|
||||
self.last_event = None
|
||||
@@ -297,31 +299,36 @@ class Job:
|
||||
)
|
||||
self.log_line(log_line)
|
||||
|
||||
def record_counted(self, group, count, rows=()):
|
||||
def record_counted(self, group, count, rows=(), *, complete=False):
|
||||
"""Count ``count`` outcomes in ``group`` at once, listing ``rows``.
|
||||
|
||||
For outcomes that need no log line each (a thumbnail job's already
|
||||
fresh previews): the counter is exact, ``rows`` (``record``'s row
|
||||
shape) are listed up to MAX_OUTCOME_ROWS, and nothing is logged.
|
||||
``complete`` also keeps every row for the report (#550: a record of
|
||||
each change that must not be cut short; the caller bounds it).
|
||||
"""
|
||||
if group not in self.counters:
|
||||
raise ValueError(f"Unknown outcome group {group!r} for {self.kind}")
|
||||
self.counters[group] += max(0, int(count))
|
||||
listed = self.outcomes[group]
|
||||
kept = self.complete.setdefault(group, []) if complete else None
|
||||
for row in rows:
|
||||
if len(listed) >= MAX_OUTCOME_ROWS:
|
||||
if kept is None and len(listed) >= MAX_OUTCOME_ROWS:
|
||||
break
|
||||
index = row.get("root_index")
|
||||
if isinstance(index, bool) or not isinstance(index, int) or index < 0:
|
||||
index = None
|
||||
listed.append(
|
||||
{
|
||||
"path": _relative_path(row.get("path")),
|
||||
"root": str(row.get("root") or ""),
|
||||
"root_index": index,
|
||||
"detail": str(row.get("detail") or ""),
|
||||
}
|
||||
)
|
||||
shaped = {
|
||||
"path": _relative_path(row.get("path")),
|
||||
"root": str(row.get("root") or ""),
|
||||
"root_index": index,
|
||||
"detail": str(row.get("detail") or ""),
|
||||
}
|
||||
if kept is not None:
|
||||
kept.append(shaped)
|
||||
if len(listed) < MAX_OUTCOME_ROWS:
|
||||
listed.append(dict(shaped))
|
||||
|
||||
def finish(self, state="finished"):
|
||||
"""End the job in ``state`` and release every subscriber.
|
||||
@@ -406,8 +413,9 @@ class Job:
|
||||
"""The downloadable report: the summary, every outcome group and the log.
|
||||
|
||||
Each group carries its exact count, its listed rows (at most
|
||||
MAX_OUTCOME_ROWS) and whether the list is cut short. Rows and log lines
|
||||
already hold root-relative paths only.
|
||||
MAX_OUTCOME_ROWS, or every row of a ``complete`` group) and whether
|
||||
the list is cut short. Rows and log lines already hold root-relative
|
||||
paths only.
|
||||
"""
|
||||
summary = self.summary()
|
||||
summary.pop("listed", None)
|
||||
@@ -417,16 +425,21 @@ class Job:
|
||||
"generated_at": self._clock(),
|
||||
"job": summary,
|
||||
"outcomes": {
|
||||
group: {
|
||||
"count": self.counters.get(group, len(rows)),
|
||||
"rows": [dict(row) for row in rows],
|
||||
"truncated": self.counters.get(group, len(rows)) > len(rows),
|
||||
}
|
||||
group: self._report_group(group, rows)
|
||||
for group, rows in self.outcomes.items()
|
||||
},
|
||||
"log": list(self.log),
|
||||
}
|
||||
|
||||
def _report_group(self, group, listed):
|
||||
rows = self.complete.get(group, listed)
|
||||
count = self.counters.get(group, len(rows))
|
||||
return {
|
||||
"count": count,
|
||||
"rows": [dict(row) for row in rows],
|
||||
"truncated": count > len(rows),
|
||||
}
|
||||
|
||||
def report_filename(self):
|
||||
"""The download name: prompt-manager-KIND-YYYYMMDD-HHMM.json (local time)."""
|
||||
stamp = time.strftime("%Y%m%d-%H%M", time.localtime(self._clock()))
|
||||
|
||||
+60
-6
@@ -1,8 +1,9 @@
|
||||
"""Maintenance routes: reviewed, read-only plans (#249 System S6).
|
||||
|
||||
``POST /prompt_manager/maintenance/preview {operation, options:{}}`` reviews
|
||||
one operation (``duplicate_prompts``, ``optimize``, ``unused_components`` or
|
||||
``missing_links``; ``database/maintenance.py``; ``duplicate_files``,
|
||||
one operation (``duplicate_prompts``, ``optimize``, ``unused_components``,
|
||||
``missing_links`` or ``pair_metadata``, #550; ``database/maintenance.py``;
|
||||
``duplicate_files``,
|
||||
``py/api/duplicate_files.py``; ``rotated_logs``, the rotated log copies,
|
||||
``py/api/log_cleanup.py``, which has its own apply job) off the event loop and
|
||||
answers a ``Plan``: ``{success, token, operation, expires_at, scope:{count,
|
||||
@@ -97,7 +98,11 @@ MAINTENANCE_TITLES = {
|
||||
"unused_components": "Unused prompt components",
|
||||
"missing_links": "Missing-file links",
|
||||
"workflow_storage": "Store each workflow once",
|
||||
# #550, beyond the concept: fills old Pair prompts' empty category and tags.
|
||||
"pair_metadata": "Pair category and tags",
|
||||
}
|
||||
# Operations whose job keeps every changed row for its report (#550).
|
||||
COMPLETE_RECORD = frozenset({"pair_metadata"})
|
||||
_OPERATION_ERROR = "operation must be one of: " + ", ".join(PREVIEW_OPERATIONS)
|
||||
MAINTENANCE_GROUPS = ("changed", "protected", "errors")
|
||||
MAX_TOKEN_LENGTH = 128
|
||||
@@ -281,9 +286,27 @@ def short_space_message(short):
|
||||
)
|
||||
|
||||
|
||||
def _changed_line(operation, count, marks=0):
|
||||
def _pair_metadata_line(result):
|
||||
count = result["changed"]
|
||||
noun = (
|
||||
"prompt from its saved pairs"
|
||||
if count == 1
|
||||
else "prompts from their saved pairs"
|
||||
)
|
||||
categories = result.get("categories", 0)
|
||||
tag_sets = result.get("tag_sets", 0)
|
||||
return (
|
||||
f"Filled {count:,} {noun} "
|
||||
f"({categories:,} {'category' if categories == 1 else 'categories'}, "
|
||||
f"{tag_sets:,} {'tag set' if tag_sets == 1 else 'tag sets'})"
|
||||
)
|
||||
|
||||
|
||||
def _changed_line(operation, count, marks=0, result=None):
|
||||
if operation == "optimize":
|
||||
return "Optimized the database"
|
||||
if operation == "pair_metadata":
|
||||
return _pair_metadata_line(result or {"changed": count})
|
||||
if operation == "duplicate_prompts":
|
||||
noun = "duplicate prompt" if count == 1 else "duplicate prompts"
|
||||
return f"Merged {count:,} {noun}"
|
||||
@@ -668,9 +691,28 @@ class MaintenanceRoutesMixin:
|
||||
rows = await self._run_in_executor(
|
||||
self._maintenance_rows, plan.operation, result["rows"]
|
||||
)
|
||||
job.record_counted("changed", result["changed"], rows)
|
||||
job.record_counted("protected", payload["protected"])
|
||||
line = _changed_line(plan.operation, result["changed"], result.get("marks", 0))
|
||||
job.record_counted(
|
||||
"changed",
|
||||
result["changed"],
|
||||
rows,
|
||||
complete=plan.operation in COMPLETE_RECORD,
|
||||
)
|
||||
if "kept" in result:
|
||||
# #550: each kept record with its reason, like the changed ones.
|
||||
kept = [
|
||||
{
|
||||
"path": "",
|
||||
"detail": f"Prompt {item['prompt_id']}: kept, "
|
||||
+ "; ".join(item["reasons"]),
|
||||
}
|
||||
for item in result["kept"]
|
||||
]
|
||||
job.record_counted("protected", len(kept), kept, complete=True)
|
||||
else:
|
||||
job.record_counted("protected", payload["protected"])
|
||||
line = _changed_line(
|
||||
plan.operation, result["changed"], result.get("marks", 0), result
|
||||
)
|
||||
job.log_line(line)
|
||||
job.update(progress=100, status_line=f"{line} · {backup_note(backup)}")
|
||||
return None
|
||||
@@ -691,6 +733,8 @@ class MaintenanceRoutesMixin:
|
||||
}
|
||||
)
|
||||
return rows
|
||||
if operation == "pair_metadata":
|
||||
return [{"path": "", "detail": _filled_detail(item)} for item in described]
|
||||
if operation == "duplicate_prompts":
|
||||
return [
|
||||
{
|
||||
@@ -708,6 +752,16 @@ class MaintenanceRoutesMixin:
|
||||
]
|
||||
|
||||
|
||||
def _filled_detail(item):
|
||||
"""``Prompt 12: category “portrait”; tags a, b``: one filled prompt (#550)."""
|
||||
parts = []
|
||||
if "category" in item:
|
||||
parts.append(f"category “{item['category']}”")
|
||||
if item.get("tags"):
|
||||
parts.append("tags " + ", ".join(item["tags"]))
|
||||
return f"Prompt {item['prompt_id']}: " + "; ".join(parts)
|
||||
|
||||
|
||||
def backup_note(backup):
|
||||
"""The status line's backup part: the file name, or that there is none."""
|
||||
return f"Safety backup {backup}" if backup else NO_BACKUP
|
||||
|
||||
@@ -64,11 +64,31 @@ function measureMaintenance(mainSelector) {
|
||||
};
|
||||
}
|
||||
|
||||
/** D102: the Metadata cache row says why its review is unavailable and its button is aria-disabled (D93's dimmed style). */
|
||||
function mapMaintenanceConcept(reference) {
|
||||
/**
|
||||
* D192 (#550, beyond the concept): Database adds "Pair category and tags" after "Unused prompt components", shaped like
|
||||
* the concept's plain Review plan row (Optimize database's).
|
||||
*/
|
||||
function mapPairRow(reference) {
|
||||
return {
|
||||
...reference,
|
||||
surfaces: reference.surfaces.map(surface => ({ ...surface, rows: surface.rows.map(row => row.title !== 'Metadata cache' ? row : {
|
||||
surfaces: reference.surfaces.map(surface => {
|
||||
const plain = surface.rows.find(row => row.title === 'Optimize database');
|
||||
const at = surface.rows.findIndex(row => row.title === 'Unused prompt components');
|
||||
if (!plain || at < 0) return surface;
|
||||
return { ...surface, tree: [...surface.tree, 'div.operation'], rows: [...surface.rows.slice(0, at + 1),
|
||||
{ ...plain, title: 'Pair category and tags', description: 'Fill an empty category or tags on prompts older ' +
|
||||
'Pair node runs saved, from their saved pairs. Only empty fields of never-edited prompts.' },
|
||||
...surface.rows.slice(at + 1)] };
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
/** D102: the Metadata cache row says why its review is unavailable and its button is aria-disabled (D93's dimmed style). */
|
||||
function mapMaintenanceConcept(reference) {
|
||||
const mapped = mapPairRow(reference);
|
||||
return {
|
||||
...mapped,
|
||||
surfaces: mapped.surfaces.map(surface => ({ ...surface, rows: surface.rows.map(row => row.title !== 'Metadata cache' ? row : {
|
||||
...row, description: `${row.description} Nothing is cached in this browser.`,
|
||||
actions: row.actions.map(action => ({ ...action, opacity: '0.55', cursor: 'not-allowed' })) }) })),
|
||||
};
|
||||
@@ -209,8 +229,8 @@ test('maintenance comparison detects mutations', async ({ page, request }) => {
|
||||
const expected = mapMaintenanceConcept(concept);
|
||||
const production = await productionMaintenance(page, 'dark');
|
||||
expect(diffInventory(expected, production)).toEqual([]);
|
||||
// Unmapped, the section differs only on the D102 row.
|
||||
expect(diffInventory(concept, production).map(item => item.path).sort()).toEqual([
|
||||
// Mapped only for the D192 row, the section differs only on the D102 row.
|
||||
expect(diffInventory(mapPairRow(concept), production).map(item => item.path).sort()).toEqual([
|
||||
'surfaces.1.rows.3.actions.0.cursor', 'surfaces.1.rows.3.actions.0.opacity', 'surfaces.1.rows.3.description']);
|
||||
const measure = () => page.evaluate(measureMaintenance, '#workspace-main');
|
||||
const paths = async () => diffInventory(expected, await measure()).map(item => item.path);
|
||||
|
||||
@@ -64,7 +64,9 @@ test('the section shows the concept rows; Metadata cache follows Q5 (D102)', asy
|
||||
item.querySelector('.row-actions > button').textContent, item.querySelector('.row-actions > button').className]),
|
||||
]))).toEqual([
|
||||
['Database', [['Integrity & relationship checks', 'Check', ''], ['Duplicate prompt text', 'Review plan', ''],
|
||||
['Optimize database', 'Review plan', ''], ['Unused prompt components', 'Review plan', 'danger']]],
|
||||
['Optimize database', 'Review plan', ''], ['Unused prompt components', 'Review plan', 'danger'],
|
||||
// D192 (#550): beyond the concept.
|
||||
['Pair category and tags', 'Review plan', '']]],
|
||||
['Images & cache', [['Missing-file links', 'Review plan', 'danger'], ['Duplicate image files', 'Find duplicates', ''],
|
||||
['Thumbnail cache', 'Review options', ''], ['Metadata cache', 'Review plan', '']]],
|
||||
]);
|
||||
@@ -177,6 +179,70 @@ test("plan dialog shows the server's scope, protected, backup policy and media l
|
||||
expect(await serverJson(request, '/__test__/system/digest')).toEqual(before);
|
||||
});
|
||||
|
||||
test('D192: Pair category and tags reviews old Pair prompts, keeps an edited one and fills the other (#550)',
|
||||
async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/system/pair-metadata')).toBeOK();
|
||||
const answers = [];
|
||||
page.on('response', response => { if (response.url().endsWith(PREVIEW_URL)) answers.push(response.json()); });
|
||||
await openMaintenance(page);
|
||||
const pairRow = row(page, 'Pair category and tags');
|
||||
await expect(pairRow.locator('p')).toHaveText('Fill an empty category or tags on prompts older Pair node runs saved, ' +
|
||||
'from their saved pairs. Only empty fields of never-edited prompts.');
|
||||
await pairRow.getByRole('button', { name: 'Review plan', exact: true }).click();
|
||||
const plan = dialog(page);
|
||||
await expect(plan.locator('#modal-body > h3')).toHaveText('Pair category and tags');
|
||||
const answer = await answers[0];
|
||||
expect(answer.operation).toBe('pair_metadata');
|
||||
expect(await gridOf(page)).toEqual([
|
||||
['Scope', '1 prompt to fill: 1 category, 1 tag set'],
|
||||
['Protected', '1 edited after it was saved'],
|
||||
['Backup', `Create and verify before changes (${answer.backup.destination_label})`],
|
||||
['Media on disk', 'No image file or saved pair is changed.'],
|
||||
]);
|
||||
await expect(plan.locator('details > summary')).toHaveText('Listed records (3)');
|
||||
await plan.locator('details > summary').click();
|
||||
// Protected rows come first, the Record row last.
|
||||
const lines = (await plan.locator('details > pre').textContent()).split('\n');
|
||||
expect(lines).toHaveLength(3);
|
||||
expect(lines[0]).toBe('S550 harbor · Kept: edited after it was saved; its empty category or tags may be intentional');
|
||||
expect(lines[1]).toMatch(/^S550 lantern · Fills category “night” and 2 tags: s550-glow, s550-warm \(pair \d+\)$/);
|
||||
expect(lines[2]).toMatch(/^Record · The job's report lists every filled field\. Download it to keep it/);
|
||||
await plan.getByRole('button', { name: 'Apply reviewed operation', exact: true }).click();
|
||||
await expect(dialog(page)).toHaveCount(0);
|
||||
await expect(page).toHaveURL(/#\/activity\/[0-9a-f]{32}$/);
|
||||
const job = await ended(request, page.url().split('/').pop());
|
||||
expect([job.title, job.state, job.counters.changed, job.counters.protected])
|
||||
.toEqual(['Pair category and tags', 'finished', 1, 1]);
|
||||
const main = page.locator('#workspace-main');
|
||||
await expect(main.locator('h1')).toHaveText('Pair category and tags');
|
||||
await expect(main.getByRole('button', { name: 'Download report', exact: true })).toBeVisible();
|
||||
// #550 live test: the Changed group lists the path-less record rows as plain text, never an open action.
|
||||
const results = main.getByRole('region', { name: 'Results', exact: true });
|
||||
await results.locator('.outcome-list > button').filter({ hasText: 'Changed' }).click();
|
||||
const records = results.locator('.outcome-files > .file-row');
|
||||
await expect(records).toHaveCount(1);
|
||||
await expect(records.first()).toHaveText(/^Prompt \d+: category “night”; tags s550-glow, s550-warm$/);
|
||||
expect(await records.first().evaluate(node => node.localName)).toBe('div');
|
||||
await expect(results.locator('.outcome-files button')).toHaveCount(0);
|
||||
await expect(results.getByRole('searchbox', { name: 'Filter results' })).toHaveAttribute('placeholder', 'Filter records…');
|
||||
// Protected lists each kept prompt with the review's reason (#550 live re-check).
|
||||
await results.locator('.outcome-list > button').filter({ hasText: 'Protected' }).click();
|
||||
await expect(records).toHaveCount(1);
|
||||
await expect(records.first()).toHaveText(
|
||||
/^Prompt \d+: kept, edited after it was saved; its empty category or tags may be intentional$/);
|
||||
// An empty group says records, not files.
|
||||
await results.locator('.outcome-list > button').filter({ hasText: 'Errors' }).click();
|
||||
await expect(results.locator('.outcome-files > p.empty-state')).toHaveText('No records in this group.');
|
||||
const library = await serverJson(request, '/prompt_manager/library');
|
||||
const lantern = library.items.find(item => item.text === 'S550 lantern');
|
||||
const harbor = library.items.find(item => item.text === 'S550 harbor');
|
||||
const filled = (await serverJson(request, `/prompt_manager/library/${lantern.id}?archive=all`)).positive;
|
||||
expect([filled.category, [...filled.tags].sort()]).toEqual(['night', ['s550-glow', 's550-warm']]);
|
||||
expect((await serverJson(request, `/prompt_manager/library/${harbor.id}?archive=all`)).positive.category).toBeFalsy();
|
||||
await page.goto(`${workspace}#/library`);
|
||||
await expect(page.locator(`article[data-prompt-id="${lantern.id}"] .card-title > .chip`)).toHaveText('night');
|
||||
});
|
||||
|
||||
test('a plan with nothing to do shows only Close, never Apply', async ({ page, request }) => {
|
||||
const applies = [];
|
||||
page.on('request', item => { if (item.url().endsWith(APPLY_URL)) applies.push(item.url()); });
|
||||
|
||||
@@ -41,6 +41,11 @@ read. These per-server hooks only change what a System test needs on top:
|
||||
- ``POST /__test__/system/duplicate-prompts`` (S7) adds two prompts whose
|
||||
text differs only in case and spacing (a duplicate prompt text group);
|
||||
``{"again": true}`` adds a third, so a reviewed plan no longer matches;
|
||||
- ``POST /__test__/system/pair-metadata`` (#550) saves two saved pairs the
|
||||
way the Pair node did before #546 (the pair holds the category and tags,
|
||||
its new positive prompt none): ``S550 lantern`` (category ``night``, tags
|
||||
``s550-glow``, ``s550-warm``) fills; ``S550 harbor`` was edited after it was
|
||||
saved, so the plan keeps it;
|
||||
- ``POST /__test__/system/duplicate-files`` (S7) writes one PNG's bytes three
|
||||
times: ``output/s7-dupes/same.png``, ``input/s7-dupes/same.png`` (the same
|
||||
relative path in the other source) and ``output/s7-dupes/copy.png``;
|
||||
@@ -344,6 +349,25 @@ def register_system_commands(app, routes, api, root: Path):
|
||||
conn.commit()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/pair-metadata")
|
||||
async def pair_metadata(request):
|
||||
def seed():
|
||||
api.db.save_prompt_pair(
|
||||
"S550 lantern",
|
||||
"S550 blurry",
|
||||
category="night",
|
||||
tags=["s550-glow", "s550-warm"],
|
||||
)
|
||||
api.db.save_prompt_pair("S550 harbor", category="night")
|
||||
with api.db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE prompts SET updated_at='2099-01-01T00:00:00+00:00' "
|
||||
"WHERE text='S550 harbor'"
|
||||
)
|
||||
|
||||
await api._run_in_executor(seed)
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.post("/__test__/system/duplicate-files")
|
||||
async def duplicate_files(request):
|
||||
body = await request.json() if request.can_read_body else {}
|
||||
|
||||
@@ -21,8 +21,23 @@ test('the section lists the concept rows in its order, labels and danger marks',
|
||||
assert.deepEqual(MAINTENANCE_SURFACES.map(([heading]) => heading), surfaces.map(text => text.replace('&', '&')));
|
||||
const production = MAINTENANCE_SURFACES.flatMap(([, rows]) => rows.map(([title, description, label, , danger = false]) =>
|
||||
[title, description, label, danger]));
|
||||
assert.deepEqual(production, concept.map(([title, ...rest]) => [title.replace('&', '&'), ...rest]));
|
||||
assert.equal(production.length, 8);
|
||||
// D192 (#550, beyond the concept): "Pair category and tags" follows "Unused prompt components" in Database.
|
||||
const expected = concept.map(([title, ...rest]) => [title.replace('&', '&'), ...rest]);
|
||||
expected.splice(expected.findIndex(([title]) => title === 'Unused prompt components') + 1, 0, PAIR_ROW);
|
||||
assert.deepEqual(production, expected);
|
||||
assert.equal(production.length, 9);
|
||||
});
|
||||
|
||||
const PAIR_ROW = Object.freeze(['Pair category and tags', 'Fill an empty category or tags on prompts older Pair node runs ' +
|
||||
'saved, from their saved pairs. Only empty fields of never-edited prompts.', 'Review plan', false]);
|
||||
|
||||
test('D192: the Pair category and tags row reviews the pair_metadata plan under its title (#550)', async () => {
|
||||
const { MAINTENANCE_SURFACES } = await maintenance;
|
||||
const { OPERATION_TITLES } = await review;
|
||||
const [, database] = MAINTENANCE_SURFACES[0];
|
||||
const row = database.find(([title]) => title === PAIR_ROW[0]);
|
||||
assert.deepEqual(row, [PAIR_ROW[0], PAIR_ROW[1], PAIR_ROW[2], 'pair_metadata']);
|
||||
assert.equal(OPERATION_TITLES.pair_metadata, 'Pair category and tags');
|
||||
});
|
||||
|
||||
const PLAN = Object.freeze({
|
||||
@@ -210,3 +225,40 @@ test('a rotated-log cleanup the log overtook offers Review again (S12)', async (
|
||||
assert.deepEqual(reviewAgainOf({ kind: 'maintenance', state: 'failed', scope: { operation: 'rotated_logs', failure: 'plan-changed' } }),
|
||||
{ operation: 'rotated_logs' });
|
||||
});
|
||||
|
||||
// #550 live test: a records operation's rows have no path, only a detail ("Prompt 12: category …").
|
||||
const RECORD_JOB = Object.freeze({
|
||||
id: 'a'.repeat(32), kind: 'maintenance', title: 'Pair category and tags', state: 'finished',
|
||||
admission: 'records', cancellable: false, progress: 100, processed: 0, total: 0, status_line: 'Filled 2 prompts',
|
||||
scope: { operation: 'pair_metadata' }, counters: { changed: 2, protected: 0, errors: 0 },
|
||||
started_at: 1, finished_at: 2,
|
||||
});
|
||||
|
||||
test('#550: path-less record rows are kept with their detail; file rows are unchanged', async () => {
|
||||
const { parseDetail, filterRows } = await jobView;
|
||||
const parsed = parseDetail({ ...RECORD_JOB, outcomes: { changed: [
|
||||
{ path: '', root: '', root_index: null, detail: 'Prompt 2325: category “asian”; tags asian' },
|
||||
{ path: '', root: '', root_index: null, detail: '' },
|
||||
{ path: 'a/b.png', root: 'output', root_index: 0, detail: 'Link removed' },
|
||||
] }, log: [] });
|
||||
assert.deepEqual(parsed.outcomes.changed, [
|
||||
{ path: '', root: '', root_index: null, detail: 'Prompt 2325: category “asian”; tags asian', record: true },
|
||||
{ path: 'a/b.png', root: 'output', root_index: 0, detail: 'Link removed', record: false },
|
||||
]);
|
||||
// The filter reads a record's detail and a file's path.
|
||||
assert.deepEqual(filterRows(parsed.outcomes.changed, 'asian').map(row => row.detail),
|
||||
['Prompt 2325: category “asian”; tags asian']);
|
||||
assert.deepEqual(filterRows(parsed.outcomes.changed, 'a/b').map(row => row.path), ['a/b.png']);
|
||||
});
|
||||
|
||||
test('#550: a records operation says records, not files, for an empty group and a filter', async () => {
|
||||
const { groupCopy, EMPTY_GROUP, NO_MATCH } = await jobView;
|
||||
assert.deepEqual(groupCopy(RECORD_JOB), { empty: 'No records in this group.', noMatch: 'No listed record matches this filter.',
|
||||
placeholder: 'Filter records…' });
|
||||
for (const operation of ['duplicate_prompts', 'unused_components']) {
|
||||
assert.equal(groupCopy({ ...RECORD_JOB, scope: { operation } }).empty, 'No records in this group.');
|
||||
}
|
||||
const files = { empty: EMPTY_GROUP, noMatch: NO_MATCH, placeholder: 'Filter by file path…' };
|
||||
assert.deepEqual(groupCopy({ ...RECORD_JOB, scope: { operation: 'missing_links' } }), files);
|
||||
assert.deepEqual(groupCopy({ ...RECORD_JOB, kind: 'scan', scope: {} }), files);
|
||||
});
|
||||
|
||||
@@ -0,0 +1,181 @@
|
||||
"""Pair category and tags repair through the maintenance routes (#550).
|
||||
|
||||
``pair_metadata`` is reviewed and applied like the other maintenance plans:
|
||||
``options`` must be ``{}``, the job is titled "Pair category and tags", the
|
||||
safety backup follows the ``maintenance.safety_backup`` policy, and the job's
|
||||
downloadable report lists every filled prompt, never cut at 200 rows.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
_mock_server = MagicMock()
|
||||
_mock_server.PromptServer.instance.routes = MagicMock()
|
||||
sys.modules.setdefault("server", _mock_server)
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from py.api.jobs import MAX_OUTCOME_ROWS # noqa: E402
|
||||
from py.api.maintenance import MAINTENANCE_TITLES # noqa: E402
|
||||
from py.config import PromptManagerConfig # noqa: E402
|
||||
|
||||
try:
|
||||
from tests.test_api_maintenance import WAIT, ApplyTestCase # noqa: E402
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_api_maintenance import WAIT, ApplyTestCase # noqa: E402
|
||||
|
||||
OPERATION = "pair_metadata"
|
||||
TITLE = "Pair category and tags"
|
||||
|
||||
|
||||
class PairMetadataAPITestCase(ApplyTestCase):
|
||||
def pair(self, positive, negative="", **metadata):
|
||||
pair_id = self.api.db.save_prompt_pair(positive, negative, **metadata)
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
prompt = conn.execute(
|
||||
"SELECT positive_prompt_id FROM prompt_pairs WHERE id=?", (pair_id,)
|
||||
).fetchone()[0]
|
||||
return pair_id, prompt
|
||||
|
||||
def category(self, prompt_id):
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
return conn.execute(
|
||||
"SELECT category FROM prompts WHERE id=?", (prompt_id,)
|
||||
).fetchone()[0]
|
||||
|
||||
|
||||
class TestPairMetadataRoutes(PairMetadataAPITestCase):
|
||||
async def test_preview_accepts_the_operation_and_no_options(self):
|
||||
self.assertEqual(MAINTENANCE_TITLES[OPERATION], TITLE)
|
||||
self.pair("a cat", category="portrait", tags=["a"])
|
||||
plan = await self.preview(OPERATION)
|
||||
self.assertEqual(plan["operation"], OPERATION)
|
||||
self.assertEqual(plan["scope"]["count"], 1)
|
||||
self.assertEqual(plan["backup"]["policy"], "on")
|
||||
refused = await self.preview(OPERATION, 400, options={"all": True})
|
||||
self.assertEqual(refused["code"], "invalid-request")
|
||||
|
||||
async def test_apply_fills_with_a_backup_and_names_it(self):
|
||||
_pair, prompt = self.pair("a cat", "neg", category="portrait", tags=["a", "b"])
|
||||
plan, summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(summary["title"], TITLE)
|
||||
self.assertEqual(summary["admission"], "records")
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(self.category(prompt), "portrait")
|
||||
[backup] = self.backups()
|
||||
self.assertEqual(
|
||||
job.status_line,
|
||||
"Filled 1 prompt from its saved pairs (1 category, 1 tag set) · "
|
||||
f"Safety backup {backup}",
|
||||
)
|
||||
self.assertEqual(
|
||||
job.outcomes["changed"][0]["detail"],
|
||||
f"Prompt {prompt}: category “portrait”; tags a, b",
|
||||
)
|
||||
self.assertEqual(job.counters["changed"], 1)
|
||||
|
||||
async def test_protected_prompts_are_counted(self):
|
||||
_a, prompt = self.pair("conflict", "n1", category="x")
|
||||
self.pair("conflict", "n2", category="y")
|
||||
self.pair("fill me", category="portrait")
|
||||
plan, _summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(plan["protected"]["reasons"], ["1 where pairs disagree"])
|
||||
self.assertEqual(job.counters["protected"], 1)
|
||||
self.assertIsNone(self.category(prompt))
|
||||
# Live test (#550): the count comes with its rows, in the review's wording.
|
||||
self.assertEqual(
|
||||
job.outcomes["protected"],
|
||||
[
|
||||
{
|
||||
"path": "",
|
||||
"root": "",
|
||||
"root_index": None,
|
||||
"detail": (
|
||||
f"Prompt {prompt}: kept, pairs disagree on category: x, y"
|
||||
),
|
||||
}
|
||||
],
|
||||
)
|
||||
report = job.report()["outcomes"]["protected"]
|
||||
self.assertEqual((report["count"], report["truncated"]), (1, False))
|
||||
|
||||
async def test_report_lists_every_kept_prompt(self):
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
count = MAX_OUTCOME_ROWS + 5
|
||||
for n in range(count):
|
||||
_pair, prompt = self.pair(f"kept {n:03d}", category="portrait")
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE prompts SET updated_at='2099-01-01T00:00:00+00:00' "
|
||||
"WHERE id=?",
|
||||
(prompt,),
|
||||
)
|
||||
_plan, _summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(job.counters["protected"], count)
|
||||
self.assertEqual(len(job.outcomes["protected"]), MAX_OUTCOME_ROWS)
|
||||
protected = job.report()["outcomes"]["protected"]
|
||||
self.assertEqual(
|
||||
(len(protected["rows"]), protected["truncated"]), (count, False)
|
||||
)
|
||||
self.assertTrue(
|
||||
protected["rows"][0]["detail"].endswith(
|
||||
": kept, edited after it was saved; its empty category or tags "
|
||||
"may be intentional"
|
||||
)
|
||||
)
|
||||
|
||||
async def test_policy_off_writes_without_a_backup(self):
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
_pair, prompt = self.pair("no backup", category="portrait")
|
||||
plan, _summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(plan["backup"]["policy"], "off")
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertIn("No safety backup (the policy is off)", "\n".join(job.log))
|
||||
self.assertTrue(job.status_line.endswith("· No safety backup"))
|
||||
self.assertEqual(self.backups(), [])
|
||||
self.assertEqual(self.category(prompt), "portrait")
|
||||
|
||||
async def test_reviewed_with_the_policy_on_keeps_the_backup(self):
|
||||
self.pair("promise", category="portrait")
|
||||
plan = await self.preview(OPERATION)
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
body = await self.apply(plan["token"])
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
self.assertEqual(len(self.backups()), 1)
|
||||
|
||||
async def test_plan_changed_writes_nothing(self):
|
||||
pair_id, prompt = self.pair("drift", category="portrait")
|
||||
plan = await self.preview(OPERATION)
|
||||
self.api.db.update_prompt_pair_metadata(pair_id, category="landscape")
|
||||
body = await self.apply(plan["token"])
|
||||
job = self.api._jobs.get(body["job"]["id"])
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertEqual(job.last_event["code"], "plan-changed")
|
||||
self.assertIsNone(self.category(prompt))
|
||||
|
||||
async def test_report_lists_every_filled_prompt(self):
|
||||
PromptManagerConfig.SAFETY_BACKUP = False
|
||||
count = MAX_OUTCOME_ROWS + 5
|
||||
for n in range(count):
|
||||
self.pair(f"bulk {n:03d}", category="portrait")
|
||||
_plan, _summary, job = await self.run_apply(OPERATION)
|
||||
self.assertEqual(job.state, "finished", job.log)
|
||||
self.assertEqual(len(job.outcomes["changed"]), MAX_OUTCOME_ROWS)
|
||||
response = await self.client.request(
|
||||
"GET", f"/prompt_manager/jobs/{job.id}/report"
|
||||
)
|
||||
self.assertEqual(response.status, 200)
|
||||
report = await response.json()
|
||||
changed = report["outcomes"]["changed"]
|
||||
self.assertEqual(changed["count"], count)
|
||||
self.assertEqual(len(changed["rows"]), count)
|
||||
self.assertFalse(changed["truncated"])
|
||||
self.assertTrue(
|
||||
all(
|
||||
row["detail"].endswith("category “portrait”") for row in changed["rows"]
|
||||
)
|
||||
)
|
||||
@@ -490,6 +490,22 @@ class TestJobOutcomesAndLog(unittest.TestCase):
|
||||
with self.assertRaises(ValueError):
|
||||
job.record_counted("bogus", 1)
|
||||
|
||||
def test_record_counted_complete_keeps_every_row_for_the_report(self):
|
||||
# #550: a repair's record of each change is never cut at the cap.
|
||||
job = Job("maintenance", "Pair category and tags", groups=("changed", "errors"))
|
||||
rows = [
|
||||
{"path": "", "detail": f"Prompt {i}"} for i in range(MAX_OUTCOME_ROWS + 5)
|
||||
]
|
||||
job.record_counted("changed", len(rows), rows, complete=True)
|
||||
self.assertEqual(len(job.outcomes["changed"]), MAX_OUTCOME_ROWS)
|
||||
report = job.report()["outcomes"]
|
||||
self.assertEqual(len(report["changed"]["rows"]), MAX_OUTCOME_ROWS + 5)
|
||||
self.assertEqual(
|
||||
report["changed"]["rows"][-1]["detail"], f"Prompt {MAX_OUTCOME_ROWS + 4}"
|
||||
)
|
||||
self.assertFalse(report["changed"]["truncated"])
|
||||
self.assertEqual(report["errors"], {"count": 0, "rows": [], "truncated": False})
|
||||
|
||||
def test_thumbnail_job_groups(self):
|
||||
self.assertEqual(
|
||||
jobs_module.THUMBNAIL_GROUP_NAMES, ("generated", "skipped", "errors")
|
||||
|
||||
@@ -0,0 +1,539 @@
|
||||
"""Pair category and tags repair (#550): the ``pair_metadata`` maintenance plan.
|
||||
|
||||
Before #546 a Pair node run saved its category and tags on the saved pair
|
||||
only, so the positive prompt it created stayed Uncategorized and untagged.
|
||||
``pair_metadata`` reviews those prompts and fills only their empty fields
|
||||
from the prompt's non-archived pairs when every pair with a value agrees.
|
||||
It never touches a negative, a saved pair or ``updated_at``, and a prompt
|
||||
the Pair node did not create, that was edited, archived, or whose pairs
|
||||
disagree, is kept (``550-design.md``).
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from database import maintenance # noqa: E402
|
||||
from database import pair_metadata_repair as repair # noqa: E402
|
||||
|
||||
try:
|
||||
from tests.test_maintenance_plans import MaintenanceDatabaseCase # noqa: E402
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_maintenance_plans import MaintenanceDatabaseCase # noqa: E402
|
||||
|
||||
OPERATION = "pair_metadata"
|
||||
|
||||
|
||||
class PairMetadataCase(MaintenanceDatabaseCase):
|
||||
def pair(self, positive, negative="", **metadata):
|
||||
"""A pre-#546 Pair save: the pair gets the metadata, the prompt none."""
|
||||
pair_id = self.db.save_prompt_pair(positive, negative, **metadata)
|
||||
return pair_id, self.positive_of(pair_id)
|
||||
|
||||
def positive_of(self, pair_id):
|
||||
return self.sql(
|
||||
"SELECT positive_prompt_id FROM prompt_pairs WHERE id=?", pair_id
|
||||
)[0][0]
|
||||
|
||||
def sql(self, statement, *args):
|
||||
with self.db.model.get_connection() as conn:
|
||||
return [tuple(row) for row in conn.execute(statement, args)]
|
||||
|
||||
def run_sql(self, statement, *args):
|
||||
with self.db.model.get_connection() as conn:
|
||||
conn.execute(statement, args)
|
||||
|
||||
def category(self, prompt_id):
|
||||
return self.sql("SELECT category FROM prompts WHERE id=?", prompt_id)[0][0]
|
||||
|
||||
def tags(self, prompt_id):
|
||||
return sorted(
|
||||
row[0]
|
||||
for row in self.sql(
|
||||
"SELECT t.name FROM prompt_tags pt JOIN tags t ON t.id=pt.tag_id "
|
||||
"WHERE pt.prompt_id=?",
|
||||
prompt_id,
|
||||
)
|
||||
)
|
||||
|
||||
def preview(self, operation=OPERATION, roots=None):
|
||||
return self.db.preview_maintenance(operation, [])
|
||||
|
||||
def apply(self, draft):
|
||||
return self.db.apply_maintenance(
|
||||
draft.operation, draft.fingerprint, draft.targets, []
|
||||
)
|
||||
|
||||
def edit_stamp(self, prompt_id, updated_at="2099-01-01T00:00:00+00:00"):
|
||||
self.run_sql(
|
||||
"UPDATE prompts SET updated_at=? WHERE id=?", updated_at, prompt_id
|
||||
)
|
||||
|
||||
def details(self, draft):
|
||||
return [(row["label"], row["detail"]) for row in draft.details]
|
||||
|
||||
|
||||
class TestClassification(PairMetadataCase):
|
||||
def test_fills_empty_category_and_tags_from_the_pair(self):
|
||||
pair_id, prompt = self.pair(
|
||||
"a cat on a sofa", "blurry", category="portrait", tags=["b", "a"]
|
||||
)
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.operation, OPERATION)
|
||||
self.assertEqual(draft.targets, [[prompt, True, True]])
|
||||
self.assertEqual(draft.scope["count"], 1)
|
||||
self.assertEqual(
|
||||
draft.scope["label"], "1 prompt to fill: 1 category, 1 tag set"
|
||||
)
|
||||
self.assertEqual(draft.protected, {"count": 0, "reasons": []})
|
||||
self.assertEqual(
|
||||
draft.media,
|
||||
{
|
||||
"deletes_files": False,
|
||||
"label": "No image file or saved pair is changed.",
|
||||
},
|
||||
)
|
||||
self.assertIn(
|
||||
(
|
||||
"a cat on a sofa",
|
||||
f"Fills category “portrait” and 2 tags: a, b (pair {pair_id})",
|
||||
),
|
||||
self.details(draft),
|
||||
)
|
||||
self.assertEqual(draft.details[-1]["label"], "Record")
|
||||
self.assertEqual(draft.details[-1]["detail"], repair.RECORD_NOTE)
|
||||
|
||||
def test_empty_plan(self):
|
||||
self.save("plain prompt")
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [])
|
||||
self.assertEqual(draft.scope["count"], 0)
|
||||
self.assertEqual(
|
||||
draft.scope["label"], "No Pair prompts are missing a category or tags"
|
||||
)
|
||||
self.assertEqual(draft.details, [])
|
||||
|
||||
def test_prompt_already_complete_is_not_counted(self):
|
||||
_pair, prompt = self.pair("done", category="portrait", tags=["a"])
|
||||
self.run_sql("UPDATE prompts SET category='portrait' WHERE id=?", prompt)
|
||||
self.db.set_prompt_tags(prompt, ["a"])
|
||||
self.edit_stamp(
|
||||
prompt, self.sql("SELECT created_at FROM prompts WHERE id=?", prompt)[0][0]
|
||||
)
|
||||
self.assertEqual(self.preview().scope["count"], 0)
|
||||
|
||||
def test_whitespace_category_is_blank_on_prompt_and_pair(self):
|
||||
_p1, blank_prompt = self.pair("ws prompt", category=" portrait ")
|
||||
self.run_sql("UPDATE prompts SET category=' ' WHERE id=?", blank_prompt)
|
||||
_p2, blank_pair = self.pair("ws pair", category=" ")
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [[blank_prompt, True, False]])
|
||||
self.assertIn("“portrait”", self.details(draft)[0][1])
|
||||
self.assertNotIn(blank_pair, [t[0] for t in draft.targets])
|
||||
|
||||
def test_mixed_fields_category_fills_while_tags_conflict(self):
|
||||
_a, prompt = self.pair("mixed", "n1", category="portrait", tags=["a"])
|
||||
self.pair("mixed", "n2", category="portrait", tags=["b"])
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [[prompt, True, False]])
|
||||
self.assertEqual(draft.scope["count"], 1)
|
||||
self.assertEqual(draft.protected["count"], 1)
|
||||
self.assertEqual(draft.protected["reasons"], ["1 where pairs disagree"])
|
||||
self.assertIn(("mixed", "Kept: pairs disagree on tags"), self.details(draft))
|
||||
|
||||
def test_mixed_fields_tags_fill_while_category_conflicts(self):
|
||||
_a, prompt = self.pair("mixed2", "n1", category="portrait", tags=["a"])
|
||||
self.pair("mixed2", "n2", category="landscape", tags=["a"])
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [[prompt, False, True]])
|
||||
self.assertIn(
|
||||
("mixed2", "Kept: pairs disagree on category: landscape, portrait"),
|
||||
self.details(draft),
|
||||
)
|
||||
|
||||
def test_category_set_and_tags_empty_fills_tags_only(self):
|
||||
_a, prompt = self.pair("has cat", category="portrait", tags=["a"])
|
||||
self.run_sql("UPDATE prompts SET category='mine' WHERE id=?", prompt)
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [[prompt, False, True]])
|
||||
|
||||
def test_pairs_without_a_value_are_ignored(self):
|
||||
_a, prompt = self.pair("partial", "n1", category="portrait", tags=["a"])
|
||||
self.pair("partial", "n2") # no category, no tags
|
||||
self.assertEqual(self.preview().targets, [[prompt, True, True]])
|
||||
|
||||
def test_equal_values_across_pairs_fill(self):
|
||||
_a, prompt = self.pair("same", "n1", category="portrait", tags=["a", "b"])
|
||||
self.pair("same", "n2", category="portrait", tags=["b", "a"])
|
||||
self.assertEqual(self.preview().targets, [[prompt, True, True]])
|
||||
|
||||
def test_category_is_case_sensitive(self):
|
||||
_a, prompt = self.pair("case", "n1", category="Portrait")
|
||||
self.pair("case", "n2", category="portrait")
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [])
|
||||
self.assertEqual(draft.protected["count"], 1)
|
||||
|
||||
def test_edited_prompt_is_protected(self):
|
||||
cases = {
|
||||
"later": "2099-01-01T00:00:00+00:00",
|
||||
"null": None,
|
||||
"legacy": "2020-01-01 10:00:00",
|
||||
"garbage": "yesterday",
|
||||
}
|
||||
for name, stamp in cases.items():
|
||||
with self.subTest(name=name):
|
||||
_a, prompt = self.pair(f"edited {name}", category="portrait")
|
||||
self.edit_stamp(prompt, stamp)
|
||||
draft = self.preview()
|
||||
self.assertNotIn(prompt, [t[0] for t in draft.targets])
|
||||
self.assertIn(
|
||||
(
|
||||
f"edited {name}",
|
||||
"Kept: edited after it was saved; its empty category or "
|
||||
"tags may be intentional",
|
||||
),
|
||||
self.details(draft),
|
||||
)
|
||||
self.run_sql(
|
||||
"DELETE FROM prompt_pairs WHERE positive_prompt_id=?", prompt
|
||||
)
|
||||
|
||||
def test_first_release_stamps_microseconds_apart_are_never_edited(self):
|
||||
_a, prompt = self.pair("caa9649f", category="portrait")
|
||||
self.run_sql(
|
||||
"UPDATE prompts SET created_at=?, updated_at=? WHERE id=?",
|
||||
"2026-10-18T10:00:00.000100+00:00",
|
||||
"2026-10-18T10:00:00.000180+00:00",
|
||||
prompt,
|
||||
)
|
||||
self.run_sql(
|
||||
"UPDATE prompt_pairs SET created_at=? WHERE positive_prompt_id=?",
|
||||
"2026-10-18T10:00:00.000400+00:00",
|
||||
prompt,
|
||||
)
|
||||
self.assertEqual(self.preview().targets, [[prompt, True, False]])
|
||||
|
||||
def test_edited_beats_conflict(self):
|
||||
_a, prompt = self.pair("both", "n1", category="portrait")
|
||||
self.pair("both", "n2", category="landscape")
|
||||
self.edit_stamp(prompt)
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.protected["reasons"], ["1 edited after it was saved"])
|
||||
|
||||
def test_archived_prompt_is_protected(self):
|
||||
_a, prompt = self.pair("archived", category="portrait")
|
||||
self.run_sql(
|
||||
"INSERT INTO prompt_archive(prompt_id, archived_at) VALUES(?, ?)",
|
||||
prompt,
|
||||
"2026-10-31T00:00:00+00:00",
|
||||
)
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [])
|
||||
self.assertEqual(draft.protected["reasons"], ["1 archived in the Library"])
|
||||
self.assertIn(
|
||||
("archived", "Kept: archived in the Library"), self.details(draft)
|
||||
)
|
||||
|
||||
def test_archived_pairs_are_no_source(self):
|
||||
pair_id, prompt = self.pair("old pair", "n1", category="portrait")
|
||||
self.db.archive_prompt_pair(pair_id)
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [])
|
||||
self.assertEqual(draft.protected["count"], 0)
|
||||
# An archived pair also never disagrees with a live one.
|
||||
self.pair("old pair", "n2", category="landscape")
|
||||
self.assertEqual(self.preview().targets, [[prompt, True, False]])
|
||||
self.assertIn("landscape", self.details(self.preview())[0][1])
|
||||
|
||||
def test_prompt_not_created_by_its_pair_is_out_of_scope(self):
|
||||
prompt = self.save("saved by the node first")
|
||||
self.run_sql(
|
||||
"UPDATE prompts SET created_at=?, updated_at=? WHERE id=?",
|
||||
"2026-01-01T00:00:00+00:00",
|
||||
"2026-01-01T00:00:00+00:00",
|
||||
prompt,
|
||||
)
|
||||
self.db.save_prompt_pair("saved by the node first", category="portrait")
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [])
|
||||
self.assertEqual(draft.protected["count"], 0)
|
||||
|
||||
def test_negative_is_never_a_target_and_positive_role_sources_only(self):
|
||||
_a, both = self.pair("dual", "shared negative", category="portrait")
|
||||
neg = self.sql("SELECT id FROM prompts WHERE text='shared negative'")[0][0]
|
||||
# The negative of pair A is the positive of pair B, made later: not
|
||||
# Pair-created as a positive, so out of scope.
|
||||
self.db.save_prompt_pair("shared negative", category="landscape")
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [[both, True, False]])
|
||||
self.assertNotIn(neg, [t[0] for t in draft.targets])
|
||||
|
||||
def test_positive_first_then_negative_is_filled_from_positive_pairs_only(self):
|
||||
_a, prompt = self.pair("pos first", category="portrait")
|
||||
self.db.save_prompt_pair("another", "pos first", category="landscape")
|
||||
draft = self.preview()
|
||||
self.assertIn([prompt, True, False], draft.targets)
|
||||
self.assertIn("“portrait”", dict(self.details(draft))["pos first"])
|
||||
|
||||
def test_more_than_twenty_tags_is_protected(self):
|
||||
tags = [f"t{n:02d}" for n in range(21)]
|
||||
_a, prompt = self.pair("many tags", category="portrait", tags=tags)
|
||||
draft = self.preview()
|
||||
self.assertEqual(draft.targets, [[prompt, True, False]])
|
||||
self.assertEqual(
|
||||
draft.protected["reasons"], ["1 whose pair has more than 20 tags"]
|
||||
)
|
||||
self.assertIn(
|
||||
("many tags", "Kept: its pair has more than 20 tags"), self.details(draft)
|
||||
)
|
||||
|
||||
def test_protected_rows_come_first_within_fifty(self):
|
||||
_c, conflict = self.pair("zz conflict", "n1", category="x")
|
||||
self.pair("zz conflict", "n2", category="y")
|
||||
for n in range(60):
|
||||
self.pair(f"fill {n:02d}", category="portrait")
|
||||
draft = self.preview()
|
||||
self.assertEqual(len(draft.details), maintenance.MAX_DETAILS)
|
||||
self.assertEqual(draft.details[0]["label"], "zz conflict")
|
||||
self.assertEqual(draft.details[-1]["label"], "Record")
|
||||
self.assertEqual(draft.scope["count"], 60)
|
||||
|
||||
def test_detail_rows_list_conflicts_before_other_kept_prompts(self):
|
||||
_a, archived = self.pair("aa archived", category="portrait")
|
||||
self.run_sql(
|
||||
"INSERT INTO prompt_archive(prompt_id, archived_at) VALUES(?, ?)",
|
||||
archived,
|
||||
"2026-10-31T00:00:00+00:00",
|
||||
)
|
||||
_b, edited = self.pair("bb edited", category="portrait")
|
||||
self.edit_stamp(edited)
|
||||
self.pair("zz conflict", "n1", category="x")
|
||||
self.pair("zz conflict", "n2", category="y")
|
||||
draft = self.preview()
|
||||
self.assertEqual(
|
||||
[label for label, _detail in self.details(draft)],
|
||||
["zz conflict", "bb edited", "aa archived"],
|
||||
)
|
||||
# The reasons keep the precedence order.
|
||||
self.assertEqual(
|
||||
draft.protected["reasons"],
|
||||
[
|
||||
"1 archived in the Library",
|
||||
"1 edited after it was saved",
|
||||
"1 where pairs disagree",
|
||||
],
|
||||
)
|
||||
|
||||
def test_same_library_same_fingerprint_and_pair_use_does_not_change_it(self):
|
||||
pair_id, _prompt = self.pair("fp", category="portrait", tags=["a"])
|
||||
first = self.preview().fingerprint
|
||||
self.db.record_prompt_pair_use(pair_id)
|
||||
self.db.update_prompt_pair_metadata(pair_id, notes="a note", title="t")
|
||||
self.assertEqual(self.preview().fingerprint, first)
|
||||
|
||||
|
||||
class TestPreviewWritesNothing(PairMetadataCase):
|
||||
def test_preview_reads_one_read_only_snapshot(self):
|
||||
self.pair("cat", category="portrait", tags=["a"])
|
||||
before = self.sql("SELECT * FROM prompts ORDER BY id")
|
||||
tags = self.sql("SELECT * FROM prompt_tags ORDER BY prompt_id, tag_id")
|
||||
connect = maintenance.sqlite3.connect
|
||||
opened = []
|
||||
|
||||
def spy(*args, **kwargs):
|
||||
opened.append(args[0])
|
||||
return connect(*args, **kwargs)
|
||||
|
||||
with patch.object(maintenance.sqlite3, "connect", side_effect=spy):
|
||||
self.preview()
|
||||
self.assertEqual(len(opened), 1)
|
||||
self.assertTrue(str(opened[0]).endswith("?mode=ro"))
|
||||
self.assertEqual(self.sql("SELECT * FROM prompts ORDER BY id"), before)
|
||||
self.assertEqual(
|
||||
self.sql("SELECT * FROM prompt_tags ORDER BY prompt_id, tag_id"), tags
|
||||
)
|
||||
|
||||
|
||||
class TestApply(PairMetadataCase):
|
||||
def snapshot_pairs(self):
|
||||
return (
|
||||
self.sql("SELECT * FROM prompt_pairs ORDER BY id"),
|
||||
self.sql("SELECT * FROM prompt_pair_tags ORDER BY pair_id, tag_id"),
|
||||
)
|
||||
|
||||
def test_apply_fills_only_empty_fields_and_leaves_pairs_alone(self):
|
||||
_a, prompt = self.pair("apply me", "neg", category="portrait", tags=["a", "b"])
|
||||
_b, partial = self.pair("partial", "n1", category="portrait", tags=["a"])
|
||||
self.pair("partial", "n2", category="portrait", tags=["c"])
|
||||
neg = self.sql("SELECT id FROM prompts WHERE text='neg'")[0][0]
|
||||
pairs = self.snapshot_pairs()
|
||||
stamps = self.sql("SELECT id, created_at, updated_at FROM prompts ORDER BY id")
|
||||
state = self.sql("SELECT * FROM prompt_library_state ORDER BY prompt_id")
|
||||
tag_rows = self.sql("SELECT id, name FROM tags ORDER BY id")
|
||||
draft = self.preview()
|
||||
result = self.apply(draft)
|
||||
self.assertEqual(result["changed"], 2)
|
||||
self.assertEqual(
|
||||
result["rows"],
|
||||
[
|
||||
{"prompt_id": prompt, "category": "portrait", "tags": ["a", "b"]},
|
||||
{"prompt_id": partial, "category": "portrait"},
|
||||
],
|
||||
)
|
||||
self.assertEqual(result["categories"], 2)
|
||||
self.assertEqual(result["tag_sets"], 1)
|
||||
self.assertEqual(
|
||||
(self.category(prompt), self.tags(prompt)), ("portrait", ["a", "b"])
|
||||
)
|
||||
self.assertEqual((self.category(partial), self.tags(partial)), ("portrait", []))
|
||||
self.assertEqual((self.category(neg), self.tags(neg)), (None, []))
|
||||
self.assertEqual(self.snapshot_pairs(), pairs)
|
||||
self.assertEqual(
|
||||
self.sql("SELECT id, created_at, updated_at FROM prompts ORDER BY id"),
|
||||
stamps,
|
||||
)
|
||||
self.assertEqual(
|
||||
self.sql("SELECT * FROM prompt_library_state ORDER BY prompt_id"), state
|
||||
)
|
||||
self.assertEqual(self.sql("SELECT id, name FROM tags ORDER BY id"), tag_rows)
|
||||
|
||||
def test_apply_returns_one_kept_row_per_protected_prompt(self):
|
||||
_a, conflict = self.pair("conflict", "n1", category="asian")
|
||||
self.pair("conflict", "n2", category="Adult")
|
||||
_b, edited = self.pair("edited", category="portrait")
|
||||
self.edit_stamp(edited)
|
||||
_c, both = self.pair(
|
||||
"both", "n1", category="x", tags=[f"t{n}" for n in range(21)]
|
||||
)
|
||||
self.pair("both", "n2", category="y", tags=[f"t{n}" for n in range(21)])
|
||||
_d, filled = self.pair("fill", category="portrait")
|
||||
draft = self.preview()
|
||||
result = self.apply(draft)
|
||||
self.assertEqual(
|
||||
result["kept"],
|
||||
[
|
||||
{
|
||||
"prompt_id": conflict,
|
||||
"reasons": ["pairs disagree on category: Adult, asian"],
|
||||
},
|
||||
{
|
||||
"prompt_id": edited,
|
||||
"reasons": [
|
||||
"edited after it was saved; its empty category or tags "
|
||||
"may be intentional"
|
||||
],
|
||||
},
|
||||
{
|
||||
"prompt_id": both,
|
||||
"reasons": [
|
||||
"pairs disagree on category: x, y",
|
||||
"its pair has more than 20 tags",
|
||||
],
|
||||
},
|
||||
],
|
||||
)
|
||||
self.assertEqual(len(result["kept"]), draft.protected["count"])
|
||||
self.assertEqual(self.category(filled), "portrait")
|
||||
|
||||
def test_kept_rows_are_never_capped(self):
|
||||
for n in range(205):
|
||||
_p, prompt = self.pair(f"kept {n:03d}", category="portrait")
|
||||
self.edit_stamp(prompt)
|
||||
result = self.apply(self.preview())
|
||||
self.assertEqual(result["changed"], 0)
|
||||
self.assertEqual(len(result["kept"]), 205)
|
||||
|
||||
def test_second_run_finds_nothing_and_a_partial_fill_completes_later(self):
|
||||
_a, prompt = self.pair("partial", "n1", category="portrait", tags=["a"])
|
||||
p2, _same = self.pair("partial", "n2", category="portrait", tags=["c"])
|
||||
self.apply(self.preview())
|
||||
again = self.preview()
|
||||
self.assertEqual(again.scope["count"], 0)
|
||||
self.assertEqual(again.protected["count"], 1) # tags still disagree
|
||||
self.db.update_prompt_pair_metadata(p2, tags=["a"])
|
||||
self.apply(self.preview())
|
||||
self.assertEqual(self.tags(prompt), ["a"])
|
||||
self.assertEqual(self.preview().protected["count"], 0)
|
||||
|
||||
def test_drift_raises_plan_changed_and_writes_nothing(self):
|
||||
def pair_category(ids):
|
||||
self.db.update_prompt_pair_metadata(ids[0], category="landscape")
|
||||
|
||||
def prompt_category(ids):
|
||||
self.run_sql("UPDATE prompts SET category='mine' WHERE id=?", ids[1])
|
||||
|
||||
def new_pair(ids):
|
||||
self.db.save_prompt_pair("drift", "other neg", category="landscape")
|
||||
|
||||
def tag_rename(ids):
|
||||
tag = self.sql("SELECT id FROM tags WHERE name='a'")[0][0]
|
||||
self.run_sql("UPDATE tags SET name='renamed' WHERE id=?", tag)
|
||||
|
||||
def edited(ids):
|
||||
self.edit_stamp(ids[1])
|
||||
|
||||
for change in (pair_category, prompt_category, new_pair, tag_rename, edited):
|
||||
with self.subTest(change=change.__name__):
|
||||
ids = self.pair("drift", "n", category="portrait", tags=["a"])
|
||||
draft = self.preview()
|
||||
change(ids)
|
||||
before = self.sql("SELECT id, category FROM prompts ORDER BY id")
|
||||
tags = self.sql("SELECT * FROM prompt_tags ORDER BY prompt_id")
|
||||
with self.assertRaises(maintenance.PlanChanged):
|
||||
self.apply(draft)
|
||||
self.assertEqual(
|
||||
self.sql("SELECT id, category FROM prompts ORDER BY id"), before
|
||||
)
|
||||
self.assertEqual(
|
||||
self.sql("SELECT * FROM prompt_tags ORDER BY prompt_id"), tags
|
||||
)
|
||||
self.run_sql("DELETE FROM prompt_pairs")
|
||||
self.run_sql("DELETE FROM prompts")
|
||||
self.run_sql("DELETE FROM tags")
|
||||
|
||||
def test_pair_run_between_preview_and_apply_still_applies(self):
|
||||
pair_id, prompt = self.pair("busy", category="portrait")
|
||||
draft = self.preview()
|
||||
self.db.record_prompt_pair_use(pair_id)
|
||||
self.apply(draft)
|
||||
self.assertEqual(self.category(prompt), "portrait")
|
||||
|
||||
def test_guard_failure_rolls_back_everything(self):
|
||||
_a, first = self.pair("first", category="portrait")
|
||||
_b, second = self.pair("second", category="portrait")
|
||||
draft = self.preview()
|
||||
real = repair._fill_categories
|
||||
|
||||
def racing(conn, fills):
|
||||
# Another writer's value appears inside the transaction.
|
||||
conn.execute("UPDATE prompts SET category='x' WHERE id=?", (second,))
|
||||
return real(conn, fills)
|
||||
|
||||
with patch.object(repair, "_fill_categories", side_effect=racing):
|
||||
with self.assertRaises(maintenance.PlanChanged):
|
||||
self.apply(draft)
|
||||
self.assertIsNone(self.category(first))
|
||||
self.assertIsNone(self.category(second))
|
||||
|
||||
def test_more_than_two_hundred_fills_are_all_returned(self):
|
||||
for n in range(205):
|
||||
self.pair(f"bulk {n:03d}", category="portrait")
|
||||
result = self.apply(self.preview())
|
||||
self.assertEqual(result["changed"], 205)
|
||||
self.assertEqual(len(result["rows"]), 205)
|
||||
|
||||
def test_batches_bind_few_ids(self):
|
||||
for n in range(5):
|
||||
self.pair(f"batch {n}", category="portrait", tags=["a"])
|
||||
with patch.object(maintenance, "ID_BATCH", 2):
|
||||
draft = self.preview()
|
||||
result = self.apply(draft)
|
||||
self.assertEqual(result["changed"], 5)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -11,6 +11,9 @@
|
||||
* - Results (`section.surface`): the outcome list (`button[aria-pressed]` with counts, in the mockup's order), a path
|
||||
* filter, and the selected outcome's listed rows (at most 200 per outcome, so "Showing N of M"). Rows under Added and
|
||||
* Already known open the file in the existing viewer through its listing identity (D57); the others are plain text.
|
||||
* - #550: a row with no path is a record (a maintenance apply's "Prompt 12: category …"): its detail is shown as plain
|
||||
* text, never opened; the filter reads it. A maintenance plan of records (every operation but Missing-file links)
|
||||
* says "records" where a file job says "files" (`groupCopy`).
|
||||
* - Log: `details.term` "Scan log", collapsed, holding the raw timestamped lines in `pre[tabindex=0]`.
|
||||
* - D89: the tiles, the outcome list, the openable rows and the log and cancel labels are the job kind's (`jobView`):
|
||||
* an image tagging job shows Tagged, Unchanged, Skipped and Errors, opens Tagged and Unchanged rows, and reads
|
||||
@@ -44,6 +47,10 @@ export const DETAIL_INTERVAL_MS = 1000;
|
||||
export const NOT_FOUND = 'This job is not in this session’s Activity. Jobs are kept until ComfyUI restarts.';
|
||||
export const EMPTY_GROUP = 'No files in this group.';
|
||||
export const NO_MATCH = 'No listed file matches this filter.';
|
||||
export const EMPTY_RECORDS = 'No records in this group.';
|
||||
export const NO_RECORD_MATCH = 'No listed record matches this filter.';
|
||||
const FILE_FILTER = 'Filter by file path…';
|
||||
const RECORD_FILTER = 'Filter records…';
|
||||
export const VIEWER_HINT = 'each row opens the image in the viewer';
|
||||
export const FILE_GONE = 'This file is no longer in Images. Rescan from Import and try again.';
|
||||
export const OPEN_FAILED = 'The file could not be opened. Try again.';
|
||||
@@ -68,10 +75,11 @@ export function parseDetail(raw) {
|
||||
const groups = raw.outcomes !== null && typeof raw.outcomes === 'object' ? raw.outcomes : {};
|
||||
for (const [group, rows] of Object.entries(groups)) {
|
||||
if (!Array.isArray(rows)) continue;
|
||||
outcomes[group] = rows.filter(row => row && isText(row.path) && row.path)
|
||||
// A file row has a path; a record row (#550) has none, only its detail.
|
||||
outcomes[group] = rows.filter(row => row && isText(row.path) && (row.path || (isText(row.detail) && row.detail)))
|
||||
.map(row => ({ path: row.path, root: isText(row.root) ? row.root : '',
|
||||
root_index: Number.isSafeInteger(row.root_index) && row.root_index >= 0 ? row.root_index : null,
|
||||
detail: isText(row.detail) ? row.detail : '' }));
|
||||
detail: isText(row.detail) ? row.detail : '', record: !row.path }));
|
||||
}
|
||||
const log = Array.isArray(raw.log) ? raw.log.filter(isText) : [];
|
||||
return { job, outcomes, log };
|
||||
@@ -129,11 +137,23 @@ export function resultsNote(shown, total, openable) {
|
||||
return parts.length === 1 && openable && shown >= total ? `${text[0].toUpperCase()}${text.slice(1)}.` : text;
|
||||
}
|
||||
|
||||
/** Rows whose path (or root label and path) holds the case-folded filter. */
|
||||
/** Rows whose path (or root label and path), or a record's detail (#550), holds the case-folded filter. */
|
||||
export function filterRows(rows, filter) {
|
||||
const wanted = filter.trim().toLocaleLowerCase();
|
||||
if (!wanted) return rows;
|
||||
return rows.filter(row => `${row.root}/${row.path}`.toLocaleLowerCase().includes(wanted));
|
||||
return rows.filter(row => (row.record ? row.detail : `${row.root}/${row.path}`).toLocaleLowerCase().includes(wanted));
|
||||
}
|
||||
|
||||
/** #550: whether a job lists records (a maintenance plan of records), not files. */
|
||||
export function listsRecords(job) {
|
||||
return job?.kind === 'maintenance' && job.scope?.operation !== 'missing_links';
|
||||
}
|
||||
|
||||
/** The empty-group, no-match and filter copy: "records" for a records job, "files" otherwise (#550). */
|
||||
export function groupCopy(job) {
|
||||
return listsRecords(job)
|
||||
? { empty: EMPTY_RECORDS, noMatch: NO_RECORD_MATCH, placeholder: RECORD_FILTER }
|
||||
: { empty: EMPTY_GROUP, noMatch: NO_MATCH, placeholder: FILE_FILTER };
|
||||
}
|
||||
|
||||
/** The first outcome with files, so a view opens on what happened; the first outcome when nothing has yet. */
|
||||
@@ -176,7 +196,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
const search = element('label', undefined, 'search-field');
|
||||
const input = element('input');
|
||||
input.type = 'search';
|
||||
input.placeholder = 'Filter by file path…';
|
||||
input.placeholder = FILE_FILTER;
|
||||
input.setAttribute('aria-label', 'Filter results');
|
||||
search.append(input);
|
||||
resultsPart.heading.append(search);
|
||||
@@ -280,9 +300,13 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
// Listed rows only ever grow, so a row keeps its node across reads: a focused row (or the viewer's trigger) stays.
|
||||
let rowNodes = new Map();
|
||||
function rowNode(row, index, openable) {
|
||||
const key = `${selected}\n${index}\n${row.root}\n${row.path}`;
|
||||
const key = `${selected}\n${index}\n${row.root}\n${row.path}\n${row.record ? row.detail : ''}`;
|
||||
let node = rowNodes.get(key);
|
||||
if (!node) {
|
||||
if (!node && row.record) {
|
||||
// #550: a record is its detail, plain text, with no open or locate action.
|
||||
node = element('div', undefined, 'file-row record-row');
|
||||
node.append(element('span', row.detail, 'why'));
|
||||
} else if (!node) {
|
||||
node = openable ? createButton('', 'file-row') : element('div', undefined, 'file-row');
|
||||
node.append(element('span', row.path, 'path'), element('span', row.detail, 'why'));
|
||||
if (openable) node.row = row;
|
||||
@@ -300,8 +324,11 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
const nodes = [...kept.values()];
|
||||
// The counter is exact; at most 200 of its files are listed, and the filter narrows those.
|
||||
const total = Math.max(job.counters[selected] ?? 0, rows.length);
|
||||
if (!nodes.length) nodes.push(element('p', rows.length ? NO_MATCH : EMPTY_GROUP, 'empty-state'));
|
||||
const note = wanted.length ? resultsNote(wanted.length, total, openable) : '';
|
||||
const copy = groupCopy(job);
|
||||
input.placeholder = copy.placeholder;
|
||||
if (!nodes.length) nodes.push(element('p', rows.length ? copy.noMatch : copy.empty, 'empty-state'));
|
||||
const opens = openable && wanted.some(row => !row.record);
|
||||
const note = wanted.length ? resultsNote(wanted.length, total, opens) : '';
|
||||
if (note) nodes.push(element('p', note, 'small'));
|
||||
for (const child of [...files.children]) if (!nodes.includes(child)) child.remove();
|
||||
nodes.forEach((node, index) => {
|
||||
|
||||
@@ -49,6 +49,8 @@ export const OPERATION_TITLES = Object.freeze({
|
||||
duplicate_prompts: 'Duplicate prompt text',
|
||||
optimize: 'Optimize database',
|
||||
unused_components: 'Unused prompt components',
|
||||
// D192 (#550): beyond the concept, System › Maintenance › Database.
|
||||
pair_metadata: 'Pair category and tags',
|
||||
missing_links: 'Missing-file links',
|
||||
duplicate_files: 'Duplicate image files',
|
||||
// Reviewed and applied through the API and seen in Activity; Maintenance has no row for it (the concept has none).
|
||||
|
||||
@@ -2,10 +2,12 @@
|
||||
* System › Maintenance (#249 System S7 Task 19): the concept's `renderSystem` "maintenance" body (app.js 688–690).
|
||||
*
|
||||
* - The notice "Check first. Review changes. Then apply." with "Run health checks" (D123), then the surfaces
|
||||
* "Database" and "Images & cache" with the concept's eight operations, in its order, labels and danger marks.
|
||||
* "Database" and "Images & cache" with the concept's eight operations, in its order, labels and danger marks, plus
|
||||
* D192 (#550, beyond the concept): "Pair category and tags" after "Unused prompt components".
|
||||
* - Integrity & relationship checks → Check runs the same read-only health check and opens the Health report (D95);
|
||||
* its "Review missing links" reviews a fresh Missing-file links plan, never a cleanup.
|
||||
* - Duplicate prompt text, Optimize database, Unused prompt components and Missing-file links → Review plan open the
|
||||
* - Duplicate prompt text, Optimize database, Unused prompt components, Pair category and tags (`pair_metadata`: fill
|
||||
* old Pair prompts' empty category and tags from their saved pairs) and Missing-file links → Review plan open the
|
||||
* D100 plan review (maintenance-review.mjs), which applies only what it showed, as an Activity job.
|
||||
* - Duplicate image files → Find duplicates starts the duplicate search as an Activity job and opens it; its view
|
||||
* offers Review groups (D101) once it found some.
|
||||
@@ -39,6 +41,9 @@ export const MAINTENANCE_SURFACES = Object.freeze([
|
||||
['Optimize database', 'Reclaim unused SQLite space. This does not delete prompts or media.', 'Review plan', 'optimize'],
|
||||
['Unused prompt components', 'Preview unreferenced components. Pair references and history remain protected.', 'Review plan',
|
||||
'unused_components', true],
|
||||
// D192 (#550): beyond the concept. Fills only empty fields, deletes nothing (no danger mark).
|
||||
['Pair category and tags', 'Fill an empty category or tags on prompts older Pair node runs saved, from their saved ' +
|
||||
'pairs. Only empty fields of never-edited prompts.', 'Review plan', 'pair_metadata'],
|
||||
]],
|
||||
['Images & cache', [
|
||||
['Missing-file links', 'Remove stale database links only. Files on disk are never deleted by this operation.', 'Review plan',
|
||||
|
||||
Reference in New Issue
Block a user