Compare commits

..
Author SHA1 Message Date
Vito Sansevero b1db059a41 test(comfyui): count #347's cached-run cases in the execution suite pin (#347)
The pin listed 19 execution-text-link cases per project; #347 adds five node
kinds and eight one-renderer cases, so the suite lists 32 per project and 118
in total with #332's image-attribution case from main.
2026-10-28 09:05:13 -07:00
Vito Sansevero 28305ef2fa Merge remote-tracking branch 'origin/main' into fix/347-cached-runtime-runs 2026-10-28 09:04:37 -07:00
Vito Sansevero fb2f8e6e46 Merge remote-tracking branch 'origin/main' into fix/347-cached-runtime-runs 2026-10-28 08:40:03 -07:00
Vito Sansevero 9e0f4744cd Merge remote-tracking branch 'origin/main' into fix/347-cached-runtime-runs 2026-10-28 06:14:06 -07:00
Vito Sansevero 5bc3bc8bf6 Merge remote-tracking branch 'origin/main' into fix/347-cached-runtime-runs 2026-10-28 04:19:31 -07:00
Vito Sansevero 6559de8255 fix(usage): never replay a node twice in one job; dedupe receipts once complete (#347)
A repeated execution_cached id or message for the same job now counts the
node once. Receipts replace equal older ones only when their job finishes
(or the next job starts), so a list still recording [A] of [A, B] no longer
drops an older [A] receipt. Node-level tests check the receipt each
execution records.
2026-10-27 19:43:48 -07:00
Vito Sansevero 1b67cb7ccb fix(usage): count cached re-runs of runtime-linked prompt text (#347)
A PromptManager-family node whose text comes from a runtime link
(PreviewAny, an LLM, a list node) counted only the job that executed it;
later jobs served from ComfyUI's output cache counted nothing.

Each execution now records a receipt (digests of its STRING outputs and
hashes of the text it stores) per run key and queue item. On
execution_cached the receipt whose digests equal the cache entry's text is
replayed once for the job, with the node's role and what the queue hook
already saw recomputed from the job's own graph. The executing node and the
replay count through one shared own_counts. Unmatched cache entries count
nothing, as before; a node that executes after its replay in the same job
skips its own count. No schema change, no backfill.
2026-10-27 19:26:13 -07:00
64 changed files with 220 additions and 4038 deletions
-6
View File
@@ -312,12 +312,6 @@ Both nodes support the same metadata fields:
Both nodes will automatically save prompts to the database and link any generated images to the prompt.
- **Save to library** (`save_to_library`, on by default; "Save to library" in the Focused, Split and Pair editors):
turn it off while iterating and PromptManager ignores that node's runs. The prompt is not saved, the run is not
counted and its images are not linked (a scan skips them too), while the node's output is unchanged. Workflows
saved before this switch existed load with it on. If any PromptManager node feeding a sampler's positive input
has it off, none of that run's images are linked, because the image cannot show which sampler produced it.
### 🖼️ Using the Output-Wide Gallery
The comprehensive gallery system provides complete access to all your ComfyUI output:
+7 -67
View File
@@ -391,10 +391,6 @@ A spec performance budget gets two tests:
`test_filter_budget` (Images filters, spec D12: p95 < 150 ms at 50,000 files
and 20,000 links).
A hot SQLite read that returns many rows aggregates them into one step
(`json_group_array`) instead of stepping once per row, so it does not wait for
the GIL once per row while ComfyUI generates (#354, `get_image_link_matches`).
The `perf budget` job in `.github/workflows/test.yml` is the reference
measurement: `ubuntu-latest`, one Python, no coverage, only the wall-clock tests.
It runs on every push to `main` and on `workflow_dispatch`, never on pull
@@ -1344,7 +1340,7 @@ How the earlier workspace Images filters moved (old → new). The legacy `/galle
| Folder | "Refine this view" → Folder (D14); shown in the scope chip |
| Media (All media / Images / Videos) | "Refine this view" → Media (D14) |
| Sort: Newest first / Oldest first / Filename A–Z / Filename Z–A | toolbar sort: Recently added / Oldest first / Name A–Z / Name Z–A, plus Highest rated (D13, D14) |
| (none) | toolbar category (the effective category, D141) and "Refine this view" → Tags, through the linked positive (D13, D14) |
| (none) | toolbar category and "Refine this view" → Tags, through the linked positive (D13, D14) |
| Refresh | Import → "Scan an output folder" → Rescan (D15) |
| Grid / List / Masonry, Comfortable / Large / Compact | the same choices in the concept filter row |
| Card image → side panel; preview icon → viewer | image → viewer; Info → side panel (D19) |
@@ -1427,8 +1423,7 @@ Recorded deviations on this page (spec §8):
`overscroll-behavior: contain`.
- **D35**: counts pluralize ("1 positive prompt", "1 image", "1 more image loaded").
- **D36**: the category list is the Library's, sorted alphabetically (case-insensitive)
by the page, whatever order the server sends (amended by D141: Images adds its
image categories).
by the page, whatever order the server sends.
- **D37**: production loads 40 images per page (the concept 12), so the continuation
and toast use 40; after Escape production returns focus to the opening image.
- **D44** (#249 PR E): Filters reads "Filters · N" while N of Source, Folder, Media and
@@ -1514,46 +1509,15 @@ Recorded deviations on this page (spec §8):
- **D39** (ruled 2026-10-21: fallback): a recipe-linked image shows its recipe's own
category when set, otherwise the linked positive prompt's, and its recipe's own tags
when it has any, otherwise the positive's; each field falls back on its own and the
two tag sets are never merged. The Images Tags filter matches the positive's tags.
The category filter matches the same inherited category as the panel (D142, #332), and
an image's own category overrides both (D141, D146). The inspection response names each
two tag sets are never merged. Set category (#259) writes the positive's category and
the Images category and Tags filters match the positive's, so the panel now agrees
with them whenever the recipe adds nothing. The inspection response names each
field's source (`library.category_scope`, `library.tags_scope`: `pair`, `positive` or
null); the category chip's visually hidden prefix (D21) says "Recipe category: " or
"Positive prompt category: " from it, and tag chips are Tags filter buttons only when
they are the positive's (#260 D2). History's media inspector uses the same rule. The
concept shows the prompt's category and tags for every image and the concept
fixture's recipes have neither, so the inspector comparison needs no mapping for it.
- **D141** (#332, amends D13 and D36): the Images category filter matches each image's
effective category: its own category (Images Set category, PR 3) when it has one,
else its inherited one (D142), else none. An image whose own category differs never
matches its prompt's category, and an unlinked or ambiguous image with an own
category is listed under it (the members-only gate now applies to the Tags filter
only). "Select all N matching" and saved views use the same value. The caption's
positive-prompt count still counts the linked positives of the images shown. With a
category set the listing reads the own categories once per source (a primary-key
range); a failed read is the existing 500 `listing-failed`, never an unfiltered page.
The link read then keeps only links whose raw recipe or positive category is the
value (a superset of those that inherit it; Python decides), and reads the links of
files whose own category is the value by path, so their rating and positive count.
`test_filter_budget` seeds 2,000 own categories on its 50,000-file fixture.
The option list adds the image categories of listed files
(`GET /prompt_manager/images/categories`, PR 3).
- **D142** (#332 MC4, amends D39's filter note): the inherited category is one rule,
`inherited_category` in `utils/inspection_response.py`: a linked recipe's own
category when it is not blank, else the linked positive's. Before #332 the filter
matched the positive's even when the chip showed the recipe's. The link reads select
the raw recipe and positive categories and never decide inheritance in SQL
(`tests/test_api_workspace_image_categories.py` checks the filter and the chip agree
on blank values stored as spaces, tabs, U+00A0 and BLOB). The concept has no recipe
categories, so this is behavior only.
- **D145** (#332, DC4): History's media inspector shows an image's own category with
the chip rule of the side panel and viewer. Only the inspection request reads it, once,
inside the History snapshot: the file is the symlink-resolved output folder plus the
receipt's subfolder and filename, keyed with `image_category_key` over the configured
output sources, as Images keys it. A temp output or a file no source holds has none;
a failed read hides the chip (DC5). The outputs list and media requests read nothing
new. The chip's hidden prefix reads "Image category: " for an own category; the
visible origin text and its comparison are D146 (PR 3).
- **D42**: the concept never shows a toast and a page alert together. Production keeps
the concept's fixed toast (centred, 22px from the bottom, 12px at 700px and below)
and, when an Import → Rescan toast leaves the Images discovery alert
@@ -4660,18 +4624,8 @@ Filters (`mode=workspace`; anything invalid is 400 "Invalid gallery filters"):
rating 5 to 1, then linked but unrated, then unlinked and ambiguous; ties break
newest first. A signed cursor covers every filter and the sort, so reusing it
with a change is 400.
- One link read (`get_image_link_matches`) serves the filters, the rating sort
and the count. SQLite returns its rows as one JSON value in one step (#354):
Python's `sqlite3` releases and retakes the GIL around every step, and while
ComfyUI generates each retake can wait on the busy thread, so a read that
stepped once per linked row took seconds. The result keeps only the listed
paths; peak memory grows with the database's matching linked rows (about
530 bytes each, freed on return). When the JSON step fails (a SQLite without
JSON functions, a BLOB value, a JSON text over SQLite's length limit) the read
streams the rows as before and logs one warning per process; a missing JSON
function stops the attempt for the rest of the process. An interrupted read
(Move library) raises as before. `tests/test_database_link_matches.py` proves both reads
equal on every filter and on odd values.
- One link read (`get_image_link_matches`, streamed and bounded to the listed
paths) serves the filters, the rating sort and the count.
`positive_count` is the number of distinct linked positives over the full matching
set, not the page:
@@ -5781,20 +5735,6 @@ rotates the epoch, so a pre-restore `expected_epoch` is refused with `storage-ch
Owner rows and their timestamps never change on a tag command; a category apply only
touches rows carrying the exact source.
Image categories (#332 §7, MC9, MC10, DC7): each `GET categories` row also carries
`image_count`, the stored `image_categories` rows holding that exact value (files
missing on disk included). A value prompts and images share appears once, ordered by
its oldest positive; a value only images hold appears once after every prompt-backed
value (`first_pid` 9223372036854775807 in the keyset cursor), ordered by value, with
`prompt_count` 0. `/library/options` keeps its own query and never lists image-only
values. Category preview and apply report `impact.images` (rows holding the source),
the target exists when an image row holds it, an image-only value can be renamed, the
apply renames the image rows in the same transaction (file keys and rowids kept, so a
merge cannot collide), and the impact token hashes the sorted image rowids, so an image
row gaining or losing the value between preview and apply is 409
`vocabulary-conflict` with nothing written. Every image-category statement reads
`idx_image_categories_category` (`test_category_reads_use_the_image_category_index`).
Two legacy routes change additively: `GET /tags` now lists unused registry tags, and
`/tags/suggest` without context appends unused names at count 0. Unused tags become
deletable only through the native Delete tag (vocabulary UI PR), not on the legacy
+41 -184
View File
@@ -7,12 +7,7 @@ import json
import math
import datetime
import os
from typing import Any, Dict, Iterable, List, Optional, Sequence, Set, Tuple
try:
from ..utils.inspection_response import inherited_category
except ImportError:
from utils.inspection_response import inherited_category
from typing import Optional, List, Dict, Any, Set, Tuple
from .models import PromptModel, normalize_image_path, utc_now_iso
from .pairs import PairDependencyError, PairOperations # noqa: F401 (re-exported)
@@ -243,17 +238,12 @@ LEFT JOIN prompt_library_state pls ON pls.prompt_id = pp.positive_prompt_id
WHERE gi.image_path IN ({marks}) OR gi.file_path IN ({marks}){fallback}"""
# One image row describes a card only when no other row claims the same file
# (as in get_image_link_summaries, several rows are ambiguous). The pair scope
# and the linked positive follow _LINK_SUMMARY_SQL; rating, tags and the
# searchable text and titles come from that positive (spec D12/D13). The raw
# recipe and positive categories are read for the inherited category, which
# Python decides (``inherited_category``, #332 MC4); SQL never filters by
# category. Every
# and the linked positive follow _LINK_SUMMARY_SQL; category, rating, tags and
# the searchable text and titles come from that positive (spec D12/D13). Every
# writer since the 3.2.4 migration stores file_path, so it is the path key.
# The prompt-level conditions are uncorrelated IN sets SQLite builds once, and
# the CASE evaluates the per-row ambiguity probe only for admitted rows.
_LINK_FILTER_SQL = """SELECT gi.file_path AS file_path, p.rating AS rating,
{hit} AS matched, p.id AS positive, pp.id IS NOT NULL AS has_pair,
pp.category AS pair_category, p.category AS category
_LINK_FILTER_SQL = """SELECT gi.file_path, p.rating, {hit}, p.id
FROM generated_images gi
LEFT JOIN prompt_pairs pp ON pp.id = (
SELECT CASE WHEN COUNT(DISTINCT ip.pair_id) = 1 THEN MIN(ip.pair_id) END
@@ -262,31 +252,10 @@ JOIN prompts p ON p.id = COALESCE(pp.positive_prompt_id, gi.prompt_id)
WHERE gi.file_path IS NOT NULL AND CASE WHEN {admit} THEN NOT EXISTS (
SELECT 1 FROM generated_images other
WHERE other.file_path = gi.file_path AND other.id <> gi.id) ELSE 0 END"""
# _LINK_FILTER_SQL's rows as one JSON array, built by SQLite in one step (#354).
# Python's sqlite3 releases and retakes the GIL around every step, so a read
# that steps once per linked row waits on a busy ComfyUI thread thousands of
# times. A BLOB value (no writer stores one) makes the value NULL (or
# json_array refuses it), and the read streams instead. The raw categories are
# put in only for a category read (#332), so plain reads stay as small as #354
# measured them.
_LINK_MATCHES_JSON_SQL = """SELECT CASE
WHEN max({blob}) THEN NULL
ELSE json_group_array(json_array({columns})) END
FROM ({select})"""
_LINK_PLAIN_COLUMNS = ("file_path", "rating", "matched", "positive")
_LINK_CATEGORY_COLUMNS = _LINK_PLAIN_COLUMNS + ("has_pair", "pair_category", "category")
# The columns a BLOB can reach: matched, positive and has_pair are computed.
_LINK_STORED_COLUMNS = ("file_path", "rating", "pair_category", "category")
# Set when this SQLite has no JSON functions: every later read streams.
_one_step_unsupported = False
# One warning per process when a read falls back to streaming.
_one_step_warned = False
# The linked positive of each given path (one unambiguous link, pair scope as
# in _LINK_FILTER_SQL) with the raw categories of its inherited category; {hit}
# as in link_filter_query. Used for the few files whose current image tags meet
# a Tags filter (#249 B3).
_LINK_FACTS_SQL = """SELECT gi.file_path, p.rating, {hit}, p.id,
pp.id IS NOT NULL, pp.category, p.category
# in _LINK_FILTER_SQL) with its category; {hit} as in link_filter_query. Used
# for the few files whose current image tags meet a Tags filter (#249 B3).
_LINK_FACTS_SQL = """SELECT gi.file_path, p.rating, {hit}, p.id, p.category
FROM generated_images gi
LEFT JOIN prompt_pairs pp ON pp.id = (
SELECT CASE WHEN COUNT(DISTINCT ip.pair_id) = 1 THEN MIN(ip.pair_id) END
@@ -315,24 +284,8 @@ _LINK_FILTER_TAG = (
)
# One unambiguously linked image's facts for the Images filters, a plain tuple
# (hit, rating, positive id, inherited category): built per linked row, so no
# NamedTuple. The category is the inherited one (#332 MC4: the recipe's when
# set, else the positive's), never the image's own category.
LinkMatch = Tuple[bool, Optional[int], int, Any]
def _link_match(rating, matched, positive, has_pair, pair_category, category):
inherited, _scope = inherited_category(category, pair_category, bool(has_pair))
return (bool(matched), rating, positive, inherited)
def link_filter_query(
q: str,
tags: List[str],
*,
every_link: bool = False,
category: Optional[str] = None,
q: str, category: Optional[str], tags: List[str], *, every_link: bool = False
) -> Tuple[str, Dict[str, Any]]:
"""The workspace filter SELECT and its named parameters (all bound).
@@ -346,9 +299,7 @@ def link_filter_query(
params["pattern"] = f"%{escape_like(q)}%"
admitted = []
if category is not None:
# Only narrows: an inherited category is one of the two raw values;
# which one is decided in Python (inherited_category, #332 MC4).
admitted.append("(p.category = :category OR pp.category = :category)")
admitted.append("p.category = :category")
params["category"] = category
for index, tag in enumerate(tags):
admitted.append(_LINK_FILTER_TAG.format(index=index))
@@ -359,50 +310,6 @@ def link_filter_query(
return sql, params
def link_matches_json_query(sql: str, categories: bool = False) -> str:
"""*sql* from link_filter_query, wrapped to return its rows as one JSON value.
With *categories* each row also carries the raw values the inherited
category is decided from; otherwise only path, rating, hit and positive.
"""
columns = _LINK_CATEGORY_COLUMNS if categories else _LINK_PLAIN_COLUMNS
blob = " OR ".join(
f"typeof({name}) = 'blob'" for name in _LINK_STORED_COLUMNS if name in columns
)
return _LINK_MATCHES_JSON_SQL.format(
blob=blob, columns=", ".join(columns), select=sql
)
def _link_matches_from_rows(
rows: Iterable[Sequence[Any]], categories: bool, wanted: Optional[Set[str]]
) -> Dict[str, LinkMatch]:
"""LinkMatch per wanted path from link rows (one-step or streamed).
Without *categories* the inherited category is None, as the caller then
never compares it.
"""
matches: Dict[str, LinkMatch] = {}
for path, rating, matched, positive, *raw in rows:
if wanted is None or path in wanted:
if categories:
matches[path] = _link_match(rating, matched, positive, *raw[:3])
else:
matches[path] = (bool(matched), rating, positive, None)
return matches
def _streamed_link_matches(
conn: sqlite3.Connection,
sql: str,
params: Dict[str, Any],
wanted: Optional[Set[str]],
categories: bool = False,
) -> Dict[str, LinkMatch]:
"""The link read stepped once per row (the fallback of the one-step read)."""
return _link_matches_from_rows(conn.execute(sql, params), categories, wanted)
_ASCII_FOLD = str.maketrans("ABCDEFGHIJKLMNOPQRSTUVWXYZ", "abcdefghijklmnopqrstuvwxyz")
@@ -2636,94 +2543,38 @@ class PromptDatabase(
def get_image_link_matches(
self,
q: str,
category: Optional[str],
tags: List[str],
*,
every_link: bool = False,
wanted: Optional[Set[str]] = None,
category: Optional[str] = None,
) -> Dict[str, LinkMatch]:
"""Map normalized paths of unambiguously linked images to a LinkMatch
(hit, rating, positive id, inherited category).
) -> Dict[str, Tuple[bool, Optional[int], int]]:
"""Map normalized paths of unambiguously linked images to
(hit, rating, positive id).
Rows are limited to images whose linked positive has every tag in
*tags* (exact names). The category filter is applied by the caller to
the effective category (#332). ``hit`` says whether *q* occurs in
Rows are limited to images whose linked positive has *category* and
every tag in *tags* (exact names). ``hit`` says whether *q* occurs in
the positive's text, Library title or a tag name; a pair's own title
names its negative variant and is not searched.
Without tags only hits are returned, unless *every_link*
Without category or tags only hits are returned, unless *every_link*
asks for every linked image (the rating sort and the positive count).
The positive id is the card's linked positive. With *category* only
links whose raw recipe or positive category is that value are read (a
superset of the links that inherit it; the caller compares the
inherited value) and the inherited category is decided per row;
otherwise it is None, which keeps the plain reads cheap.
One SELECT, returned as one JSON value (#354): SQLite steps once, so
the read takes the GIL back a constant two or three times instead of
once per linked row while ComfyUI generates. Peak memory therefore
grows with the database's matching linked rows (about 530 bytes each,
more with the raw categories of a category read, freed on return); the
result is bounded by *wanted* (the listing's normalized paths) when
given. Where the JSON step fails (no JSON functions, a BLOB value, a
JSON text over SQLite's length limit) the rows are streamed as before.
The positive id is the card's linked positive. One SELECT whose rows
are streamed; with *wanted* (the listing's normalized paths) only those
paths are kept, so memory is bounded by the listing, not the database.
"""
sql, params = link_filter_query(
q, tags, every_link=every_link, category=category
)
categories = category is not None
sql, params = link_filter_query(q, category, tags, every_link=every_link)
matches: Dict[str, Tuple[bool, Optional[int], int]] = {}
with self.model.get_connection() as conn:
rows = None
if not _one_step_unsupported:
rows = self._one_step_link_rows(conn, sql, params, categories)
if rows is None:
return _streamed_link_matches(conn, sql, params, wanted, categories)
return _link_matches_from_rows(rows, categories, wanted)
def _one_step_link_rows(
self,
conn: sqlite3.Connection,
sql: str,
params: Dict[str, Any],
categories: bool = False,
) -> Optional[List[List[Any]]]:
"""*sql*'s rows from one JSON value, or None when this read must stream.
An interrupted read (Move library retiring the connection) is raised
as before, never retried row by row.
"""
global _one_step_unsupported, _one_step_warned
error: Optional[str] = None
try:
cursor = conn.execute(link_matches_json_query(sql, categories), params)
try:
((document,),) = cursor.fetchall()
finally:
cursor.close() # resets the statement: its snapshot ends now
except (sqlite3.OperationalError, sqlite3.DataError) as failure:
# DataError: the JSON text is longer than SQLITE_LIMIT_LENGTH.
error = str(failure)
if error == "interrupted":
raise
if error.startswith("no such function"):
_one_step_unsupported = True
else:
if document is not None:
return json.loads(document)
error = "a BLOB value"
if not _one_step_warned:
_one_step_warned = True
self.logger.warning(
"Reading image links row by row (slower while ComfyUI "
"generates); the one-step read failed: %s",
error,
)
return None
for path, rating, matched, positive in conn.execute(sql, params):
if wanted is None or path in wanted:
matches[path] = (bool(matched), rating, positive)
return matches
def get_image_link_facts(
self, paths: List[str], q: str = ""
) -> Dict[str, Tuple[bool, Optional[int], int, Any, Tuple[str, ...]]]:
) -> Dict[str, Tuple[bool, Optional[int], int, Optional[str], Tuple[str, ...]]]:
"""Map each normalized path with one unambiguous link to its positive's
(hit, rating, positive id, inherited category, tag names).
(hit, rating, positive id, category, tag names).
The link is resolved as in get_image_link_matches, and ``hit`` says
whether *q* occurs in the positive's text, Library title or a tag
@@ -2734,7 +2585,7 @@ class PromptDatabase(
if q:
hit = _LINK_FILTER_HIT
params["pattern"] = f"%{escape_like(q)}%"
rows: Dict[str, LinkMatch] = {}
rows: Dict[str, Tuple[bool, Optional[int], int, Optional[str]]] = {}
names: Dict[int, List[str]] = {}
unique = list(dict.fromkeys(paths))
with self.model.get_connection() as conn:
@@ -2743,8 +2594,10 @@ class PromptDatabase(
bound = {f"path{i}": path for i, path in enumerate(chunk)}
marks = ",".join(f":{key}" for key in bound)
sql = _LINK_FACTS_SQL.format(hit=hit, marks=marks)
for path, *row in conn.execute(sql, {**params, **bound}):
rows[path] = _link_match(*row)
for path, rating, matched, positive, category in conn.execute(
sql, {**params, **bound}
):
rows[path] = (bool(matched), rating, positive, category)
positives = sorted({row[2] for row in rows.values()})
for start in range(0, len(positives), _FACTS_CHUNK):
chunk = positives[start : start + _FACTS_CHUNK]
@@ -2757,16 +2610,20 @@ class PromptDatabase(
path: (*row, tuple(names.get(row[2], ()))) for path, row in rows.items()
}
def get_image_paths_matching(self, q: str, tags: List[str]) -> Set[str]:
"""Normalized paths whose linked positive matches *q* and *tags*.
def get_image_paths_matching(
self, q: str, category: Optional[str], tags: List[str]
) -> Set[str]:
"""Normalized paths whose linked positive matches *q*, *category* and *tags*.
See get_image_link_matches; an empty *q* matches every linked image
the tags admit.
the category and tags admit.
"""
return {
path
for path, match in self.get_image_link_matches(q, tags).items()
if match[0] or not q
for path, (hit, _rating, _positive) in self.get_image_link_matches(
q, category, tags
).items()
if hit or not q
}
def get_prompt_id_for_image(self, image_path: str) -> Optional[int]:
+15 -55
View File
@@ -13,9 +13,7 @@ same connection before commit. Owner rows and their timestamps never change.
Category consolidation is previewed in one ``library_snapshot`` and applied in
one ``_pair_transaction`` that recomputes the owner projection and requires the
preview's ``impact_token``; only rows carrying the exact source change. Image
rows holding the source (``image_categories``, #332) are owners too: they are
counted, hashed into the token and renamed with the prompts and pairs.
preview's ``impact_token``; only rows carrying the exact source change.
"""
import sqlite3
@@ -99,30 +97,16 @@ def _validate_tag_id(tag_id: object) -> None:
raise ValueError("Invalid tag ID")
# Category values in Tags & categories (#332 §7): the prompt-backed values
# (nonblank values on eligible parents or on active pairs, grouped exactly, so
# whitespace and case variants stay distinct; /library/options lists the same
# prompt-backed set with its own query) plus the values only image rows hold.
# A prompt-backed value is ordered by its oldest owning positive; an image-only
# value sorts after all of them (first_pid is SQLite's INTEGER maximum, above
# any prompt id), then by value. ``image_count`` counts stored rows, including
# files missing on disk. No FULL OUTER JOIN, so it runs on SQLite 3.31+.
_IMAGE_ONLY_FIRST_PID = 2**63 - 1
_CATEGORY_ROWS = (
# The /library/options universe: nonblank values on eligible parents or on
# active pairs, grouped exactly (whitespace and case variants stay distinct).
_CATEGORY_GROUPS = (
"WITH owners(value,pid) AS ("
"SELECT p.category,p.id FROM prompts p WHERE p.category IS NOT NULL "
"AND TRIM(p.category)!='' AND " + _ELIGIBLE + " "
"UNION SELECT pp.category,pp.positive_prompt_id FROM prompt_pairs pp "
"WHERE pp.archived=0 AND pp.category IS NOT NULL AND TRIM(pp.category)!=''),"
"image_values(value,image_count) AS ("
"SELECT category,COUNT(*) FROM image_categories GROUP BY category),"
"grouped(value,first_pid,prompt_count) AS ("
"SELECT value,MIN(pid),COUNT(DISTINCT pid) FROM owners GROUP BY value "
"UNION ALL SELECT value," + str(_IMAGE_ONLY_FIRST_PID) + ",0 FROM image_values "
"WHERE value NOT IN (SELECT value FROM owners)),"
"counted AS (SELECT g.value,g.first_pid,g.prompt_count,"
"COALESCE(i.image_count,0) AS image_count FROM grouped g "
"LEFT JOIN image_values i ON i.value=g.value) "
"grouped AS (SELECT value,MIN(pid) AS first_pid,"
"COUNT(DISTINCT pid) AS prompt_count FROM owners GROUP BY value) "
)
@@ -133,23 +117,15 @@ _CATEGORY_COMPONENTS = (
"WHERE p.category=? ORDER BY p.id"
)
_CATEGORY_PAIRS = "SELECT id,archived FROM prompt_pairs WHERE category=? ORDER BY id"
# Image rows holding one exact value (#332 DC7), an index search.
_IMAGE_CATEGORY_ROWIDS = (
"SELECT rowid FROM image_categories WHERE category=? ORDER BY rowid"
)
_CATEGORY_EXISTS = (
"SELECT EXISTS(SELECT 1 FROM prompts WHERE category=:v)"
" OR EXISTS(SELECT 1 FROM prompt_pairs WHERE category=:v)"
" OR EXISTS(SELECT 1 FROM image_categories WHERE category=:v)"
)
# Pairs after components and image rows last, so a failing update rolls every
# earlier one back. An image row is keyed by its file, never by its value, so
# merging into a value other files already hold cannot conflict.
# Pairs after components, so a failing pair update rolls both back.
_CATEGORY_UPDATES = (
"UPDATE prompts SET category=?,updated_at=? WHERE category=?",
"UPDATE prompt_pairs SET category=?,updated_at=? WHERE category=?",
"UPDATE image_categories SET category=?,updated_at=? WHERE category=?",
)
@@ -160,15 +136,8 @@ def category_impact_token(
target_exists: bool,
components: list,
pairs: list,
*,
images: list = (),
) -> str:
"""Hash a category projection like ``library_edit_token`` (no secret).
``images`` are the rowids of the image rows holding ``source`` (#332 DC7).
With none the preimage has no ``images`` key, so a prompt-only token keeps
the value it had before image categories existed.
"""
"""Hash a category projection like ``library_edit_token`` (no secret)."""
preimage = {
"epoch": epoch,
"source": source,
@@ -177,8 +146,6 @@ def category_impact_token(
"components": sorted([int(i), bool(flag)] for i, flag in components),
"pairs": sorted([int(i), bool(flag)] for i, flag in pairs),
}
if images:
preimage["images"] = sorted(int(i) for i in images)
return canonical_token(_CATEGORY_DOMAIN, preimage)
@@ -358,8 +325,7 @@ class VocabularyOperations:
pairs = [
[row[0], bool(row[1])] for row in conn.execute(_CATEGORY_PAIRS, (source,))
]
images = [row[0] for row in conn.execute(_IMAGE_CATEGORY_ROWIDS, (source,))]
if not components and not pairs and not images:
if not components and not pairs:
raise LibraryCommandError("vocabulary-conflict", epoch)
target_exists = bool(
conn.execute(_CATEGORY_EXISTS, {"v": target}).fetchone()[0]
@@ -373,10 +339,9 @@ class VocabularyOperations:
"other_prompts": len(components) - eligible,
"negative_variants": len(pairs) - archived,
"archived_negative_variants": archived,
"images": len(images),
},
"impact_token": category_impact_token(
epoch, source, target, target_exists, components, pairs, images=images
epoch, source, target, target_exists, components, pairs
),
}
@@ -400,8 +365,7 @@ class VocabularyOperations:
def apply_vocabulary_category(
self, *, expected_epoch: str, source: str, target: str, impact_token: str
) -> dict:
"""Set the exact ``source`` to ``target`` on every component, pair
and image row.
"""Set the exact ``source`` to ``target`` on every component and pair.
The recomputed projection must still match the preview's token; a
mismatch is ``vocabulary-conflict`` with nothing written.
@@ -458,15 +422,15 @@ class VocabularyOperations:
_limit(limit)
key = _category_after(after)
total = conn.execute(
_CATEGORY_ROWS + "SELECT COUNT(*) FROM counted"
_CATEGORY_GROUPS + "SELECT COUNT(*) FROM grouped"
).fetchone()[0]
where, params = "", []
if key is not None:
where = " WHERE first_pid>? OR (first_pid=? AND value>?)"
params = [key[0], key[0], key[1]]
rows = conn.execute(
_CATEGORY_ROWS
+ "SELECT value,first_pid,prompt_count,image_count FROM counted"
_CATEGORY_GROUPS
+ "SELECT value,first_pid,prompt_count FROM grouped"
+ where
+ " ORDER BY first_pid,value LIMIT ?",
[*params, limit + 1],
@@ -476,11 +440,7 @@ class VocabularyOperations:
return {
"library_epoch": epoch,
"categories": [
{
"value": row["value"],
"prompt_count": row["prompt_count"],
"image_count": row["image_count"],
}
{"value": row["value"], "prompt_count": row["prompt_count"]}
for row in rows
],
"total": total,
+6 -13
View File
@@ -26,13 +26,13 @@ except ImportError:
InputTypeDict = dict
try:
from .prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
from .prompt_manager_base import PromptManagerBase
except ImportError:
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
from prompt_manager_base import PromptManagerBase
class PromptManager(PromptManagerBase, ComfyNodeABC):
@@ -115,7 +115,6 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
)
},
),
"save_to_library": SAVE_TO_LIBRARY_INPUT,
},
"hidden": {
# The queued graph and this node's id: used to tell the positive
@@ -153,7 +152,6 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
append_text: str = "",
prompt: Optional[Dict[str, Any]] = None,
unique_id: Optional[str] = None,
save_to_library: bool = True,
) -> Tuple[Any]:
"""
Encode the text prompt and save it to the database.
@@ -166,7 +164,6 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
search_text: Text to search for in past prompts
prepend_text: Text to prepend to the main prompt
append_text: Text to append to the main prompt
save_to_library: Off: nothing is saved, counted or tracked (#90)
Returns:
Tuple containing the conditioning for the diffusion model and the final
@@ -208,9 +205,7 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
# Save prompt to database and set execution context for gallery tracking
prompt_id = None
extended_tags = []
if not save_to_library:
self._skip_not_saved(prompt, unique_id)
elif storage_text and storage_text.strip():
if storage_text and storage_text.strip():
self.logger.debug(f"Processing prompt text: {storage_text[:100]}...")
# Add prepend/append info to tags if they exist
@@ -254,11 +249,6 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
tokens = clip.tokenize(encoding_text)
conditioning = clip.encode_from_tokens_scheduled(tokens)
self.logger.info(f"CLIP encoding completed, text: {repr(encoding_text)[:80]}")
result = (conditioning, encoding_text)
if not save_to_library:
return result # #90: no metadata registration, no history text
# Register with ComfyUI integration for standard metadata compatibility
node_id = f"promptmanager_{int(time.time() * 1000)}"
self.comfyui_integration.register_prompt(
@@ -272,6 +262,9 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
"append_text": append_text.strip() if append_text else None,
},
)
self.logger.info(f"CLIP encoding completed, text: {repr(encoding_text)[:80]}")
result = (conditioning, encoding_text)
record_runtime_context(
type(self).__name__, text, prepend_text, append_text, encoding_text
)
+2 -38
View File
@@ -23,7 +23,7 @@ try:
from .utils.image_monitor import get_image_monitor
from .utils.prompt_tracker import get_prompt_tracker
from .utils.hashing import generate_prompt_hash
from .utils.prompt_graph import resolve_text, run_prompt_nodes, save_setting
from .utils.prompt_graph import resolve_text, run_prompt_nodes
from .utils.run_keys import executing_run_id, prompt_node_key
from .utils.usage_tracking import PENDING_FIRST_USE
from .utils.cached_runs import (
@@ -42,7 +42,7 @@ except ImportError:
from utils.image_monitor import get_image_monitor
from utils.prompt_tracker import get_prompt_tracker
from utils.hashing import generate_prompt_hash
from utils.prompt_graph import resolve_text, run_prompt_nodes, save_setting
from utils.prompt_graph import resolve_text, run_prompt_nodes
from utils.run_keys import executing_run_id, prompt_node_key
from utils.usage_tracking import PENDING_FIRST_USE
from utils.cached_runs import (
@@ -54,22 +54,6 @@ except ImportError:
)
# The optional input every PromptManager node declares last (#90): declared
# last so no widget value of a workflow saved before it shifts.
SAVE_TO_LIBRARY_INPUT = (
"BOOLEAN",
{
"default": True,
"label_on": "save",
"label_off": "skip",
"tooltip": (
"Save this node's prompt to the library, count its runs and link its "
"images. Off: PromptManager ignores the run; the output is unchanged."
),
},
)
class PromptManagerBase:
"""Mixin providing shared prompt management logic for ComfyUI nodes.
@@ -172,9 +156,7 @@ class PromptManagerBase:
resolved = resolve_text(prompt_graph, node_id)
return {
"is_run_prompt": node_id in run_prompt_nodes(prompt_graph),
# The hook skips a switch it cannot read (#90): the node counts
"seen_at_queue": resolved is not None
and save_setting(prompt_graph, node_id) is True
and generate_prompt_hash(resolved) == generate_prompt_hash(text),
"known": True,
}
@@ -263,24 +245,6 @@ class PromptManagerBase:
"run_id": executing_run_id(),
}
def _skip_not_saved(self, prompt_graph: Any = None, unique_id: Any = None) -> None:
"""``save_to_library`` is off: nothing is saved, counted or tracked (#90).
A literal off makes the node no run prompt (``utils.prompt_graph``), so
no image looks up its run key. A switch wired from a node the graph
cannot read keeps the node a run prompt: its run key is marked like an
empty editor's, so an earlier run's entry of that key never links this
run's images.
"""
message = (
"Save to library is off for this node: its prompt is not saved, the "
"run is not counted and its images are not linked"
)
if isinstance(prompt_graph, dict) and unique_id is not None:
self._mark_untracked(prompt_graph, unique_id, message)
else:
self.logger.info(message)
def _skip_empty_prompt(self, prompt_graph: Any, unique_id: Any) -> None:
"""The ``text`` editor is empty: save and track nothing (#342).
+4 -23
View File
@@ -22,11 +22,7 @@ try:
)
from .utils.hashing import generate_prompt_hash
from .utils.pair_execution import record_pair_execution
from .utils.prompt_graph import (
intrinsic_pair_nodes,
run_prompt_components,
save_setting,
)
from .utils.prompt_graph import intrinsic_pair_nodes, run_prompt_components
from .utils.prompt_pair_resolver import resolve_text_input
from .utils.usage_tracking import (
PENDING_FIRST_USE,
@@ -45,11 +41,7 @@ except ImportError:
)
from utils.hashing import generate_prompt_hash
from utils.pair_execution import record_pair_execution
from utils.prompt_graph import (
intrinsic_pair_nodes,
run_prompt_components,
save_setting,
)
from utils.prompt_graph import intrinsic_pair_nodes, run_prompt_components
from utils.prompt_pair_resolver import resolve_text_input
from utils.usage_tracking import (
PENDING_FIRST_USE,
@@ -98,13 +90,8 @@ class PromptManagerPair(PromptManagerFocused):
prompt: Optional[Dict[str, Any]] = None,
unique_id: Optional[str] = None,
extra_pnginfo: Optional[Dict[str, Any]] = None,
save_to_library: bool = True,
) -> Tuple[Any, Any, str, str]:
"""Encode both sides once; database and tracking failures never block CLIP.
With ``save_to_library`` off nothing is saved, counted, tracked or
recorded (#90); the four outputs are the same.
"""
"""Encode both sides once; database and tracking failures never block CLIP."""
if clip is None:
raise RuntimeError("ERROR: clip input is invalid: None")
if not isinstance(text, str) or not isinstance(negative_text, str):
@@ -122,10 +109,6 @@ class PromptManagerPair(PromptManagerFocused):
negative_conditioning = clip.encode_from_tokens_scheduled(
clip.tokenize(negative_text)
)
result = positive_conditioning, negative_conditioning, positive, negative_text
if not save_to_library:
self._skip_not_saved(prompt, unique_id)
return result
record_pair_execution(
extra_pnginfo,
unique_id,
@@ -186,6 +169,7 @@ class PromptManagerPair(PromptManagerFocused):
)
except Exception as error:
self.logger.warning(f"Could not register paired metadata: {error}")
result = positive_conditioning, negative_conditioning, positive, negative_text
record_runtime_context(
type(self).__name__,
text,
@@ -235,9 +219,6 @@ class PromptManagerPair(PromptManagerFocused):
resolved_negative = (
resolve_text_input(prompt, node_id, "negative_text") if known else None
)
if known and save_setting(prompt, node_id) is not True:
# The hook skips a switch it cannot read (#90): the node counts
resolved_positive = resolved_negative = None
positive_seen = resolved_positive is not None and generate_prompt_hash(
resolved_positive
) == generate_prompt_hash(text)
+6 -13
View File
@@ -26,13 +26,13 @@ except ImportError:
InputTypeDict = dict
try:
from .prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
from .prompt_manager_base import PromptManagerBase
except ImportError:
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
from prompt_manager_base import PromptManagerBase
class PromptManagerText(PromptManagerBase, ComfyNodeABC):
@@ -115,7 +115,6 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
),
},
),
"save_to_library": SAVE_TO_LIBRARY_INPUT,
},
"hidden": {
# The queued graph and this node's id: used to tell the positive
@@ -150,7 +149,6 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
append_text: str = "",
prompt: Optional[Dict[str, Any]] = None,
unique_id: Optional[str] = None,
save_to_library: bool = True,
) -> Tuple[str]:
"""
Process the text prompt and save it to the database.
@@ -162,7 +160,6 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
search_text: Text to search for in past prompts
prepend_text: Text to prepend to the main prompt
append_text: Text to append to the main prompt
save_to_library: Off: nothing is saved, counted or tracked (#90)
Returns:
Tuple containing the final processed text string
@@ -187,9 +184,7 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
# Save prompt to database and set execution context for gallery tracking
prompt_id = None
extended_tags = []
if not save_to_library:
self._skip_not_saved(prompt, unique_id)
elif storage_text and storage_text.strip():
if storage_text and storage_text.strip():
self.logger.debug(f"Processing prompt text: {storage_text[:100]}...")
# Add prepend/append info to tags if they exist
@@ -225,11 +220,6 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
else:
self._skip_empty_prompt(prompt, unique_id)
self.logger.debug(f"Text processing completed: {final_text[:100]}...")
result = (final_text,)
if not save_to_library:
return result # #90: no metadata registration, no history text
# Register with ComfyUI integration for standard metadata compatibility
node_id = f"promptmanagertext_{int(time.time() * 1000)}"
self.comfyui_integration.register_prompt(
@@ -243,6 +233,9 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
"append_text": append_text.strip() if append_text else None,
},
)
self.logger.debug(f"Text processing completed: {final_text[:100]}...")
result = (final_text,)
record_runtime_context(
type(self).__name__, text, prepend_text, append_text, final_text
)
-31
View File
@@ -52,7 +52,6 @@ try:
extract_generation_params,
parse_page_json,
)
from ...utils.prompt_graph import metadata_opted_out, side_opted_out
from ...utils.prompt_pair_resolver import resolve_prompt_pair, sampler_less_prompt
from ...utils.review_capture import reviewable
except ImportError:
@@ -68,7 +67,6 @@ except ImportError:
extract_generation_params,
parse_page_json,
)
from utils.prompt_graph import metadata_opted_out, side_opted_out
from utils.prompt_pair_resolver import resolve_prompt_pair, sampler_less_prompt
from utils.review_capture import reviewable
@@ -262,11 +260,6 @@ SCAN_DETAILS = {
"failed": "Error: the prompt could not be stored",
"error": "Error: the file could not be processed (see the server log)",
"found": "Metadata found, but no prompt text to store",
"not_saved": "Skipped · its PromptManager node does not save to the library",
"not_saved_wired": (
"Skipped · its Save to library switch is wired from another node, so only "
"the run itself can link it"
),
None: "No prompt metadata in the file",
}
@@ -1727,34 +1720,10 @@ class AdminRoutesMixin:
except (ValueError, TypeError):
# Preserve malformed metadata for conservative resolver rejection.
pass
if metadata_opted_out(canonical):
# Save to library was off for the run's prompt: no prompt, link,
# recorded-run fallback or Needs review item (#90)
return "not_saved"
if metadata_opted_out(canonical, unreadable=True):
# The switch is wired from a node the file cannot show: only the
# live run (the node saw the value) can link it, never a scan
return "not_saved_wired"
if not canonical.get("prompt") and not canonical.get("parameters"):
return self._link_recorded_run(media_file, canonical, detailed, history)
_with_file_size(canonical, media_file)
resolution = resolve_prompt_pair(canonical)
if resolution["state"] == "resolved" and side_opted_out(
canonical.get("prompt"), resolution["negative"]
):
# Only the negative's node is off: store the positive alone (#90)
prompt_text = (
resolution["positive"]
.get("storage_text", resolution["positive"]["text"])
.strip()
)
status = self._store_scanned_prompt(media_file, canonical, prompt_text)
if not detailed or status not in SCAN_TITLED_DETAILS:
return status
return {
"status": status,
"title": self._scan_title(media_file, prompt_text),
}
if blank_positive(resolution):
# Only prepend/append text: no stored prompt can own it (#342)
resolution = {**resolution, "state": "unresolved"}
-17
View File
@@ -86,23 +86,6 @@ def unavailable_message(model_name: str) -> str:
return f"{model_name} is not downloaded. {UNAVAILABLE_HINT}"
def model_refusal(status: dict, model_type: str, model_name: str):
"""Why *model_type* cannot run, from ``get_models_status()``, else None.
Not downloaded first (``unavailable_message``), then a backend package
that cannot be imported (#424). Both are a 409 ``model-unavailable``.
"""
entry = status.get(model_type) or {}
if not entry.get("downloaded"):
return unavailable_message(model_name)
missing = entry.get("missing_packages") or []
if missing:
from ..autotag import missing_packages_message
return missing_packages_message(model_type, [str(name) for name in missing])
return None
def run_suggestions(service, capture_path: str, model_type: str, options: dict):
"""Blocking: load (or swap to) *model_type*, then suggest for the capture.
+5 -9
View File
@@ -21,9 +21,9 @@ from aiohttp import web
from .annotation_suggest import (
SUGGEST_FIELDS,
OptionError,
model_refusal,
run_suggestions,
suggestion_rows,
unavailable_message,
validate_options,
)
from .autotag_routes import model_busy_response
@@ -595,7 +595,7 @@ class AnnotationRoutesMixin:
it, including a legacy batch), never downloads (``model-unavailable``)
and never writes.
"""
from ..autotag import MODELS, MissingBackendPackages, get_autotag_service
from ..autotag import MODELS, get_autotag_service
from ..config import MAX_SUGGESTION_LIMIT, AutotagConfig
data, error = await self._annotation_body(request, SUGGEST_FIELDS)
@@ -633,10 +633,9 @@ class AnnotationRoutesMixin:
return error
service = get_autotag_service()
status = await self._run_in_executor(service.get_models_status)
name = MODELS.get(model_type, {}).get("name", model_type)
refusal = model_refusal(status, model_type, name)
if refusal is not None:
return _coded("model-unavailable", refusal)
if not status.get(model_type, {}).get("downloaded"):
name = MODELS.get(model_type, {}).get("name", model_type)
return _coded("model-unavailable", unavailable_message(name))
owner = model_gate.new_owner("image-annotations")
if not model_gate.try_acquire(owner):
return model_busy_response()
@@ -655,9 +654,6 @@ class AnnotationRoutesMixin:
options,
on_skipped=functools.partial(store.release_use, token),
)
except MissingBackendPackages as exc:
# A package gone since the status read (load_model logged it).
return _coded("model-unavailable", str(exc))
except Exception:
self.logger.exception("Tag suggestion failed")
return _error(
+4 -20
View File
@@ -102,17 +102,6 @@ def size_label(size: str) -> str:
return f"{match.group(1)} {match.group(2)}" if match else str(size or "")
def package_entry(flags: dict) -> dict:
"""The public package fields of one status entry (#424): the pip names
that cannot be imported, the install command and note (strings or null)."""
missing = flags.get("missing_packages") or []
text = {
key: flags[key] if isinstance(flags.get(key), str) else None
for key in ("install_command", "install_note")
}
return {"missing_packages": [str(name) for name in missing], **text}
def public_models(status: dict, loaded_type: Optional[str]) -> dict:
"""Each engine model's public entry: names and sizes from ``MODELS``, the
flags from ``status``; never a path (``model_path`` is dropped)."""
@@ -133,7 +122,6 @@ def public_models(status: dict, loaded_type: Optional[str]) -> dict:
for key in ("model_exists", "mmproj_exists"):
if key in flags:
entry[key] = bool(flags[key])
entry.update(package_entry(flags))
models[model_type] = entry
return models
@@ -762,7 +750,6 @@ class AutotagRoutesMixin:
"downloaded": bool(
(status.get(model_type) or {}).get("downloaded")
),
**package_entry(status.get(model_type) or {}),
},
"existing": existing,
"review_required": existing == "replace_after_review",
@@ -1086,8 +1073,8 @@ class AutotagRoutesMixin:
return self.db.current_image_tags({locator: fingerprint}).get(locator, [])
async def _suggest_single(self, target, model_type, options, base):
from ..autotag import MODELS, MissingBackendPackages, get_autotag_service
from .annotation_suggest import model_refusal
from ..autotag import MODELS, get_autotag_service
from .annotation_suggest import unavailable_message
from .annotations import _pinned_suggestions, _settled, _submit
try:
@@ -1097,8 +1084,8 @@ class AutotagRoutesMixin:
service = get_autotag_service()
status = await self._run_in_executor(service.get_models_status)
message = model_refusal(status, model_type, MODELS[model_type]["name"])
if message is not None:
if not (status.get(model_type) or {}).get("downloaded"):
message = unavailable_message(MODELS[model_type]["name"])
return self._coded_error(409, "model-unavailable", message)
owner = model_gate.new_owner("legacy-single")
if not model_gate.try_acquire(owner):
@@ -1144,9 +1131,6 @@ class AutotagRoutesMixin:
options,
on_skipped=functools.partial(store.release_use, capture.token),
)
except MissingBackendPackages as exc:
# A package gone since the status read (load_model logged it).
return self._coded_error(409, "model-unavailable", str(exc))
finally:
store.release(capture.token)
finally:
+5 -53
View File
@@ -5,7 +5,6 @@ import base64
import binascii
from functools import wraps
import json
import os
import re
import sqlite3
from typing import Callable
@@ -28,7 +27,6 @@ from .history_current import (
snapshot_current_output_scope,
)
from .inspection_service import CurrentFileChanged, inspect_current_file
from .images import image_category_key
try:
from ...database.execution_history import HistoryEpochChanged
@@ -41,11 +39,7 @@ try:
WindowPosition,
)
from ...utils.history_read_fence import HistoryReadFence
from ...utils.inspection_response import (
OWN_CATEGORY_UNREAD,
apply_own_category,
encode_history_inspection_response,
)
from ...utils.inspection_response import encode_history_inspection_response
except ImportError:
from database.execution_history import HistoryEpochChanged
from database.history_gallery import read_history_gallery
@@ -57,11 +51,7 @@ except ImportError:
WindowPosition,
)
from utils.history_read_fence import HistoryReadFence
from utils.inspection_response import (
OWN_CATEGORY_UNREAD,
apply_own_category,
encode_history_inspection_response,
)
from utils.inspection_response import encode_history_inspection_response
_RUN_MAX_BYTES = 8 * 1024 * 1024
@@ -687,32 +677,6 @@ class HistoryRoutesMixin:
) -> web.Response:
return await self._history_current(request, "inspection")
def _history_own_category(self, reference: dict, scope):
"""The recorded output's own category (#332), inside the snapshot.
The path is the symlink-resolved output folder plus the receipt's
subfolder and filename, keyed by ``image_category_key`` over the
configured output sources as Images keys it. A temp output or a path
no source holds has none. A failed read is OWN_CATEGORY_UNREAD.
"""
if (
reference.get("folder_type") != "output"
or scope.canonical_output is None
or candidate_locator(reference, scope) is None
):
return None
path = os.path.join(
str(scope.canonical_output), reference["subfolder"], reference["filename"]
)
try:
key = image_category_key(self._get_all_output_dirs(), path)
return None if key is None else self.db.image_category(*key)
except Exception:
self.logger.warning(
"History image category could not be read", exc_info=True
)
return OWN_CATEGORY_UNREAD
async def _history_current(
self, request: web.Request, kind: str
) -> web.StreamResponse:
@@ -737,7 +701,6 @@ class HistoryRoutesMixin:
if scope.current_scope != current_scope:
raise _current_stale()
result = db.get_history_occurrence(run_id, node, ordinal)
own = None
if result is not None:
receipt = result["receipt"]
path = candidate_locator(receipt["reference"], scope)
@@ -745,10 +708,6 @@ class HistoryRoutesMixin:
db.model.get_connection(),
(path,) if path is not None else (),
)
if kind == "inspection":
own = self._history_own_category(
receipt["reference"], scope
)
finally:
db.model.close()
if result is None:
@@ -767,16 +726,9 @@ class HistoryRoutesMixin:
resolved.canonical_target,
source_label=resolved.public["source_label"],
saved=saved.get("saved"),
library=apply_own_category(
saved.get(
"library",
{
"status": (
"unavailable" if saved_unavailable else "none"
)
},
),
own,
library=saved.get(
"library",
{"status": "unavailable" if saved_unavailable else "none"},
),
recorded=recorded_output_lookup(
db, close=True, reference=receipt["reference"]
+3 -16
View File
@@ -29,11 +29,9 @@ from .prompts import bad_request
try:
from ...database.image_categories import MAX_CATEGORY_LENGTH, MAX_KEY_LENGTH
from ...database.models import normalize_image_path
from ...utils.validators import validate_category
except ImportError:
from database.image_categories import MAX_CATEGORY_LENGTH, MAX_KEY_LENGTH
from database.models import normalize_image_path
from utils.validators import validate_category
CATEGORY_MEDIA_MESSAGE = "Only media files can have a category"
@@ -145,7 +143,7 @@ class ImageCategoryRoutesMixin:
# -- reading ---------------------------------------------------------------
def _own_category_map(self, sources, listings) -> dict:
"""``{normalized listed path: own category}`` for the listed files (blocking).
"""``{root / listed path: own category}`` for the listed files (blocking).
One primary-key range read per source; each stored row is matched to
the listing of every selected root that holds its source, so the work
@@ -164,30 +162,19 @@ class ImageCategoryRoutesMixin:
for _index, root, files in listings:
listing = self._favorite_listing(root, files)
root_key = _favorite_root_key(root)
# normalize_image_path(root / listed) without a call per file: the
# listed spellings are clean relative POSIX paths. Windows abspath
# also drops a component's trailing dots and spaces, so it keeps
# the call there.
base = os.path.join(normalize_image_path(str(root)), "")
def normalized(listed, base=base, root=root):
if os.name == "nt":
return normalize_image_path(str(root / listed))
return base + os.path.normcase(listed.replace("/", os.sep))
nested = [
item for item in _nested_sources(root, sources) if item[1] != root_key
]
for relative, category in by_source.get(root_key, {}).items():
listed = listing.get(relative)
if listed is not None and _source_prefix(relative, nested) is None:
found[normalized(listed)] = category
found[root / listed] = category
for prefix, source_key in nested:
for relative, category in by_source.get(source_key, {}).items():
full = f"{prefix}/{relative}"
listed = listing.get(full)
if listed is not None and _source_prefix(full, nested) == prefix:
found[normalized(listed)] = category
found[root / listed] = category
return found
async def _source_listings(self, roots) -> list[tuple]:
+9 -12
View File
@@ -42,7 +42,7 @@ from .. import model_gate
from .admin import _plural, _time_left
from .annotation_suggest import (
OptionError,
model_refusal,
unavailable_message,
validate_options,
)
from .annotations import (
@@ -510,8 +510,10 @@ class ImageTagJobsMixin:
)
)
label = MODELS[model_type]["label"]
message = model_refusal(status, model_type, label)
if message is not None:
if not (status.get(model_type) or {}).get("downloaded"):
message = (
f"{label} is not downloaded. Download it in System › Auto-tagging."
)
return None, _refusal(409, "model-unavailable", message)
if backup_on and short is not None:
return None, _refusal(507, "insufficient-space", short_space_message(short))
@@ -786,13 +788,13 @@ class ImageTagJobsMixin:
async def _prepare_tagging_model(self, job, spec: _Spec, owner):
"""The loaded service, or None (logged) when the job cannot run."""
from ..autotag import MODELS, MissingBackendPackages, get_autotag_service
from ..autotag import MODELS, get_autotag_service
service = get_autotag_service()
status = await self._run_in_executor(service.get_models_status)
name = MODELS.get(spec.model_type, {}).get("name", spec.model_type)
message = model_refusal(status, spec.model_type, name)
if message is not None:
if not status.get(spec.model_type, {}).get("downloaded"):
name = MODELS.get(spec.model_type, {}).get("name", spec.model_type)
message = unavailable_message(name)
job.update(status_line=message)
job.log_line(f"! {message}")
return None
@@ -804,11 +806,6 @@ class ImageTagJobsMixin:
spec.model_type,
spec.options["use_gpu"],
)
except MissingBackendPackages as exc:
# A package gone since the status read (load_model logged it).
job.update(status_line=str(exc))
job.log_line(f"! {exc}")
return None
except Exception:
self.logger.exception("The auto-tag job could not load its model")
job.update(status_line=INFERENCE_FAILED)
+25 -97
View File
@@ -66,8 +66,6 @@ try:
from ...utils.parallel import map_parallel
from ...utils.json_strict import dumps_strict
from ...utils.inspection_response import (
OWN_CATEGORY_UNREAD,
apply_own_category,
encode_inspection_response,
library_metadata,
)
@@ -89,8 +87,6 @@ except ImportError:
from utils.parallel import map_parallel
from utils.json_strict import dumps_strict
from utils.inspection_response import (
OWN_CATEGORY_UNREAD,
apply_own_category,
encode_inspection_response,
library_metadata,
)
@@ -569,8 +565,7 @@ class _ImageTagMatch(NamedTuple):
tags: bool # every filter tag is in its image tags or linked positive tags
query: bool # ``q`` is a case-folded substring of one image tag name
# (hit, rating, positive id, inherited category) of its unambiguous link
link: tuple | None
link: tuple | None # (hit, rating, positive id) of its unambiguous link
def _current_fingerprint(path: Path) -> tuple[int, int] | None:
@@ -1225,27 +1220,20 @@ def _admitted_entries(
links: dict | None,
favorites: frozenset | None = None,
image_tags: dict | None = None,
own_categories: dict | None = None,
):
"""Yield ``(position, entry, link match)`` for each entry the filters admit.
*links* maps normalized paths to (hit, rating, positive id, inherited
category) for every unambiguously linked image (see
PromptDatabase.get_image_link_matches); a tag filter admits only those,
and ``q`` admits a filename match or a linked-positive hit. A category
admits a file whose effective category equals it (#332 MC5): its own
category (*own_categories*, ``{normalized path: category}``) when it has one,
else the inherited one, so an unlinked file with its own category is
admitted. *favorites*, when given, admits only those listed
*links* maps normalized paths to (hit, rating, positive id) for every
unambiguously linked image (see PromptDatabase.get_image_link_matches); a
category or tag filter admits only those, and ``q`` admits a filename match
or a linked-positive hit. *favorites*, when given, admits only those listed
paths (``favorites=1``). *image_tags* maps normalized paths to their
_ImageTagMatch: a tag filter also admits a file whose current image tags
(with its linked positive's tags) carry every tag, and ``q`` also admits an
image tag name hit.
"""
query, media, folder = filters["query"], filters["media"], filters["folder"]
category = filters["category"]
members_only = bool(filters["tags"])
own = own_categories or {}
members_only = filters["category"] is not None or bool(filters["tags"])
native_folder = folder.replace("/", os.sep)
nested = native_folder + os.sep
exact = filters.get("path", "")
@@ -1264,12 +1252,6 @@ def _admitted_entries(
image = None if image_tags is None else image_tags.get(entry.normalized)
if members_only and match is None and not (image and image.tags):
continue
if category is not None:
effective = own.get(entry.normalized) if own else None
if effective is None and match is not None:
effective = match[3]
if effective != category:
continue
if (
query
and query not in entry.folded
@@ -1288,7 +1270,6 @@ def _filtered_page(
links: dict | None,
favorites: frozenset | None = None,
image_tags: dict | None = None,
own_categories: dict | None = None,
) -> tuple[int, list, int | None]:
"""Count the *entries* the filters admit and take *size* from *start* on.
@@ -1298,7 +1279,7 @@ def _filtered_page(
"""
total, page, positives = 0, [], set()
for position, entry, match in _admitted_entries(
entries, filters, links, favorites, image_tags, own_categories
entries, filters, links, favorites, image_tags
):
total += 1
if match is not None:
@@ -2238,10 +2219,9 @@ class ImageRoutesMixin:
stat'ed (never every listed file), and a candidate's tags count only
while a byte record is verified at its present ``(size, mtime_ns)``
(spec amendment S1). Under a Tags filter the candidates' linked
positives are read too, so their tags combine; the category is checked
once, on the effective category, by _admitted_entries (#332).
positives are read too, so their tags and category combine.
"""
tags, query = filters["tags"], filters["query"]
tags, query, category = filters["tags"], filters["query"], filters["category"]
wanted = set(tags)
located = self._workspace_locators(listings)
# The tag index answers which files carry a relevant name (#379); no
@@ -2271,9 +2251,11 @@ class ImageRoutesMixin:
link = facts.get(entry.normalized)
carried = set(names).union(link[4] if link else ())
matches[entry.normalized] = _ImageTagMatch(
bool(wanted) and wanted <= carried,
bool(wanted)
and wanted <= carried
and (category is None or (link is not None and link[3] == category)),
bool(query) and any(query in name.casefold() for name in names),
tuple(link[:4]) if link else None,
link[:3] if link else None,
)
return matches
@@ -2282,14 +2264,10 @@ class ImageRoutesMixin:
) -> tuple:
"""The ordered entries and what admits them (blocking; run in executor).
Returns ``(entries, key, links, image_tags, favorites, own)``: the
listing in the filters' order, its keyset key, and the link, image-tag,
favorite and own-category maps _admitted_entries reads. One SELECT
(get_image_link_matches) reads every linked positive when *read_links*;
under a category it keeps only links whose raw recipe or positive
category is that value (the inherited one is decided in Python), and
the links of files whose own category is that value are read by path,
so their rating and positive still count (#332). A failed read raises when the
Returns ``(entries, key, links, image_tags, favorites)``: the listing in
the filters' order, its keyset key, and the link, image-tag and favorite
maps _admitted_entries reads. One SELECT (get_image_link_matches) reads
every linked positive when *read_links*. A failed read raises when the
filters need it (category, tags, the rating sort) or *strict*; else the
listing stays usable without links (a search then matches filenames
only). A failed image-tag read raises under a Tags filter; a search then
@@ -2301,10 +2279,10 @@ class ImageRoutesMixin:
try:
links = self.db.get_image_link_matches(
filters["text"],
category,
tags,
every_link=True,
wanted=self._workspace_wanted(listings),
category=category,
)
except Exception:
if strict or category is not None or tags or order == "rating":
@@ -2318,13 +2296,6 @@ class ImageRoutesMixin:
),
exc_info=True,
)
own = None
if category is not None:
# A failed read raises: an unfiltered page must never pass for the
# category (DC6).
own = self._own_category_map(self._get_all_output_dirs(), listings)
if links is not None:
links = self._own_category_links(links, own, filters)
image_tags = None
if filters["query"] or tags:
try:
@@ -2371,30 +2342,7 @@ class ImageRoutesMixin:
favorites = (
self._workspace_favorite_paths(listings) if filters["favorites"] else None
)
return entries, key, links, image_tags, favorites, own
def _own_category_links(self, links: dict, own: dict, filters: dict) -> dict:
"""*links* plus the links of files whose own category is the filter's.
Their inherited category differs, so the category's link read left
them out; read by path (get_image_link_facts), kept only when their
positive has every filter tag, as the link read would.
"""
category, wanted = filters["category"], set(filters["tags"])
missing = [
path
for path, value in own.items()
if value == category and path not in links
]
if not missing:
return links
facts = self.db.get_image_link_facts(missing, filters["text"])
extra = {
path: tuple(fact[:4])
for path, fact in facts.items()
if wanted <= set(fact[4])
}
return {**links, **extra} if extra else links
return entries, key, links, image_tags, favorites
def _select_workspace_page(
self,
@@ -2423,7 +2371,7 @@ class ImageRoutesMixin:
or bool(filters["tags"])
or order == "rating"
)
entries, key, links, image_tags, favorites, own = self._workspace_admission(
entries, key, links, image_tags, favorites = self._workspace_admission(
listings, filters, read_links=needed or after is None
)
start = 0
@@ -2434,7 +2382,7 @@ class ImageRoutesMixin:
else:
start = bisect_right(entries, after, key=key)
total, page, positive_count = _filtered_page(
entries, start, limit + 1, filters, links, favorites, image_tags, own
entries, start, limit + 1, filters, links, favorites, image_tags
)
if not needed and after is not None:
positive_count = _COUNT_OMITTED
@@ -2456,12 +2404,12 @@ class ImageRoutesMixin:
The links are read strictly: the positives must be the listing's own.
"""
entries, _key, links, image_tags, favorites, own = self._workspace_admission(
entries, _key, links, image_tags, favorites = self._workspace_admission(
listings, filters, read_links=True, strict=True
)
targets, positives, unlinked, scopes = [], {}, 0, {}
for _position, entry, match in _admitted_entries(
entries, filters, links, favorites, image_tags, own
entries, filters, links, favorites, image_tags
):
if len(targets) == cap:
return ListingSelection(targets, list(positives), unlinked, True)
@@ -3780,20 +3728,13 @@ class ImageRoutesMixin:
except Exception:
# The link stays known; only its title is withheld.
self.logger.warning("Inspection link title could not be read")
# An output file's own category (#332); LoRA roots have none.
own = (
self._read_own_category(outputs, path)
if index < len(outputs)
else None
)
try:
observed = inspect_current_file(
path,
source_label=_public_path(roots[index]),
saved=saved,
library=apply_own_category(
self._inspection_library(saved, unavailable=unavailable),
own,
library=self._inspection_library(
saved, unavailable=unavailable
),
image_tags=self._inspection_image_tags(
roots[index], Path(relative).parts
@@ -3861,19 +3802,6 @@ class ImageRoutesMixin:
return read
def _read_own_category(self, sources, path):
"""One file's own category, None, or OWN_CATEGORY_UNREAD (blocking).
Keyed by ``image_category_key`` over the output sources, as Images
writes it. A failed read is logged and hides the category (DC5).
"""
try:
key = image_category_key(sources, path)
return None if key is None else self.db.image_category(*key)
except Exception:
self.logger.warning("Image category could not be read", exc_info=True)
return OWN_CATEGORY_UNREAD
def _inspection_library(self, saved: dict | None, *, unavailable: bool) -> dict:
"""The inspector's Library facts; the route sets the title afterwards."""
result = {
+7 -121
View File
@@ -7,7 +7,6 @@ Refactored from standalone_tagger.py for integration with PromptManager API.
import errno
import gc
import importlib.util
import inspect
import os
import sys
@@ -156,91 +155,6 @@ MODELS = {
},
}
# The Python modules each backend imports when it loads, with their pip names
# (#424). None of them is a PromptManager requirement: they are optional, and
# llama-cpp-python needs a build for the user's GPU. Nothing is ever installed.
BACKEND_PACKAGES: Dict[str, Tuple[Tuple[str, str], ...]] = {
"gguf": (("llama_cpp", "llama-cpp-python"),),
# The default 8-bit load needs bitsandbytes, and device_map needs accelerate.
"hf": (
("torch", "torch"),
("transformers", "transformers"),
("accelerate", "accelerate"),
("bitsandbytes", "bitsandbytes"),
),
"wd14-swinv2": (("onnxruntime", "onnxruntime"),),
"wd14-vit": (("onnxruntime", "onnxruntime"),),
}
_INSTALL_NOTES = {
"gguf": (
'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python'
),
"wd14-swinv2": "Use onnxruntime-gpu for CUDA.",
"wd14-vit": "Use onnxruntime-gpu for CUDA.",
}
_PIP_NAMES = {
module: pip for packages in BACKEND_PACKAGES.values() for module, pip in packages
}
def _importable(module: str) -> bool:
"""Whether *module* (a top-level name) can be imported; imports nothing."""
if sys.modules.get(module) is not None:
return True
try:
return importlib.util.find_spec(module) is not None
except (ImportError, ValueError, OSError):
return False
def missing_packages(model_type: str) -> List[str]:
"""The pip names of the packages *model_type* needs and cannot import."""
return [
pip
for module, pip in BACKEND_PACKAGES.get(model_type, ())
if not _importable(module)
]
def package_fields(model_type: str, missing: List[str]) -> Dict[str, Any]:
"""The status fields of *missing*: pip names, install command and note."""
missing = list(missing)
return {
"missing_packages": missing,
"install_command": "pip install " + " ".join(missing) if missing else None,
"install_note": _INSTALL_NOTES.get(model_type) if missing else None,
}
def _joined(names: List[str]) -> str:
if len(names) <= 1:
return "".join(names)
return ", ".join(names[:-1]) + " and " + names[-1]
def missing_packages_message(model_type: str, missing: List[str]) -> str:
"""The one sentence every refusal of a model with missing packages uses."""
label = MODELS.get(model_type, {}).get("label", model_type)
fields = package_fields(model_type, missing)
pronoun = "it" if len(missing) == 1 else "them"
message = (
f"{label} needs {_joined(missing)}. Install {pronoun} into ComfyUI's "
f"Python: {fields['install_command']}."
)
if fields["install_note"]:
message += " " + fields["install_note"]
return message
class MissingBackendPackages(RuntimeError):
"""A model whose backend packages cannot be imported (#424)."""
def __init__(self, model_type: str, missing: List[str]):
self.model_type = model_type
self.missing = list(missing)
super().__init__(missing_packages_message(model_type, self.missing))
# Default prompts
DEFAULT_PROMPT = DEFAULT_TAG_TEMPLATE
@@ -253,10 +167,6 @@ SYSTEM_PROMPT = (
)
def _none(_model_type: str) -> List[str]:
return []
def _load_rgb(image_path: str) -> Image.Image:
"""Decode the image as RGB and close the file before inference.
@@ -293,7 +203,6 @@ class AutoTagService:
self,
models_dir: Optional[Path] = None,
model_loader: Optional[ModelLoader] = None,
package_probe: Optional[Callable[[str], List[str]]] = None,
):
"""Initialize the AutoTag service.
@@ -304,16 +213,9 @@ class AutoTagService:
``(kind, backend)`` tuple used by ``generate_tags``.
Defaults to ``_load_model``, which imports the ML
backends lazily. Tests inject a fake here.
package_probe: Callable ``(model_type)`` returning the pip names
of the backend packages that cannot be imported.
Defaults to ``missing_packages`` with the default
loader; an injected loader brings its own backend.
"""
self.logger = get_logger("autotag.service")
self._model_loader: ModelLoader = model_loader or self._load_model
if package_probe is None:
package_probe = missing_packages if model_loader is None else _none
self._package_probe = package_probe
# Determine models directory
if models_dir:
@@ -418,9 +320,6 @@ class AutoTagService:
# Get the actual path (local or cache)
model_status["model_path"] = str(self._get_hf_model_path())
model_status.update(
package_fields(model_type, self._package_probe(model_type))
)
status[model_type] = model_status
return status
@@ -556,9 +455,13 @@ class AutoTagService:
def _load_wd14_tagger(self, model_type: str, use_gpu: bool = True):
"""Load WD14 ONNX model and tag list."""
# A missing onnxruntime is refused by load_model before this runs
# (#424); a ModuleNotFoundError here becomes the same refusal.
import onnxruntime as ort
try:
import onnxruntime as ort
except ImportError:
raise RuntimeError(
"onnxruntime is required for WD14 models. "
"Install with: pip install onnxruntime (or onnxruntime-gpu for CUDA)"
)
import csv
config = MODELS[model_type]
@@ -791,7 +694,6 @@ class AutoTagService:
Raises:
ValueError: If model_type is invalid
RuntimeError: If model not downloaded or loading fails
MissingBackendPackages: If its backend packages cannot be imported
"""
if model_type not in MODELS:
raise ValueError(f"Invalid model type: {model_type}")
@@ -803,11 +705,6 @@ class AutoTagService:
status = self.get_models_status()
if not status[model_type]["downloaded"]:
raise RuntimeError(f"Model {model_type} not downloaded")
missing = status[model_type].get("missing_packages") or []
if missing:
refusal = MissingBackendPackages(model_type, missing)
self.logger.warning(str(refusal))
raise refusal
try:
self._tagger = self._model_loader(model_type, use_gpu)
@@ -815,17 +712,6 @@ class AutoTagService:
self.logger.info(f"Model {model_type} loaded successfully")
return True
except ModuleNotFoundError as e:
# A package find_spec found but that does not import (a broken
# install), or a dependency of one: the same refusal, no traceback.
self._tagger = None
self._current_model_type = None
module = (e.name or "").split(".")[0] or str(e)
refusal = MissingBackendPackages(
model_type, [_PIP_NAMES.get(module, module)]
)
self.logger.warning(str(refusal))
raise refusal from None
except Exception as e:
self.logger.error(f"Failed to load model {model_type}: {e}")
self._tagger = None
+1 -9
View File
@@ -19,7 +19,7 @@ test.beforeEach(async ({page, browserName}) => {
await import('/extensions/promptmanager/prompt_manager.js');
class Node {
constructor(){
this.widgets=[...['text','category','tags','search_text'].map(name=>({name,value:'',inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true}];
this.widgets=['text','category','tags','search_text'].map(name=>({name,value:'',inputEl:document.createElement('textarea')}));
this.widgets_values=[]; this.inputs=[];
}
addDOMWidget(name,type,element){document.body.append(element);return {element};}
@@ -36,14 +36,6 @@ test.afterEach(async ({page,browserName}) => {
expect(page.classicErrors).toEqual([]);
});
test('Classic loads an older workflow with Save to library on and keeps a saved off (#90)',async({page})=>{
// A workflow saved before the switch held the search UI's slot ('') at the switch's index
const loaded=await page.evaluate(()=>{const widget=node.widgets.find(w=>w.name==='save_to_library');widget.value='';node.configure({});return widget.value;});
expect(loaded).toBe(true);
const off=await page.evaluate(()=>{const widget=node.widgets.find(w=>w.name==='save_to_library');widget.value=false;node.configure({});return [widget.value,node.serialize().widgets_values[4]];});
expect(off).toEqual([false,false]);
});
test('Classic keeps immediate selection, search query and metadata serialization',async({page})=>{
await page.route('**/prompt_manager/search?**', route=>route.fulfill({json:{success:true,results:[{text:'A new portrait',category:'Portrait',tags:['film grain'],rating:5}]}}));
await page.evaluate(()=>{node.widgets.find(w=>w.name==='search_text').value='tag:"film grain" portrait';});
+2 -28
View File
@@ -26,7 +26,7 @@ test.beforeEach(async({page,browserName})=>{
window.fixtureApp={registerExtension(extension){this.extension=extension;},ui:{settings:new EventTarget()},graph:{setDirtyCanvas(){}},canvas:{}};
await import('/extensions/promptmanager/prompt_manager.js');
class Node {
constructor(){this.widgets=[...['text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true,options:{}}];this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}})),{name:'save_to_library',type:'BOOLEAN',link:null,widget:{name:'save_to_library'}}];this.properties={};this.size=[550,700];}
constructor(){this.widgets=['text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}}))];this.properties={};this.size=[550,700];}
findInputByType(type){const index=this.inputs.findIndex(input=>input.type===type&&input.link==null);return index<0?undefined:{index,slot:this.inputs[index]};}
addWidget(type,name,value,callback,options){const widget={type,name,value,callback,options};this.widgets.push(widget);return widget;}
addDOMWidget(name,type,element){const host=document.createElement('div');host.style.height='650px';host.className='fixture-dom-widget';host.append(element);document.body.append(host);return {element};}
@@ -41,32 +41,6 @@ test.beforeEach(async({page,browserName})=>{
});
test.afterEach(async({page,browserName})=>{await saveCoverage(page,browserName);expect(page.focusedErrors).toEqual([]);});
test(`${variant} Save to library switch saves its value and loads older workflows on (#90)`, async ({page}) => {
const toggle=page.getByRole('checkbox',{name:'Save to library',exact:true});
await expect(toggle).toBeChecked();
expect(await page.evaluate(()=>node.widgets.map(widget=>widget.name))).toEqual(['text','category','tags','search_text','prepend_text','append_text','save_to_library']);
// The native switch is drawn in the toolbar only: the host keeps it hidden and zero-height, like the text fallbacks
expect(await page.evaluate(()=>{const widget=node.widgets.find(item=>item.name==='save_to_library');return {type:widget.type,hidden:widget.hidden,size:widget.computeSize()};})).toEqual({type:'hidden',hidden:true,size:[0,-4]});
expect(await page.evaluate(()=>node.widgets.find(item=>item.name==='text').computeSize())).toEqual([0,-4]);
await toggle.uncheck();
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Save to library is off: runs of this node are not added to the library.');
const saved=await page.evaluate(()=>node.serialize());
expect(saved.widgets_values[6]).toBe(false);
await page.evaluate(saved=>node.configure(saved),saved);
await expect(toggle).not.toBeChecked();
await page.getByLabel('Main prompt',{exact:true}).fill('Typed while off');
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Save to library is off: runs of this node are not added to the library.');
// A workflow saved before the switch held the editor's own slot ('') at its index
await page.evaluate(()=>node.configure({widgets_values:['Old text','','','','','',''],properties:{}}));
await expect(toggle).toBeChecked();
expect(await page.evaluate(()=>node.widgets.find(widget=>widget.name==='save_to_library').value)).toBe(true);
await toggle.check();
await page.getByLabel('Main prompt',{exact:true}).fill('Typed while on');
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Main prompt is saved when this node runs.');
await page.evaluate(()=>{node.inputs.find(input=>input.name==='save_to_library').link=9;node.onConnectionsChange?.();});
await expect(toggle).toBeDisabled();
});
test(`${variant} library autocompletes filters and searches without changing editor metadata`, async ({page}) => {
const requests=[];
await page.route('**/prompt_manager/categories',route=>route.fulfill({json:{success:true,categories:['Portrait']}}));
@@ -283,7 +257,7 @@ test(`${variant} restores workflow values and disclosures and protects connected
test(`${variant} offers text as a link socket, keeps typed text while linked and through a reload (#343)`,async({page})=>{
const sockets=()=>page.evaluate(()=>Object.fromEntries(node.inputs.map(input=>[input.name,input.widget?.name??null])));
// The hidden text widget no longer owns the input, so the host draws it as a socket; the other widget inputs are unchanged.
expect(await sockets()).toEqual({clip:null,prepend_text:null,append_text:null,text:null,category:'category',tags:'tags',search_text:'search_text',save_to_library:'save_to_library'});
expect(await sockets()).toEqual({clip:null,prepend_text:null,append_text:null,text:null,category:'category',tags:'tags',search_text:'search_text'});
expect(await page.evaluate(()=>node.findInputByType('STRING').slot.name)).toBe('text');
expect(await page.evaluate(()=>node.findInputByType('CLIP').slot.name)).toBe('clip');
const main=page.getByRole('textbox',{name:'Main prompt',exact:true});
+1 -13
View File
@@ -17,7 +17,7 @@ test.beforeEach(async({page,browserName})=>{
window.fixtureApp={registerExtension(extension){this.extension=extension;},ui:{settings},graph:{_nodes:[],setDirtyCanvas(){}},canvas:{},graphToPrompt:async()=>({})};
await import('/extensions/promptmanager/prompt_manager.js');
class Node {
constructor(){this.id=1;this.type='PromptManagerPair';this.widgets=[...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true,options:{}}];this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}})),{name:'save_to_library',type:'BOOLEAN',link:null,widget:{name:'save_to_library'}}];this.properties={};this.size=[1100,700];}
constructor(){this.id=1;this.type='PromptManagerPair';this.widgets=['text','negative_text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}}))];this.properties={};this.size=[1100,700];}
findInputByType(type){const index=this.inputs.findIndex(input=>input.type===type&&input.link==null);return index<0?undefined:{index,slot:this.inputs[index]};}
addWidget(type,name,value,callback,options){const widget={type,name,value,callback,options};this.widgets.push(widget);return widget;}
addDOMWidget(name,type,element){const host=document.createElement('div');host.style.cssText='height:700px;width:1100px;max-width:100%';host.className='fixture-dom-widget';host.append(element);document.body.append(host);return {element};}
@@ -34,18 +34,6 @@ test.afterEach(async({page,browserName})=>{await saveCoverage(page,browserName);
async function selectPair(page){await page.getByRole('button',{name:'Recent',exact:true}).click();await page.locator('.pm-editor-result').click();}
async function draft(page,positive='Draft positive',negative='Draft negative'){await page.getByRole('textbox',{name:'Positive prompt',exact:true}).fill(positive);await page.getByRole('textbox',{name:'Negative prompt',exact:true}).fill(negative);}
test('paired editor Save to library switch keeps the widget order and loads older workflows on (#90)',async({page})=>{
await expect(page.locator('.pm-pair')).toBeVisible();
const toggle=page.getByRole('checkbox',{name:'Save to library',exact:true});
await expect(toggle).toBeChecked();
expect(await page.evaluate(()=>node.widgets.map(widget=>widget.name))).toEqual(['text','negative_text','category','tags','search_text','prepend_text','append_text','save_to_library']);
await toggle.uncheck();
expect(await page.evaluate(()=>node.serialize().widgets_values[7])).toBe(false);
await expect(page.locator('.pm-pair > .pm-editor-status[role=status]').last()).toHaveText('Save to library is off: runs of this node are not added to the library.');
await page.evaluate(()=>node.configure({widgets_values:['P','N','','','','','',''],properties:{}}));
await expect(toggle).toBeChecked();
});
test('paired editor previews both texts before explicitly replacing both',async({page})=>{
await expect(page.locator('.pm-pair')).toBeVisible();
await draft(page);
-5
View File
@@ -75,8 +75,6 @@ class TestModel:
loaded = None
# Models the Tag image dialog workflows mark as not downloaded.
unavailable: set = set()
# Pip names per model whose backend packages "cannot be imported" (#424).
missing_packages: dict = {}
# Scored names the concept comparisons answer instead of the fixed list
# (``/__test__/tagging/suggestions``); None keeps the fixed list.
suggestions: list | None = None
@@ -84,13 +82,10 @@ class TestModel:
models_config: dict = {}
def get_models_status(self) -> dict:
from py.autotag import package_fields
return {
name: {
"downloaded": name not in self.unavailable,
"loaded": self.loaded == name,
**package_fields(name, self.missing_packages.get(name, [])),
}
for name in self.models_config
}
@@ -123,32 +123,6 @@ test('model options are the four real engines with size and the description show
await expect(actionOf(page, 'Unload')).toBeDisabled();
});
const NEEDS_LLAMA = 'GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI\'s Python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python';
test('a downloaded model whose backend package is missing says so and is not offered as ready (#424)', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/models', { data: { unavailable: ['wd14-vit'],
missing_packages: { gguf: ['llama-cpp-python'], 'wd14-vit': ['onnxruntime'] } } })).toBeOK();
await openTagging(page);
const line = modelRow(page).locator('p').first();
await page.locator('#tag-model').selectOption('gguf');
await expect(line).toHaveText('Downloaded. Needs llama-cpp-python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python');
// Files and packages are separate: Download (again) stays offered.
await expect(actionOf(page, 'Download again')).not.toHaveAttribute('aria-disabled', 'true');
// Review explains instead of reviewing; nothing is previewed.
await expect(reviewButton(page)).toHaveAttribute('aria-disabled', 'true');
await reviewButton(page).dispatchEvent('click');
await expect(toast(page)).toHaveText(NEEDS_LLAMA);
expect(page.modelCalls.filter(call => call.endsWith('/autotag/batch/preview'))).toEqual([]);
// Not downloaded reads differently and still names what it will need.
await page.locator('#tag-model').selectOption('wd14-vit');
await expect(line).toHaveText('Not downloaded. Download size ~400 MB. Also needs onnxruntime.');
await page.locator('#tag-model').selectOption('wd14-swinv2');
await expect(line).toHaveText('Downloaded. Not loaded.');
await expect(reviewButton(page)).not.toHaveAttribute('aria-disabled', 'true');
});
test('entering the section makes no model call besides status', async ({ page }) => {
await openTagging(page);
await page.locator('#workspace-main .section-nav').getByRole('button', { name: 'Integrations', exact: true }).click();
@@ -186,26 +186,6 @@ test('a model that is not downloaded keeps Start disabled and says why', async (
expect(posted).toBe(0);
});
const NEEDS_LLAMA = 'GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI\'s Python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python';
test('a model missing its backend package keeps Start disabled and names the install command (#424)', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/models', { data: { unavailable: [],
missing_packages: { gguf: ['llama-cpp-python'] } } })).toBeOK();
await select(page, FIRST);
await barButton(page, 'Auto-tag selected').click();
const status = modal(page).locator('#tag-generation-status');
const start = actionButtons(page).last();
await modal(page).getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('gguf');
await expect(status).toHaveText(NEEDS_LLAMA);
await expect(start).toHaveAttribute('aria-disabled', 'true');
await modal(page).getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('wd14-vit');
await expect(status).toHaveText('');
await expect(start).not.toHaveAttribute('aria-disabled', 'true');
await actionButtons(page).first().click();
await expect(modal(page)).toHaveCount(0);
});
test('Suggestions field prefills from the setting and caps chips at the entered count', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/defaults', { data: { general: 0.5, character: 0.9, limit: 7 } })).toBeOK();
await select(page, FIRST);
-12
View File
@@ -260,18 +260,6 @@ test('unavailable model shows inline status and never downloads', async ({ page,
await expect(dialog(page).locator('.dialog-actions .ca-primary')).toBeFocused();
});
test('a missing backend package is the server\'s install sentence, no traceback page (#424)', async ({ page, request }) => {
await expect(await request.post('/__test__/tagging/models', { data: { unavailable: [],
missing_packages: { 'wd14-swinv2': ['onnxruntime'] } } })).toBeOK();
await openTag(page);
const answer = await generated(page);
expect(answer.status()).toBe(409);
expect(await answer.json()).toMatchObject({ success: false, code: 'model-unavailable' });
await expect(status(page)).toHaveText('WD14 SwinV2 needs onnxruntime. Install it into ComfyUI\'s Python: '
+ 'pip install onnxruntime. Use onnxruntime-gpu for CUDA.');
await expect(chips(page)).toHaveCount(0);
});
test('video tag is manual only', async ({ page, request }) => {
await openTag(page, VIDEO);
const preview = dialog(page).locator('video.tag-image-preview');
+2 -23
View File
@@ -41,18 +41,6 @@ FILES = (
("loose/unlinked.png", "unlinked"),
)
MODELS = ("wd14-swinv2", "wd14-vit", "gguf", "hf")
def _valid_missing(value) -> bool:
"""``{"<model>": ["<pip name>", ...]}`` with known models and string names."""
return isinstance(value, dict) and all(
name in MODELS
and isinstance(names, list)
and all(isinstance(pip, str) and pip for pip in names)
for name, names in value.items()
)
# Unlinked copies enough to virtualize the Images grid (it renders all up to 120).
MAX_EXTRA_IMAGES = 160
_TABLES = (
@@ -229,7 +217,6 @@ def register_tagging_commands(
if not 0 <= extra <= MAX_EXTRA_IMAGES:
raise web.HTTPBadRequest()
model.unavailable = set()
model.missing_packages = {}
model.suggestions = None
model.loaded = None
restore_download()
@@ -248,25 +235,17 @@ def register_tagging_commands(
@routes.post("/__test__/tagging/models")
async def models(request: web.Request) -> web.Response:
"""Mark fake models as not downloaded (``{"unavailable": [...]}``) and,
optionally, as missing backend packages (``"missing_packages":
{"<model>": ["<pip name>", ...]}``, #424)."""
"""Mark fake models as not downloaded (``{"unavailable": [...]}``)."""
data = await request.json() if request.can_read_body else None
if (
request.query
or not isinstance(data, dict)
or "unavailable" not in data
or set(data) - {"unavailable", "missing_packages"}
or set(data) != {"unavailable"}
or not isinstance(data["unavailable"], list)
or any(name not in MODELS for name in data["unavailable"])
or not _valid_missing(data.get("missing_packages", {}))
):
raise web.HTTPBadRequest()
model.unavailable = set(data["unavailable"])
model.missing_packages = {
name: list(names)
for name, names in data.get("missing_packages", {}).items()
}
return web.json_response({"success": True})
@routes.post("/__test__/tagging/loaded")
+3 -45
View File
@@ -16,8 +16,8 @@ installHostHooks(test, ['PromptManager', 'PromptManagerText', 'PromptManagerFocu
// `primitive`: a core string node the queue-time resolver follows. `legacy`: the frontend's virtual Primitive node,
// whose value is queued as literal text. `preview`: PrimitiveString -> PreviewAny, the shape of the maintainer's
// TextGenerate -> PreviewAny chain, whose text is only known when the node runs.
function build(page, { kind, positive, negative, prefix, saveToLibrary }) {
return page.evaluate(async ({ kind, positive, negative, prefix, saveToLibrary }) => {
function build(page, { kind, positive, negative, prefix }) {
return page.evaluate(async ({ kind, positive, negative, prefix }) => {
const { app } = await import('/scripts/app.js');
app.graph.clear();
const add = (type, x, y) => { const node = LiteGraph.createNode(type); app.graph.add(node); node.pos = [x, y]; return node; };
@@ -28,7 +28,6 @@ function build(page, { kind, positive, negative, prefix, saveToLibrary }) {
const set = (name, value) => { const widget = prompt.widgets.find(item => item.name === name); widget.value = value; if (widget.inputEl) widget.inputEl.value = value; };
set('text', positive.typed);
if (negative) set('negative_text', negative.typed);
if (saveToLibrary !== undefined) prompt.widgets.find(item => item.name === 'save_to_library').value = saveToLibrary;
const drive = (name, side, y) => {
if (!side.via) return;
if (side.via === 'legacy') {
@@ -74,7 +73,7 @@ function build(page, { kind, positive, negative, prefix, saveToLibrary }) {
await prompt._pmTextSerializerReady;
const { output, workflow } = await app.graphToPrompt();
return { output, workflow, promptNode: String(prompt.id) };
}, { kind, positive, negative, prefix, saveToLibrary });
}, { kind, positive, negative, prefix });
}
async function run(execution, page, testCase, built) {
@@ -196,47 +195,6 @@ for (const [label, linked, negativeVia] of [['positive', ['text'], 'primitive'],
});
}
// #90: with Save to library off the run still produces its image, but PromptManager saves, counts and links nothing,
// and the run's history records no prompt text for it.
for (const kind of ['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerSplit', 'PromptManagerPair']) {
test(`${kind} with Save to library off saves, counts and links nothing (#90)`, async ({ execution, page }) => {
test.setTimeout(90000);
const text = unique(`Not saved ${kind}`);
const negative = kind === 'PromptManagerPair' ? { typed: unique('Not saved negative') } : undefined;
const prefix = `notsaved/${randomUUID()}`;
const { output, workflow, promptNode } = await build(page, { kind, positive: { typed: text }, negative, prefix, saveToLibrary: false });
expect(output[promptNode].inputs.save_to_library).toBe(false);
const id = randomUUID();
const response = await execution.request.post('/prompt', { data: { prompt: output, prompt_id: id, extra_data: { extra_pnginfo: { workflow } } } });
expect(response.ok(), await response.text()).toBe(true);
let item;
await expect.poll(async () => {
item = (await (await execution.request.get(`/history/${id}`)).json())[id];
return item?.status?.status_str ?? null;
}, { timeout: 20000 }).toBe('success');
let result;
await expect.poll(async () => {
result = await readExecutionRuns(execution.databasePath, id);
const run = result.rows[0];
return Boolean(run && result.captures[run.run_id]);
}, { timeout: 10000, message: 'the run reaches SQLite' }).toBe(true);
const run = result.rows[0];
for (const context of result.captures[run.run_id].contexts || []) {
expect(context.positive_prompt_id ?? null).toBeNull();
expect(context.pair_id ?? null).toBeNull();
}
expect((result.usage[run.run_id] || []).every(entry => entry.run_count === 0)).toBe(true);
const files = Object.values(item.outputs).flatMap(value => value.images || []).map(image => image.filename);
expect(files.length).toBeGreaterThan(0);
await scan(execution);
expect(Object.values(await linkedPrompts(execution, files))).toEqual(files.map(() => null));
for (const query of [text, negative?.typed].filter(Boolean)) {
const found = await (await execution.request.get(`/prompt_manager/search?text=${encodeURIComponent(query)}`)).json();
expect(found.results ?? [], query).toEqual([]);
}
});
}
// #347: a cached PromptManager node counts every job, as an executing one does. Jobs after the first change only
// the sampler seed, so ComfyUI serves the PromptManager node (and its PreviewAny source) from its output cache.
const COUNTS = `import json, sqlite3, sys, pathlib
+1 -10
View File
@@ -370,7 +370,6 @@ test('workflows saved before #343 load unchanged and their linked text keeps wor
const nodes = Object.fromEntries(app.graph._nodes.map(node => [node.id, node]));
const reloaded = app.graph.serialize();
return {
saves: Object.fromEntries(Object.entries(output).filter(([, node]) => node.class_type.startsWith('PromptManager')).map(([id, node]) => [id, node.inputs.save_to_library])),
values: Object.fromEntries(reloaded.nodes.map(node => [node.id, node.widgets_values])),
links: Object.fromEntries(Object.entries(output).map(([id, node]) => [id, { text: node.inputs.text, negative_text: node.inputs.negative_text }])),
widgetBound: Object.fromEntries(Object.values(nodes).filter(node => node.type.startsWith('PromptManager')).map(node => [node.id, node.inputs.filter(input => ['text', 'negative_text'].includes(input.name) && input.widget).map(input => input.name)])),
@@ -378,16 +377,8 @@ test('workflows saved before #343 load unchanged and their linked text keeps wor
}, workflow);
for (const node of workflow.nodes) {
// Every saved widget value comes back as it was saved, except the typed text main lost under a link (null).
// #90: the last saved value is the editor's own slot (''), where the Save to library switch is now: it loads on.
if (node.widgets_values?.[0] === null) continue;
if (!node.type.startsWith('PromptManager')) { expect(result.values[node.id], `${node.type} ${node.id}`).toEqual(node.widgets_values); continue; }
const saved = node.widgets_values.length - 1;
expect(result.values[node.id].slice(0, saved), `${node.type} ${node.id}`).toEqual(node.widgets_values.slice(0, saved));
expect(result.values[node.id][saved], `${node.type} ${node.id} save_to_library`).toBe(true);
if (node.widgets_values?.[0] !== null) expect(result.values[node.id], `${node.type} ${node.id}`).toEqual(node.widgets_values);
}
// Every PromptManager node of the old workflow queues with Save to library on.
for (const save of Object.values(result.saves)) expect(save).toBe(true);
expect(Object.keys(result.saves).length).toBeGreaterThan(0);
expect(result.links).toMatchObject({
1: { text: 'Saved PromptManager text' }, 2: { text: 'Saved PromptManagerText text' }, 3: { text: 'Saved PromptManagerFocused text' },
4: { text: 'Saved PromptManagerSplit text' }, 5: { text: 'Saved PromptManagerPair text', negative_text: 'Saved pair negative' },
-68
View File
@@ -1,68 +0,0 @@
// #90: the node's save_to_library switch (web/js/save-to-library.js).
const test = require('node:test');
const assert = require('node:assert/strict');
const source = import('../../web/js/save-to-library.js');
function classicNode(saved) {
const names = ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text', 'save_to_library', 'prompt_manager_search_ui'];
const widgets = names.map(name => ({name, value: name === 'save_to_library' ? true : ''}));
const node = {widgets, widgets_values: saved ? [...saved] : undefined};
// What a host configure does: each saved value goes to the widget at its index.
(saved || []).forEach((value, index) => { if (widgets[index]) widgets[index].value = value; });
return node;
}
test('an older workflow stored the editor slot where the switch is now: it loads on', async () => {
const {normalizeSaveWidget, savesToLibrary} = await source;
const node = classicNode(['a cat', 'c', 't', '', '', '', '']);
assert.equal(node.widgets[6].value, '', 'the host put the old DOM slot value into the switch');
assert.equal(normalizeSaveWidget(node), true);
assert.equal(node.widgets[6].value, true);
assert.equal(node.widgets_values[6], true);
assert.equal(savesToLibrary(node), true);
assert.deepEqual(node.widgets_values.slice(0, 6), ['a cat', 'c', 't', '', '', ''], 'no earlier value moved');
});
test('null, undefined and missing slots load on; saved booleans are kept', async () => {
const {normalizeSaveWidget, savesToLibrary} = await source;
for (const value of [null, undefined, 0, 'false']) {
const node = classicNode(['a', '', '', '', '', '', value]);
normalizeSaveWidget(node);
assert.equal(savesToLibrary(node), true, String(value));
}
const short = classicNode(['a', '', '', '', '', '']);
assert.equal(normalizeSaveWidget(short), false);
assert.equal(savesToLibrary(short), true);
const off = classicNode(['a', '', '', '', '', '', false]);
assert.equal(normalizeSaveWidget(off), false);
assert.equal(savesToLibrary(off), false);
});
test('a node without the widget (an older backend) saves', async () => {
const {normalizeSaveWidget, savesToLibrary} = await source;
assert.equal(savesToLibrary({widgets: [{name: 'text', value: ''}]}), true);
assert.equal(normalizeSaveWidget({widgets: []}), false);
assert.equal(savesToLibrary(undefined), true);
});
test('the switch moves after the editor fallbacks so every frontend has one order', async () => {
const {placeSaveWidgetLast} = await source;
const widgets = ['text', 'category', 'tags', 'search_text', 'save_to_library', 'prepend_text', 'append_text'].map(name => ({name}));
const node = {widgets};
assert.equal(placeSaveWidgetLast(node), true);
assert.deepEqual(node.widgets.map(widget => widget.name), ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text', 'save_to_library']);
assert.equal(placeSaveWidgetLast(node), false, 'already last');
assert.equal(placeSaveWidgetLast({widgets: [{name: 'text'}]}), false);
});
test('setting the switch updates the widget, its saved value and notifies the host', async () => {
const {setSaveToLibrary, savesToLibrary} = await source;
const calls = [];
const node = classicNode(['a', '', '', '', '', '', true]);
node.widgets[6].callback = value => calls.push(value);
setSaveToLibrary(node, false);
assert.equal(savesToLibrary(node), false);
assert.equal(node.widgets_values[6], false);
assert.deepEqual(calls, [false]);
setSaveToLibrary({widgets: []}, false);
});
-29
View File
@@ -383,32 +383,3 @@ test('the D117 wrap rule targets only the settings summary button', async () =>
assert.deepEqual(rules.map(line => line.trim()), ['.ca-tools #viewer-details .settings-summary { white-space: normal; /* D117 */ }']);
assert.doesNotMatch(css, /#viewer-details > div > button/);
});
test('category chip names its origin and shows an own category for any link status (#332)', async () => {
const base = await localRecord(localPng('a.png'));
const chipTexts = record => {
const {f, done} = renderFull(record);
try {
return f.walk(f.container, n => /(^| )chip( |$)/.test(n.className ?? '')).map(n => n.textContent);
} finally { done(); }
};
const cases = [
[{status:'none', category:'Scans', category_scope:'image'}, 'Image category: Scans'],
[{status:'ambiguous', category:'Scans', category_scope:'image'}, 'Image category: Scans'],
[{status:'linked', category:'Portraits', category_scope:'image'}, 'Image category: Portraits'],
[{status:'linked', category:'Landscapes', category_scope:'positive'}, 'Positive prompt category: Landscapes'],
[{status:'linked', category:'Recipes', category_scope:'pair'}, 'Recipe category: Recipes'],
];
for (const [library, text] of cases) {
const chips = chipTexts({...base, library:{...base.library, tags:[], tags_scope:null, ...library}});
assert.ok(chips.includes(text), `${JSON.stringify(library)} -> ${chips}`);
}
for (const library of [
{status:'none', category:'Inherited', category_scope:'positive'},
{status:'linked', category:null, category_scope:null},
{status:'none', category:null, category_scope:null},
]) {
const chips = chipTexts({...base, library:{...base.library, tags:[], tags_scope:null, ...library}});
assert.equal(chips.some(chip => /category: /.test(chip)), false, JSON.stringify(library));
}
});
@@ -135,41 +135,6 @@ test('a model that is not downloaded is named with the server sentence', () => {
assert.throws(() => modelUnavailable({}, 'nope'), TypeError);
});
test('a downloaded model whose backend package is missing names the package and the command (#424)', () => {
const { modelUnavailable, needsPackages } = state;
const gguf = { downloaded: true, name: 'GGUF (Recommended)', label: 'GGUF JoyCaption', missing_packages: ['llama-cpp-python'],
install_command: 'pip install llama-cpp-python',
install_note: 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python' };
const sentence = 'GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI\'s Python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python';
assert.equal(modelUnavailable({ gguf }, 'gguf'), sentence);
// Not downloaded is named first, in its own sentence.
assert.equal(modelUnavailable({ gguf: { ...gguf, downloaded: false } }, 'gguf'),
'GGUF (Recommended) is not downloaded. Download it in System › Auto-tagging.');
assert.equal(modelUnavailable({ gguf: { ...gguf, missing_packages: [] } }, 'gguf'), '');
assert.equal(needsPackages('HuggingFace JoyCaption', { missing_packages: ['accelerate', 'bitsandbytes'] }),
'HuggingFace JoyCaption needs accelerate and bitsandbytes. Install them into ComfyUI\'s Python: pip install accelerate bitsandbytes.');
assert.equal(needsPackages('X', { missing_packages: ['a', 'b', 'c'], install_command: 'pip install a b c' }),
'X needs a, b and c. Install them into ComfyUI\'s Python: pip install a b c.');
assert.equal(needsPackages('X', null), '');
assert.equal(needsPackages('X', { missing_packages: [1, ''] }), '');
// Without the server's label the option's own name is used.
assert.ok(modelUnavailable({ 'wd14-vit': { downloaded: true, missing_packages: ['onnxruntime'] } }, 'wd14-vit')
.startsWith('WD14 ViT needs onnxruntime.'));
});
test('the bulk dialog keeps Start disabled while the chosen model misses a package (#424)', async () => {
const { createTaggingOptionsController } = state;
const models = { 'wd14-vit': { downloaded: true, label: 'WD14 ViT', missing_packages: ['onnxruntime'],
install_command: 'pip install onnxruntime' } };
const options = createTaggingOptionsController({ api: { get: async () => ({ success: true, models, model_type: 'wd14-vit' }) } });
await options.open();
assert.equal(options.state.model, 'wd14-vit');
assert.equal(options.state.canStart, false);
assert.equal(options.state.unavailable, 'WD14 ViT needs onnxruntime. Install it into ComfyUI\'s Python: pip install onnxruntime.');
options.dispose();
});
test('the bulk options start from the server defaults and wait for them before Start', async () => {
const { createTaggingOptionsController } = state;
let answer;
@@ -278,7 +278,6 @@ test('record rejects root path media and invalid versioned fields with retryable
data => { data.resolution.sampler_id = {}; },
data => { data.library.tags_scope = 'recipe'; },
data => { data.library.category_scope = 'recipe'; },
data => { data.library.tags_scope = 'image'; },
data => { delete data.library.category_scope; },
data => { delete data.library.tags_scope; },
]) {
@@ -943,14 +942,3 @@ test("names the server quotes (parentheses, '!*') still match their original and
assert.equal(record.download.url, entry.url);
assert.deepEqual(record.preview, { url: entry.thumbnail_url });
});
test('record accepts an image\'s own category on any link status (#332)', () => {
const toRecord = contract('toInspectionRecord');
for (const status of ['linked', 'none', 'ambiguous', 'unavailable']) {
const data = envelope();
Object.assign(data.library, { status, category: 'Scans', category_scope: 'image' });
const record = toRecord(data, image(), extractGenerationFacts);
assert.equal(record.library.category, 'Scans');
assert.equal(record.library.category_scope, 'image');
}
});
-54
View File
@@ -482,60 +482,6 @@ test('Tag output images › Review previews the drafts, else says why; a refused
} finally { scope.dispose(); dom.restore(); }
});
// #424: GGUF on disk without llama-cpp-python.
const NEEDS_LLAMA = { missing_packages: ['llama-cpp-python'], install_command: 'pip install llama-cpp-python',
install_note: 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python' };
const NEEDS_SENTENCE = 'GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI\'s Python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python';
test('a model on disk whose backend package is missing never reads as ready (#424)', async () => {
const { modelDescription, readStatus, batchCloseNote, STATUS_COPY } = await tagging;
const status = readStatus({ ...MODELS, models: { ...MODELS.models,
gguf: entry('GGUF JoyCaption', '~4.5 GB', { ...NEEDS_LLAMA }),
'wd14-vit': entry('WD14 ViT', '~400 MB', { downloaded: false, missing_packages: ['onnxruntime'],
install_command: 'pip install onnxruntime', install_note: 'Use onnxruntime-gpu for CUDA.' }),
hf: entry('HuggingFace JoyCaption', '~16 GB', { downloaded: false, model_exists: undefined,
missing_packages: ['accelerate', 'bitsandbytes'], install_command: 'pip install accelerate bitsandbytes' }) } });
const ready = modelDescription(status, 'gguf');
assert.notEqual(ready, STATUS_COPY.downloaded);
assert.equal(ready, 'Downloaded. Needs llama-cpp-python: pip install llama-cpp-python. '
+ 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python');
// Not downloaded reads differently, and still says what it will need.
assert.equal(modelDescription(status, 'wd14-vit'), 'Not downloaded. Download size ~400 MB. Also needs onnxruntime.');
assert.equal(modelDescription(status, 'hf'), 'Not downloaded. Download size ~16 GB. Also needs accelerate and bitsandbytes.');
const partial = readStatus({ ...MODELS, models: { ...MODELS.models, gguf: entry('GGUF JoyCaption', '~4.5 GB',
{ downloaded: false, model_exists: true, mmproj_exists: false, ...NEEDS_LLAMA }) } });
assert.equal(modelDescription(partial, 'gguf'), 'Partly downloaded. Download again to finish. Also needs llama-cpp-python.');
// Untrusted shapes are ignored; a missing command is built from the names.
const odd = readStatus({ ...MODELS, models: { ...MODELS.models, gguf: entry('GGUF JoyCaption', '~4.5 GB',
{ missing_packages: ['llama-cpp-python', 3, ''], install_command: 7, install_note: {} }) } });
assert.equal(modelDescription(odd, 'gguf'), 'Downloaded. Needs llama-cpp-python: pip install llama-cpp-python.');
const junk = readStatus({ ...MODELS, models: { ...MODELS.models, gguf: entry('GGUF JoyCaption', '~4.5 GB',
{ missing_packages: 'llama-cpp-python' }) } });
assert.equal(modelDescription(junk, 'gguf'), STATUS_COPY.downloaded);
// The batch review says why it cannot start, in the server's sentence.
assert.equal(batchCloseNote({ ...PLAN, model: { type: 'gguf', label: 'GGUF JoyCaption', downloaded: true, ...NEEDS_LLAMA } }),
NEEDS_SENTENCE);
assert.equal(batchCloseNote({ ...PLAN, model: { ...PLAN.model, downloaded: false, ...NEEDS_LLAMA } }),
'WD14 SwinV2 is not downloaded. Download it first.');
});
test('Review explains a missing backend package instead of reviewing (#424)', async () => {
const models = { ...MODELS, models: { ...MODELS.models, gguf: entry('GGUF JoyCaption', '~4.5 GB', { ...NEEDS_LLAMA }) } };
const { dom, scope, button, byId, calls } = await mounted({ models });
try {
byId('tag-model').value = 'gguf'; await byId('tag-model').dispatch('change');
const review = button('Review');
assert.equal(review.getAttribute('aria-disabled'), 'true');
assert.equal(review.dataset.reason, NEEDS_SENTENCE);
await review.dispatch('click');
assert.equal(calls.toasts.at(-1), NEEDS_SENTENCE);
assert.deepEqual(calls.post, []);
// Download stays offered: files and packages are separate.
assert.equal(button('Download again').getAttribute('aria-disabled'), null);
} finally { scope.dispose(); dom.restore(); }
});
test('settingRefusal marks only an Auto-tagging save\'s plain 400 as invalid-setting (CUJ 2a)', async () => {
const { settingRefusal } = await import('../../web/js/workspace/system.mjs');
const refusal = (extra = {}) => Object.assign(new Error('General-tag confidence must be between 0 and 1.'),
+1 -44
View File
@@ -29,7 +29,6 @@ except ImportError:
model_gate = None
from py import config as config_module
from py.autotag import MissingBackendPackages, package_fields
AutotagConfig = getattr(config_module, "AutotagConfig", None)
@@ -38,11 +37,6 @@ WAIT = 30.0
WD14 = {"general_threshold": 0.35, "character_threshold": 0.85}
LLM = {"instructions": "Generate tags."}
UNAVAILABLE = "Download it in System › Auto-tagging."
NEEDS_LLAMA = (
"GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI's Python: "
'pip install llama-cpp-python. For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" '
"pip install llama-cpp-python"
)
SCORED = [("sandstone", 0.98), ("desert", 0.96)] + [
(f"tag_{i:03d}", round(0.9 - i * 0.005, 4)) for i in range(150)
@@ -65,8 +59,6 @@ class FakeTagger:
"hf": {},
}
self.downloaded = {name: True for name in self.models_config}
# pip names of the backend packages that cannot be imported (#424)
self.missing = {name: [] for name in self.models_config}
self.loaded = None
self.load_calls = []
self.unload_calls = 0
@@ -79,11 +71,7 @@ class FakeTagger:
def get_models_status(self):
return {
name: {
"downloaded": self.downloaded[name],
"model_path": None,
**package_fields(name, self.missing[name]),
}
name: {"downloaded": self.downloaded[name], "model_path": None}
for name in self.models_config
}
@@ -327,37 +315,6 @@ class TestModelAvailability(SuggestTestCase):
self.assertEqual(self.tagger.calls, [])
self.assertIsNone(model_gate.holder())
async def test_missing_backend_package_is_refused_before_the_model(self):
observed = await self.observed()
self.tagger.missing["gguf"] = ["llama-cpp-python"]
status, body = await self.suggest(self.suggest_body(observed, "gguf"))
self.assertEqual((status, body["code"]), (409, "model-unavailable"))
self.assertEqual(body["success"], False)
self.assertEqual(body["error"], NEEDS_LLAMA)
self.assertEqual(self.tagger.load_calls, [])
self.assertIsNone(model_gate.holder())
# Not downloaded still reads as before, whatever the packages.
self.tagger.downloaded["gguf"] = False
status, body = await self.suggest(self.suggest_body(observed, "gguf"))
self.assertIn("is not downloaded", body["error"])
async def test_a_package_lost_after_the_check_is_the_same_409_without_a_traceback(
self,
):
observed = await self.observed()
self.tagger.load_error = MissingBackendPackages("gguf", ["llama-cpp-python"])
with self.assertNoLogs(self.api.logger, level="ERROR"):
status, body = await self.suggest(self.suggest_body(observed, "gguf"))
self.assertEqual((status, body["code"]), (409, "model-unavailable"))
self.assertEqual(body["error"], NEEDS_LLAMA)
self.assertIsNone(model_gate.holder())
entry = self.api._captures._entries[observed["capture"]["token"]]
self.assertEqual(entry.uses, 0)
async def test_load_failure_is_a_generic_error_and_frees_everything(self):
observed = await self.observed()
self.tagger.load_error = RuntimeError(f"cannot load {self.output_dir}")
-24
View File
@@ -73,7 +73,6 @@ class FakeAutotagService:
self.status_calls = 0
self.status_threads = []
self.status_paths = {}
self.status_extra = {}
self.suggest_calls = 0
def get_models_status(self):
@@ -83,7 +82,6 @@ class FakeAutotagService:
name: {
"downloaded": self.downloaded,
"model_path": self.status_paths.get(name),
**self.status_extra.get(name, {}),
}
for name in self.models_config
}
@@ -1113,24 +1111,6 @@ class TestModelStatus(AutotagAPITestCase):
leaked = [text for text in _strings(data) if _absolute(text)]
self.assertEqual(leaked, [])
async def test_models_status_names_missing_backend_packages(self):
fields = {
"missing_packages": ["llama-cpp-python"],
"install_command": "pip install llama-cpp-python",
"install_note": "For a CUDA build: ...",
}
self.service.status_extra = {"gguf": fields}
models = (await self._models())["models"]
self.assertTrue(models["gguf"]["downloaded"])
self.assertEqual(
{key: models["gguf"][key] for key in fields},
fields,
)
self.assertEqual(
{key: models["wd14-vit"][key] for key in fields},
{"missing_packages": [], "install_command": None, "install_note": None},
)
async def test_models_status_adds_label_size_label_and_loaded(self):
self.service._loaded_type = "wd14-vit"
data = await self._models()
@@ -1689,10 +1669,6 @@ class TestTagOutputImagesPreview(BatchJobTestCase):
"label": "WD14 ViT",
"size_label": "~400 MB",
"downloaded": True,
# #424: the backend packages it cannot import, none here.
"missing_packages": [],
"install_command": None,
"install_note": None,
},
)
self.assertEqual(skip["existing"], "skip")
+2 -74
View File
@@ -62,16 +62,12 @@ class TestVocabularyAPI(ImageAPITestCase):
},
)
self.assertEqual(
categories["categories"],
[{"value": "Animal", "prompt_count": 1, "image_count": 0}],
categories["categories"], [{"value": "Animal", "prompt_count": 1}]
)
_, rest = await self.get(
"categories", limit=1, cursor=categories["next_cursor"]
)
self.assertEqual(
rest["categories"],
[{"value": "Pet", "prompt_count": 1, "image_count": 0}],
)
self.assertEqual(rest["categories"], [{"value": "Pet", "prompt_count": 1}])
self.assertFalse(rest["has_more"])
async def test_vocabulary_cursor_is_signed_epoch_scoped(self):
@@ -492,7 +488,6 @@ class TestVocabularyAPI(ImageAPITestCase):
"other_prompts": 0,
"negative_variants": 1,
"archived_negative_variants": 0,
"images": 0,
}
self.assertEqual(
preview,
@@ -533,73 +528,6 @@ class TestVocabularyAPI(ImageAPITestCase):
self.assertEqual(response.status, 409)
self.assertEqual(replay["code"], "vocabulary-conflict")
async def test_category_rows_report_images_and_image_only_values(self):
self._save_prompt("Owner", category="Prompt value")
_, options_before = await self.get_options()
self.api.db.set_image_categories(
[("root", "a.png"), ("root", "b.png")], "Prompt value"
)
self.api.db.set_image_categories([("root", "c.png")], "Image value")
response, first = await self.get("categories", limit=1)
self.assertEqual(response.status, 200)
self.assertEqual(
first["categories"],
[{"value": "Prompt value", "prompt_count": 1, "image_count": 2}],
)
self.assertEqual(first["total"], 2)
_, rest = await self.get("categories", limit=1, cursor=first["next_cursor"])
self.assertEqual(
rest["categories"],
[{"value": "Image value", "prompt_count": 0, "image_count": 1}],
)
self.assertFalse(rest["has_more"])
# The Prompt library's category list does not gain image values.
_, options_after = await self.get_options()
self.assertEqual(options_after, options_before)
self.assertEqual(options_after["categories"], ["Prompt value"])
async def get_options(self):
response = await self.client.get("/prompt_manager/library/options")
return response, await response.json()
async def test_category_routes_rename_image_only_value(self):
epoch = self.api.db.get_execution_epoch()
self._save_prompt("Bystander", category="Kept")
self.api.db.set_image_categories(
[("root", "a.png"), ("second", "b.png")], "Image value"
)
prompts_before = self.category_rows()
body = {"expected_epoch": epoch, "source": "Image value", "target": "Moved"}
response, preview = await self.send("POST", "categories/preview", body)
self.assertEqual(response.status, 200, preview)
impact = {
"positive_prompts": 0,
"other_prompts": 0,
"negative_variants": 0,
"archived_negative_variants": 0,
"images": 2,
}
self.assertEqual(preview["impact"], impact)
stale = preview["impact_token"]
self.api.db.set_image_categories([("root", "late.png")], "Image value")
response, refused = await self.send(
"POST", "categories/apply", {**body, "impact_token": stale}
)
self.assertEqual(response.status, 409)
self.assertEqual(refused["code"], "vocabulary-conflict")
self.assertEqual(self.api.db.image_category("root", "a.png"), "Image value")
_, preview = await self.send("POST", "categories/preview", body)
response, applied = await self.send(
"POST",
"categories/apply",
{**body, "impact_token": preview["impact_token"]},
)
self.assertEqual(response.status, 200, applied)
self.assertEqual(applied["impact"], {**impact, "images": 3})
for key in (("root", "a.png"), ("second", "b.png"), ("root", "late.png")):
self.assertEqual(self.api.db.image_category(*key), "Moved")
self.assertEqual(self.category_rows(), prompts_before)
async def test_preview_validation(self):
epoch = self.api.db.get_execution_epoch()
self._save_prompt("Owner", category="Scenery")
@@ -1,262 +0,0 @@
"""Images filters by the effective category (#332 MC4, MC5, DC6).
An image's effective category is its own category when it has one, else the
category its link supplies (the recipe's when set, else the positive's), else
none. The category filter, "select all N matching" and saved views match it.
"""
import os
import sqlite3
from unittest.mock import patch
try:
import test_api_workspace_images as workspace
from test_image_categories import category_checksum
except ImportError:
from tests import test_api_workspace_images as workspace
from tests.test_image_categories import category_checksum
from py.api.images import image_category_key, parse_listing_filter
class WorkspaceCategoryCase(workspace.ImageAPITestCase):
"""Listing helpers plus own-category writes keyed as the routes key them."""
media = workspace.TestWorkspaceImages.media
get = workspace.TestWorkspaceImages.get
tag = workspace.TestWorkspaceImageTags.tag
names = workspace.TestWorkspaceImageTags.names
async def tearDownAsync(self):
self.api.db.close_all()
workspace.assert_closed(self, self._temp_db.name, self._output_tmp.name)
await super().tearDownAsync()
def own(self, path, value, sources=None):
sources = sources or self.api._get_all_output_dirs()
self.api.db.set_image_categories([image_category_key(sources, path)], value)
async def listed(self, **params):
response, body = await self.get(sort="name", **params)
self.assertEqual(response.status, 200, body)
return self.names(body), body
class TestEffectiveCategoryFilter(WorkspaceCategoryCase):
def batch(self):
"""Four images of "castle at dusk" (Landscapes); one is overridden."""
castle = self._save_prompt("castle at dusk", category="Landscapes", rating=4)
files = [self.media(f"castle_{n}.png", mtime=100 + n) for n in range(4)]
for path in files:
self.api.db.link_image_to_prompt(castle, str(path))
self.own(files[0], "Portraits")
return castle, files
async def test_category_filter_prefers_own_category(self):
castle, _files = self.batch()
conn = self.api.db.model.get_connection()
before = category_checksum(conn)
names, body = await self.listed(category="Portraits")
self.assertEqual(names, ["castle_0.png"])
# The caption still counts the prompt behind the image shown.
self.assertEqual(body["positive_count"], 1)
names, body = await self.listed(category="Landscapes")
self.assertEqual(names, ["castle_1.png", "castle_2.png", "castle_3.png"])
self.assertEqual(body["total"], 3)
names, _ = await self.listed()
self.assertEqual(len(names), 4)
self.assertEqual(category_checksum(conn), before)
self.assertEqual(self.api.db.get_prompt_by_id(castle)["category"], "Landscapes")
async def test_category_filter_lists_unlinked_with_own(self):
scans = [self.media(f"scan_{n}.png", mtime=100 + n) for n in range(2)]
self.media("plain.png", mtime=50)
for path in scans:
self.own(path, "Scans")
names, body = await self.listed(category="Scans")
self.assertEqual(names, ["scan_0.png", "scan_1.png"])
self.assertEqual(body["positive_count"], 0)
async def test_recipe_category_is_inherited(self):
"""MC4: a recipe's own category, not the positive's, is inherited."""
db = self.api.db
self._save_prompt("Pair positive", category="Night")
pair = db.save_prompt_pair("Pair positive", "Noise", category="Recipes")
blank = db.save_prompt_pair("Pair positive", "Other noise", category=" ")
recipe = self.media("recipe.png", mtime=103)
blank_recipe = self.media("blank_recipe.png", mtime=102)
positive = db.get_prompt_pair(pair)["positive_prompt_id"]
for path, pair_id in ((recipe, pair), (blank_recipe, blank)):
image = db.link_image_to_prompt(positive, str(path))
db.link_image_to_pair(pair_id, image, {"source": "fixture"})
self.assertEqual((await self.listed(category="Recipes"))[0], ["recipe.png"])
# A blank recipe category falls back to the positive's.
self.assertEqual((await self.listed(category="Night"))[0], ["blank_recipe.png"])
# An own category still wins over the recipe's.
self.own(recipe, "Mine")
self.assertEqual((await self.listed(category="Recipes"))[0], [])
self.assertEqual((await self.listed(category="Mine"))[0], ["recipe.png"])
async def test_ambiguous_files_inherit_nothing_but_keep_their_own(self):
first = self._save_prompt("Owner one", category="Night")
second = self._save_prompt("Owner two", category="Night")
ambiguous = self.media("ambiguous.png")
for owner in (first, second):
self.api.db.link_image_to_prompt(owner, str(ambiguous))
self.assertEqual((await self.listed(category="Night"))[0], [])
self.own(ambiguous, "Night")
self.assertEqual((await self.listed(category="Night"))[0], ["ambiguous.png"])
async def test_combines_with_tags_search_and_rating(self):
stone = self._save_prompt(
"castle at dusk", category="Landscapes", tags=["stone"], rating=4
)
files = [self.media(f"castle_{n}.png", mtime=100 + n) for n in range(2)]
for path in files:
self.api.db.link_image_to_prompt(stone, str(path))
self.own(files[0], "Portraits")
other = self._save_prompt("harbor at noon", category="Portraits", rating=2)
harbor = self.media("harbor.png", mtime=90)
self.api.db.link_image_to_prompt(other, str(harbor))
names, _ = await self.listed(category="Portraits", tags="stone")
self.assertEqual(names, ["castle_0.png"])
names, _ = await self.listed(category="Portraits", q="harbor")
self.assertEqual(names, ["harbor.png"])
names, _ = await self.listed(category="Portraits", q="castle_0")
self.assertEqual(names, ["castle_0.png"])
response, body = await self.get(category="Portraits", sort="rating")
self.assertEqual(response.status, 200, body)
self.assertEqual(self.names(body), ["castle_0.png", "harbor.png"])
self.assertEqual(body["positive_count"], 2)
async def test_image_tag_admitted_file_keeps_its_inherited_category(self):
night = self._save_prompt("Night owl", category="Night", tags=["a"])
tagged = self.media("tagged.png", mtime=103)
unlinked = self.media("unlinked.png", mtime=102)
self.api.db.link_image_to_prompt(night, str(tagged))
self.tag(tagged, ["b"])
self.tag(unlinked, ["a", "b"])
names, _ = await self.listed(tags="a,b", category="Night")
self.assertEqual(names, ["tagged.png"])
self.own(unlinked, "Night")
names, _ = await self.listed(tags="a,b", category="Night")
self.assertEqual(names, ["tagged.png", "unlinked.png"])
async def test_cursor_pages_walk_only_the_category(self):
files = [self.media(f"img_{n:02}.png", mtime=100 + n) for n in range(7)]
for path in files[::2]:
self.own(path, "Even")
seen, cursor = [], None
while True:
params = {"category": "Even", "limit": 2, "sort": "name"}
if cursor:
params["cursor"] = cursor
response, body = await self.get(**params)
self.assertEqual(response.status, 200, body)
seen += self.names(body)
self.assertEqual(body["total"], 4)
cursor = body["next_cursor"]
if not cursor:
break
self.assertEqual(seen, ["img_00.png", "img_02.png", "img_04.png", "img_06.png"])
async def test_an_outer_source_lists_a_nested_sources_own_category(self):
inner = self.output_dir / "nested"
nested = self.media("nested/a.png")
self.media("b.png")
self.api._get_all_output_dirs = lambda: [self.output_dir, inner]
self.own(nested, "Scans")
names, _ = await self.listed(category="Scans", root="0")
self.assertEqual(names, ["nested/a.png"])
names, _ = await self.listed(category="Scans", root="1")
self.assertEqual(names, ["a.png"])
# The outer source's own key for that path is not the file's key.
from py.api.images import _favorite_root_key
self.api.db.clear_image_categories([image_category_key([inner], nested)])
self.api.db.set_image_categories(
[(_favorite_root_key(self.output_dir), "nested/a.png")], "Stale"
)
self.assertEqual((await self.listed(category="Stale"))[0], [])
async def test_a_gone_files_category_is_not_listed(self):
gone = self.media("gone.png")
self.own(gone, "Scans")
os.unlink(gone)
self.api.invalidate_gallery_cache()
self.assertEqual((await self.listed(category="Scans"))[0], [])
async def test_a_failed_own_category_read_is_listing_failed(self):
self.media("a.png")
def broken(*_args, **_kwargs):
raise sqlite3.OperationalError("secret detail")
with patch.object(self.api.db, "own_categories_of", side_effect=broken):
with self.assertLogs("prompt_manager", level="ERROR"):
response, body = await self.get(category="Scans")
self.assertEqual(response.status, 500)
self.assertNotIn("secret", str(body))
# Without a category nothing new is read.
response, body = await self.get()
self.assertEqual(response.status, 200, body)
async def test_select_all_matching_uses_the_effective_category(self):
_castle, files = self.batch()
resolved = await self.api.resolve_listing_filter(
parse_listing_filter({"category": "Portraits"}), 100
)
self.assertEqual([t[2] for t in resolved.targets], ["castle_0.png"])
resolved = await self.api.resolve_listing_filter(
parse_listing_filter({"category": "Landscapes"}), 100
)
self.assertEqual(len(resolved.targets), 3)
self.assertEqual(len(files), 4)
async def test_saved_view_category_means_the_effective_category(self):
"""CUJ 8: a saved view's category is the listing's query value."""
_castle, files = self.batch()
self.own(files[1], "Portraits")
names, _ = await self.listed(category="Portraits")
self.assertEqual(names, ["castle_0.png", "castle_1.png"])
class TestFilterAndChipAgreeOnBlanks(WorkspaceCategoryCase):
"""The filter and the inspector use one inherited-category rule."""
async def chip(self, path):
response = await self.client.get(
f"/prompt_manager/images/inspect/{path}", params={"root": "0"}
)
body = await response.json()
self.assertEqual(response.status, 200, body)
return body["library"]["category"]
async def test_blank_recipe_values_agree(self):
db = self.api.db
positive = self._save_prompt("Pair positive", category="Night")
blanks = {"spaces": " ", "tab": "\t", "nbsp": " ", "set": "Recipes"}
for index, (label, value) in enumerate(blanks.items()):
pair = db.save_prompt_pair("Pair positive", f"noise {label}")
path = self.media(f"{label}.png", mtime=100 + index)
image = db.link_image_to_prompt(positive, str(path))
db.link_image_to_pair(pair, image, {"source": "fixture"})
with db.model.get_connection() as conn:
conn.execute(
"UPDATE prompt_pairs SET category=? WHERE id=?", (value, pair)
)
blob = db.save_prompt_pair("Pair positive", "noise blob")
path = self.media("blob.png", mtime=90)
image = db.link_image_to_prompt(positive, str(path))
db.link_image_to_pair(blob, image, {"source": "fixture"})
with db.model.get_connection() as conn:
conn.execute(
"UPDATE prompt_pairs SET category=CAST(' ' AS BLOB) WHERE id=?",
(blob,),
)
for name in ("spaces", "tab", "nbsp", "set", "blob"):
with self.subTest(name=name):
shown = await self.chip(f"{name}.png")
for value in ("Night", "Recipes"):
listed = f"{name}.png" in (await self.listed(category=value))[0]
self.assertEqual(listed, shown == value, (name, value, shown))
+13 -69
View File
@@ -1347,7 +1347,7 @@ class TestWorkspaceImages(ImageAPITestCase):
_, data = await self.get(q="harbor")
self.assertEqual(self._names(data), ["night_a.png"])
self.assertEqual(
self.api.db.get_image_paths_matching("zebra", []),
self.api.db.get_image_paths_matching("zebra", None, []),
{
normalize_image_path(str(files[name]))
for name in ("night_ab", "day_ab", "titled")
@@ -1407,10 +1407,8 @@ class TestWorkspaceImages(ImageAPITestCase):
for query in statements
if query.lstrip().upper().startswith(("SELECT", "WITH"))
]
# Plus the one stream of tagged image records q and tags read (B3),
# and the category's own-category read of the one source (#332).
self.assertEqual(len(selects), 5, selects)
self.assertEqual(sum("image_categories" in q for q in selects), 1, selects)
# Plus the one stream of tagged image records q and tags read (B3).
self.assertEqual(len(selects), 4, selects)
self.assertEqual(sum("image_favorites" in q for q in selects), 1, selects)
self.assertEqual(sum("media_image_tags" in q for q in selects), 1, selects)
@@ -1530,16 +1528,15 @@ class TestWorkspaceImages(ImageAPITestCase):
elsewhere = self.api.db.get_prompt_id_for_image("/elsewhere/7.png")
self.assertEqual(
self.api.db.get_image_link_matches(
"", [], every_link=True, wanted=wanted, category="Night"
"", "Night", [], every_link=True, wanted=wanted
),
{
normalize_image_path(str(files["night_ab"])): (
True,
5,
self.api.db.get_prompt_id_for_image(str(files["night_ab"])),
"Night",
),
"/elsewhere/7.png": (True, 5, elsewhere, "Night"),
"/elsewhere/7.png": (True, 5, elsewhere),
},
)
@@ -1591,24 +1588,6 @@ class TestWorkspaceImages(ImageAPITestCase):
for i in range(links)
],
)
# 2,000 images (one in 25 at 50,000 files) carry their own category
# (#332), so a category filter also reads and matches those.
from py.api.images import _favorite_root_key
root_key = _favorite_root_key(self.output_dir)
with self.api.db.model.get_connection() as conn:
conn.executemany(
"INSERT INTO image_categories VALUES(?,?,?,?)",
[
(
root_key,
listed[i][0].relative_to(self.output_dir).as_posix(),
f"cat-{i % 10}",
"2026-10-27T00:00:00+00:00",
)
for i in range(1, files, 25)[:2_000]
],
)
return listed
_BUDGET_REQUESTS = (
@@ -1636,7 +1615,7 @@ class TestWorkspaceImages(ImageAPITestCase):
import math
from py.api import images as images_module
from database.operations import link_filter_query, link_matches_json_query
from database.operations import link_filter_query
listed = await asyncio.get_running_loop().run_in_executor(
None, self._budget_fixture, 5_000, 2_000
@@ -1699,13 +1678,7 @@ class TestWorkspaceImages(ImageAPITestCase):
# q and tags also stream the tagged image records once
# (#249 B3); none are tagged here, so nothing is stat'ed.
streamed = "q" in params or "tags" in params
# A category also reads the own categories, one primary
# key range per source (#332).
own = "category" in params
self.assertEqual(len(found), 1 + streamed + own, found)
self.assertEqual(
sum("image_categories" in query for query in found), int(own)
)
self.assertEqual(len(found), 2 if streamed else 1, found)
self.assertEqual(
sum("media_image_tags" in query for query in found),
int(streamed),
@@ -1765,42 +1738,13 @@ class TestWorkspaceImages(ImageAPITestCase):
self.assertEqual(self.api._workspace_listed_cache, {})
self.assertEqual(self.api._workspace_order_cache, {})
# The per-row ambiguity probe is an index search, never a scan.
# The read runs it wrapped in one JSON aggregate (#354): same plan.
sql, params = link_filter_query("lantern", ["tag-1"], every_link=True)
for query in (sql, link_matches_json_query(sql)):
with self.subTest(wrapped=query is not sql):
with self.api.db.model.get_connection() as conn:
plan = [
row[3]
for row in conn.execute("EXPLAIN QUERY PLAN " + query, params)
]
self.assertIn(
"SEARCH other USING COVERING INDEX idx_image_file_path "
"(file_path=?)",
plan,
)
self.assertFalse(
[step for step in plan if step.startswith("SCAN other")]
)
# The category's narrowing clause (#332) keeps every join an index
# search, plain and wrapped with the raw categories.
sql, params = link_filter_query(
"", ["tag-1"], every_link=True, category="cat-3"
sql, params = link_filter_query("lantern", "cat-3", ["tag-1"], every_link=True)
with self.api.db.model.get_connection() as conn:
plan = [row[3] for row in conn.execute("EXPLAIN QUERY PLAN " + sql, params)]
self.assertIn(
"SEARCH other USING COVERING INDEX idx_image_file_path (file_path=?)", plan
)
for query in (sql, link_matches_json_query(sql, categories=True)):
with self.subTest(category_wrapped=query is not sql):
with self.api.db.model.get_connection() as conn:
plan = [
row[3]
for row in conn.execute("EXPLAIN QUERY PLAN " + query, params)
]
self.assertIn(
"SEARCH other USING COVERING INDEX idx_image_file_path "
"(file_path=?)",
plan,
)
scans = [step for step in plan if step.startswith("SCAN")]
self.assertTrue(all(step.startswith("SCAN gi") for step in scans), plan)
self.assertFalse([step for step in plan if step.startswith("SCAN other")])
@unittest.skipUnless(
PERF_BUDGET and not TRACED,
-43
View File
@@ -553,35 +553,6 @@ class TestStartRefusals(BatchTestCase):
self.assertEqual(self.api._jobs.list(), [])
self.assertIsNone(model_gate.holder())
async def test_a_missing_backend_package_refuses_the_start(self):
self.tagger.missing["gguf"] = ["llama-cpp-python"]
status, data = await self.start_batch(model_type="gguf")
self.assertEqual((status, data["code"]), (409, "model-unavailable"))
self.assertEqual(data["success"], False)
self.assertTrue(
data["error"].startswith("GGUF JoyCaption needs llama-cpp-python."), data
)
self.assertIn("pip install llama-cpp-python", data["error"])
self.assertEqual(self.api._jobs.list(), [])
self.assertEqual(self.tagger.load_calls, [])
self.assertIsNone(model_gate.holder())
async def test_a_package_lost_after_the_start_fails_the_job_without_a_traceback(
self,
):
from py.autotag import MissingBackendPackages
self.image("a.png", (1, 1, 1))
self.tagger.load_error = MissingBackendPackages("gguf", ["llama-cpp-python"])
with self.assertNoLogs(self.api.logger, level="ERROR"):
job = await self.run_batch(model_type="gguf")
self.assertEqual(job.state, "failed")
self.assertTrue(
job.status_line.startswith("GGUF JoyCaption needs"), job.status_line
)
self.assertTrue(any("pip install llama-cpp-python" in line for line in job.log))
self.assertIsNone(model_gate.holder())
async def test_model_busy_when_the_gate_is_held(self):
owner = model_gate.new_owner("image-annotations")
self.assertTrue(model_gate.try_acquire(owner))
@@ -687,20 +658,6 @@ class TestPreview(BatchTestCase):
self.assertIsNone(llm["estimate_device"])
self.assertEqual([s for s in _strings(skip) if _absolute(s)], [])
async def test_preview_names_missing_backend_packages(self):
self.image("a.png", (1, 1, 1))
data = (await self.preview(model_type="gguf"))[1]
self.assertEqual(data["model"]["missing_packages"], [])
self.assertIsNone(data["model"]["install_command"])
self.tagger.missing["gguf"] = ["llama-cpp-python"]
data = (await self.preview(model_type="gguf"))[1]
self.assertTrue(data["model"]["downloaded"])
self.assertEqual(data["model"]["missing_packages"], ["llama-cpp-python"])
self.assertEqual(
data["model"]["install_command"], "pip install llama-cpp-python"
)
self.assertIn("DGGML_CUDA", data["model"]["install_note"])
async def test_preview_uses_the_graphics_card_rate_when_wd14_runs_on_cuda(self):
self.image("a.png", (1, 1, 1))
self.tagger.wd14_device = lambda: "gpu"
+1 -5
View File
@@ -800,12 +800,8 @@ class TestDefaultLoaders(EngineTestCase):
transformers, model, processor, inputs = self._fake_transformers()
pixel = inputs["pixel_values"]
service = AutoTagService(models_dir=self.models_dir)
# The 8-bit load also needs these two (#424); stand-ins satisfy the check.
stubs = {
name: types.ModuleType(name) for name in ("accelerate", "bitsandbytes")
}
with _patch_modules({"torch": torch, "transformers": transformers, **stubs}):
with _patch_modules({"torch": torch, "transformers": transformers}):
service.load_model("hf")
tags = service.generate_tags(str(self.image_path))
-33
View File
@@ -161,39 +161,6 @@ class TestLegacyBatch(LegacyTestCase):
self.assertEqual(self.tagger.calls[-1]["prompt"], AutotagConfig.TEMPLATE)
self.assertEqual(self.tagger.custom_prompt, "legacy prompt")
async def test_a_missing_backend_package_is_one_error_frame(self):
self.tagger.missing["gguf"] = ["llama-cpp-python"]
_, frames = await self.legacy("model_type=gguf")
self.assertEqual(len(frames), 1)
self.assertEqual(frames[0]["type"], "error")
self.assertTrue(
frames[0]["message"].startswith("GGUF JoyCaption needs llama-cpp-python.")
)
self.assertEqual(self.tagger.load_calls, [])
async def test_legacy_single_refuses_a_missing_backend_package(self):
from py.autotag import MissingBackendPackages
self.image("a.png", (1, 1, 1))
await self.relist()
self.tagger.missing["gguf"] = ["llama-cpp-python"]
status, data = await self.single(
{"path": "a.png", "root_index": 0, "model_type": "gguf"}
)
self.assertEqual((status, data["code"]), (409, "model-unavailable"))
self.assertIn("pip install llama-cpp-python", data["error"])
self.assertEqual(self.tagger.load_calls, [])
# Lost after the check: the same 409, no ERROR record, nothing held.
self.tagger.missing["gguf"] = []
self.tagger.load_error = MissingBackendPackages("gguf", ["llama-cpp-python"])
with self.assertNoLogs(self.api.logger, level="ERROR"):
status, data = await self.single(
{"path": "a.png", "root_index": 0, "model_type": "gguf"}
)
self.assertEqual((status, data["code"]), (409, "model-unavailable"))
self.assertIn("pip install llama-cpp-python", data["error"])
self.assertIsNone(model_gate.holder())
async def test_each_refusal_is_one_error_frame(self):
self.tagger.downloaded["wd14-vit"] = False
_, frames = await self.legacy()
-248
View File
@@ -1,248 +0,0 @@
"""#424: each tagging backend reports the Python packages it cannot import.
``find_spec`` is patched throughout: no ML backend is installed or imported.
"""
import os
import sys
import types
import unittest
from unittest.mock import patch
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
try:
from test_autotag_engine import (
EngineTestCase,
FakeGgufModel,
_patch_modules,
_touch_model_files,
)
except ImportError: # pragma: no cover - run as a package
from tests.test_autotag_engine import (
EngineTestCase,
FakeGgufModel,
_patch_modules,
_touch_model_files,
)
from py import autotag # noqa: E402
from py.autotag import ( # noqa: E402
BACKEND_PACKAGES,
MODELS,
AutoTagService,
MissingBackendPackages,
missing_packages,
package_fields,
)
GGUF_CUDA = 'For a CUDA build: CMAKE_ARGS="-DGGML_CUDA=on" pip install llama-cpp-python'
def _find_spec_without(*absent):
"""A ``find_spec`` that knows every module except *absent*."""
def find_spec(name, *args):
return None if name in absent else object()
return find_spec
def _missing_from(*absent):
return patch.object(
autotag.importlib.util, "find_spec", side_effect=_find_spec_without(*absent)
)
class TestMissingPackages(unittest.TestCase):
def test_every_backend_names_its_modules_and_pip_names(self):
self.assertEqual(BACKEND_PACKAGES["gguf"], (("llama_cpp", "llama-cpp-python"),))
self.assertEqual(
[pip for _module, pip in BACKEND_PACKAGES["hf"]],
["torch", "transformers", "accelerate", "bitsandbytes"],
)
for model in ("wd14-swinv2", "wd14-vit"):
self.assertEqual(BACKEND_PACKAGES[model], (("onnxruntime", "onnxruntime"),))
self.assertEqual(set(BACKEND_PACKAGES), set(MODELS))
def test_reports_pip_names_of_modules_find_spec_cannot_find(self):
with _missing_from("llama_cpp", "accelerate", "bitsandbytes"):
self.assertEqual(missing_packages("gguf"), ["llama-cpp-python"])
self.assertEqual(missing_packages("hf"), ["accelerate", "bitsandbytes"])
self.assertEqual(missing_packages("wd14-vit"), [])
def test_find_spec_errors_count_as_missing(self):
for error in (ValueError("no spec"), ImportError("broken"), OSError("io")):
with self.subTest(error=type(error).__name__):
with patch.object(
autotag.importlib.util, "find_spec", side_effect=error
):
self.assertEqual(missing_packages("wd14-vit"), ["onnxruntime"])
def test_a_module_already_imported_is_present_without_a_lookup(self):
with _patch_modules({"llama_cpp": types.ModuleType("llama_cpp")}):
with patch.object(
autotag.importlib.util, "find_spec", side_effect=AssertionError
):
self.assertEqual(missing_packages("gguf"), [])
def test_the_check_imports_nothing(self):
before = set(sys.modules)
missing_packages("gguf")
missing_packages("hf")
missing_packages("wd14-vit")
imported = set(sys.modules) - before
heavy = {"llama_cpp", "torch", "transformers", "onnxruntime"}
self.assertFalse(imported & heavy, imported)
def test_unknown_model_has_no_packages(self):
self.assertEqual(missing_packages("nope"), [])
def test_package_fields_and_the_one_sentence(self):
self.assertEqual(
package_fields("gguf", []),
{"missing_packages": [], "install_command": None, "install_note": None},
)
fields = package_fields("gguf", ["llama-cpp-python"])
self.assertEqual(fields["missing_packages"], ["llama-cpp-python"])
self.assertEqual(fields["install_command"], "pip install llama-cpp-python")
self.assertEqual(fields["install_note"], GGUF_CUDA)
self.assertEqual(
package_fields("hf", ["torch", "bitsandbytes"])["install_command"],
"pip install torch bitsandbytes",
)
self.assertIsNone(package_fields("hf", ["torch"])["install_note"])
self.assertIn(
"onnxruntime-gpu",
package_fields("wd14-vit", ["onnxruntime"])["install_note"],
)
error = MissingBackendPackages("gguf", ["llama-cpp-python"])
self.assertIsInstance(error, RuntimeError)
self.assertEqual(
str(error),
"GGUF JoyCaption needs llama-cpp-python. Install it into ComfyUI's "
"Python: pip install llama-cpp-python. " + GGUF_CUDA,
)
self.assertEqual(
str(MissingBackendPackages("hf", ["accelerate", "bitsandbytes"])),
"HuggingFace JoyCaption needs accelerate and bitsandbytes. Install them "
"into ComfyUI's Python: pip install accelerate bitsandbytes.",
)
class TestStatusReportsPackages(EngineTestCase):
def test_downloaded_keeps_its_files_only_meaning(self):
_touch_model_files(self.models_dir, "gguf")
_touch_model_files(self.models_dir, "wd14-vit")
service = AutoTagService(models_dir=self.models_dir)
with _missing_from("llama_cpp"):
status = service.get_models_status()
self.assertTrue(status["gguf"]["downloaded"])
self.assertEqual(status["gguf"]["missing_packages"], ["llama-cpp-python"])
self.assertEqual(
status["gguf"]["install_command"], "pip install llama-cpp-python"
)
self.assertEqual(status["gguf"]["install_note"], GGUF_CUDA)
self.assertEqual(status["wd14-vit"]["missing_packages"], [])
self.assertIsNone(status["wd14-vit"]["install_command"])
# A model not on disk still says which packages it will need.
self.assertFalse(status["wd14-swinv2"]["downloaded"])
self.assertIn("missing_packages", status["hf"])
def test_an_injected_loader_brings_its_own_backend(self):
service = self._service(("gguf", FakeGgufModel("x")))
with _missing_from("llama_cpp"):
self.assertEqual(
service.get_models_status()["gguf"]["missing_packages"], []
)
def test_an_injected_probe_is_used_for_status_and_load(self):
_touch_model_files(self.models_dir, "gguf")
service = AutoTagService(
models_dir=self.models_dir,
model_loader=lambda *_: ("gguf", FakeGgufModel("x")),
package_probe=lambda model_type: ["llama-cpp-python"],
)
self.assertEqual(
service.get_models_status()["gguf"]["missing_packages"],
["llama-cpp-python"],
)
with self.assertRaises(MissingBackendPackages):
service.load_model("gguf")
class TestLoadFailsFast(EngineTestCase):
def test_missing_package_refuses_before_the_loader_without_an_error_log(self):
_touch_model_files(self.models_dir, "gguf")
service = AutoTagService(models_dir=self.models_dir)
calls = []
service._model_loader = lambda *args: calls.append(args)
with (
_missing_from("llama_cpp"),
self.assertLogs(service.logger, level="DEBUG") as logs,
):
with self.assertRaises(MissingBackendPackages) as ctx:
service.load_model("gguf")
self.assertEqual(calls, [])
self.assertEqual(ctx.exception.model_type, "gguf")
self.assertEqual(ctx.exception.missing, ["llama-cpp-python"])
self.assertIn("pip install llama-cpp-python", str(ctx.exception))
self.assertFalse(service.is_model_loaded())
levels = {record.levelname for record in logs.records}
self.assertNotIn("ERROR", levels)
self.assertIn("WARNING", levels)
self.assertTrue(all(record.exc_info is None for record in logs.records))
def test_not_downloaded_is_reported_before_packages(self):
service = AutoTagService(models_dir=self.models_dir)
with _missing_from("llama_cpp"):
with self.assertRaises(RuntimeError) as ctx:
service.load_model("gguf")
self.assertNotIsInstance(ctx.exception, MissingBackendPackages)
def test_a_module_not_found_from_the_loader_becomes_the_same_refusal(self):
_touch_model_files(self.models_dir, "gguf")
def loader(model_type, use_gpu):
raise ModuleNotFoundError("No module named 'llama_cpp'", name="llama_cpp")
service = AutoTagService(
models_dir=self.models_dir,
model_loader=loader,
package_probe=lambda _model: [],
)
with self.assertLogs(service.logger, level="DEBUG") as logs:
with self.assertRaises(MissingBackendPackages) as ctx:
service.load_model("gguf")
self.assertEqual(ctx.exception.missing, ["llama-cpp-python"])
self.assertNotIn("ERROR", {record.levelname for record in logs.records})
self.assertFalse(service.is_model_loaded())
def test_an_unknown_module_not_found_names_the_module(self):
def loader(model_type, use_gpu):
raise ModuleNotFoundError(
"No module named 'sentencepiece'", name="sentencepiece"
)
hf_dir = self.models_dir / MODELS["hf"]["subdir"]
hf_dir.mkdir(parents=True, exist_ok=True)
(hf_dir / "config.json").write_text("{}")
service = AutoTagService(
models_dir=self.models_dir,
model_loader=loader,
package_probe=lambda _model: [],
)
with self.assertRaises(MissingBackendPackages) as ctx:
service.load_model("hf")
self.assertEqual(ctx.exception.missing, ["sentencepiece"])
if __name__ == "__main__":
unittest.main()
+3 -43
View File
@@ -210,24 +210,13 @@ class JobCase(unittest.TestCase):
node = self.node(kind)
self.context = (self.context[0], index)
clip = mock.Mock()
# The node receives its switch as the queued graph holds it (#90)
save = g[node_id]["inputs"].get("save_to_library", True)
if kind == "PromptManagerText":
return node.process_text(
text, prompt=g, unique_id=node_id, save_to_library=save
)
return node.process_text(text, prompt=g, unique_id=node_id)
if kind == "PromptManagerPair":
return node.encode_prompt(
clip,
text,
negative_text=negative,
prompt=g,
unique_id=node_id,
save_to_library=save,
clip, text, negative_text=negative, prompt=g, unique_id=node_id
)
return node.encode_prompt(
clip, text, prompt=g, unique_id=node_id, save_to_library=save
)
return node.encode_prompt(clip, text, prompt=g, unique_id=node_id)
def cache(self, node_id, *results):
"""Store executed results (one per list index) as ComfyUI's cache entry."""
@@ -324,35 +313,6 @@ class TestCachedJobsCount(JobCase):
self.job(g, cached=["pm"])
self.assertEqual(self.db.search_prompts(), [])
def test_disabled_node_then_cached_jobs_counts_0(self):
# #90 x #347: a node with Save to library off records no receipt and is
# no run prompt, so its cached jobs replay nothing. The prompt exists
# (an enabled run saved it), so any replayed count would show.
for kind in (
"PromptManager",
"PromptManagerFocused",
"PromptManagerSplit",
"PromptManagerText",
"PromptManagerPair",
):
with self.subTest(kind=kind):
text = f"{LLM} not saved by {kind}"
negative = "blurry" if kind == "PromptManagerPair" else ""
self.first(graph(kind, negative_text=negative), text, negative)
before = (self.count(text), self.pair_count(text, negative))
def off(seed):
g = graph(kind, negative_text=negative, seed=seed)
g["pm"]["inputs"]["save_to_library"] = False
return g
self.first(off(1), text, negative)
self.job(off(2), cached=["pm"])
self.job(off(3), cached=["pm"])
after = (self.count(text), self.pair_count(text, negative))
self.assertEqual(before[0], 1)
self.assertEqual(after, before)
def test_alternating_upstream_values_each_count_their_own_prompt(self):
g = graph()
self.first(g, text="value A")
-456
View File
@@ -1,456 +0,0 @@
"""The prompt-link read of the workspace Images page (#354).
``PromptDatabase.get_image_link_matches`` returns every row of its filter
SELECT in one SQLite step (one JSON value) instead of one step per row, so the
read does not have to win the GIL back once per linked image while ComfyUI
generates. Its result must equal the streamed read it replaces, which stays as
the fallback, on every filter and on odd values.
"""
import itertools
import os
import sqlite3
import sys
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
from database import operations # noqa: E402
from database.operations import PromptDatabase, link_filter_query # noqa: E402
try:
from tests.open_handles import assert_closed # noqa: E402
except ImportError: # discovered with tests/ as the top-level directory
from open_handles import assert_closed # noqa: E402
LOGGER = "prompt_manager.database"
MISSING_FUNCTION_SQL = (
"SELECT pm_missing_function(json_group_array(json_array("
"file_path, rating, matched, positive))) FROM ({select})"
)
QUERIES = ("", "lantern", "LANTERN title", "no-such-text")
CATEGORIES = (None, "cat-a", "cat-b", "cat-none")
TAG_SETS = ([], ["t1"], ["t1", "t2"], ["t-none"])
PROMPTS = (
# id, text, category, rating, tags, library title
(1, "red lantern at dusk", "cat-a", 5, ("t1", "t2"), None),
(2, "blue harbor", "cat-a", None, ("t1",), None),
(3, "green field", "cat-b", 3, (), "Lantern title"),
(4, "pair positive", "cat-b", 4, ("t2",), None),
(5, "pair negative lantern", "cat-a", 1, ("t1",), None),
)
QUOTED = '/out/it\'s "quoted".png'
BACKSLASHED = "C:\\out\\back\\slash.png"
UNICODE = "/out/ünïcødé 画像.png"
NEWLINE = "/out/new\nline.png"
IMAGES = (
# prompt id, file path
(1, "/out/a.png"),
(2, QUOTED),
(2, BACKSLASHED),
(3, UNICODE),
(1, NEWLINE),
(1, "/out/ambiguous.png"),
(2, "/out/ambiguous.png"),
(5, "/out/pair.png"), # linked through pair 1: its positive is prompt 4
(3, "/out/two-pairs.png"), # two pairs: no single pair, so prompt 3
(1, None), # no file path: never listed
)
class LinkMatchesTestCase(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory()
self.root = Path(os.path.realpath(self.temp.name))
self.db = PromptDatabase(str(self.root / "prompts.db"))
for name in ("_one_step_unsupported", "_one_step_warned"):
patcher = patch.object(operations, name, False)
patcher.start()
self.addCleanup(patcher.stop)
self.statements = []
original = self.db.model.get_connection
def traced_connection():
connection = original()
connection.set_trace_callback(self.statements.append)
return connection
patcher = patch.object(
self.db.model, "get_connection", side_effect=traced_connection
)
patcher.start()
self.addCleanup(patcher.stop)
def tearDown(self):
self.db.close_all()
assert_closed(self, self.root)
self.temp.cleanup()
def seed(self):
with self.db.model.get_connection() as conn:
for prompt_id, text, category, rating, tags, title in PROMPTS:
conn.execute(
"INSERT INTO prompts(id,text,category,rating,hash) "
"VALUES(?,?,?,?,?)",
(prompt_id, text, category, rating, f"hash-{prompt_id}"),
)
for tag in tags:
conn.execute("INSERT OR IGNORE INTO tags(name) VALUES(?)", (tag,))
conn.execute(
"INSERT INTO prompt_tags(prompt_id,tag_id) "
"SELECT ?, id FROM tags WHERE name=?",
(prompt_id, tag),
)
if title:
conn.execute(
"INSERT INTO prompt_library_state(prompt_id,title) "
"VALUES(?,?)",
(prompt_id, title),
)
ids = {}
for prompt_id, path in IMAGES:
cursor = conn.execute(
"INSERT INTO generated_images"
"(prompt_id,image_path,filename,file_path) VALUES(?,?,?,?)",
(prompt_id, path or "/elsewhere/x.png", "x.png", path),
)
ids.setdefault(path, cursor.lastrowid)
conn.execute(
"INSERT INTO prompt_pairs(id,positive_prompt_id,negative_prompt_id) "
"VALUES(1,4,5),(2,3,5),(3,3,NULL)"
)
conn.executemany(
"INSERT INTO image_prompt_pairs(image_id,pair_id,provenance) "
"VALUES(?,?,'{}')",
[
(ids["/out/pair.png"], 1),
(ids["/out/two-pairs.png"], 2),
(ids["/out/two-pairs.png"], 3),
],
)
def add_image(self, prompt_id, path):
with self.db.model.get_connection() as conn:
conn.execute(
"INSERT INTO generated_images(prompt_id,image_path,filename,file_path)"
" VALUES(?,?,?,?)",
(prompt_id, "/blob/x.png", "x.png", path),
)
def streamed(self, q, category, tags, every_link=False, wanted=None):
"""Today's read, called directly (the fallback)."""
sql, params = link_filter_query(
q, tags, every_link=every_link, category=category
)
with self.db.model.get_connection() as conn:
return operations._streamed_link_matches(
conn, sql, params, wanted, category is not None
)
def selects(self):
found = [s for s in self.statements if s.lstrip().upper().startswith("SELECT")]
self.statements.clear()
return found
class TestOneStepLinkRead(LinkMatchesTestCase):
def test_plain_read_returns_every_unambiguous_link(self):
self.seed()
matches = self.db.get_image_link_matches("", [], every_link=True)
self.assertEqual(
matches,
{
"/out/a.png": (True, 5, 1, None),
QUOTED: (True, None, 2, None),
BACKSLASHED: (True, None, 2, None),
UNICODE: (True, 3, 3, None),
NEWLINE: (True, 5, 1, None),
"/out/pair.png": (True, 4, 4, None),
"/out/two-pairs.png": (True, 3, 3, None),
},
)
self.assertIsNone(matches[QUOTED][1])
self.assertIsInstance(matches["/out/a.png"][1], int)
self.assertIsInstance(matches["/out/a.png"][2], int)
def test_search_hits_follow_text_title_and_tags(self):
self.seed()
self.assertEqual(
self.db.get_image_link_matches("lantern", []),
{
"/out/a.png": (True, 5, 1, None),
NEWLINE: (True, 5, 1, None),
UNICODE: (True, 3, 3, None),
"/out/two-pairs.png": (True, 3, 3, None),
},
)
self.assertEqual(
self.db.get_image_link_matches("lantern", [], category="cat-a"),
{
"/out/a.png": (True, 5, 1, "cat-a"),
NEWLINE: (True, 5, 1, "cat-a"),
QUOTED: (False, None, 2, "cat-a"),
BACKSLASHED: (False, None, 2, "cat-a"),
},
)
def test_every_filter_equals_the_streamed_read(self):
self.seed()
wanted_sets = (None, {"/out/a.png", UNICODE, "/out/pair.png", "/absent.png"})
for q, category, tags, every_link, wanted in itertools.product(
QUERIES, CATEGORIES, TAG_SETS, (False, True), wanted_sets
):
with self.subTest(
q=q, category=category, tags=tags, every=every_link, wanted=wanted
):
expected = self.streamed(q, category, tags, every_link, wanted)
self.statements.clear()
actual = self.db.get_image_link_matches(
q,
list(tags),
every_link=every_link,
wanted=wanted,
category=category,
)
self.assertEqual(actual, expected)
self.assertEqual(
{path: tuple(map(type, row)) for path, row in actual.items()},
{path: tuple(map(type, row)) for path, row in expected.items()},
)
found = self.selects()
self.assertEqual(len(found), 1, found)
self.assertIn("json_group_array", found[0])
def test_empty_database_returns_nothing_in_one_statement(self):
self.assertEqual(self.db.get_image_link_matches("", [], every_link=True), {})
self.assertEqual(self.db.get_image_link_matches("x", ["t"], category="c"), {})
found = self.selects()
self.assertEqual(len(found), 2, found)
self.assertTrue(all("json_group_array" in query for query in found))
def test_wanted_limits_the_result_to_the_listing(self):
self.seed()
self.assertEqual(
self.db.get_image_link_matches(
"", [], every_link=True, wanted={"/out/pair.png", "/x.png"}
),
{"/out/pair.png": (True, 4, 4, None)},
)
self.assertEqual(
self.db.get_image_link_matches("", [], every_link=True, wanted=set()),
{},
)
def test_category_read_carries_the_inherited_category_in_one_step(self):
# A recipe's own category wins over its positive's (#332 MC4); the
# read only narrows to links holding the value, the caller compares.
self.seed()
with self.db.model.get_connection() as conn:
conn.execute("UPDATE prompt_pairs SET category='cat-a' WHERE id=1")
self.statements.clear()
for category, expected in (
("cat-a", {"/out/pair.png": (True, 4, 4, "cat-a")}),
("cat-b", {"/out/pair.png": (True, 4, 4, "cat-a")}),
):
with self.subTest(category=category):
actual = self.db.get_image_link_matches(
"", [], every_link=True, category=category, wanted={"/out/pair.png"}
)
self.assertEqual(actual, expected)
self.assertEqual(
actual, self.streamed("", category, [], True, {"/out/pair.png"})
)
found = [query for query in self.selects() if "json_group_array" in query]
self.assertEqual(len(found), 2, found)
self.assertTrue(all("pair_category" in query for query in found))
def test_plain_read_leaves_the_categories_out_of_the_json(self):
self.seed()
self.statements.clear()
self.db.get_image_link_matches("", [], every_link=True)
(query,) = self.selects()
aggregate = query[query.index("json_array(") :].split(")", 1)[0]
self.assertEqual(aggregate, "json_array(file_path, rating, matched, positive")
def test_read_releases_its_snapshot(self):
# A statement left open after the read would pin its WAL snapshot, and
# a TRUNCATE checkpoint from another connection would report busy.
self.seed()
self.assertTrue(self.db.get_image_link_matches("", [], every_link=True))
other = sqlite3.connect(str(self.root / "prompts.db"))
try:
other.execute("UPDATE prompts SET rating=2 WHERE id=1")
other.commit()
busy, _log, _done = other.execute(
"PRAGMA wal_checkpoint(TRUNCATE)"
).fetchone()
finally:
other.close()
self.assertEqual(busy, 0)
self.assertEqual(os.path.getsize(self.root / "prompts.db-wal"), 0)
def test_wrapped_query_keeps_the_ambiguity_probe_an_index_search(self):
self.seed()
sql, params = link_filter_query(
"lantern", ["t1"], every_link=True, category="cat-a"
)
with self.db.model.get_connection() as conn:
plan = [
row[3]
for row in conn.execute(
"EXPLAIN QUERY PLAN "
+ operations.link_matches_json_query(sql, categories=True),
params,
)
]
self.assertIn(
"SEARCH other USING COVERING INDEX idx_image_file_path (file_path=?)", plan
)
self.assertFalse([step for step in plan if step.startswith("SCAN other")])
class TestStreamedFallback(LinkMatchesTestCase):
def test_missing_json_functions_stream_and_stop_trying(self):
self.seed()
expected = self.streamed("", None, [], True)
self.statements.clear()
wrapper = operations.link_matches_json_query
with (
patch.object(operations, "_LINK_MATCHES_JSON_SQL", MISSING_FUNCTION_SQL),
patch.object(
operations, "link_matches_json_query", side_effect=wrapper
) as attempts,
self.assertLogs(LOGGER, "WARNING") as logged,
):
first = self.db.get_image_link_matches("", [], every_link=True)
found = self.selects()
self.assertEqual(attempts.call_count, 1)
second = self.db.get_image_link_matches("", [], every_link=True)
again = self.selects()
self.assertEqual(first, expected)
self.assertEqual(second, expected)
self.assertEqual(len(logged.records), 1, logged.output)
self.assertIn("no such function: pm_missing_function", logged.output[0])
self.assertTrue(operations._one_step_unsupported)
# SQLite refuses the wrapped statement before it runs (not traced).
self.assertEqual(len(found), 1, found)
self.assertNotIn("json_group_array", found[0])
# The next read skips the attempt for the rest of the process.
self.assertEqual(attempts.call_count, 1)
self.assertEqual(len(again), 1, again)
self.assertNotIn("json_group_array", again[0])
def test_blob_path_streams_that_read_and_keeps_trying(self):
self.seed()
self.add_image(2, b"/out/blob.png")
expected = self.streamed("", None, [], True)
self.assertEqual(expected[b"/out/blob.png"], (True, None, 2, None))
self.statements.clear()
with self.assertLogs(LOGGER, "WARNING") as logged:
first = self.db.get_image_link_matches("", [], every_link=True)
second = self.db.get_image_link_matches("", [], every_link=True)
self.assertEqual(first, expected)
self.assertEqual(second, expected)
self.assertEqual(len(logged.records), 1, logged.output)
self.assertFalse(operations._one_step_unsupported)
found = self.selects()
self.assertEqual(sum("json_group_array" in query for query in found), 2)
self.assertEqual(len(found), 4, found)
def test_blob_that_reads_as_json_still_streams(self):
# SQLite 3.45+ reads a BLOB that is valid JSONB as JSON (b"\x00" is
# null) instead of refusing it; the streamed read keeps the bytes.
self.seed()
self.add_image(2, b"\x00")
expected = self.streamed("", None, [], True)
self.assertIn(b"\x00", expected)
with self.assertLogs(LOGGER, "WARNING"):
self.assertEqual(
self.db.get_image_link_matches("", [], every_link=True), expected
)
self.assertFalse(operations._one_step_unsupported)
def test_blob_rating_streams(self):
self.seed()
with self.db.model.get_connection() as conn:
# Only a database written around the CHECK can hold one.
conn.execute("PRAGMA ignore_check_constraints=ON")
conn.execute("UPDATE prompts SET rating=? WHERE id=3", (b"\x00",))
conn.execute("PRAGMA ignore_check_constraints=OFF")
expected = self.streamed("", None, [], True)
self.assertEqual(expected[UNICODE], (True, b"\x00", 3, None))
with self.assertLogs(LOGGER, "WARNING"):
self.assertEqual(
self.db.get_image_link_matches("", [], every_link=True), expected
)
def test_blob_category_streams_a_category_read(self):
self.seed()
with self.db.model.get_connection() as conn:
# Admitted by its recipe's text category; the positive's is a BLOB.
conn.execute("UPDATE prompts SET category=? WHERE id=4", (b"cat-b",))
conn.execute("UPDATE prompt_pairs SET category='cat-b' WHERE id=1")
expected = self.streamed("", "cat-b", [], True)
self.assertEqual(expected["/out/pair.png"], (True, 4, 4, "cat-b"))
with self.assertLogs(LOGGER, "WARNING") as logged:
actual = self.db.get_image_link_matches(
"", [], every_link=True, category="cat-b"
)
self.assertEqual(actual, expected)
self.assertIn("BLOB", logged.output[0])
self.assertFalse(operations._one_step_unsupported)
def test_json_text_over_the_length_limit_streams(self):
self.seed()
expected = self.streamed("", None, [], True)
conn = self.db.model.get_connection()
limit = conn.setlimit(sqlite3.SQLITE_LIMIT_LENGTH, 200)
try:
with self.assertLogs(LOGGER, "WARNING") as logged:
actual = self.db.get_image_link_matches("", [], every_link=True)
finally:
conn.setlimit(sqlite3.SQLITE_LIMIT_LENGTH, limit)
self.assertEqual(actual, expected)
self.assertIn("too big", logged.output[0])
self.assertFalse(operations._one_step_unsupported)
def test_an_interrupted_read_raises_without_a_fallback(self):
# Move library interrupts a retired connection: the read fails as
# before, and the one warning is kept for a real fallback.
self.seed()
conn = self.db.model.get_connection()
conn.set_progress_handler(lambda: 1, 1)
self.statements.clear()
try:
with (
self.assertNoLogs(LOGGER, "WARNING"),
self.assertRaisesRegex(sqlite3.OperationalError, "^interrupted$"),
):
self.db.get_image_link_matches("", [], every_link=True)
finally:
conn.set_progress_handler(None, 1)
self.assertFalse(operations._one_step_warned)
self.assertFalse(operations._one_step_unsupported)
self.assertFalse([s for s in self.statements if "json_group_array" not in s])
def test_the_maintenance_link_rows_reader_is_not_shadowed(self):
from database.maintenance import MaintenanceOperations
self.assertIs(PromptDatabase._link_rows, MaintenanceOperations._link_rows)
def test_a_failed_streamed_read_still_raises(self):
with (
patch.object(operations, "_LINK_MATCHES_JSON_SQL", MISSING_FUNCTION_SQL),
patch.object(operations, "_LINK_FILTER_SQL", "SELECT broken FROM nowhere"),
self.assertLogs(LOGGER, "WARNING"),
):
with self.assertRaises(sqlite3.OperationalError):
self.db.get_image_link_matches("", [], every_link=True)
if __name__ == "__main__":
unittest.main()
-13
View File
@@ -299,19 +299,6 @@ class TestAutoTag(ImageTagJobTestCase):
self.tagger.download_model.assert_not_called()
self.assertIsNone(model_gate.holder())
async def test_a_missing_backend_package_fails_the_job_before_the_first_file(self):
self.png("a.png")
self.tagger.missing["wd14-vit"] = ["onnxruntime"]
with self.assertNoLogs(self.api.logger, level="ERROR"):
job = await self.run_job(self.auto_body(["a.png"]))
self.assertEqual(job.state, "failed")
self.assertEqual(job.processed, 0)
self.assertTrue(job.status_line.startswith("WD14 ViT needs onnxruntime."))
self.assertIn("pip install onnxruntime", job.status_line)
self.assertEqual(self.tagger.load_calls, [])
async def test_model_unavailable_fails_the_job_before_the_first_file(self):
self.png("a.png")
self.tagger.downloaded["wd14-vit"] = False
-277
View File
@@ -1,277 +0,0 @@
"""The inspector and History show an image's own category (#332 §5, DC4, DC5)."""
import os
import sqlite3
import unittest
from unittest.mock import patch
from tests import test_api_history_outputs as history
from tests.test_api_images import make_png
from py.api.images import _favorite_root_key, image_category_key
from utils.inspection_response import (
OWN_CATEGORY_UNREAD,
_library_projection,
apply_own_category,
)
class InspectCase(history.ImageAPITestCase):
async def tearDownAsync(self):
self.api.db.close_all()
await super().tearDownAsync()
def own(self, path, value, sources=None):
sources = sources or self.api._get_all_output_dirs()
self.api.db.set_image_categories([image_category_key(sources, path)], value)
async def library(self, path, root="0"):
response = await self.client.get(
f"/prompt_manager/images/inspect/{path}", params={"root": root}
)
body = await response.json()
self.assertEqual(response.status, 200, body)
return body["library"]
class TestInspectorOwnCategory(InspectCase):
async def test_inspect_reports_image_scope(self):
path = make_png(self.output_dir / "image.png")
self.own(path, "Portraits")
library = await self.library("image.png")
self.assertEqual(library["status"], "none")
self.assertEqual(
(library["category"], library["category_scope"]), ("Portraits", "image")
)
async def test_own_overrides_linked_recipe_and_ambiguous(self):
db = self.api.db
linked = make_png(self.output_dir / "linked.png")
self._link_image(linked, "castle at dusk")
prompt = db.get_prompt_id_for_image(str(linked))
with db.model.get_connection() as conn:
conn.execute(
"UPDATE prompts SET category='Landscapes' WHERE id=?", (prompt,)
)
library = await self.library("linked.png")
self.assertEqual(
(library["category"], library["category_scope"]),
("Landscapes", "positive"),
)
self.own(linked, "Portraits")
library = await self.library("linked.png")
self.assertEqual(library["status"], "linked")
self.assertEqual(
(library["category"], library["category_scope"]), ("Portraits", "image")
)
recipe = make_png(self.output_dir / "recipe.png")
self._save_prompt("Pair positive", category="Night")
pair = db.save_prompt_pair("Pair positive", "Noise", category="Recipes")
positive = db.get_prompt_pair(pair)["positive_prompt_id"]
image = db.link_image_to_prompt(positive, str(recipe))
db.link_image_to_pair(pair, image, {"source": "fixture"})
library = await self.library("recipe.png")
self.assertEqual(
(library["category"], library["category_scope"]), ("Recipes", "pair")
)
self.own(recipe, "Mine")
library = await self.library("recipe.png")
self.assertEqual(
(library["category"], library["category_scope"]), ("Mine", "image")
)
ambiguous = make_png(self.output_dir / "ambiguous.png")
self._link_image(ambiguous, "owner one")
self._link_image(ambiguous, "owner two")
self.own(ambiguous, "Scans")
library = await self.library("ambiguous.png")
self.assertEqual(library["status"], "ambiguous")
self.assertEqual(
(library["category"], library["category_scope"]), ("Scans", "image")
)
async def test_a_failed_read_hides_the_category(self):
linked = make_png(self.output_dir / "linked.png")
self._link_image(linked, "castle at dusk")
def broken(*_args, **_kwargs):
raise sqlite3.OperationalError("secret")
with patch.object(self.api.db, "image_category", side_effect=broken):
with self.assertLogs("prompt_manager", level="WARNING"):
library = await self.library("linked.png")
self.assertEqual(library["status"], "linked")
self.assertIsNone(library["category"])
self.assertIsNone(library["category_scope"])
async def test_a_lora_root_reads_no_own_category(self):
lora = self.comfy_dir / "loras"
make_png(lora / "preview.png")
roots = [self.output_dir, lora]
reads = []
real = self.api.db.image_category
def recorded(*args):
reads.append(args)
return real(*args)
with (
patch.object(self.api, "_media_roots", lambda outputs: roots),
patch.object(self.api, "_linked_only_error", self._no_error),
patch.object(self.api.db, "image_category", side_effect=recorded),
):
response = await self.client.get(
"/prompt_manager/images/inspect/preview.png", params={"root": "1"}
)
body = await response.json()
self.assertEqual(response.status, 200, body)
self.assertEqual(reads, [])
self.assertIsNone(body["library"]["category"])
@staticmethod
async def _no_error(*_args):
return None
class TestProjection(unittest.TestCase):
def test_scope_image_is_kept_for_category_only(self):
base = {"status": "none", "scope": "saved-library", "tags": []}
projected = _library_projection(
{**base, "category": "Scans", "category_scope": "image"}
)
self.assertEqual(projected["category_scope"], "image")
projected = _library_projection({**base, "tags_scope": "image"})
self.assertNotIn("tags_scope", projected)
def test_apply_own_category_three_cases(self):
library = {"status": "linked", "category": "L", "category_scope": "positive"}
self.assertEqual(
apply_own_category(library, "Mine"),
{"status": "linked", "category": "Mine", "category_scope": "image"},
)
self.assertEqual(apply_own_category(library, None), library)
self.assertEqual(
apply_own_category(library, OWN_CATEGORY_UNREAD),
{"status": "linked", "category": None, "category_scope": None},
)
self.assertEqual(library["category"], "L") # never mutated
class TestHistoryOwnCategory(history.ImageAPITestCase):
"""Only the inspection request reads the own category, keyed as Images."""
# The History fixture's helpers, without inheriting its tests.
_base = history.HistoryOutputAPITests
ref = _base.ref
capture = _base.capture
outputs = _base.outputs
current = _base.current
media = _base.media
async def get_application(self):
app = await super().get_application()
self.temp_dir = self.comfy_dir / "temp"
self.temp_dir.mkdir()
self.host_output = str(self.output_dir)
self.host_temp = str(self.temp_dir)
self.patches = history.ExitStack()
self.addCleanup(self.patches.close)
for name, value in (
("folder_paths.get_output_directory", lambda: self.host_output),
("folder_paths.get_temp_directory", lambda: self.host_temp),
):
self.patches.enter_context(patch(name, value, create=True))
self.patches.enter_context(
patch.object(
history.GalleryConfig,
"allowed_gallery_parents",
lambda: [str(self.comfy_dir)],
)
)
return app
async def tearDownAsync(self):
self.api.close_history_reads()
self.patches.close()
await super().tearDownAsync()
async def inspect(self, start):
descriptor = await self.media(start)
response = await self.current(descriptor["inspection_url"])
body = await response.json()
self.assertEqual(response.status, 200, body)
return body["library"]
async def test_history_inspection_shows_own_category(self):
path = make_png(self.output_dir / "image.png")
self.api.db.set_image_categories(
[image_category_key([self.output_dir], path)], "Portraits"
)
start = self.capture([self.ref()])
library = await self.inspect(start)
self.assertEqual(
(library["category"], library["category_scope"]), ("Portraits", "image")
)
async def test_nested_sources_give_the_images_key(self):
inner = self.output_dir / "nested"
path = make_png(inner / "image.png")
self.api._get_all_output_dirs = lambda: [self.output_dir, inner]
self.api.db.set_image_categories(
[(_favorite_root_key(inner), "image.png")], "S"
)
start = self.capture([self.ref(subfolder="nested")])
library = await self.inspect(start)
self.assertEqual(library["category"], "S")
self.assertEqual(
image_category_key([self.output_dir, inner], path),
(_favorite_root_key(inner), "image.png"),
)
async def test_a_symlinked_output_folder_still_shows_it(self):
real = self.comfy_dir / "real-output"
real.mkdir()
link = self.comfy_dir / "linked-output"
self._make_symlink(real, link, target_is_directory=True)
make_png(real / "image.png")
self.host_output = str(link)
self.api._get_all_output_dirs = lambda: [real]
self.api.db.set_image_categories([(_favorite_root_key(real), "image.png")], "S")
start = self.capture([self.ref()])
library = await self.inspect(start)
self.assertEqual(library["category"], "S")
async def test_outputs_and_media_read_no_own_category(self):
make_png(self.output_dir / "image.png")
start = self.capture([self.ref()])
reads = []
real = self.api.db.image_category
def recorded(*args):
reads.append(args)
return real(*args)
with patch.object(self.api.db, "image_category", side_effect=recorded):
descriptor = await self.media(start)
for field in ("url", "thumbnail_url"):
response = await self.current(descriptor[field])
await response.read()
self.assertEqual(response.status, 200)
self.assertEqual(reads, [])
await self.current(descriptor["inspection_url"])
self.assertEqual(len(reads), 1)
async def test_a_temp_output_has_none(self):
reference = {"folder_type": "temp", "subfolder": "", "filename": "a.png"}
self.assertIsNone(self.api._history_own_category(reference, None))
async def test_a_failed_read_hides_it(self):
make_png(self.output_dir / "image.png")
start = self.capture([self.ref()])
def broken(*_args):
raise sqlite3.OperationalError("secret")
with patch.object(self.api.db, "image_category", side_effect=broken):
with self.assertLogs("prompt_manager", level="WARNING"):
library = await self.inspect(start)
self.assertIsNone(library["category"])
self.assertTrue(os.path.exists(self.output_dir / "image.png"))
-29
View File
@@ -1426,32 +1426,3 @@ class TestPerRecordBudgets(unittest.TestCase):
self.assertEqual(metadata["record_formats"], {"workflow": "limited"})
self.assertEqual(metadata["workflow_graph"]["status"], "limited")
self.assertEqual(metadata["prompt_graph"]["status"], "absent")
class TestInheritedCategory(unittest.TestCase):
"""#332 MC4: one inherited-category rule for the chip, filter and History."""
def test_recipe_set_blank_and_absent(self):
from utils.inspection_response import inherited_category, library_metadata
cases = [
(("Night", "Recipes", True), ("Recipes", "pair")),
(("Night", " ", True), ("Night", "positive")),
(("Night", " \t", True), ("Night", "positive")),
(("Night", b" ", True), ("Night", "positive")),
(("Night", None, True), ("Night", "positive")),
(("Night", "Recipes", False), ("Night", "positive")),
((" ", None, False), (" ", "positive")),
((None, None, False), (None, "positive")),
]
for args, expected in cases:
with self.subTest(args=args):
self.assertEqual(inherited_category(*args), expected)
positive, recipe, has_recipe = args
metadata = library_metadata(
positive, ["t"], (recipe, []) if has_recipe else None
)
self.assertEqual(
(metadata["category"], metadata["category_scope"]), expected
)
self.assertEqual(metadata["tags_scope"], "positive")
-508
View File
@@ -1,508 +0,0 @@
"""#90 item 3: a node's ``save_to_library`` switch.
Off: PromptManager ignores the run completely (no prompt saved, no run count,
no image link), while the node still returns its normal outputs. On (the
default, and every workflow or image made before the switch existed): as before.
"""
import logging
import os
import sys
import unittest
from pathlib import Path
from unittest import mock
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
try:
from tests.test_prompt_attribution import MonitorCase, NodeCase
from tests.test_execution_contexts import load_module
except ImportError: # discovered with tests/ as the top-level directory
from test_prompt_attribution import MonitorCase, NodeCase
from test_execution_contexts import load_module
from prompt_manager import PromptManager, PromptManagerFocused, PromptManagerSplit
from prompt_manager_pair import PromptManagerPair
from prompt_manager_text import PromptManagerText
from prompt_search_list import PromptSearchList
from utils.hashing import generate_prompt_hash
from utils.prompt_graph import (
intrinsic_pair_nodes,
positive_opt_outs,
run_prompt_components,
run_prompt_nodes,
save_setting,
)
from utils.run_keys import prompt_node_key
from utils.usage_tracking import (
PendingFirstUse,
QueuedRuns,
handle_queued_prompt,
on_dequeue,
queued_pair_runs,
queued_prompt_runs,
)
POSITIVE = "a lighthouse at dusk"
NEGATIVE = "blurry"
NODE_CLASSES = (
PromptManager,
PromptManagerText,
PromptManagerFocused,
PromptManagerSplit,
PromptManagerPair,
)
def run_graph(save=None, kind="PromptManager", negative_save=None):
"""A PromptManager positive feeding a KSampler; ``save`` None leaves it out."""
inputs = {"text": POSITIVE, "clip": ["4", 1]}
if save is not None:
inputs["save_to_library"] = save
negative_inputs = {"text": NEGATIVE, "clip": ["4", 1]}
if negative_save is not None:
negative_inputs["save_to_library"] = negative_save
return {
"1": {"class_type": kind, "inputs": inputs},
"2": {"class_type": "PromptManager", "inputs": negative_inputs},
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
"3": {
"class_type": "KSampler",
"inputs": {
"model": ["4", 0],
"positive": ["1", 0],
"negative": ["2", 0],
"latent_image": ["6", 0],
},
},
"6": {"class_type": "EmptyLatentImage", "inputs": {"batch_size": 1}},
}
def pair_graph(save=None):
inputs = {"text": POSITIVE, "negative_text": NEGATIVE, "clip": ["4", 1]}
if save is not None:
inputs["save_to_library"] = save
return {
"1": {"class_type": "PromptManagerPair", "inputs": inputs},
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
"3": {
"class_type": "KSampler",
"inputs": {
"model": ["4", 0],
"positive": ["1", 0],
"negative": ["1", 1],
"latent_image": ["6", 0],
},
},
"6": {"class_type": "EmptyLatentImage", "inputs": {"batch_size": 1}},
}
class TestInputDeclaration(unittest.TestCase):
def test_every_prompt_manager_node_declares_a_boolean_defaulting_on(self):
for cls in NODE_CLASSES:
optional = cls.INPUT_TYPES()["optional"]
kind, options = optional["save_to_library"]
self.assertEqual(kind, "BOOLEAN", cls.__name__)
self.assertIs(options["default"], True, cls.__name__)
# Declared last: no saved widget value of an older workflow shifts
self.assertEqual(list(optional)[-1], "save_to_library", cls.__name__)
def test_search_list_has_no_switch(self):
inputs = PromptSearchList.INPUT_TYPES()
self.assertNotIn("save_to_library", inputs.get("optional", {}))
self.assertNotIn("save_to_library", inputs.get("required", {}))
class TestSaveSetting(unittest.TestCase):
def test_missing_input_is_on(self):
self.assertIs(save_setting(run_graph(), "1"), True)
def test_literals_follow_comfyui_boolean_coercion(self):
for value, expected in (
(True, True),
(False, False),
("", False),
(0, False),
({"__value__": False}, False),
({"__value__": True}, True),
):
with self.subTest(value=value):
self.assertIs(save_setting(run_graph(save=value), "1"), expected)
def test_link_to_primitive_boolean_is_read(self):
for value in (True, False):
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "PrimitiveBoolean", "inputs": {"value": value}}
self.assertIs(save_setting(graph, "1"), value)
def test_link_to_an_unknown_node_is_not_knowable(self):
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
self.assertIsNone(save_setting(graph, "1"))
self.assertIsNone(save_setting(run_graph(save=["missing", 0]), "1"))
class TestGraphRoles(unittest.TestCase):
def test_opted_out_positive_is_not_a_run_prompt(self):
graph = run_graph(save=False)
self.assertEqual(run_prompt_nodes(graph), [])
self.assertEqual(run_prompt_components(graph), [])
self.assertEqual(positive_opt_outs(graph), ["1"])
def test_enabled_positive_is_unchanged(self):
for save in (None, True):
graph = run_graph(save=save)
self.assertEqual(run_prompt_nodes(graph), ["1"])
self.assertEqual(positive_opt_outs(graph), [])
def test_opted_out_negative_is_no_positive_opt_out(self):
graph = run_graph(negative_save=False)
self.assertEqual(run_prompt_nodes(graph), ["1"])
self.assertEqual(positive_opt_outs(graph), [])
def test_walk_stops_at_an_opted_out_node(self):
graph = run_graph(save=False)
graph["1"]["inputs"]["text"] = ["7", 0]
graph["7"] = {"class_type": "PromptManagerText", "inputs": {"text": "x"}}
self.assertEqual(run_prompt_nodes(graph), [])
def test_sampler_less_fallback_skips_opted_out_nodes(self):
graph = {
"1": {
"class_type": "PromptManagerText",
"inputs": {"text": "a", "save_to_library": False},
},
"2": {"class_type": "PromptManagerText", "inputs": {"text": "b"}},
}
self.assertEqual(run_prompt_nodes(graph), ["2"])
self.assertEqual(run_prompt_components(graph), [("2", "text")])
self.assertEqual(positive_opt_outs(graph), ["1"])
def test_opted_out_pair_is_not_an_intrinsic_pair(self):
self.assertEqual(intrinsic_pair_nodes(pair_graph()), ["1"])
self.assertEqual(intrinsic_pair_nodes(pair_graph(save=False)), [])
class TestQueueHook(unittest.TestCase):
def test_opted_out_node_is_never_stashed(self):
self.assertEqual(queued_prompt_runs(run_graph(save=False)), [])
self.assertEqual(queued_pair_runs(pair_graph(save=False)), [])
def test_unknown_setting_is_left_to_the_node(self):
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
self.assertEqual(queued_prompt_runs(graph), [])
pair = pair_graph(save=["9", 0])
pair["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
self.assertEqual(queued_pair_runs(pair), [])
def test_primitive_boolean_on_still_counts(self):
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "PrimitiveBoolean", "inputs": {"value": True}}
self.assertEqual(
queued_prompt_runs(graph), [(generate_prompt_hash(POSITIVE), POSITIVE)]
)
def test_dequeue_counts_nothing_for_a_cached_opted_out_node(self):
"""A cached node never executes: only the queued graph can say it is off."""
db = mock.Mock()
db.get_prompt_by_hash.return_value = {"id": 5}
queued, pending = QueuedRuns(), PendingFirstUse()
request = {"prompt": run_graph(save=False), "prompt_id": "job-1"}
handle_queued_prompt(request, queued, pair_queued=QueuedRuns())
on_dequeue("job-1", queued, lambda: db, pending, pair_queued=QueuedRuns())
db.record_prompt_use.assert_not_called()
self.assertEqual(pending.consume(generate_prompt_hash(POSITIVE)), 0)
class TestNodesSaveNothing(NodeCase):
def run_node(self, cls, graph, save):
node = self.node(cls)
node.record_pair = mock.Mock()
kwargs = {"prompt": graph, "unique_id": "1", "save_to_library": save}
if cls is PromptManagerText:
return node, node.process_text(POSITIVE, **kwargs)
if cls is PromptManagerPair:
return node, node.encode_prompt(
self.clip(), POSITIVE, negative_text=NEGATIVE, **kwargs
)
return node, node.encode_prompt(self.clip(), POSITIVE, **kwargs)
def test_off_saves_counts_and_tracks_nothing_but_returns_outputs(self):
for cls in NODE_CLASSES:
with self.subTest(node=cls.__name__):
graph = (
pair_graph(save=False)
if cls is PromptManagerPair
else run_graph(save=False, kind=cls.__name__)
)
node, result = self.run_node(cls, graph, False)
self.assertEqual(self.count_rows("prompts"), 0)
self.assertEqual(self.count_rows("prompt_pairs"), 0)
self.assertEqual(self.tracker.keyed_snapshot(), {})
self.assertEqual(self.tracker._prompt_queue, [])
node.comfyui_integration.register_prompt.assert_not_called()
if cls is PromptManagerText:
self.assertEqual(result, (POSITIVE,))
elif cls is PromptManagerPair:
self.assertEqual(result[0], ("cond", POSITIVE))
self.assertEqual(result[1], ("cond", NEGATIVE))
self.assertEqual(result[2:], (POSITIVE, NEGATIVE))
else:
self.assertEqual(result, (("cond", POSITIVE), POSITIVE))
def test_off_without_graph_context_still_saves_nothing(self):
node = self.node(PromptManager)
node.encode_prompt(self.clip(), POSITIVE, save_to_library=False)
self.assertEqual(self.count_rows("prompts"), 0)
self.assertEqual(self.tracker._prompt_queue, [])
def test_off_leaves_an_existing_prompt_uncounted(self):
prompt_id = self.save(POSITIVE)
self.run_node(PromptManager, run_graph(save=False), False)
row = self.db.get_prompt_by_id(prompt_id)
self.assertFalse(row.get("run_count") or 0)
def test_on_is_unchanged(self):
self.run_node(PromptManager, run_graph(save=True), True)
self.assertIsNotNone(self.db.get_prompt_by_hash(generate_prompt_hash(POSITIVE)))
self.assertIn(
prompt_node_key(run_graph(save=True), "1"), self.tracker.keyed_snapshot()
)
def test_unknown_setting_on_at_run_time_counts_at_execution(self):
"""The hook could not read a wired switch, so the node counts its run."""
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
prompt_id = self.save(POSITIVE)
self.run_node(PromptManager, graph, True)
self.assertEqual(self.db.get_prompt_by_id(prompt_id)["run_count"], 1)
class TestWiredSwitchOffMarksItsRun(NodeCase):
"""Review finding: a wired switch off at run time keeps the node a run prompt."""
def wired_graph(self):
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
return graph
def test_an_earlier_entry_of_the_key_never_stands_in(self):
graph = self.wired_graph()
key = prompt_node_key(graph, "1")
node = self.node(PromptManager)
node.encode_prompt(
self.clip(), POSITIVE, prompt=graph, unique_id="1", save_to_library=True
)
node.encode_prompt(
self.clip(), POSITIVE, prompt=graph, unique_id="1", save_to_library=False
)
latest, _run, _count = self.tracker.keyed_snapshot()[key]
self.assertTrue(latest.get("placeholder"))
def test_literal_off_marks_nothing(self):
node = self.node(PromptManager)
node.encode_prompt(
self.clip(),
POSITIVE,
prompt=run_graph(save=False),
unique_id="1",
save_to_library=False,
)
self.assertEqual(self.tracker.keyed_snapshot(), {})
class TestMonitorLeavesOptedOutFilesUnlinked(MonitorCase):
def test_existing_prompt_is_not_linked_from_metadata(self):
self.save(POSITIVE)
self.assertIsNone(self.process({"prompt": run_graph(save=False)}))
def test_find_prompt_reports_opted_out(self):
self.save(POSITIVE)
state, prompt = self.handler._find_prompt_from_metadata(
{"prompt": run_graph(save=False)}
)
self.assertEqual((state, prompt), ("opted_out", None))
def test_mixed_positive_settings_leave_the_file_unlinked(self):
self.save(POSITIVE)
graph = run_graph(save=True)
graph["8"] = {
"class_type": "PromptManager",
"inputs": {"text": "other", "save_to_library": False},
}
graph["5"] = {
"class_type": "KSampler",
"inputs": {
"model": ["4", 0],
"positive": ["8", 0],
"negative": ["2", 0],
"latent_image": ["6", 0],
},
}
self.assertIsNone(self.process({"prompt": graph}))
def test_workflow_only_metadata_with_switch_off_is_opted_out(self):
self.save(POSITIVE)
metadata = {
"text_encoder_nodes": [
{
"type": "PromptManager",
"inputs": [],
"widgets_values": [POSITIVE, "", "", "", "", "", False],
}
]
}
state, _prompt = self.handler._find_prompt_from_metadata(metadata)
self.assertEqual(state, "opted_out")
def test_workflow_only_ignores_muted_nodes_and_reads_subgraphs(self):
from utils.prompt_graph import metadata_opted_out
node = {
"type": "PromptManager",
"widgets_values": [POSITIVE, "", "", "", "", "", False],
}
self.assertFalse(
metadata_opted_out({"workflow": {"nodes": [{**node, "mode": 4}]}})
)
self.assertTrue(
metadata_opted_out(
{
"workflow": {
"nodes": [],
"definitions": {"subgraphs": [{"nodes": [node]}]},
}
}
)
)
def test_unknown_setting_is_left_to_the_run(self):
self.save(POSITIVE)
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
state, _prompt = self.handler._find_prompt_from_metadata({"prompt": graph})
self.assertEqual(state, "unknown")
def test_enabled_positive_still_links(self):
prompt_id = self.save(POSITIVE)
self.assertEqual(self.process({"prompt": run_graph(save=True)}), prompt_id)
class TestLinkKeepsOptedOutNegativeOut(MonitorCase):
def setUp(self):
super().setUp()
# The real link, not the recording stub of MonitorCase
del self.handler.link_image_to_prompt
def test_opted_out_negative_is_never_stored_as_a_pair(self):
prompt_id = self.save(POSITIVE)
metadata = {"prompt": run_graph(save=True, negative_save=False)}
self.handler.link_image_to_prompt(self.image, {"id": prompt_id}, metadata)
self.assertIsNone(self.db.get_prompt_by_hash(generate_prompt_hash(NEGATIVE)))
self.assertEqual(self.count_pairs(), 0)
linked = (
self.db.model.get_connection()
.execute("SELECT prompt_id FROM generated_images")
.fetchall()
)
self.assertEqual([row[0] for row in linked], [prompt_id])
def test_opted_out_positive_links_nothing(self):
prompt_id = self.save(POSITIVE)
metadata = {"prompt": run_graph(save=False)}
self.handler.link_image_to_prompt(self.image, {"id": prompt_id}, metadata)
count = (
self.db.model.get_connection()
.execute("SELECT COUNT(*) FROM generated_images")
.fetchone()[0]
)
self.assertEqual(count, 0)
def count_pairs(self):
conn = self.db.model.get_connection()
return conn.execute("SELECT COUNT(*) FROM prompt_pairs").fetchone()[0]
class TestScanSkipsOptedOutFiles(NodeCase):
def setUp(self):
super().setUp()
from py.api import admin
self.mixin = admin.AdminRoutesMixin.__new__(admin.AdminRoutesMixin)
self.mixin.db = self.db
self.mixin.logger = logging.getLogger("test.save_to_library.scan")
self.file = Path(self.tmp.name) / "out.png"
def test_opted_out_file_stores_nothing(self):
status = self.mixin._ingest_scanned_file(
self.file, {"prompt": run_graph(save=False)}
)
self.assertEqual(status, "not_saved")
self.assertEqual(self.count_rows("prompts"), 0)
self.assertEqual(self.count_rows("generated_images"), 0)
def test_opted_out_file_is_never_queued_for_review(self):
ambiguous = []
graph = run_graph(save=False)
graph["5"] = dict(graph["3"]) # a second sampler: ambiguous otherwise
graph["5"]["inputs"] = {**graph["3"]["inputs"], "positive": ["2", 0]}
status = self.mixin._ingest_scanned_file(
self.file, {"prompt": graph}, ambiguous=ambiguous
)
self.assertEqual(status, "not_saved")
self.assertEqual(ambiguous, [])
def test_opted_out_negative_stores_the_positive_only(self):
status = self.mixin._ingest_scanned_file(
self.file, {"prompt": run_graph(save=True, negative_save=False)}
)
self.assertEqual(status, "added")
self.assertIsNotNone(self.db.get_prompt_by_hash(generate_prompt_hash(POSITIVE)))
self.assertIsNone(self.db.get_prompt_by_hash(generate_prompt_hash(NEGATIVE)))
self.assertEqual(self.count_rows("prompt_pairs"), 0)
def test_wired_switch_is_left_to_the_live_run(self):
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
status = self.mixin._ingest_scanned_file(self.file, {"prompt": graph})
self.assertEqual(status, "not_saved_wired")
self.assertEqual(self.count_rows("prompts"), 0)
def test_status_has_a_scan_detail(self):
from py.api import admin
group, row, _line = admin._scan_outcome(self.file, [], "not_saved")
self.assertEqual(group, "no_metadata")
self.assertIn("does not save", row["detail"])
class TestHistoryRecordsNoText(unittest.TestCase):
def setUp(self):
self.m = load_module("execution_context_graph")
def test_opted_out_role_has_no_sources_and_no_text(self):
recipe = self.m.project_contexts(run_graph(save=False), None).recipes[0]
self.assertEqual(recipe.positive.sources, ())
self.assertIsNone(recipe.positive.inferred)
self.assertEqual(recipe.positive.reason, "not_saved")
# The enabled negative is recorded as before
self.assertEqual(recipe.negative.inferred.raw.value, NEGATIVE)
def test_unknown_setting_records_no_text_either(self):
graph = run_graph(save=["9", 0])
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
recipe = self.m.project_contexts(graph, None).recipes[0]
self.assertIsNone(recipe.positive.inferred)
self.assertEqual(recipe.positive.reason, "not_saved")
def test_enabled_role_is_unchanged(self):
recipe = self.m.project_contexts(run_graph(save=True), None).recipes[0]
self.assertEqual(recipe.positive.inferred.raw.value, POSITIVE)
self.assertEqual(len(recipe.positive.sources), 1)
if __name__ == "__main__":
unittest.main()
+1 -291
View File
@@ -13,11 +13,6 @@ from database.library_commands import LibraryCommandError
from database.operations import PromptDatabase
from utils.hashing import generate_prompt_hash
try:
from test_image_categories import category_checksum
except ImportError:
from tests.test_image_categories import category_checksum
class TestVocabularyReads(unittest.TestCase):
def setUp(self):
@@ -173,8 +168,7 @@ class TestVocabularyReads(unittest.TestCase):
self.assertEqual(page["total"], 4)
self.assertFalse(page["has_more"])
for row in page["categories"]:
self.assertEqual(set(row), {"value", "prompt_count", "image_count"})
self.assertEqual(row["image_count"], 0)
self.assertEqual(set(row), {"value", "prompt_count"})
self.assertEqual(
row["prompt_count"],
self.db.list_library(category=row["value"])["total"],
@@ -244,111 +238,6 @@ class TestVocabularyReads(unittest.TestCase):
with self.assertRaises(ValueError):
self.read("list_vocabulary_tags", **options)
def own(self, value, *paths, root="root"):
"""Give each relative path under ``root`` the own category ``value``."""
self.db.set_image_categories([(root, path) for path in paths], value)
def all_categories(self, limit=200):
rows, pages, after = [], [], None
while True:
page = self.read("list_vocabulary_categories", limit=limit, after=after)
pages.append(page)
rows.extend(page["categories"])
if not page["has_more"]:
return rows, pages
after = page["next_key"]
def test_category_list_includes_image_only_values_last(self):
self.save("Oldest", category="Zeta")
self.save("Newer", category="Alpha")
self.own("Zeta", "a.png", "b.png")
self.own("Only images B", "c.png")
self.own("Only images A", "d.png", "e.png", "gone.png")
rows, _pages = self.all_categories()
self.assertEqual(
rows,
[
{"value": "Zeta", "prompt_count": 1, "image_count": 2},
{"value": "Alpha", "prompt_count": 1, "image_count": 0},
{"value": "Only images A", "prompt_count": 0, "image_count": 3},
{"value": "Only images B", "prompt_count": 0, "image_count": 1},
],
)
page = self.read("list_vocabulary_categories")
self.assertEqual(page["total"], 4)
def test_category_cursor_crosses_into_image_only_values(self):
for index in range(3):
self.save(f"Owner {index}", category=f"P{index}")
self.own("P1", "shared.png")
self.own("I-b", "x.png")
self.own("I-a", "y.png")
self.own("I-c", "z.png", root="other")
for limit in (1, 2, 3, 4):
with self.subTest(limit=limit):
rows, pages = self.all_categories(limit=limit)
self.assertEqual(
[row["value"] for row in rows],
["P0", "P1", "P2", "I-a", "I-b", "I-c"],
)
self.assertEqual({page["total"] for page in pages}, {6})
self.assertEqual(
[row["image_count"] for row in rows], [0, 1, 0, 1, 1, 1]
)
first = self.read("list_vocabulary_categories", limit=3)
self.assertEqual(first["next_key"], [3, "P2"])
second = self.read("list_vocabulary_categories", limit=1, after=[3, "P2"])
self.assertEqual(second["next_key"], [2**63 - 1, "I-a"])
def test_image_rows_on_a_prompt_value_count_once(self):
self.save("Owner", category="Shared")
self.db.save_prompt_pair("Owner", "Noise", category="Shared")
self.own("Shared", *[f"{n}.png" for n in range(5)])
self.own("Shared", "elsewhere.png", root="second")
rows, _pages = self.all_categories()
self.assertEqual(
rows, [{"value": "Shared", "prompt_count": 1, "image_count": 6}]
)
def test_library_options_ignore_image_categories(self):
self.save("Owner", category="Prompt value")
before = self.db.get_library_options()
self.own("Image value", "a.png")
self.own("Prompt value", "b.png")
self.assertEqual(self.db.get_library_options(), before)
self.assertEqual(before["categories"], ["Prompt value"])
def test_category_reads_use_the_image_category_index(self):
from database import vocabulary
self.own("Value", "a.png")
statements = [
vocabulary._CATEGORY_ROWS + "SELECT COUNT(*) FROM counted",
vocabulary._CATEGORY_ROWS
+ "SELECT value,first_pid,prompt_count,image_count FROM counted "
"WHERE first_pid>? OR (first_pid=? AND value>?) "
"ORDER BY first_pid,value LIMIT ?",
]
plans = []
with self.db.model.get_connection() as conn:
for sql in statements:
params = [1] * sql.count("?")
plans.extend(
row[-1] for row in conn.execute("EXPLAIN QUERY PLAN " + sql, params)
)
for sql in (
vocabulary._IMAGE_CATEGORY_ROWIDS,
*vocabulary._CATEGORY_UPDATES,
):
params = [1] * sql.count("?")
plans.extend(
row[-1] for row in conn.execute("EXPLAIN QUERY PLAN " + sql, params)
)
image_steps = [step for step in plans if "image_categories" in step]
self.assertTrue(image_steps)
for step in image_steps:
self.assertIn("idx_image_categories_category", step, step)
def test_reads_share_one_snapshot_and_report_epoch(self):
self.save("Cat", category="Animal", tags=["cat"])
epoch = self.db.get_execution_epoch()
@@ -852,7 +741,6 @@ class TestVocabularyCategoryCommands(unittest.TestCase):
"other_prompts": 1,
"negative_variants": 2,
"archived_negative_variants": 1,
"images": 0,
},
)
self.assertFalse(result["target_exists"])
@@ -922,7 +810,6 @@ class TestVocabularyCategoryCommands(unittest.TestCase):
"other_prompts": 0,
"negative_variants": 0,
"archived_negative_variants": 0,
"images": 0,
},
)
self.assertEqual(self.rows("prompts")[variant]["category"], "Other")
@@ -1015,171 +902,6 @@ class TestVocabularyCategoryCommands(unittest.TestCase):
self.apply(bad)
self.assertEqual(self.snapshot(), before)
def own(self, value, *paths, root="root"):
self.db.set_image_categories([(root, path) for path in paths], value)
def image_rows(self):
with self.db.model.get_connection() as conn:
return {
(row[0], row[1]): (row[2], row[3], row[4])
for row in conn.execute(
"SELECT root_key,relative_path,category,updated_at,rowid "
"FROM image_categories ORDER BY rowid"
)
}
def checksum(self):
with self.db.model.get_connection() as conn:
return category_checksum(conn)
def test_preview_counts_image_rows(self):
self.owners()
self.own("Scenery", "a.png", "b.png", "missing.png")
self.own("Scenery", "c.png", root="second")
self.own("Other", "d.png")
before = self.image_rows()
result = self.preview()
self.assertEqual(result["impact"]["images"], 4)
self.assertEqual(self.image_rows(), before)
self.assertTrue(self.preview(target="Other")["target_exists"])
def test_apply_renames_image_categories(self):
self.owners()
bystander = self.save("Bystander", category="Other")
self.own("Scenery", "a.png", "b.png")
self.own("Scenery", "c.png", root="second")
self.own(" Scenery ", "padded.png")
self.own("Other", "d.png")
self.age_rows()
with self.db.model.get_connection() as conn:
conn.execute("UPDATE image_categories SET updated_at=?", (_OLD_STAMP,))
images_before = self.image_rows()
prompts_before, pairs_before = self.snapshot()
preview = self.preview()
receipt = self.apply(preview["impact_token"])
self.assertEqual(receipt["impact"]["images"], 3)
images_after = self.image_rows()
self.assertEqual(set(images_after), set(images_before))
renamed = {("root", "a.png"), ("root", "b.png"), ("second", "c.png")}
for key, (value, stamp, rowid) in images_after.items():
old_value, old_stamp, old_rowid = images_before[key]
self.assertEqual(rowid, old_rowid, key)
if key in renamed:
self.assertEqual(value, "Landscape")
self.assertNotEqual(stamp, _OLD_STAMP)
else:
self.assertEqual((value, stamp), (old_value, old_stamp))
prompts_after, pairs_after = self.snapshot()
self.assertEqual(prompts_after[bystander], prompts_before[bystander])
for before, after in (
(prompts_before, prompts_after),
(pairs_before, pairs_after),
):
for row_id, row in after.items():
if before[row_id]["category"] != "Scenery":
self.assertEqual(row, before[row_id])
# Merging into a value images already hold keeps every file key.
merge = self.preview(source="Landscape", target="Other")
self.assertTrue(merge["target_exists"])
self.assertEqual(merge["impact"]["images"], 3)
receipt = self.apply(merge["impact_token"], source="Landscape", target="Other")
self.assertTrue(receipt["consolidated"])
merged = self.image_rows()
self.assertEqual(set(merged), set(images_before))
self.assertEqual(
sorted(key for key, row in merged.items() if row[0] == "Other"),
sorted([*renamed, ("root", "d.png")]),
)
def test_image_only_value_can_be_renamed(self):
self.save("Bystander", category="Kept")
self.db.save_prompt_pair("Pair", "Noise", category="Kept")
self.own("Image only", "a.png", "b.png")
self.age_rows()
before = self.checksum()
preview = self.preview(source="Image only", target="Renamed")
self.assertEqual(
preview["impact"],
{
"positive_prompts": 0,
"other_prompts": 0,
"negative_variants": 0,
"archived_negative_variants": 0,
"images": 2,
},
)
self.assertFalse(preview["target_exists"])
receipt = self.apply(
preview["impact_token"], source="Image only", target="Renamed"
)
self.assertFalse(receipt["consolidated"])
self.assertEqual({row[0] for row in self.image_rows().values()}, {"Renamed"})
self.assertEqual(self.checksum(), before)
# An image-only target counts as existing.
self.own("Image target", "c.png")
self.assertTrue(
self.preview(source="Renamed", target="Image target")["target_exists"]
)
# Nothing holds the value any more: refused.
with self.assertRaises(LibraryCommandError) as caught:
self.preview(source="Image only", target="Again")
self.assertEqual(caught.exception.code, "vocabulary-conflict")
def test_token_refused_after_an_image_row_gains_or_loses_the_value(self):
self.owners()
self.own("Scenery", "a.png", "b.png")
def gain():
self.own("Scenery", "late.png")
def lose():
self.db.clear_image_categories([("root", "a.png")])
def move_away():
self.own("Elsewhere", "b.png")
def target_on_image():
self.own("Landscape", "target.png")
for mutate in (gain, lose, move_away, target_on_image):
with self.subTest(mutate.__name__):
token = self.preview()["impact_token"]
mutate()
before = self.snapshot(), self.image_rows()
with self.assertRaises(LibraryCommandError) as caught:
self.apply(token)
self.assertEqual(caught.exception.code, "vocabulary-conflict")
self.assertEqual((self.snapshot(), self.image_rows()), before)
def test_image_category_failure_rolls_back_prompts(self):
self.owners()
self.own("Scenery", "a.png")
token = self.preview()["impact_token"]
with self.db.model.get_connection() as conn:
conn.execute(
"CREATE TRIGGER abort_images BEFORE UPDATE ON image_categories "
"BEGIN SELECT RAISE(ABORT, 'boom'); END"
)
before = self.snapshot(), self.image_rows()
with self.assertRaises(sqlite3.Error):
self.apply(token)
self.assertEqual((self.snapshot(), self.image_rows()), before)
def test_category_exists_reads_the_image_index(self):
from database import vocabulary
with self.db.model.get_connection() as conn:
plan = [
row[-1]
for row in conn.execute(
"EXPLAIN QUERY PLAN " + vocabulary._CATEGORY_EXISTS, {"v": "x"}
)
]
steps = [step for step in plan if "image_categories" in step]
self.assertTrue(steps, plan)
for step in steps:
self.assertIn("idx_image_categories_category", step)
def test_category_apply_failure_rolls_back(self):
self.owners()
token = self.preview()["impact_token"]
@@ -1220,18 +942,6 @@ class TestSharedTokenHashing(unittest.TestCase):
),
"cefe7f6a5544b7f6a203f5527810899c6af568aa7abc1b75adffbfb5ae6251ae",
)
with_images = category_impact_token(
"e", "S", "T", True, [(1, True)], [], images=[9, 4]
)
self.assertRegex(with_images, r"\A[0-9a-f]{64}\Z")
self.assertEqual(
with_images,
category_impact_token("e", "S", "T", True, [(1, True)], [], images=[4, 9]),
)
self.assertNotEqual(
with_images,
category_impact_token("e", "S", "T", True, [(1, True)], [], images=[4]),
)
if __name__ == "__main__":
+3 -4
View File
@@ -35,16 +35,15 @@ test('execution script discovers only the new matrix and preserves all 78 origin
assert.match(executions, /\[nodes2-start\]/);
// + the fresh-host library location case, + #342 image attribution, + #343 linked text,
// + #332 image category routes on a generated output (image-attribution.spec.mjs, 2 projects),
// + #347 cached runtime-linked runs: five node kinds and eight one-renderer cases (execution-text-link.spec.mjs, 2 projects),
// + #90 Save to library off (five node types)
assert.match(executions, /Total: 128 tests in 5 files/);
// + #347 cached runtime-linked runs: five node kinds and eight one-renderer cases (execution-text-link.spec.mjs, 2 projects)
assert.match(executions, /Total: 118 tests in 5 files/);
for (const project of ['classic-start', 'nodes2-start']) {
const cases = executions.split('\n').filter(line => line.includes(`[${project}]`));
assert.equal(cases.filter(line => line.includes('execution-history.spec.mjs')).length, 15);
assert.equal(cases.filter(line => line.includes('execution-outputs.spec.mjs')).length, 8);
assert.equal(cases.filter(line => line.includes('library-location.spec.mjs')).length, 1);
assert.equal(cases.filter(line => line.includes('image-attribution.spec.mjs')).length, 3);
assert.equal(cases.filter(line => line.includes('execution-text-link.spec.mjs')).length, 37);
assert.equal(cases.filter(line => line.includes('execution-text-link.spec.mjs')).length, 32);
}
assert.doesNotMatch(executions, /prompt-manager.spec/);
});
-6
View File
@@ -107,12 +107,6 @@
"functions": 98,
"statements": 98
},
"save-to-library.js": {
"lines": 98,
"branches": 98,
"functions": 98,
"statements": 98
},
"scan-progress.js": {
"lines": 98,
"branches": 98,
-3
View File
@@ -11,7 +11,6 @@ import weakref
from .execution_context_budget import ContextBudget, PayloadLease, SealedContext
from .execution_context_graph import (
NOT_SAVED,
GraphProjection,
GraphRole,
SourceRef,
@@ -326,8 +325,6 @@ class ContextCapture:
nonlocal ambiguous
if role.reason == "source-transformed":
return _unknown("unsupported")
if role.reason == NOT_SAVED:
return _unknown("unavailable")
if not role.sources:
return role.inferred or _unknown("unsupported")
values = [
-10
View File
@@ -3,13 +3,8 @@
import re
from dataclasses import dataclass
from .execution_contexts import PromptSnapshot, TextEvidence, text_size
from .prompt_graph import save_setting
from .prompt_pair_resolver import _DEFAULT_ADAPTERS
# A role whose PromptManager source has save_to_library off, or wired from a
# node the graph cannot read: no text is recorded for it (#90)
NOT_SAVED = "not_saved"
@dataclass(frozen=True)
class SourceRef:
@@ -337,11 +332,6 @@ class _Projector:
self.transformed = False
# Only the capture-source-to-consumer path determines transforms.
sources = self.discover_sources(value, "conditioning")
if any(
save_setting(self.graph, source.node_id) is not True
for source in sources
):
return GraphRole((), None, NOT_SAVED)
snapshot, _ = self.text(value, "conditioning")
return GraphRole(
sources, snapshot, "source-transformed" if self.transformed else ""
+3 -36
View File
@@ -37,13 +37,7 @@ from .hashing import generate_prompt_hash
from .metadata_extractor import ComfyUIMetadataExtractor
from .parallel import note_refusal, refused_at_shutdown
from .logging_config import get_logger
from .prompt_graph import (
metadata_opted_out,
resolve_text,
run_prompt_nodes,
save_setting,
side_opted_out,
)
from .prompt_graph import resolve_text, run_prompt_nodes
from .prompt_pair_resolver import resolve_prompt_pair
from .run_keys import image_run_keys
from .usage_tracking import survive_switch
@@ -456,12 +450,6 @@ class ImageGenerationHandler(FileSystemEventHandler):
# that produced this image, independent of node caching and of any
# negative-prompt PromptManager nodes in the workflow.
state, current_prompt = self._find_prompt_from_metadata(metadata)
if state == "opted_out":
self.logger.info(
"Save to library is off for this run's prompt, skipping image: "
f"{image_path}"
)
return
if state == "foreign":
# Another workflow saved this image: it belongs to no prompt of
# ours, and must not consume a batch queue entry.
@@ -727,27 +715,16 @@ class ImageGenerationHandler(FileSystemEventHandler):
``("linked", prompt)`` with a context dict holding 'id' and 'text';
``("foreign", None)`` when the workflow has no positive PromptManager
node, so the image belongs to no prompt of ours;
``("opted_out", None)`` when a PromptManager node that would be the
run's prompt has its ``save_to_library`` switch off (#90): the image
is linked to no prompt, by no fallback;
``("unknown", None)`` when metadata can't decide (missing, a batch
item whose text came from another node, or a lookup failure) and the
caller's queue and tracker fallbacks apply.
"""
if not metadata:
return "unknown", None
if metadata_opted_out(metadata):
return "opted_out", None
graph = metadata.get("prompt")
if isinstance(graph, dict) and any(
save_setting(graph, node_id) is None for node_id in run_prompt_nodes(graph)
):
# A switch wired from a node the graph cannot read: only the node's
# own run (it saw the real value) can link the image (#90)
return "unknown", None
prompt_text = None
resolution = resolve_prompt_pair(metadata)
graph = metadata.get("prompt")
if resolution["state"] == "resolved" and isinstance(graph, dict):
positive = resolution["positive"]
path = positive["provenance"].get("path", [])
@@ -888,23 +865,13 @@ class ImageGenerationHandler(FileSystemEventHandler):
metadata: Extracted metadata from the image file (workflow, parameters,
etc.)
"""
if metadata_opted_out(metadata):
# Reached from a run or history: the file's own graph says no (#90)
self.logger.info(
f"Save to library is off for {os.path.basename(image_path)}; "
"left unlinked"
)
return
try:
resolution = resolve_prompt_pair(metadata)
graph = metadata.get("prompt") if isinstance(metadata, dict) else None
# An opted-out negative is never stored: link the positive alone
pair_allowed = not side_opted_out(graph, resolution.get("negative"))
def link():
# One write; run again on the moved library when a Move
# library switched while it waited (#249 S6b).
if resolution["state"] == "resolved" and pair_allowed:
if resolution["state"] == "resolved":
positive = resolution["positive"]
expected = self.db_manager.get_prompt_by_hash(
generate_prompt_hash(
+8 -56
View File
@@ -300,49 +300,6 @@ def _blank(value: str | bytes | None) -> bool:
return value is None or not value.strip()
def inherited_category(
positive_category: str | bytes | None,
recipe_category: str | bytes | None,
has_recipe: bool,
) -> tuple:
"""``(value, scope)`` of the category an image's link supplies (#332 MC4).
The one rule for the inspector chip, the Images category filter and
History (D39): a linked recipe's own category when it is not blank,
``(recipe category, "pair")``; otherwise the positive's value unchanged,
blank or not, ``(positive category, "positive")``.
"""
if has_recipe and not _blank(recipe_category):
return recipe_category, "pair"
return positive_category, "positive"
class _OwnCategoryUnread:
"""The own-category read failed: hide the category (#332 DC5)."""
def __repr__(self) -> str:
return "OWN_CATEGORY_UNREAD"
OWN_CATEGORY_UNREAD = _OwnCategoryUnread()
def apply_own_category(library: dict, own) -> dict:
"""*library* with an image's own category applied (#332 §5).
``own`` is the own category (a string, shown with scope ``"image"``
whatever the link status), None (the inherited values stay) or
OWN_CATEGORY_UNREAD (category and scope hidden rather than possibly
wrong). Returns a new dict.
"""
result = dict(library)
if own is OWN_CATEGORY_UNREAD:
result.update(category=None, category_scope=None)
elif isinstance(own, str):
result.update(category=own, category_scope="image")
return result
def library_metadata(
positive_category: str | bytes | None,
positive_tags: list,
@@ -355,14 +312,13 @@ def library_metadata(
the linked positive prompt's, the values Set category writes and the Images
filters match. Tag sets are never merged. ``recipe`` is (category, tags).
"""
category, category_scope = inherited_category(
positive_category,
recipe[0] if recipe is not None else None,
recipe is not None,
)
category, category_scope = positive_category, "positive"
tags, tags_scope = positive_tags, "positive"
if recipe is not None and recipe[1]:
tags, tags_scope = recipe[1], "pair"
if recipe is not None:
if not _blank(recipe[0]):
category, category_scope = recipe[0], "pair"
if recipe[1]:
tags, tags_scope = recipe[1], "pair"
return {
"category": category,
"category_scope": category_scope,
@@ -389,13 +345,9 @@ def _library_projection(library: dict) -> dict:
value = library.get(key)
if value is None or isinstance(value, str):
result[key] = value
# An image's own category (#332) has the scope "image"; tags never do.
for key, scopes in (
("category_scope", ("pair", "positive", "image", None)),
("tags_scope", ("pair", "positive", None)),
):
for key in ("category_scope", "tags_scope"):
scope = library.get(key)
if scope in scopes:
if scope in ("pair", "positive", None):
result[key] = scope
tags = library.get("tags")
result["tags"] = (
+18 -184
View File
@@ -21,11 +21,6 @@ PROMPT_MANAGER_TYPES = frozenset(
}
)
# A node's switch (#90): False means PromptManager ignores that node's runs
SAVE_INPUT = "save_to_library"
# Core nodes whose output is a known boolean: (class_type) -> input holding it
_BOOLEAN_SOURCES = {"PrimitiveBoolean": "value"}
# Upper bound on nodes expanded per graph; queued and embedded graphs are untrusted
MAX_VISITS = 500
# Longest text resolve_text will build; StringConcatenate chains can double
@@ -64,116 +59,6 @@ def _inputs(node: Any) -> Dict[str, Any]:
return inputs if isinstance(inputs, dict) else {}
def _as_boolean(value: Any) -> bool:
"""ComfyUI's coercion of a BOOLEAN input before the node runs."""
if isinstance(value, dict) and "__value__" in value:
value = value["__value__"]
return bool(value)
def save_setting(graph: Any, node_id: Any) -> Optional[bool]:
"""A PromptManager node's ``save_to_library`` as the node will receive it.
True when the input is missing (every workflow and image from before the
switch), the literal coerced like ComfyUI does, the value of a linked core
``PrimitiveBoolean``, and None when the switch is wired from a node whose
output cannot be known from the graph.
"""
node = graph.get(str(node_id)) if isinstance(graph, dict) else None
inputs = _inputs(node)
if SAVE_INPUT not in inputs:
return True
value = inputs[SAVE_INPUT]
if not _is_link(value):
return _as_boolean(value)
source = graph.get(str(value[0]))
if not isinstance(source, dict) or value[1] != 0:
return None
key = _BOOLEAN_SOURCES.get(str(source.get("class_type", "")))
source_inputs = _inputs(source)
if key is None or key not in source_inputs or _is_link(source_inputs[key]):
return None
return _as_boolean(source_inputs[key])
def opted_out(graph: Any, node_id: Any) -> bool:
"""Whether a PromptManager node's switch is off: PromptManager ignores it."""
return save_setting(graph, node_id) is False
def side_opted_out(graph: Any, side: Any) -> bool:
"""Whether a resolved side's path passes a PromptManager node that is off.
``side`` is a resolver side (``positive`` or ``negative`` of a resolved
pair): its text must not be stored when any PromptManager node on its
provenance path has its switch off or wired from an unreadable node (#90).
"""
if not isinstance(graph, dict) or not isinstance(side, dict):
return False
provenance = side.get("provenance")
path = provenance.get("path") if isinstance(provenance, dict) else None
for step in path if isinstance(path, list) else []:
node_id = step.get("node_id") if isinstance(step, dict) else None
node = graph.get(str(node_id)) if node_id is not None else None
if (
isinstance(node, dict)
and node.get("class_type") in PROMPT_MANAGER_TYPES
and save_setting(graph, node_id) is not True
):
return True
return False
def _workflow_node_opted_out(node: Any) -> bool:
"""A saved-workflow PromptManager node whose widget values hold ``False``.
No other PromptManager widget holds a boolean, so the value is the switch.
"""
if not isinstance(node, dict):
return False
if (node.get("type") or node.get("class_type")) not in PROMPT_MANAGER_TYPES:
return False
if node.get("mode") in (2, 4):
return False # muted or bypassed: it did not run
values = node.get("widgets_values")
return isinstance(values, list) and any(value is False for value in values)
def metadata_opted_out(metadata: Any, *, unreadable: bool = False) -> bool:
"""Whether a file's metadata says its run's prompt does not save (#90).
The executed graph (``prompt``) decides when present: a node of
``positive_opt_outs``, or with ``unreadable`` also a run prompt whose
switch is wired from a node the graph cannot read. Without one, the
saved workflow (``workflow`` nodes or the extractor's
``text_encoder_nodes``) is opted out when a PromptManager node stores
``False``.
"""
if not isinstance(metadata, dict):
return False
graph = metadata.get("prompt")
if isinstance(graph, dict):
if positive_opt_outs(graph):
return True
return unreadable and any(
save_setting(graph, node_id) is None for node_id in run_prompt_nodes(graph)
)
workflow = metadata.get("workflow")
nodes = list(metadata.get("text_encoder_nodes") or [])
if isinstance(workflow, dict) and isinstance(workflow.get("nodes"), list):
nodes.extend(workflow["nodes"])
definitions = workflow.get("definitions")
subgraphs = (
definitions.get("subgraphs") if isinstance(definitions, dict) else None
)
for subgraph in subgraphs if isinstance(subgraphs, list) else []:
inner = subgraph.get("nodes") if isinstance(subgraph, dict) else None
nodes.extend(inner if isinstance(inner, list) else [])
# Without the executed graph the node's role is unknown: any opted-out
# PromptManager node opts the file out (never a guessed link).
return any(_workflow_node_opted_out(node) for node in nodes[:MAX_VISITS])
def _positive_roots(graph: Dict[str, Any]) -> List[Any]:
"""Links into samplers' positive inputs (KSampler, CFGGuider, BasicGuider, ...).
@@ -215,21 +100,11 @@ def has_sampler(graph: Any) -> bool:
def positive_prompt_components(graph: Any) -> List[Tuple[str, str]]:
"""Node/input identities reaching actual sampler positive wires.
A node whose switch is off is not one, and nothing upstream of it is
followed (#90): see ``positive_opt_outs``.
"""
return _walk_positive(graph)[0]
def _walk_positive(graph: Any) -> Tuple[List[Tuple[str, str]], List[str]]:
"""(components of enabled nodes, opted-out nodes) on sampler positive wires."""
"""Node/input identities reaching actual sampler positive wires."""
if not isinstance(graph, dict):
return [], []
return []
found: List[Tuple[str, str]] = []
skipped: List[str] = []
seen = set()
stack = list(reversed(_positive_roots(graph)))
while stack and len(seen) < MAX_VISITS:
@@ -242,10 +117,6 @@ def _walk_positive(graph: Any) -> Tuple[List[Tuple[str, str]], List[str]]:
if not isinstance(node, dict):
continue
if node.get("class_type") in PROMPT_MANAGER_TYPES:
if opted_out(graph, node_id):
if node_id not in skipped:
skipped.append(node_id)
continue
field = "text"
if node.get("class_type") == "PromptManagerPair":
field = {
@@ -261,33 +132,7 @@ def _walk_positive(graph: Any) -> Tuple[List[Tuple[str, str]], List[str]]:
found.append(identity)
continue
stack.extend(reversed(_upstream_links(_inputs(node))))
return found, skipped
def _sampler_less_nodes(graph: Dict[str, Any]) -> List[str]:
"""Every PromptManager node of a graph without a recognisable sampler."""
return [
str(node_id)
for node_id, node in graph.items()
if isinstance(node, dict) and node.get("class_type") in PROMPT_MANAGER_TYPES
]
def positive_opt_outs(graph: Any) -> List[str]:
"""PromptManager nodes with their switch off that would be a run's prompt.
Those on a sampler's positive wires, or, in a graph without a recognisable
sampler, any. A run with one of them links no file to any prompt (#90):
the file's graph cannot always say which sampler wrote it.
"""
if not isinstance(graph, dict):
return []
found, skipped = _walk_positive(graph)
if found or skipped or has_sampler(graph):
return skipped
return [
node_id for node_id in _sampler_less_nodes(graph) if opted_out(graph, node_id)
]
return found
def positive_prompt_nodes(graph: Any) -> List[str]:
@@ -303,13 +148,13 @@ def positive_prompt_nodes(graph: Any) -> List[str]:
def run_prompt_components(graph: Any) -> List[Tuple[str, str]]:
"""Actual positive components, retaining standalone main-editor fallback."""
components, skipped = _walk_positive(graph)
if components or skipped or not isinstance(graph, dict) or has_sampler(graph):
components = positive_prompt_components(graph)
if components or not isinstance(graph, dict) or has_sampler(graph):
return components
return [
(node_id, "text")
for node_id in _sampler_less_nodes(graph)
if not opted_out(graph, node_id)
(str(node_id), "text")
for node_id, node in graph.items()
if isinstance(node, dict) and node.get("class_type") in PROMPT_MANAGER_TYPES
]
@@ -320,16 +165,16 @@ def run_prompt_nodes(graph: Any) -> List[str]:
say), every PromptManager node counts, as before 3.2.4, so an exotic workflow
never silently loses usage counting or image linking. When a sampler exists
and no PromptManager node feeds its positive input, nothing counts: the only
PromptManager nodes are negative prompts. A node whose switch is off never
counts (#90).
PromptManager nodes are negative prompts.
"""
return list(
dict.fromkeys(
node_id
for node_id, field in run_prompt_components(graph)
if field == "text"
)
)
positive = positive_prompt_nodes(graph)
if positive or not isinstance(graph, dict) or has_sampler(graph):
return positive
return [
str(node_id)
for node_id, node in graph.items()
if isinstance(node, dict) and node.get("class_type") in PROMPT_MANAGER_TYPES
]
def _join_text_inputs(
@@ -425,18 +270,7 @@ def resolve_text(graph: Any, node_id: str) -> Optional[str]:
def intrinsic_pair_nodes(graph: Any) -> List[str]:
"""Paired nodes whose matching outputs feed the same sampler, or standalone.
A pair whose switch is off is never one (#90).
"""
return [
node_id
for node_id in _intrinsic_pair_nodes(graph)
if not opted_out(graph, node_id)
]
def _intrinsic_pair_nodes(graph: Any) -> List[str]:
"""Paired nodes whose matching outputs feed the same sampler, or standalone."""
if not isinstance(graph, dict) or len(graph) > MAX_VISITS:
return []
if not has_sampler(graph):
+1 -13
View File
@@ -30,12 +30,7 @@ from typing import Any, Callable, List, Optional, Tuple
try:
from .hashing import generate_prompt_hash
from .logging_config import get_logger
from .prompt_graph import (
intrinsic_pair_nodes,
resolve_text,
run_prompt_components,
save_setting,
)
from .prompt_graph import intrinsic_pair_nodes, resolve_text, run_prompt_components
from .prompt_pair_resolver import resolve_text_input
from .pair_execution import clear_pair_execution
from .queue_observers import subscribe_dequeue
@@ -46,7 +41,6 @@ except ImportError:
intrinsic_pair_nodes,
resolve_text,
run_prompt_components,
save_setting,
)
from utils.prompt_pair_resolver import resolve_text_input
from utils.pair_execution import clear_pair_execution
@@ -133,14 +127,10 @@ def queued_prompt_runs(graph: Any) -> List[Run]:
"""(hash, text) of each positive PromptManager node whose text is knowable.
Each hash once: the same text in two positive nodes is one run of the prompt.
A node whose switch is off is never counted; one whose switch is wired from
a node the graph cannot read counts itself when it executes (#90).
"""
runs: List[Run] = []
seen = set()
for node_id, field in run_prompt_components(graph):
if save_setting(graph, node_id) is not True:
continue
node = graph.get(node_id, {}) if isinstance(graph, dict) else {}
text = (
resolve_text_input(graph, node_id, field)
@@ -166,8 +156,6 @@ def queued_pair_runs(graph: Any) -> List[Tuple[str, str, str]]:
"""Resolve raw paired recipes only when both outputs form the used pair."""
found, seen = [], set()
for node_id in intrinsic_pair_nodes(graph):
if save_setting(graph, node_id) is not True:
continue # wired switch: the node counts itself (#90)
positive = resolve_text_input(graph, node_id, "text")
negative = resolve_text_input(graph, node_id, "negative_text")
if positive is None or not positive.strip() or negative is None:
-3
View File
@@ -6,7 +6,6 @@ const { installClassicControls } = await import(api.fileURL('/prompt_manager/js/
const { installFocusedNode, installSplitNode, installPairNode } = await import(api.fileURL('/prompt_manager/js/focused-node.js'));
const { installPromptBridge } = await import(api.fileURL('/prompt_manager/js/prompt-bridge-host.js'));
const { collapseLinkedWidget, isLinked, linkNoteText, primitiveTextValue, storedWidgetValue } = await import(api.fileURL('/prompt_manager/js/text-links.js'));
const { normalizeSaveWidget } = await import(api.fileURL('/prompt_manager/js/save-to-library.js'));
const promptManagerTypes = new Set(['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerSplit', 'PromptManagerPair']);
// ============================================================================
@@ -526,8 +525,6 @@ app.registerExtension({
if (originalConfigure) {
originalConfigure.call(this, data);
}
// #90: an older workflow loads with Save to library on
normalizeSaveWidget(this);
if (data._userHasResized) {
this._userHasResized = data._userHasResized;
}
+4 -31
View File
@@ -2,13 +2,9 @@ import {createScope, createButton, createField, mountTheme} from './shared/toolk
import {createPromptLibrary} from './prompt-library.js';
import {attachMetadataAutocomplete} from './metadata-autocomplete.js';
import {exposeTextSockets, linkNoteText, preferTextInputs, textInputAtPoint} from './text-links.js';
import {SAVE_WIDGET, normalizeSaveWidget, placeSaveWidgetLast, saveWidget, savesToLibrary, setSaveToLibrary} from './save-to-library.js';
const names = ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text'];
let nextEditorId = 0;
// The editor draws these widgets itself; the host keeps them as hidden, zero-height values.
function hiddenWidgetSize() { return [0, -4]; }
function hideNativeWidget(widget) { widget.hidden = true; widget.type = 'hidden'; widget.computeSize = hiddenWidgetSize; }
function loadStyles() {
const href = new URL('./prompt-editor.css', import.meta.url).href;
@@ -47,15 +43,10 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
for (const name of pair ? [...names,'negative_text'] : names) {
let widget = this.widgets?.find(item => item.name === name);
if (!widget) widget = this.addWidget('text', name, '', () => {}, {});
hideNativeWidget(widget);
widget.hidden = true; widget.type = 'hidden'; widget.computeSize = () => [0, -4];
widgets.set(name, widget);
}
exposeTextSockets(this, textInputs);
// #90: the switch follows the hidden fallbacks above (one widget order on every frontend) and is drawn in the
// toolbar, like the other hidden widgets are drawn by the editor.
placeSaveWidgetLast(this);
const nativeSave = saveWidget(this);
if (nativeSave) hideNativeWidget(nativeSave);
const element = document.createElement('div'); element.className = `ca-tools pm-focused${split ? ' pm-split' : ''}${pair ? ' pm-pair' : ''}`;
element.setAttribute('aria-label',pair ? 'Paired prompt editor' : split ? 'Split prompt editor' : 'Focused prompt editor');
loadStyles(); mountTheme(element, app.ui?.settings, scope);
@@ -67,11 +58,6 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
toolbar.append(web);
const save = pair ? createButton('Save pair','ca-primary') : null;
if (save) toolbar.append(save);
const saveSwitch = document.createElement('input'); saveSwitch.type = 'checkbox'; saveSwitch.dataset.saveToLibrary = '';
const saveLabel = document.createElement('label'); saveLabel.className = 'pm-editor-save';
saveLabel.title = 'Off: runs of this node are not saved, counted or linked to their images. The output is unchanged.';
saveLabel.append(saveSwitch, document.createTextNode('Save to library'));
if (nativeSave) toolbar.append(saveLabel);
let saving = false;
const workspace = document.createElement('div'); workspace.className = 'pm-editor-workspace';
const editor = document.createElement('div'); editor.className = 'pm-editor-compose';
@@ -152,11 +138,7 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
}
const composed = disclosure('preview','Composed text preview');
const preview = document.createElement('p'); preview.className = 'pm-editor-preview'; preview.dataset.composedPreview = ''; composed.append(preview);
const footer = document.createElement('p'); footer.className = 'pm-editor-status'; footer.setAttribute('role','status');
const SAVE_OFF = 'Save to library is off: runs of this node are not added to the library.';
const SAVE_LINKED = 'Save to library comes from a link: runs are added to the library only while it is on.';
function runNote(saved) { return isConnected(SAVE_WIDGET) ? SAVE_LINKED : savesToLibrary(thisNode) ? saved : SAVE_OFF; }
footer.textContent = runNote(pair ? 'The prompt pair is saved when this node runs, or use Save pair.' : 'Main prompt is saved when this node runs.');
const footer = document.createElement('p'); footer.className = 'pm-editor-status'; footer.setAttribute('role','status'); footer.textContent = pair ? 'The prompt pair is saved when this node runs, or use Save pair.' : 'Main prompt is saved when this node runs.';
const saveConnectionNotice = pair ? document.createElement('p') : null;
if (saveConnectionNotice) {
saveConnectionNotice.className = 'pm-editor-status'; saveConnectionNotice.setAttribute('role','status');
@@ -184,7 +166,6 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
mainField.hidden = Boolean(isConnected('text')); connected.hidden = !isConnected('text');
if (negative) { negative.field.hidden = Boolean(isConnected('negative_text')); negative.note.hidden = !isConnected('negative_text'); }
if (save) { save.disabled = saving || hasConnectedPairValues(); saveConnectionNotice.hidden = !hasConnectedPairValues(); }
saveSwitch.checked = savesToLibrary(thisNode); saveSwitch.disabled = isConnected(SAVE_WIDGET);
controls.forEach((input,name) => {
input.value = value(name);
if (name !== 'text' && name !== 'negative_text') {
@@ -204,7 +185,7 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
write('category',prompt.category || '');
write('tags',Array.isArray(prompt.tags) ? prompt.tags.join(', ') : prompt.tags || '');
}
render(); footer.setAttribute('role','status'); footer.textContent = runNote(pair ? 'Loaded pair. Saved when this node runs.' : 'Loaded prompt. Saved when this node runs.'); main.focus();
render(); footer.setAttribute('role','status'); footer.textContent = pair ? 'Loaded pair. Saved when this node runs.' : 'Loaded prompt. Saved when this node runs.'; main.focus();
}});
workspace.append(library.element); element.append(footer);
function snapshot() {
@@ -216,13 +197,7 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
library.element.hidden = !split && !state.libraryOpen; browse.textContent = state.libraryOpen ? 'Hide library' : 'Browse library'; browse.setAttribute('aria-expanded', String(Boolean(state.libraryOpen)));
library.setQuery(value('search_text')); library.restoreFilters(state.libraryFilters); render();
}
controls.forEach((input,name) => scope.listen(input,'input',() => {write(name,input.value);render();footer.setAttribute('role','status');footer.textContent=runNote(pair?'The pair is saved when this node runs.':'Main prompt is saved when this node runs.');}));
scope.listen(saveSwitch,'change',() => {
if (isConnected(SAVE_WIDGET)) return;
setSaveToLibrary(thisNode, saveSwitch.checked); render();
footer.setAttribute('role','status'); footer.textContent = runNote(pair ? 'The pair is saved when this node runs.' : 'Main prompt is saved when this node runs.');
thisNode.setDirtyCanvas(true,true); app.graph?.setDirtyCanvas(true,true);
});
controls.forEach((input,name) => scope.listen(input,'input',() => {write(name,input.value);render();footer.setAttribute('role','status');footer.textContent=pair?'The pair is saved when this node runs.':'Main prompt is saved when this node runs.';}));
scope.listen(browse,'click',() => {
library.element.hidden = !library.element.hidden;
library.closeCompletions();
@@ -273,8 +248,6 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
const result = configure?.apply(this,args);
// A saved workflow restores its inputs with their widget bindings; keep the text inputs as sockets.
exposeTextSockets(this, textInputs);
// #90: an older workflow loads with Save to library on
normalizeSaveWidget(this);
this._focusedView?.restore();
return result;
};
-2
View File
@@ -2,8 +2,6 @@
.pm-focused [hidden] { display: none !important; }
.pm-editor-row { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; }
.pm-editor-toolbar > strong { margin-inline-end: auto; font-weight: 500; }
.pm-editor-save { display: inline-flex; align-items: center; gap: 6px; color: var(--ca-muted); cursor: pointer; }
.pm-editor-save:has(input:checked) { color: inherit; }
.pm-focused .ca-field-label { align-self: flex-start; margin-inline: 0; text-align: start; }
.pm-editor-workspace, .pm-editor-compose { display: contents; }
.pm-focused .pm-editor-main { display: flex; flex: 1; min-block-size: 180px; flex-direction: column; align-items: stretch; gap: 5px; text-align: start; }
-47
View File
@@ -1,47 +0,0 @@
// #90: a PromptManager node's `save_to_library` switch. Off: PromptManager ignores the node's runs (nothing saved,
// counted or linked); the node's output is unchanged.
//
// The input is declared last, so it takes the widget index an older workflow used for the editor's own
// (non-serialized) DOM widget, where it stored `''` or nothing. ComfyUI coerces a BOOLEAN with `bool(value)`, so that
// `''` would turn saving off: after a workflow loads, anything that is not a boolean is the default, on.
export const SAVE_WIDGET = 'save_to_library';
export function saveWidget(node) {
return node?.widgets?.find(widget => widget?.name === SAVE_WIDGET);
}
/** Whether the node saves its runs (a missing widget, as on an older backend, is on). */
export function savesToLibrary(node) {
return saveWidget(node)?.value !== false;
}
/** Set a non-boolean switch value (from an older workflow) to on. Returns whether it changed. */
export function normalizeSaveWidget(node) {
const widget = saveWidget(node);
if (!widget || typeof widget.value === 'boolean') return false;
widget.value = true;
const index = node.widgets.indexOf(widget);
if (Array.isArray(node.widgets_values) && index >= 0) node.widgets_values[index] = true;
return true;
}
/** Move the switch after every other widget the node has now, so its index is the same on every frontend. */
export function placeSaveWidgetLast(node) {
const widget = saveWidget(node);
const widgets = node?.widgets;
if (!widget || widgets.at(-1) === widget) return false;
widgets.splice(widgets.indexOf(widget), 1);
widgets.push(widget);
return true;
}
/** Turn the switch on or off the way a native widget edit does. */
export function setSaveToLibrary(node, value) {
const widget = saveWidget(node);
if (!widget) return;
widget.value = Boolean(value);
const index = node.widgets.indexOf(widget);
if (Array.isArray(node.widgets_values) && index >= 0) node.widgets_values[index] = widget.value;
widget.callback?.(widget.value, undefined, node);
}
+1 -6
View File
@@ -20,8 +20,6 @@ const REPRESENTATIONS = {
limited: 'Record exceeds inspection limits; text is unavailable',
};
const TEXTLESS = new Set(['invalid', 'limited']);
// #332: the hidden prefix that names where a category chip's value came from.
const CATEGORY_PREFIX = {image: 'Image category: ', positive: 'Positive prompt category: ', pair: 'Recipe category: '};
const REVEAL_URL = '/prompt_manager/images/reveal';
// D38 Show in folder: the route's answers, in the maintainer's words.
const REVEAL_OPENED = 'Opened the folder on this computer.';
@@ -267,10 +265,7 @@ export function mountFileInspector(container, {api, sender, parentScope, variant
inner.append(heading, el('p', filename, 'small'));
const badges = el('div', undefined, 'tags'), badge = evidenceBadge(record.prompts);
if (badge) badges.append(el('span', badge, 'chip blue'));
// #332: an image's own category shows for any link status; an inherited one only when linked.
if (library?.category && (library.category_scope === 'image' || linked)) {
badges.append(prefixedChip(CATEGORY_PREFIX[library.category_scope] ?? CATEGORY_PREFIX.positive, library.category));
}
if (linked && library.category) badges.append(prefixedChip(library.category_scope === 'pair' ? 'Recipe category: ' : 'Positive prompt category: ', library.category));
if (badges.childNodes.length) inner.append(badges);
const prompts = el('div'); inner.append(prompts);
const blocks = mountPromptBlocks(prompts, {sender, parentScope:contentScope, includePairAction:false, contextLabel:promptContext, title});
+4 -35
View File
@@ -203,48 +203,17 @@ async function readDefaults(api, signal) {
}
export const UNAVAILABLE_HINT = 'Download it in System › Auto-tagging.';
const nonEmpty = value => typeof value === 'string' && value.trim() !== '';
const joined = names => (names.length > 1 ? `${names.slice(0, -1).join(', ')} and ${names.at(-1)}` : names.join(''));
/** The pip names a `/autotag/models` entry says its backend cannot import (#424); only non-empty strings count. */
export const missingPackages = entry => (object(entry) && Array.isArray(entry.missing_packages)
? entry.missing_packages.filter(nonEmpty) : []);
/** `pip install …` for `entry`'s missing packages: the server's command, else built from the names; '' when none. */
export function installCommand(entry) {
const missing = missingPackages(entry);
if (!missing.length) return '';
return nonEmpty(entry.install_command) ? entry.install_command : `pip install ${missing.join(' ')}`;
}
/**
* The server's missing-package sentence (`missing_packages_message`, #424) for `name`, else '' (nothing missing):
* "<name> needs llama-cpp-python. Install it into ComfyUI's Python: pip install llama-cpp-python. <note>".
*/
export function needsPackages(name, entry) {
const missing = missingPackages(entry);
if (!missing.length) return '';
const note = nonEmpty(entry.install_note) ? ` ${entry.install_note}` : '';
return `${name} needs ${joined(missing)}. Install ${missing.length === 1 ? 'it' : 'them'} into ComfyUI's Python: `
+ `${installCommand(entry)}.${note}`;
}
/** "a, b and c": the missing package names as the sentences list them. */
export const packageList = entry => joined(missingPackages(entry));
/**
* Why `model` cannot run, from the `/autotag/models` status (the server's own `model-unavailable` sentence), else ''
* (ready, or the status is unknown: the job then reports a missing model itself). Not downloaded is named first, then
* a backend package that cannot be imported (#424).
* (downloaded, or the status is unknown: the job then reports a missing model itself).
*/
export function modelUnavailable(models, model) {
const option = modelOf(model);
requireValue(option, 'Unknown tagging model');
const status = object(models) && object(models[model]) ? models[model] : null;
if (!status) return '';
const fallback = option.label.split(' · ')[0];
if (status.downloaded === false) return `${nonEmpty(status.name) ? status.name : fallback} is not downloaded. ${UNAVAILABLE_HINT}`;
return needsPackages(nonEmpty(status.label) ? status.label : fallback.replace(' (Recommended)', ''), status);
if (!status || status.downloaded !== false) return '';
const name = typeof status.name === 'string' && status.name.trim() ? status.name : option.label.split(' · ')[0];
return `${name} is not downloaded. ${UNAVAILABLE_HINT}`;
}
function initialState(source, preset) {
+1 -3
View File
@@ -201,9 +201,7 @@ function resolutionFor(value) {
function libraryFor(value) {
if (!object(value) || !['linked', 'none', 'ambiguous', 'unavailable', 'limited'].includes(value.status) || value.scope !== 'saved-library' ||
!Array.isArray(value.tags) || value.tags.some(tag => typeof tag !== 'string') ||
![null, 'pair', 'positive', 'image'].includes(value.category_scope) ||
![null, 'pair', 'positive'].includes(value.tags_scope)) throw new TypeError('Invalid inspection library');
// #332: an image's own category (scope "image") may sit on any link status; tags never have that scope.
![value.category_scope, value.tags_scope].every(scope => [null, 'pair', 'positive'].includes(scope))) throw new TypeError('Invalid inspection library');
// D39: each field names its own source, so a recipe's category can sit beside its positive's tags.
const library = { status: value.status, scope: value.scope, tags: [...value.tags], category_scope: value.category_scope, tags_scope: value.tags_scope };
for (const key of ['positive_id', 'pair_id', 'title', 'category']) {
+7 -16
View File
@@ -29,7 +29,7 @@ import { HOST_UNAVAILABLE } from './system-appearance.mjs';
import { openCloseOnlyReview, openReviewDialog } from './workspace-dialogs.mjs';
import { BACKUP_COPY, backupText, holding, reviewRefusal } from './maintenance-review.mjs';
import { busyToast, downloadDatabaseBackup } from './activity-scan.mjs';
import { MODEL_OPTIONS, installCommand, missingPackages, needsPackages, packageList } from './image-tagging-state.mjs';
import { MODEL_OPTIONS } from './image-tagging-state.mjs';
import { openImageTagging } from './image-tagging.mjs';
import { selectedImage } from './images.mjs';
import { NO_SELECTION } from './vocabulary.mjs';
@@ -80,7 +80,6 @@ export const STATUS_COPY = Object.freeze({
loaded: 'Downloaded and loaded.',
partial: 'Partly downloaded. Download again to finish.',
missing: size => `Not downloaded. Download size ${size}.`,
needs: (names, command, note) => `Downloaded. Needs ${names}: ${command}.${typeof note === 'string' && note.trim() ? ` ${note}` : ''}`,
});
/** The messages that name a setting the section cannot use (the server's own wording). */
@@ -141,21 +140,15 @@ export function modelOptions(models) {
/**
* D132: the Tagging model description for `status` (`{phase: loading|loaded|failed, models}`) and the chosen `model`.
* GGUF with only one of its two files on disk is partly downloaded. A backend package the server cannot import (#424)
* turns "Downloaded" into "Downloaded. Needs <pkg>: pip install <pkg>." and adds "Also needs <pkg>." otherwise.
* GGUF with only one of its two files on disk is partly downloaded.
*/
export function modelDescription(status, model) {
if (status?.phase === 'loading') return STATUS_COPY.loading;
const entry = status?.models?.[model];
if (status?.phase !== 'loaded' || !object(entry)) return STATUS_COPY.failed;
// #424: a backend package that cannot be imported; a model on disk without it never reads as ready.
const needs = missingPackages(entry).length ? packageList(entry) : '';
if (entry.downloaded === true) {
if (needs) return STATUS_COPY.needs(needs, installCommand(entry), entry.install_note);
return entry.loaded === true ? STATUS_COPY.loaded : STATUS_COPY.downloaded;
}
const files = partlyDownloaded(model, entry) ? STATUS_COPY.partial : STATUS_COPY.missing(modelSize(model, entry));
return needs ? `${files} Also needs ${needs}.` : files;
if (entry.downloaded === true) return entry.loaded === true ? STATUS_COPY.loaded : STATUS_COPY.downloaded;
if (partlyDownloaded(model, entry)) return STATUS_COPY.partial;
return STATUS_COPY.missing(modelSize(model, entry));
}
/** GGUF with only one of its two files on disk (a download that did not finish). */
@@ -323,7 +316,6 @@ export function diskShortNote(disk) {
export function batchCloseNote(plan) {
if (plan.review_required === true) return REPLACE_NOTE;
if (plan.model?.downloaded === false) return notDownloaded(modelLabel(plan.model.type, plan.model));
if (missingPackages(plan.model).length) return needsPackages(modelLabel(plan.model.type, plan.model), plan.model);
if (plan.disk?.sufficient === false) return diskShortNote(plan.disk);
if (!Number.isSafeInteger(plan.images) || plan.images === 0) return NO_IMAGES;
if (plan.to_tag === 0) return NOTHING_TO_TAG;
@@ -492,9 +484,8 @@ export function mountTagging(container, { api, activity, scope, drafts, notify =
unload.disabled = !(status.phase === 'loaded' && status.loadedType);
if (review.textContent === 'Reviewing…') return;
const batchProblem = known() ? draftProblem(values, { limit: false }) : unknown;
const label = modelLabel(values.autotag_model_type, entry);
setAvailable(review, entry?.downloaded === false ? notDownloaded(label)
: needsPackages(label, entry) || batchProblem?.message);
setAvailable(review, entry?.downloaded === false ? notDownloaded(modelLabel(values.autotag_model_type, entry))
: batchProblem?.message);
};
let reads = 0;