Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4fa67923b5 | ||
|
|
5393f9f1fb | ||
|
|
6439d8558b | ||
|
|
3112eea855 | ||
|
|
be73993cbe | ||
|
|
c0f7c4e33a | ||
|
|
d1e5042e2a | ||
|
|
5bc3bc8bf6 | ||
|
|
a941f71142 | ||
|
|
6559de8255 | ||
|
|
1b67cb7ccb | ||
|
|
49a6a5a2a8 | ||
|
|
bb0a8bf5ed |
@@ -27,15 +27,6 @@ concurrency:
|
||||
# coverage shards 7m34s/6m04s/5m22s (tests 6m38s/4m53s/4m04s), merge 54s, firefox 7m57s/5m21s (tests
|
||||
# 7m08s/4m34s), webkit 8m16s/5m03s (tests 7m26s/3m58s); a cache miss adds up to 5m to npm ci, which
|
||||
# the 20-minute limits absorb.
|
||||
# Hangs (#425): a job killed at its limit keeps no log and skips its upload steps, so each browser
|
||||
# test step has its own timeout-minutes (18) below the job's 20. At the step limit, SIGTERM makes
|
||||
# tools/vendor/running-tests-reporter.mjs print the tests still running and hand over to Playwright's
|
||||
# interrupt, which writes the blob report; the always() uploads then run.
|
||||
# Measured on main (runs 40725289094, 40719375442, 40700439399): setup before the test step <= 52s,
|
||||
# uploads <= 35s; test steps up to 14m42s (webkit shard 1/2) and 13m05s for coverage (shard 1/3:
|
||||
# 12m53s, node units and the viewer suite before 11.8m of workflows), so coverage is the tight one
|
||||
# in its 18 minutes. This holds only when the npm cache hits: a cold cache adds up to 5m to npm ci,
|
||||
# and then the job limit can still end the job before the step limit does.
|
||||
jobs:
|
||||
frontend-coverage:
|
||||
name: frontend coverage (shard ${{ matrix.shard }}/3)
|
||||
@@ -88,7 +79,6 @@ jobs:
|
||||
- if: matrix.shard == 3
|
||||
run: npm run --prefix tools/vendor test
|
||||
- name: Page workflows with coverage, shard ${{ matrix.shard }}/3
|
||||
timeout-minutes: 18
|
||||
run: npm run --prefix tools/vendor test:coverage:shard -- ${{ matrix.shard }}/3
|
||||
- name: Upload raw coverage
|
||||
uses: actions/upload-artifact@v7
|
||||
@@ -97,7 +87,6 @@ jobs:
|
||||
path: coverage/js/raw/
|
||||
if-no-files-found: error
|
||||
retention-days: 3
|
||||
# Traces and the blob report (test-results/blob/) of a failed, timed-out or cancelled step.
|
||||
- name: Upload Chromium and touch traces
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v7
|
||||
@@ -222,9 +211,7 @@ jobs:
|
||||
key: npm-${{ runner.os }}-playwright-v1.64.1-${{ hashFiles('tools/vendor/package-lock.json') }}
|
||||
- run: npm ci --prefix tools/vendor --ignore-scripts
|
||||
- name: Page workflows in ${{ matrix.browser }}, shard ${{ matrix.shard }}/2
|
||||
timeout-minutes: 18
|
||||
run: npm run --prefix tools/vendor test:workflows -- --project ${{ matrix.browser }} --shard ${{ matrix.shard }}/2
|
||||
# Traces and the blob report (test-results/blob/) of a failed, timed-out or cancelled step.
|
||||
- name: Upload browser traces
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v7
|
||||
|
||||
@@ -312,6 +312,12 @@ Both nodes support the same metadata fields:
|
||||
|
||||
Both nodes will automatically save prompts to the database and link any generated images to the prompt.
|
||||
|
||||
- **Save to library** (`save_to_library`, on by default; "Save to library" in the Focused, Split and Pair editors):
|
||||
turn it off while iterating and PromptManager ignores that node's runs. The prompt is not saved, the run is not
|
||||
counted and its images are not linked (a scan skips them too), while the node's output is unchanged. Workflows
|
||||
saved before this switch existed load with it on. If any PromptManager node feeding a sampler's positive input
|
||||
has it off, none of that run's images are linked, because the image cannot show which sampler produced it.
|
||||
|
||||
### 🖼️ Using the Output-Wide Gallery
|
||||
|
||||
The comprehensive gallery system provides complete access to all your ComfyUI output:
|
||||
|
||||
+5
-76
@@ -262,31 +262,6 @@ watchdog dump, is a failure like any other: read the dump, find the blocking
|
||||
call, fix it. Never raise a limit or rerun to get green; a limit changes only
|
||||
with new measurements written next to it.
|
||||
|
||||
Browser jobs (#425): GitHub keeps no log of a job killed at its limit and skips
|
||||
its upload steps, so the test step in each browser job in `viewerjs-check.yml`
|
||||
has its own `timeout-minutes` (18, below the job's 20). At the step limit
|
||||
GitHub sends SIGTERM, which would end Playwright without any report; under `CI`
|
||||
`tools/vendor/running-tests-reporter.mjs` catches the first one, prints
|
||||
`Still running when the run ended (SIGTERM): N` and one line per unfinished
|
||||
test (path relative to the suite's `rootDir`), then sends Playwright its own
|
||||
SIGINT, so the workers stop, the run ends interrupted and the blob report is
|
||||
written to `tools/vendor/test-results/blob/`. In the coverage job npm signals
|
||||
only `frontend-coverage.mjs`, which forwards SIGTERM and SIGINT to the child it
|
||||
is running (started in its own process group, so a group-wide signal still
|
||||
arrives once). Delivery is proven locally in the pinned image for a SIGTERM to
|
||||
the whole process group (`timeout -s TERM`). A SIGTERM to npm alone
|
||||
(`timeout --foreground`) reaches nothing: npm forwards it to the `sh -c` that
|
||||
runs the script, and that shell exits without passing it on. Delivery in CI,
|
||||
where the step runs through `docker exec`, is not yet observed; the next real
|
||||
step timeout will show it. That directory and the traces
|
||||
upload with `if: always()`; `npx playwright merge-reports` opens the blob.
|
||||
Each test keeps the 30 s default and there is no `globalTimeout`. Two limits
|
||||
of this: it assumes the npm cache hits (a cold cache adds up to 5 minutes to
|
||||
`npm ci`, and the job limit can then still win), and coverage shard 1 (12m53s
|
||||
measured) is the tight one in its 18-minute step. Local runs keep the `list`
|
||||
reporter only; reproduce with `CI=1 timeout -s TERM 40 npm run --prefix
|
||||
tools/vendor test:workflows -- --project=firefox`.
|
||||
|
||||
### After a push
|
||||
|
||||
Follow the run for the commit you pushed, never an older one:
|
||||
@@ -1369,7 +1344,7 @@ How the earlier workspace Images filters moved (old → new). The legacy `/galle
|
||||
| Folder | "Refine this view" → Folder (D14); shown in the scope chip |
|
||||
| Media (All media / Images / Videos) | "Refine this view" → Media (D14) |
|
||||
| Sort: Newest first / Oldest first / Filename A–Z / Filename Z–A | toolbar sort: Recently added / Oldest first / Name A–Z / Name Z–A, plus Highest rated (D13, D14) |
|
||||
| (none) | toolbar category (the effective category, D141) and "Refine this view" → Tags, through the linked positive (D13, D14) |
|
||||
| (none) | toolbar category and "Refine this view" → Tags, through the linked positive (D13, D14) |
|
||||
| Refresh | Import → "Scan an output folder" → Rescan (D15) |
|
||||
| Grid / List / Masonry, Comfortable / Large / Compact | the same choices in the concept filter row |
|
||||
| Card image → side panel; preview icon → viewer | image → viewer; Info → side panel (D19) |
|
||||
@@ -1452,8 +1427,7 @@ Recorded deviations on this page (spec §8):
|
||||
`overscroll-behavior: contain`.
|
||||
- **D35**: counts pluralize ("1 positive prompt", "1 image", "1 more image loaded").
|
||||
- **D36**: the category list is the Library's, sorted alphabetically (case-insensitive)
|
||||
by the page, whatever order the server sends (amended by D141: Images adds its
|
||||
image categories).
|
||||
by the page, whatever order the server sends.
|
||||
- **D37**: production loads 40 images per page (the concept 12), so the continuation
|
||||
and toast use 40; after Escape production returns focus to the opening image.
|
||||
- **D44** (#249 PR E): Filters reads "Filters · N" while N of Source, Folder, Media and
|
||||
@@ -1539,46 +1513,15 @@ Recorded deviations on this page (spec §8):
|
||||
- **D39** (ruled 2026-10-21: fallback): a recipe-linked image shows its recipe's own
|
||||
category when set, otherwise the linked positive prompt's, and its recipe's own tags
|
||||
when it has any, otherwise the positive's; each field falls back on its own and the
|
||||
two tag sets are never merged. The Images Tags filter matches the positive's tags.
|
||||
The category filter matches the same inherited category as the panel (D142, #332), and
|
||||
an image's own category overrides both (D141, D146). The inspection response names each
|
||||
two tag sets are never merged. Set category (#259) writes the positive's category and
|
||||
the Images category and Tags filters match the positive's, so the panel now agrees
|
||||
with them whenever the recipe adds nothing. The inspection response names each
|
||||
field's source (`library.category_scope`, `library.tags_scope`: `pair`, `positive` or
|
||||
null); the category chip's visually hidden prefix (D21) says "Recipe category: " or
|
||||
"Positive prompt category: " from it, and tag chips are Tags filter buttons only when
|
||||
they are the positive's (#260 D2). History's media inspector uses the same rule. The
|
||||
concept shows the prompt's category and tags for every image and the concept
|
||||
fixture's recipes have neither, so the inspector comparison needs no mapping for it.
|
||||
- **D141** (#332, amends D13 and D36): the Images category filter matches each image's
|
||||
effective category: its own category (Images Set category, PR 3) when it has one,
|
||||
else its inherited one (D142), else none. An image whose own category differs never
|
||||
matches its prompt's category, and an unlinked or ambiguous image with an own
|
||||
category is listed under it (the members-only gate now applies to the Tags filter
|
||||
only). "Select all N matching" and saved views use the same value. The caption's
|
||||
positive-prompt count still counts the linked positives of the images shown. With a
|
||||
category set the listing reads the own categories once per source (a primary-key
|
||||
range); a failed read is the existing 500 `listing-failed`, never an unfiltered page.
|
||||
The link read then keeps only links whose raw recipe or positive category is the
|
||||
value (a superset of those that inherit it; Python decides), and reads the links of
|
||||
files whose own category is the value by path, so their rating and positive count.
|
||||
`test_filter_budget` seeds 2,000 own categories on its 50,000-file fixture.
|
||||
The option list adds the image categories of listed files
|
||||
(`GET /prompt_manager/images/categories`, PR 3).
|
||||
- **D142** (#332 MC4, amends D39's filter note): the inherited category is one rule,
|
||||
`inherited_category` in `utils/inspection_response.py`: a linked recipe's own
|
||||
category when it is not blank, else the linked positive's. Before #332 the filter
|
||||
matched the positive's even when the chip showed the recipe's. The link reads select
|
||||
the raw recipe and positive categories and never decide inheritance in SQL
|
||||
(`tests/test_api_workspace_image_categories.py` checks the filter and the chip agree
|
||||
on blank values stored as spaces, tabs, U+00A0 and BLOB). The concept has no recipe
|
||||
categories, so this is behavior only.
|
||||
- **D145** (#332, DC4): History's media inspector shows an image's own category with
|
||||
the chip rule of the side panel and viewer. Only the inspection request reads it, once,
|
||||
inside the History snapshot: the file is the symlink-resolved output folder plus the
|
||||
receipt's subfolder and filename, keyed with `image_category_key` over the configured
|
||||
output sources, as Images keys it. A temp output or a file no source holds has none;
|
||||
a failed read hides the chip (DC5). The outputs list and media requests read nothing
|
||||
new. The chip's hidden prefix reads "Image category: " for an own category; the
|
||||
visible origin text and its comparison are D146 (PR 3).
|
||||
- **D42**: the concept never shows a toast and a page alert together. Production keeps
|
||||
the concept's fixed toast (centred, 22px from the bottom, 12px at 700px and below)
|
||||
and, when an Import → Rescan toast leaves the Images discovery alert
|
||||
@@ -5806,20 +5749,6 @@ rotates the epoch, so a pre-restore `expected_epoch` is refused with `storage-ch
|
||||
Owner rows and their timestamps never change on a tag command; a category apply only
|
||||
touches rows carrying the exact source.
|
||||
|
||||
Image categories (#332 §7, MC9, MC10, DC7): each `GET categories` row also carries
|
||||
`image_count`, the stored `image_categories` rows holding that exact value (files
|
||||
missing on disk included). A value prompts and images share appears once, ordered by
|
||||
its oldest positive; a value only images hold appears once after every prompt-backed
|
||||
value (`first_pid` 9223372036854775807 in the keyset cursor), ordered by value, with
|
||||
`prompt_count` 0. `/library/options` keeps its own query and never lists image-only
|
||||
values. Category preview and apply report `impact.images` (rows holding the source),
|
||||
the target exists when an image row holds it, an image-only value can be renamed, the
|
||||
apply renames the image rows in the same transaction (file keys and rowids kept, so a
|
||||
merge cannot collide), and the impact token hashes the sorted image rowids, so an image
|
||||
row gaining or losing the value between preview and apply is 409
|
||||
`vocabulary-conflict` with nothing written. Every image-category statement reads
|
||||
`idx_image_categories_category` (`test_category_reads_use_the_image_category_index`).
|
||||
|
||||
Two legacy routes change additively: `GET /tags` now lists unused registry tags, and
|
||||
`/tags/suggest` without context appends unused names at count 0. Unused tags become
|
||||
deletable only through the native Delete tag (vocabulary UI PR), not on the legacy
|
||||
|
||||
+60
-122
@@ -7,12 +7,7 @@ import json
|
||||
import math
|
||||
import datetime
|
||||
import os
|
||||
from typing import Any, Dict, Iterable, List, Optional, Sequence, Set, Tuple
|
||||
|
||||
try:
|
||||
from ..utils.inspection_response import inherited_category
|
||||
except ImportError:
|
||||
from utils.inspection_response import inherited_category
|
||||
from typing import Optional, List, Dict, Any, Set, Tuple
|
||||
|
||||
from .models import PromptModel, normalize_image_path, utc_now_iso
|
||||
from .pairs import PairDependencyError, PairOperations # noqa: F401 (re-exported)
|
||||
@@ -243,17 +238,13 @@ LEFT JOIN prompt_library_state pls ON pls.prompt_id = pp.positive_prompt_id
|
||||
WHERE gi.image_path IN ({marks}) OR gi.file_path IN ({marks}){fallback}"""
|
||||
# One image row describes a card only when no other row claims the same file
|
||||
# (as in get_image_link_summaries, several rows are ambiguous). The pair scope
|
||||
# and the linked positive follow _LINK_SUMMARY_SQL; rating, tags and the
|
||||
# searchable text and titles come from that positive (spec D12/D13). The raw
|
||||
# recipe and positive categories are read for the inherited category, which
|
||||
# Python decides (``inherited_category``, #332 MC4); SQL never filters by
|
||||
# category. Every
|
||||
# and the linked positive follow _LINK_SUMMARY_SQL; category, rating, tags and
|
||||
# the searchable text and titles come from that positive (spec D12/D13). Every
|
||||
# writer since the 3.2.4 migration stores file_path, so it is the path key.
|
||||
# The prompt-level conditions are uncorrelated IN sets SQLite builds once, and
|
||||
# the CASE evaluates the per-row ambiguity probe only for admitted rows.
|
||||
_LINK_FILTER_SQL = """SELECT gi.file_path AS file_path, p.rating AS rating,
|
||||
{hit} AS matched, p.id AS positive, pp.id IS NOT NULL AS has_pair,
|
||||
pp.category AS pair_category, p.category AS category
|
||||
{hit} AS matched, p.id AS positive
|
||||
FROM generated_images gi
|
||||
LEFT JOIN prompt_pairs pp ON pp.id = (
|
||||
SELECT CASE WHEN COUNT(DISTINCT ip.pair_id) = 1 THEN MIN(ip.pair_id) END
|
||||
@@ -265,28 +256,20 @@ WHERE gi.file_path IS NOT NULL AND CASE WHEN {admit} THEN NOT EXISTS (
|
||||
# _LINK_FILTER_SQL's rows as one JSON array, built by SQLite in one step (#354).
|
||||
# Python's sqlite3 releases and retakes the GIL around every step, so a read
|
||||
# that steps once per linked row waits on a busy ComfyUI thread thousands of
|
||||
# times. A BLOB value (no writer stores one) makes the value NULL (or
|
||||
# json_array refuses it), and the read streams instead. The raw categories are
|
||||
# put in only for a category read (#332), so plain reads stay as small as #354
|
||||
# measured them.
|
||||
# times. A BLOB file_path or rating (no writer stores one) makes the value NULL
|
||||
# (or json_array refuses it), and the read streams instead.
|
||||
_LINK_MATCHES_JSON_SQL = """SELECT CASE
|
||||
WHEN max({blob}) THEN NULL
|
||||
ELSE json_group_array(json_array({columns})) END
|
||||
WHEN max(typeof(file_path) = 'blob' OR typeof(rating) = 'blob') THEN NULL
|
||||
ELSE json_group_array(json_array(file_path, rating, matched, positive)) END
|
||||
FROM ({select})"""
|
||||
_LINK_PLAIN_COLUMNS = ("file_path", "rating", "matched", "positive")
|
||||
_LINK_CATEGORY_COLUMNS = _LINK_PLAIN_COLUMNS + ("has_pair", "pair_category", "category")
|
||||
# The columns a BLOB can reach: matched, positive and has_pair are computed.
|
||||
_LINK_STORED_COLUMNS = ("file_path", "rating", "pair_category", "category")
|
||||
# Set when this SQLite has no JSON functions: every later read streams.
|
||||
_one_step_unsupported = False
|
||||
# One warning per process when a read falls back to streaming.
|
||||
_one_step_warned = False
|
||||
# The linked positive of each given path (one unambiguous link, pair scope as
|
||||
# in _LINK_FILTER_SQL) with the raw categories of its inherited category; {hit}
|
||||
# as in link_filter_query. Used for the few files whose current image tags meet
|
||||
# a Tags filter (#249 B3).
|
||||
_LINK_FACTS_SQL = """SELECT gi.file_path, p.rating, {hit}, p.id,
|
||||
pp.id IS NOT NULL, pp.category, p.category
|
||||
# in _LINK_FILTER_SQL) with its category; {hit} as in link_filter_query. Used
|
||||
# for the few files whose current image tags meet a Tags filter (#249 B3).
|
||||
_LINK_FACTS_SQL = """SELECT gi.file_path, p.rating, {hit}, p.id, p.category
|
||||
FROM generated_images gi
|
||||
LEFT JOIN prompt_pairs pp ON pp.id = (
|
||||
SELECT CASE WHEN COUNT(DISTINCT ip.pair_id) = 1 THEN MIN(ip.pair_id) END
|
||||
@@ -315,24 +298,8 @@ _LINK_FILTER_TAG = (
|
||||
)
|
||||
|
||||
|
||||
# One unambiguously linked image's facts for the Images filters, a plain tuple
|
||||
# (hit, rating, positive id, inherited category): built per linked row, so no
|
||||
# NamedTuple. The category is the inherited one (#332 MC4: the recipe's when
|
||||
# set, else the positive's), never the image's own category.
|
||||
LinkMatch = Tuple[bool, Optional[int], int, Any]
|
||||
|
||||
|
||||
def _link_match(rating, matched, positive, has_pair, pair_category, category):
|
||||
inherited, _scope = inherited_category(category, pair_category, bool(has_pair))
|
||||
return (bool(matched), rating, positive, inherited)
|
||||
|
||||
|
||||
def link_filter_query(
|
||||
q: str,
|
||||
tags: List[str],
|
||||
*,
|
||||
every_link: bool = False,
|
||||
category: Optional[str] = None,
|
||||
q: str, category: Optional[str], tags: List[str], *, every_link: bool = False
|
||||
) -> Tuple[str, Dict[str, Any]]:
|
||||
"""The workspace filter SELECT and its named parameters (all bound).
|
||||
|
||||
@@ -346,9 +313,7 @@ def link_filter_query(
|
||||
params["pattern"] = f"%{escape_like(q)}%"
|
||||
admitted = []
|
||||
if category is not None:
|
||||
# Only narrows: an inherited category is one of the two raw values;
|
||||
# which one is decided in Python (inherited_category, #332 MC4).
|
||||
admitted.append("(p.category = :category OR pp.category = :category)")
|
||||
admitted.append("p.category = :category")
|
||||
params["category"] = category
|
||||
for index, tag in enumerate(tags):
|
||||
admitted.append(_LINK_FILTER_TAG.format(index=index))
|
||||
@@ -359,37 +324,9 @@ def link_filter_query(
|
||||
return sql, params
|
||||
|
||||
|
||||
def link_matches_json_query(sql: str, categories: bool = False) -> str:
|
||||
"""*sql* from link_filter_query, wrapped to return its rows as one JSON value.
|
||||
|
||||
With *categories* each row also carries the raw values the inherited
|
||||
category is decided from; otherwise only path, rating, hit and positive.
|
||||
"""
|
||||
columns = _LINK_CATEGORY_COLUMNS if categories else _LINK_PLAIN_COLUMNS
|
||||
blob = " OR ".join(
|
||||
f"typeof({name}) = 'blob'" for name in _LINK_STORED_COLUMNS if name in columns
|
||||
)
|
||||
return _LINK_MATCHES_JSON_SQL.format(
|
||||
blob=blob, columns=", ".join(columns), select=sql
|
||||
)
|
||||
|
||||
|
||||
def _link_matches_from_rows(
|
||||
rows: Iterable[Sequence[Any]], categories: bool, wanted: Optional[Set[str]]
|
||||
) -> Dict[str, LinkMatch]:
|
||||
"""LinkMatch per wanted path from link rows (one-step or streamed).
|
||||
|
||||
Without *categories* the inherited category is None, as the caller then
|
||||
never compares it.
|
||||
"""
|
||||
matches: Dict[str, LinkMatch] = {}
|
||||
for path, rating, matched, positive, *raw in rows:
|
||||
if wanted is None or path in wanted:
|
||||
if categories:
|
||||
matches[path] = _link_match(rating, matched, positive, *raw[:3])
|
||||
else:
|
||||
matches[path] = (bool(matched), rating, positive, None)
|
||||
return matches
|
||||
def link_matches_json_query(sql: str) -> str:
|
||||
"""*sql* from link_filter_query, wrapped to return its rows as one JSON value."""
|
||||
return _LINK_MATCHES_JSON_SQL.format(select=sql)
|
||||
|
||||
|
||||
def _streamed_link_matches(
|
||||
@@ -397,10 +334,13 @@ def _streamed_link_matches(
|
||||
sql: str,
|
||||
params: Dict[str, Any],
|
||||
wanted: Optional[Set[str]],
|
||||
categories: bool = False,
|
||||
) -> Dict[str, LinkMatch]:
|
||||
) -> Dict[str, Tuple[bool, Optional[int], int]]:
|
||||
"""The link read stepped once per row (the fallback of the one-step read)."""
|
||||
return _link_matches_from_rows(conn.execute(sql, params), categories, wanted)
|
||||
matches: Dict[str, Tuple[bool, Optional[int], int]] = {}
|
||||
for path, rating, matched, positive in conn.execute(sql, params):
|
||||
if wanted is None or path in wanted:
|
||||
matches[path] = (bool(matched), rating, positive)
|
||||
return matches
|
||||
|
||||
|
||||
_ASCII_FOLD = str.maketrans("ABCDEFGHIJKLMNOPQRSTUVWXYZ", "abcdefghijklmnopqrstuvwxyz")
|
||||
@@ -2636,55 +2576,47 @@ class PromptDatabase(
|
||||
def get_image_link_matches(
|
||||
self,
|
||||
q: str,
|
||||
category: Optional[str],
|
||||
tags: List[str],
|
||||
*,
|
||||
every_link: bool = False,
|
||||
wanted: Optional[Set[str]] = None,
|
||||
category: Optional[str] = None,
|
||||
) -> Dict[str, LinkMatch]:
|
||||
"""Map normalized paths of unambiguously linked images to a LinkMatch
|
||||
(hit, rating, positive id, inherited category).
|
||||
) -> Dict[str, Tuple[bool, Optional[int], int]]:
|
||||
"""Map normalized paths of unambiguously linked images to
|
||||
(hit, rating, positive id).
|
||||
|
||||
Rows are limited to images whose linked positive has every tag in
|
||||
*tags* (exact names). The category filter is applied by the caller to
|
||||
the effective category (#332). ``hit`` says whether *q* occurs in
|
||||
Rows are limited to images whose linked positive has *category* and
|
||||
every tag in *tags* (exact names). ``hit`` says whether *q* occurs in
|
||||
the positive's text, Library title or a tag name; a pair's own title
|
||||
names its negative variant and is not searched.
|
||||
Without tags only hits are returned, unless *every_link*
|
||||
Without category or tags only hits are returned, unless *every_link*
|
||||
asks for every linked image (the rating sort and the positive count).
|
||||
The positive id is the card's linked positive. With *category* only
|
||||
links whose raw recipe or positive category is that value are read (a
|
||||
superset of the links that inherit it; the caller compares the
|
||||
inherited value) and the inherited category is decided per row;
|
||||
otherwise it is None, which keeps the plain reads cheap.
|
||||
The positive id is the card's linked positive.
|
||||
|
||||
One SELECT, returned as one JSON value (#354): SQLite steps once, so
|
||||
the read takes the GIL back a constant two or three times instead of
|
||||
once per linked row while ComfyUI generates. Peak memory therefore
|
||||
grows with the database's matching linked rows (about 530 bytes each,
|
||||
more with the raw categories of a category read, freed on return); the
|
||||
result is bounded by *wanted* (the listing's normalized paths) when
|
||||
given. Where the JSON step fails (no JSON functions, a BLOB value, a
|
||||
JSON text over SQLite's length limit) the rows are streamed as before.
|
||||
freed on return); the result is bounded by *wanted* (the listing's
|
||||
normalized paths) when given. Where the JSON step fails (no JSON
|
||||
functions, a BLOB value, a JSON text over SQLite's length limit) the
|
||||
rows are streamed as before.
|
||||
"""
|
||||
sql, params = link_filter_query(
|
||||
q, tags, every_link=every_link, category=category
|
||||
)
|
||||
categories = category is not None
|
||||
sql, params = link_filter_query(q, category, tags, every_link=every_link)
|
||||
with self.model.get_connection() as conn:
|
||||
rows = None
|
||||
if not _one_step_unsupported:
|
||||
rows = self._one_step_link_rows(conn, sql, params, categories)
|
||||
rows = self._one_step_link_rows(conn, sql, params)
|
||||
if rows is None:
|
||||
return _streamed_link_matches(conn, sql, params, wanted, categories)
|
||||
return _link_matches_from_rows(rows, categories, wanted)
|
||||
return _streamed_link_matches(conn, sql, params, wanted)
|
||||
return {
|
||||
path: (bool(matched), rating, positive)
|
||||
for path, rating, matched, positive in rows
|
||||
if wanted is None or path in wanted
|
||||
}
|
||||
|
||||
def _one_step_link_rows(
|
||||
self,
|
||||
conn: sqlite3.Connection,
|
||||
sql: str,
|
||||
params: Dict[str, Any],
|
||||
categories: bool = False,
|
||||
self, conn: sqlite3.Connection, sql: str, params: Dict[str, Any]
|
||||
) -> Optional[List[List[Any]]]:
|
||||
"""*sql*'s rows from one JSON value, or None when this read must stream.
|
||||
|
||||
@@ -2694,7 +2626,7 @@ class PromptDatabase(
|
||||
global _one_step_unsupported, _one_step_warned
|
||||
error: Optional[str] = None
|
||||
try:
|
||||
cursor = conn.execute(link_matches_json_query(sql, categories), params)
|
||||
cursor = conn.execute(link_matches_json_query(sql), params)
|
||||
try:
|
||||
((document,),) = cursor.fetchall()
|
||||
finally:
|
||||
@@ -2721,9 +2653,9 @@ class PromptDatabase(
|
||||
|
||||
def get_image_link_facts(
|
||||
self, paths: List[str], q: str = ""
|
||||
) -> Dict[str, Tuple[bool, Optional[int], int, Any, Tuple[str, ...]]]:
|
||||
) -> Dict[str, Tuple[bool, Optional[int], int, Optional[str], Tuple[str, ...]]]:
|
||||
"""Map each normalized path with one unambiguous link to its positive's
|
||||
(hit, rating, positive id, inherited category, tag names).
|
||||
(hit, rating, positive id, category, tag names).
|
||||
|
||||
The link is resolved as in get_image_link_matches, and ``hit`` says
|
||||
whether *q* occurs in the positive's text, Library title or a tag
|
||||
@@ -2734,7 +2666,7 @@ class PromptDatabase(
|
||||
if q:
|
||||
hit = _LINK_FILTER_HIT
|
||||
params["pattern"] = f"%{escape_like(q)}%"
|
||||
rows: Dict[str, LinkMatch] = {}
|
||||
rows: Dict[str, Tuple[bool, Optional[int], int, Optional[str]]] = {}
|
||||
names: Dict[int, List[str]] = {}
|
||||
unique = list(dict.fromkeys(paths))
|
||||
with self.model.get_connection() as conn:
|
||||
@@ -2743,8 +2675,10 @@ class PromptDatabase(
|
||||
bound = {f"path{i}": path for i, path in enumerate(chunk)}
|
||||
marks = ",".join(f":{key}" for key in bound)
|
||||
sql = _LINK_FACTS_SQL.format(hit=hit, marks=marks)
|
||||
for path, *row in conn.execute(sql, {**params, **bound}):
|
||||
rows[path] = _link_match(*row)
|
||||
for path, rating, matched, positive, category in conn.execute(
|
||||
sql, {**params, **bound}
|
||||
):
|
||||
rows[path] = (bool(matched), rating, positive, category)
|
||||
positives = sorted({row[2] for row in rows.values()})
|
||||
for start in range(0, len(positives), _FACTS_CHUNK):
|
||||
chunk = positives[start : start + _FACTS_CHUNK]
|
||||
@@ -2757,16 +2691,20 @@ class PromptDatabase(
|
||||
path: (*row, tuple(names.get(row[2], ()))) for path, row in rows.items()
|
||||
}
|
||||
|
||||
def get_image_paths_matching(self, q: str, tags: List[str]) -> Set[str]:
|
||||
"""Normalized paths whose linked positive matches *q* and *tags*.
|
||||
def get_image_paths_matching(
|
||||
self, q: str, category: Optional[str], tags: List[str]
|
||||
) -> Set[str]:
|
||||
"""Normalized paths whose linked positive matches *q*, *category* and *tags*.
|
||||
|
||||
See get_image_link_matches; an empty *q* matches every linked image
|
||||
the tags admit.
|
||||
the category and tags admit.
|
||||
"""
|
||||
return {
|
||||
path
|
||||
for path, match in self.get_image_link_matches(q, tags).items()
|
||||
if match[0] or not q
|
||||
for path, (hit, _rating, _positive) in self.get_image_link_matches(
|
||||
q, category, tags
|
||||
).items()
|
||||
if hit or not q
|
||||
}
|
||||
|
||||
def get_prompt_id_for_image(self, image_path: str) -> Optional[int]:
|
||||
|
||||
+15
-55
@@ -13,9 +13,7 @@ same connection before commit. Owner rows and their timestamps never change.
|
||||
|
||||
Category consolidation is previewed in one ``library_snapshot`` and applied in
|
||||
one ``_pair_transaction`` that recomputes the owner projection and requires the
|
||||
preview's ``impact_token``; only rows carrying the exact source change. Image
|
||||
rows holding the source (``image_categories``, #332) are owners too: they are
|
||||
counted, hashed into the token and renamed with the prompts and pairs.
|
||||
preview's ``impact_token``; only rows carrying the exact source change.
|
||||
"""
|
||||
|
||||
import sqlite3
|
||||
@@ -99,30 +97,16 @@ def _validate_tag_id(tag_id: object) -> None:
|
||||
raise ValueError("Invalid tag ID")
|
||||
|
||||
|
||||
# Category values in Tags & categories (#332 §7): the prompt-backed values
|
||||
# (nonblank values on eligible parents or on active pairs, grouped exactly, so
|
||||
# whitespace and case variants stay distinct; /library/options lists the same
|
||||
# prompt-backed set with its own query) plus the values only image rows hold.
|
||||
# A prompt-backed value is ordered by its oldest owning positive; an image-only
|
||||
# value sorts after all of them (first_pid is SQLite's INTEGER maximum, above
|
||||
# any prompt id), then by value. ``image_count`` counts stored rows, including
|
||||
# files missing on disk. No FULL OUTER JOIN, so it runs on SQLite 3.31+.
|
||||
_IMAGE_ONLY_FIRST_PID = 2**63 - 1
|
||||
_CATEGORY_ROWS = (
|
||||
# The /library/options universe: nonblank values on eligible parents or on
|
||||
# active pairs, grouped exactly (whitespace and case variants stay distinct).
|
||||
_CATEGORY_GROUPS = (
|
||||
"WITH owners(value,pid) AS ("
|
||||
"SELECT p.category,p.id FROM prompts p WHERE p.category IS NOT NULL "
|
||||
"AND TRIM(p.category)!='' AND " + _ELIGIBLE + " "
|
||||
"UNION SELECT pp.category,pp.positive_prompt_id FROM prompt_pairs pp "
|
||||
"WHERE pp.archived=0 AND pp.category IS NOT NULL AND TRIM(pp.category)!=''),"
|
||||
"image_values(value,image_count) AS ("
|
||||
"SELECT category,COUNT(*) FROM image_categories GROUP BY category),"
|
||||
"grouped(value,first_pid,prompt_count) AS ("
|
||||
"SELECT value,MIN(pid),COUNT(DISTINCT pid) FROM owners GROUP BY value "
|
||||
"UNION ALL SELECT value," + str(_IMAGE_ONLY_FIRST_PID) + ",0 FROM image_values "
|
||||
"WHERE value NOT IN (SELECT value FROM owners)),"
|
||||
"counted AS (SELECT g.value,g.first_pid,g.prompt_count,"
|
||||
"COALESCE(i.image_count,0) AS image_count FROM grouped g "
|
||||
"LEFT JOIN image_values i ON i.value=g.value) "
|
||||
"grouped AS (SELECT value,MIN(pid) AS first_pid,"
|
||||
"COUNT(DISTINCT pid) AS prompt_count FROM owners GROUP BY value) "
|
||||
)
|
||||
|
||||
|
||||
@@ -133,23 +117,15 @@ _CATEGORY_COMPONENTS = (
|
||||
"WHERE p.category=? ORDER BY p.id"
|
||||
)
|
||||
_CATEGORY_PAIRS = "SELECT id,archived FROM prompt_pairs WHERE category=? ORDER BY id"
|
||||
# Image rows holding one exact value (#332 DC7), an index search.
|
||||
_IMAGE_CATEGORY_ROWIDS = (
|
||||
"SELECT rowid FROM image_categories WHERE category=? ORDER BY rowid"
|
||||
)
|
||||
_CATEGORY_EXISTS = (
|
||||
"SELECT EXISTS(SELECT 1 FROM prompts WHERE category=:v)"
|
||||
" OR EXISTS(SELECT 1 FROM prompt_pairs WHERE category=:v)"
|
||||
" OR EXISTS(SELECT 1 FROM image_categories WHERE category=:v)"
|
||||
)
|
||||
|
||||
# Pairs after components and image rows last, so a failing update rolls every
|
||||
# earlier one back. An image row is keyed by its file, never by its value, so
|
||||
# merging into a value other files already hold cannot conflict.
|
||||
# Pairs after components, so a failing pair update rolls both back.
|
||||
_CATEGORY_UPDATES = (
|
||||
"UPDATE prompts SET category=?,updated_at=? WHERE category=?",
|
||||
"UPDATE prompt_pairs SET category=?,updated_at=? WHERE category=?",
|
||||
"UPDATE image_categories SET category=?,updated_at=? WHERE category=?",
|
||||
)
|
||||
|
||||
|
||||
@@ -160,15 +136,8 @@ def category_impact_token(
|
||||
target_exists: bool,
|
||||
components: list,
|
||||
pairs: list,
|
||||
*,
|
||||
images: list = (),
|
||||
) -> str:
|
||||
"""Hash a category projection like ``library_edit_token`` (no secret).
|
||||
|
||||
``images`` are the rowids of the image rows holding ``source`` (#332 DC7).
|
||||
With none the preimage has no ``images`` key, so a prompt-only token keeps
|
||||
the value it had before image categories existed.
|
||||
"""
|
||||
"""Hash a category projection like ``library_edit_token`` (no secret)."""
|
||||
preimage = {
|
||||
"epoch": epoch,
|
||||
"source": source,
|
||||
@@ -177,8 +146,6 @@ def category_impact_token(
|
||||
"components": sorted([int(i), bool(flag)] for i, flag in components),
|
||||
"pairs": sorted([int(i), bool(flag)] for i, flag in pairs),
|
||||
}
|
||||
if images:
|
||||
preimage["images"] = sorted(int(i) for i in images)
|
||||
return canonical_token(_CATEGORY_DOMAIN, preimage)
|
||||
|
||||
|
||||
@@ -358,8 +325,7 @@ class VocabularyOperations:
|
||||
pairs = [
|
||||
[row[0], bool(row[1])] for row in conn.execute(_CATEGORY_PAIRS, (source,))
|
||||
]
|
||||
images = [row[0] for row in conn.execute(_IMAGE_CATEGORY_ROWIDS, (source,))]
|
||||
if not components and not pairs and not images:
|
||||
if not components and not pairs:
|
||||
raise LibraryCommandError("vocabulary-conflict", epoch)
|
||||
target_exists = bool(
|
||||
conn.execute(_CATEGORY_EXISTS, {"v": target}).fetchone()[0]
|
||||
@@ -373,10 +339,9 @@ class VocabularyOperations:
|
||||
"other_prompts": len(components) - eligible,
|
||||
"negative_variants": len(pairs) - archived,
|
||||
"archived_negative_variants": archived,
|
||||
"images": len(images),
|
||||
},
|
||||
"impact_token": category_impact_token(
|
||||
epoch, source, target, target_exists, components, pairs, images=images
|
||||
epoch, source, target, target_exists, components, pairs
|
||||
),
|
||||
}
|
||||
|
||||
@@ -400,8 +365,7 @@ class VocabularyOperations:
|
||||
def apply_vocabulary_category(
|
||||
self, *, expected_epoch: str, source: str, target: str, impact_token: str
|
||||
) -> dict:
|
||||
"""Set the exact ``source`` to ``target`` on every component, pair
|
||||
and image row.
|
||||
"""Set the exact ``source`` to ``target`` on every component and pair.
|
||||
|
||||
The recomputed projection must still match the preview's token; a
|
||||
mismatch is ``vocabulary-conflict`` with nothing written.
|
||||
@@ -458,15 +422,15 @@ class VocabularyOperations:
|
||||
_limit(limit)
|
||||
key = _category_after(after)
|
||||
total = conn.execute(
|
||||
_CATEGORY_ROWS + "SELECT COUNT(*) FROM counted"
|
||||
_CATEGORY_GROUPS + "SELECT COUNT(*) FROM grouped"
|
||||
).fetchone()[0]
|
||||
where, params = "", []
|
||||
if key is not None:
|
||||
where = " WHERE first_pid>? OR (first_pid=? AND value>?)"
|
||||
params = [key[0], key[0], key[1]]
|
||||
rows = conn.execute(
|
||||
_CATEGORY_ROWS
|
||||
+ "SELECT value,first_pid,prompt_count,image_count FROM counted"
|
||||
_CATEGORY_GROUPS
|
||||
+ "SELECT value,first_pid,prompt_count FROM grouped"
|
||||
+ where
|
||||
+ " ORDER BY first_pid,value LIMIT ?",
|
||||
[*params, limit + 1],
|
||||
@@ -476,11 +440,7 @@ class VocabularyOperations:
|
||||
return {
|
||||
"library_epoch": epoch,
|
||||
"categories": [
|
||||
{
|
||||
"value": row["value"],
|
||||
"prompt_count": row["prompt_count"],
|
||||
"image_count": row["image_count"],
|
||||
}
|
||||
{"value": row["value"], "prompt_count": row["prompt_count"]}
|
||||
for row in rows
|
||||
],
|
||||
"total": total,
|
||||
|
||||
+13
-6
@@ -26,13 +26,13 @@ except ImportError:
|
||||
InputTypeDict = dict
|
||||
|
||||
try:
|
||||
from .prompt_manager_base import PromptManagerBase
|
||||
from .prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
|
||||
except ImportError:
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from prompt_manager_base import PromptManagerBase
|
||||
from prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
|
||||
|
||||
|
||||
class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
@@ -115,6 +115,7 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
)
|
||||
},
|
||||
),
|
||||
"save_to_library": SAVE_TO_LIBRARY_INPUT,
|
||||
},
|
||||
"hidden": {
|
||||
# The queued graph and this node's id: used to tell the positive
|
||||
@@ -152,6 +153,7 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
append_text: str = "",
|
||||
prompt: Optional[Dict[str, Any]] = None,
|
||||
unique_id: Optional[str] = None,
|
||||
save_to_library: bool = True,
|
||||
) -> Tuple[Any]:
|
||||
"""
|
||||
Encode the text prompt and save it to the database.
|
||||
@@ -164,6 +166,7 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
search_text: Text to search for in past prompts
|
||||
prepend_text: Text to prepend to the main prompt
|
||||
append_text: Text to append to the main prompt
|
||||
save_to_library: Off: nothing is saved, counted or tracked (#90)
|
||||
|
||||
Returns:
|
||||
Tuple containing the conditioning for the diffusion model and the final
|
||||
@@ -205,7 +208,9 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
# Save prompt to database and set execution context for gallery tracking
|
||||
prompt_id = None
|
||||
extended_tags = []
|
||||
if storage_text and storage_text.strip():
|
||||
if not save_to_library:
|
||||
self._skip_not_saved(prompt, unique_id)
|
||||
elif storage_text and storage_text.strip():
|
||||
self.logger.debug(f"Processing prompt text: {storage_text[:100]}...")
|
||||
|
||||
# Add prepend/append info to tags if they exist
|
||||
@@ -249,6 +254,11 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
tokens = clip.tokenize(encoding_text)
|
||||
conditioning = clip.encode_from_tokens_scheduled(tokens)
|
||||
|
||||
self.logger.info(f"CLIP encoding completed, text: {repr(encoding_text)[:80]}")
|
||||
result = (conditioning, encoding_text)
|
||||
if not save_to_library:
|
||||
return result # #90: no metadata registration, no history text
|
||||
|
||||
# Register with ComfyUI integration for standard metadata compatibility
|
||||
node_id = f"promptmanager_{int(time.time() * 1000)}"
|
||||
self.comfyui_integration.register_prompt(
|
||||
@@ -262,9 +272,6 @@ class PromptManager(PromptManagerBase, ComfyNodeABC):
|
||||
"append_text": append_text.strip() if append_text else None,
|
||||
},
|
||||
)
|
||||
|
||||
self.logger.info(f"CLIP encoding completed, text: {repr(encoding_text)[:80]}")
|
||||
result = (conditioning, encoding_text)
|
||||
record_runtime_context(
|
||||
type(self).__name__, text, prepend_text, append_text, encoding_text
|
||||
)
|
||||
|
||||
+38
-2
@@ -23,7 +23,7 @@ try:
|
||||
from .utils.image_monitor import get_image_monitor
|
||||
from .utils.prompt_tracker import get_prompt_tracker
|
||||
from .utils.hashing import generate_prompt_hash
|
||||
from .utils.prompt_graph import resolve_text, run_prompt_nodes
|
||||
from .utils.prompt_graph import resolve_text, run_prompt_nodes, save_setting
|
||||
from .utils.run_keys import executing_run_id, prompt_node_key
|
||||
from .utils.usage_tracking import PENDING_FIRST_USE
|
||||
from .utils.cached_runs import (
|
||||
@@ -42,7 +42,7 @@ except ImportError:
|
||||
from utils.image_monitor import get_image_monitor
|
||||
from utils.prompt_tracker import get_prompt_tracker
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.prompt_graph import resolve_text, run_prompt_nodes
|
||||
from utils.prompt_graph import resolve_text, run_prompt_nodes, save_setting
|
||||
from utils.run_keys import executing_run_id, prompt_node_key
|
||||
from utils.usage_tracking import PENDING_FIRST_USE
|
||||
from utils.cached_runs import (
|
||||
@@ -54,6 +54,22 @@ except ImportError:
|
||||
)
|
||||
|
||||
|
||||
# The optional input every PromptManager node declares last (#90): declared
|
||||
# last so no widget value of a workflow saved before it shifts.
|
||||
SAVE_TO_LIBRARY_INPUT = (
|
||||
"BOOLEAN",
|
||||
{
|
||||
"default": True,
|
||||
"label_on": "save",
|
||||
"label_off": "skip",
|
||||
"tooltip": (
|
||||
"Save this node's prompt to the library, count its runs and link its "
|
||||
"images. Off: PromptManager ignores the run; the output is unchanged."
|
||||
),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
class PromptManagerBase:
|
||||
"""Mixin providing shared prompt management logic for ComfyUI nodes.
|
||||
|
||||
@@ -156,7 +172,9 @@ class PromptManagerBase:
|
||||
resolved = resolve_text(prompt_graph, node_id)
|
||||
return {
|
||||
"is_run_prompt": node_id in run_prompt_nodes(prompt_graph),
|
||||
# The hook skips a switch it cannot read (#90): the node counts
|
||||
"seen_at_queue": resolved is not None
|
||||
and save_setting(prompt_graph, node_id) is True
|
||||
and generate_prompt_hash(resolved) == generate_prompt_hash(text),
|
||||
"known": True,
|
||||
}
|
||||
@@ -245,6 +263,24 @@ class PromptManagerBase:
|
||||
"run_id": executing_run_id(),
|
||||
}
|
||||
|
||||
def _skip_not_saved(self, prompt_graph: Any = None, unique_id: Any = None) -> None:
|
||||
"""``save_to_library`` is off: nothing is saved, counted or tracked (#90).
|
||||
|
||||
A literal off makes the node no run prompt (``utils.prompt_graph``), so
|
||||
no image looks up its run key. A switch wired from a node the graph
|
||||
cannot read keeps the node a run prompt: its run key is marked like an
|
||||
empty editor's, so an earlier run's entry of that key never links this
|
||||
run's images.
|
||||
"""
|
||||
message = (
|
||||
"Save to library is off for this node: its prompt is not saved, the "
|
||||
"run is not counted and its images are not linked"
|
||||
)
|
||||
if isinstance(prompt_graph, dict) and unique_id is not None:
|
||||
self._mark_untracked(prompt_graph, unique_id, message)
|
||||
else:
|
||||
self.logger.info(message)
|
||||
|
||||
def _skip_empty_prompt(self, prompt_graph: Any, unique_id: Any) -> None:
|
||||
"""The ``text`` editor is empty: save and track nothing (#342).
|
||||
|
||||
|
||||
+23
-4
@@ -22,7 +22,11 @@ try:
|
||||
)
|
||||
from .utils.hashing import generate_prompt_hash
|
||||
from .utils.pair_execution import record_pair_execution
|
||||
from .utils.prompt_graph import intrinsic_pair_nodes, run_prompt_components
|
||||
from .utils.prompt_graph import (
|
||||
intrinsic_pair_nodes,
|
||||
run_prompt_components,
|
||||
save_setting,
|
||||
)
|
||||
from .utils.prompt_pair_resolver import resolve_text_input
|
||||
from .utils.usage_tracking import (
|
||||
PENDING_FIRST_USE,
|
||||
@@ -41,7 +45,11 @@ except ImportError:
|
||||
)
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.pair_execution import record_pair_execution
|
||||
from utils.prompt_graph import intrinsic_pair_nodes, run_prompt_components
|
||||
from utils.prompt_graph import (
|
||||
intrinsic_pair_nodes,
|
||||
run_prompt_components,
|
||||
save_setting,
|
||||
)
|
||||
from utils.prompt_pair_resolver import resolve_text_input
|
||||
from utils.usage_tracking import (
|
||||
PENDING_FIRST_USE,
|
||||
@@ -90,8 +98,13 @@ class PromptManagerPair(PromptManagerFocused):
|
||||
prompt: Optional[Dict[str, Any]] = None,
|
||||
unique_id: Optional[str] = None,
|
||||
extra_pnginfo: Optional[Dict[str, Any]] = None,
|
||||
save_to_library: bool = True,
|
||||
) -> Tuple[Any, Any, str, str]:
|
||||
"""Encode both sides once; database and tracking failures never block CLIP."""
|
||||
"""Encode both sides once; database and tracking failures never block CLIP.
|
||||
|
||||
With ``save_to_library`` off nothing is saved, counted, tracked or
|
||||
recorded (#90); the four outputs are the same.
|
||||
"""
|
||||
if clip is None:
|
||||
raise RuntimeError("ERROR: clip input is invalid: None")
|
||||
if not isinstance(text, str) or not isinstance(negative_text, str):
|
||||
@@ -109,6 +122,10 @@ class PromptManagerPair(PromptManagerFocused):
|
||||
negative_conditioning = clip.encode_from_tokens_scheduled(
|
||||
clip.tokenize(negative_text)
|
||||
)
|
||||
result = positive_conditioning, negative_conditioning, positive, negative_text
|
||||
if not save_to_library:
|
||||
self._skip_not_saved(prompt, unique_id)
|
||||
return result
|
||||
record_pair_execution(
|
||||
extra_pnginfo,
|
||||
unique_id,
|
||||
@@ -169,7 +186,6 @@ class PromptManagerPair(PromptManagerFocused):
|
||||
)
|
||||
except Exception as error:
|
||||
self.logger.warning(f"Could not register paired metadata: {error}")
|
||||
result = positive_conditioning, negative_conditioning, positive, negative_text
|
||||
record_runtime_context(
|
||||
type(self).__name__,
|
||||
text,
|
||||
@@ -219,6 +235,9 @@ class PromptManagerPair(PromptManagerFocused):
|
||||
resolved_negative = (
|
||||
resolve_text_input(prompt, node_id, "negative_text") if known else None
|
||||
)
|
||||
if known and save_setting(prompt, node_id) is not True:
|
||||
# The hook skips a switch it cannot read (#90): the node counts
|
||||
resolved_positive = resolved_negative = None
|
||||
positive_seen = resolved_positive is not None and generate_prompt_hash(
|
||||
resolved_positive
|
||||
) == generate_prompt_hash(text)
|
||||
|
||||
+13
-6
@@ -26,13 +26,13 @@ except ImportError:
|
||||
InputTypeDict = dict
|
||||
|
||||
try:
|
||||
from .prompt_manager_base import PromptManagerBase
|
||||
from .prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
|
||||
except ImportError:
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
||||
from prompt_manager_base import PromptManagerBase
|
||||
from prompt_manager_base import SAVE_TO_LIBRARY_INPUT, PromptManagerBase
|
||||
|
||||
|
||||
class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
@@ -115,6 +115,7 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
),
|
||||
},
|
||||
),
|
||||
"save_to_library": SAVE_TO_LIBRARY_INPUT,
|
||||
},
|
||||
"hidden": {
|
||||
# The queued graph and this node's id: used to tell the positive
|
||||
@@ -149,6 +150,7 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
append_text: str = "",
|
||||
prompt: Optional[Dict[str, Any]] = None,
|
||||
unique_id: Optional[str] = None,
|
||||
save_to_library: bool = True,
|
||||
) -> Tuple[str]:
|
||||
"""
|
||||
Process the text prompt and save it to the database.
|
||||
@@ -160,6 +162,7 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
search_text: Text to search for in past prompts
|
||||
prepend_text: Text to prepend to the main prompt
|
||||
append_text: Text to append to the main prompt
|
||||
save_to_library: Off: nothing is saved, counted or tracked (#90)
|
||||
|
||||
Returns:
|
||||
Tuple containing the final processed text string
|
||||
@@ -184,7 +187,9 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
# Save prompt to database and set execution context for gallery tracking
|
||||
prompt_id = None
|
||||
extended_tags = []
|
||||
if storage_text and storage_text.strip():
|
||||
if not save_to_library:
|
||||
self._skip_not_saved(prompt, unique_id)
|
||||
elif storage_text and storage_text.strip():
|
||||
self.logger.debug(f"Processing prompt text: {storage_text[:100]}...")
|
||||
|
||||
# Add prepend/append info to tags if they exist
|
||||
@@ -220,6 +225,11 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
else:
|
||||
self._skip_empty_prompt(prompt, unique_id)
|
||||
|
||||
self.logger.debug(f"Text processing completed: {final_text[:100]}...")
|
||||
result = (final_text,)
|
||||
if not save_to_library:
|
||||
return result # #90: no metadata registration, no history text
|
||||
|
||||
# Register with ComfyUI integration for standard metadata compatibility
|
||||
node_id = f"promptmanagertext_{int(time.time() * 1000)}"
|
||||
self.comfyui_integration.register_prompt(
|
||||
@@ -233,9 +243,6 @@ class PromptManagerText(PromptManagerBase, ComfyNodeABC):
|
||||
"append_text": append_text.strip() if append_text else None,
|
||||
},
|
||||
)
|
||||
|
||||
self.logger.debug(f"Text processing completed: {final_text[:100]}...")
|
||||
result = (final_text,)
|
||||
record_runtime_context(
|
||||
type(self).__name__, text, prepend_text, append_text, final_text
|
||||
)
|
||||
|
||||
@@ -52,6 +52,7 @@ try:
|
||||
extract_generation_params,
|
||||
parse_page_json,
|
||||
)
|
||||
from ...utils.prompt_graph import metadata_opted_out, side_opted_out
|
||||
from ...utils.prompt_pair_resolver import resolve_prompt_pair, sampler_less_prompt
|
||||
from ...utils.review_capture import reviewable
|
||||
except ImportError:
|
||||
@@ -67,6 +68,7 @@ except ImportError:
|
||||
extract_generation_params,
|
||||
parse_page_json,
|
||||
)
|
||||
from utils.prompt_graph import metadata_opted_out, side_opted_out
|
||||
from utils.prompt_pair_resolver import resolve_prompt_pair, sampler_less_prompt
|
||||
from utils.review_capture import reviewable
|
||||
|
||||
@@ -260,6 +262,11 @@ SCAN_DETAILS = {
|
||||
"failed": "Error: the prompt could not be stored",
|
||||
"error": "Error: the file could not be processed (see the server log)",
|
||||
"found": "Metadata found, but no prompt text to store",
|
||||
"not_saved": "Skipped · its PromptManager node does not save to the library",
|
||||
"not_saved_wired": (
|
||||
"Skipped · its Save to library switch is wired from another node, so only "
|
||||
"the run itself can link it"
|
||||
),
|
||||
None: "No prompt metadata in the file",
|
||||
}
|
||||
|
||||
@@ -1720,10 +1727,34 @@ class AdminRoutesMixin:
|
||||
except (ValueError, TypeError):
|
||||
# Preserve malformed metadata for conservative resolver rejection.
|
||||
pass
|
||||
if metadata_opted_out(canonical):
|
||||
# Save to library was off for the run's prompt: no prompt, link,
|
||||
# recorded-run fallback or Needs review item (#90)
|
||||
return "not_saved"
|
||||
if metadata_opted_out(canonical, unreadable=True):
|
||||
# The switch is wired from a node the file cannot show: only the
|
||||
# live run (the node saw the value) can link it, never a scan
|
||||
return "not_saved_wired"
|
||||
if not canonical.get("prompt") and not canonical.get("parameters"):
|
||||
return self._link_recorded_run(media_file, canonical, detailed, history)
|
||||
_with_file_size(canonical, media_file)
|
||||
resolution = resolve_prompt_pair(canonical)
|
||||
if resolution["state"] == "resolved" and side_opted_out(
|
||||
canonical.get("prompt"), resolution["negative"]
|
||||
):
|
||||
# Only the negative's node is off: store the positive alone (#90)
|
||||
prompt_text = (
|
||||
resolution["positive"]
|
||||
.get("storage_text", resolution["positive"]["text"])
|
||||
.strip()
|
||||
)
|
||||
status = self._store_scanned_prompt(media_file, canonical, prompt_text)
|
||||
if not detailed or status not in SCAN_TITLED_DETAILS:
|
||||
return status
|
||||
return {
|
||||
"status": status,
|
||||
"title": self._scan_title(media_file, prompt_text),
|
||||
}
|
||||
if blank_positive(resolution):
|
||||
# Only prepend/append text: no stored prompt can own it (#342)
|
||||
resolution = {**resolution, "state": "unresolved"}
|
||||
|
||||
+5
-53
@@ -5,7 +5,6 @@ import base64
|
||||
import binascii
|
||||
from functools import wraps
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sqlite3
|
||||
from typing import Callable
|
||||
@@ -28,7 +27,6 @@ from .history_current import (
|
||||
snapshot_current_output_scope,
|
||||
)
|
||||
from .inspection_service import CurrentFileChanged, inspect_current_file
|
||||
from .images import image_category_key
|
||||
|
||||
try:
|
||||
from ...database.execution_history import HistoryEpochChanged
|
||||
@@ -41,11 +39,7 @@ try:
|
||||
WindowPosition,
|
||||
)
|
||||
from ...utils.history_read_fence import HistoryReadFence
|
||||
from ...utils.inspection_response import (
|
||||
OWN_CATEGORY_UNREAD,
|
||||
apply_own_category,
|
||||
encode_history_inspection_response,
|
||||
)
|
||||
from ...utils.inspection_response import encode_history_inspection_response
|
||||
except ImportError:
|
||||
from database.execution_history import HistoryEpochChanged
|
||||
from database.history_gallery import read_history_gallery
|
||||
@@ -57,11 +51,7 @@ except ImportError:
|
||||
WindowPosition,
|
||||
)
|
||||
from utils.history_read_fence import HistoryReadFence
|
||||
from utils.inspection_response import (
|
||||
OWN_CATEGORY_UNREAD,
|
||||
apply_own_category,
|
||||
encode_history_inspection_response,
|
||||
)
|
||||
from utils.inspection_response import encode_history_inspection_response
|
||||
|
||||
|
||||
_RUN_MAX_BYTES = 8 * 1024 * 1024
|
||||
@@ -687,32 +677,6 @@ class HistoryRoutesMixin:
|
||||
) -> web.Response:
|
||||
return await self._history_current(request, "inspection")
|
||||
|
||||
def _history_own_category(self, reference: dict, scope):
|
||||
"""The recorded output's own category (#332), inside the snapshot.
|
||||
|
||||
The path is the symlink-resolved output folder plus the receipt's
|
||||
subfolder and filename, keyed by ``image_category_key`` over the
|
||||
configured output sources as Images keys it. A temp output or a path
|
||||
no source holds has none. A failed read is OWN_CATEGORY_UNREAD.
|
||||
"""
|
||||
if (
|
||||
reference.get("folder_type") != "output"
|
||||
or scope.canonical_output is None
|
||||
or candidate_locator(reference, scope) is None
|
||||
):
|
||||
return None
|
||||
path = os.path.join(
|
||||
str(scope.canonical_output), reference["subfolder"], reference["filename"]
|
||||
)
|
||||
try:
|
||||
key = image_category_key(self._get_all_output_dirs(), path)
|
||||
return None if key is None else self.db.image_category(*key)
|
||||
except Exception:
|
||||
self.logger.warning(
|
||||
"History image category could not be read", exc_info=True
|
||||
)
|
||||
return OWN_CATEGORY_UNREAD
|
||||
|
||||
async def _history_current(
|
||||
self, request: web.Request, kind: str
|
||||
) -> web.StreamResponse:
|
||||
@@ -737,7 +701,6 @@ class HistoryRoutesMixin:
|
||||
if scope.current_scope != current_scope:
|
||||
raise _current_stale()
|
||||
result = db.get_history_occurrence(run_id, node, ordinal)
|
||||
own = None
|
||||
if result is not None:
|
||||
receipt = result["receipt"]
|
||||
path = candidate_locator(receipt["reference"], scope)
|
||||
@@ -745,10 +708,6 @@ class HistoryRoutesMixin:
|
||||
db.model.get_connection(),
|
||||
(path,) if path is not None else (),
|
||||
)
|
||||
if kind == "inspection":
|
||||
own = self._history_own_category(
|
||||
receipt["reference"], scope
|
||||
)
|
||||
finally:
|
||||
db.model.close()
|
||||
if result is None:
|
||||
@@ -767,16 +726,9 @@ class HistoryRoutesMixin:
|
||||
resolved.canonical_target,
|
||||
source_label=resolved.public["source_label"],
|
||||
saved=saved.get("saved"),
|
||||
library=apply_own_category(
|
||||
saved.get(
|
||||
"library",
|
||||
{
|
||||
"status": (
|
||||
"unavailable" if saved_unavailable else "none"
|
||||
)
|
||||
},
|
||||
),
|
||||
own,
|
||||
library=saved.get(
|
||||
"library",
|
||||
{"status": "unavailable" if saved_unavailable else "none"},
|
||||
),
|
||||
recorded=recorded_output_lookup(
|
||||
db, close=True, reference=receipt["reference"]
|
||||
|
||||
@@ -29,11 +29,9 @@ from .prompts import bad_request
|
||||
|
||||
try:
|
||||
from ...database.image_categories import MAX_CATEGORY_LENGTH, MAX_KEY_LENGTH
|
||||
from ...database.models import normalize_image_path
|
||||
from ...utils.validators import validate_category
|
||||
except ImportError:
|
||||
from database.image_categories import MAX_CATEGORY_LENGTH, MAX_KEY_LENGTH
|
||||
from database.models import normalize_image_path
|
||||
from utils.validators import validate_category
|
||||
|
||||
CATEGORY_MEDIA_MESSAGE = "Only media files can have a category"
|
||||
@@ -145,7 +143,7 @@ class ImageCategoryRoutesMixin:
|
||||
# -- reading ---------------------------------------------------------------
|
||||
|
||||
def _own_category_map(self, sources, listings) -> dict:
|
||||
"""``{normalized listed path: own category}`` for the listed files (blocking).
|
||||
"""``{root / listed path: own category}`` for the listed files (blocking).
|
||||
|
||||
One primary-key range read per source; each stored row is matched to
|
||||
the listing of every selected root that holds its source, so the work
|
||||
@@ -164,30 +162,19 @@ class ImageCategoryRoutesMixin:
|
||||
for _index, root, files in listings:
|
||||
listing = self._favorite_listing(root, files)
|
||||
root_key = _favorite_root_key(root)
|
||||
# normalize_image_path(root / listed) without a call per file: the
|
||||
# listed spellings are clean relative POSIX paths. Windows abspath
|
||||
# also drops a component's trailing dots and spaces, so it keeps
|
||||
# the call there.
|
||||
base = os.path.join(normalize_image_path(str(root)), "")
|
||||
|
||||
def normalized(listed, base=base, root=root):
|
||||
if os.name == "nt":
|
||||
return normalize_image_path(str(root / listed))
|
||||
return base + os.path.normcase(listed.replace("/", os.sep))
|
||||
|
||||
nested = [
|
||||
item for item in _nested_sources(root, sources) if item[1] != root_key
|
||||
]
|
||||
for relative, category in by_source.get(root_key, {}).items():
|
||||
listed = listing.get(relative)
|
||||
if listed is not None and _source_prefix(relative, nested) is None:
|
||||
found[normalized(listed)] = category
|
||||
found[root / listed] = category
|
||||
for prefix, source_key in nested:
|
||||
for relative, category in by_source.get(source_key, {}).items():
|
||||
full = f"{prefix}/{relative}"
|
||||
listed = listing.get(full)
|
||||
if listed is not None and _source_prefix(full, nested) == prefix:
|
||||
found[normalized(listed)] = category
|
||||
found[root / listed] = category
|
||||
return found
|
||||
|
||||
async def _source_listings(self, roots) -> list[tuple]:
|
||||
|
||||
+25
-97
@@ -66,8 +66,6 @@ try:
|
||||
from ...utils.parallel import map_parallel
|
||||
from ...utils.json_strict import dumps_strict
|
||||
from ...utils.inspection_response import (
|
||||
OWN_CATEGORY_UNREAD,
|
||||
apply_own_category,
|
||||
encode_inspection_response,
|
||||
library_metadata,
|
||||
)
|
||||
@@ -89,8 +87,6 @@ except ImportError:
|
||||
from utils.parallel import map_parallel
|
||||
from utils.json_strict import dumps_strict
|
||||
from utils.inspection_response import (
|
||||
OWN_CATEGORY_UNREAD,
|
||||
apply_own_category,
|
||||
encode_inspection_response,
|
||||
library_metadata,
|
||||
)
|
||||
@@ -569,8 +565,7 @@ class _ImageTagMatch(NamedTuple):
|
||||
|
||||
tags: bool # every filter tag is in its image tags or linked positive tags
|
||||
query: bool # ``q`` is a case-folded substring of one image tag name
|
||||
# (hit, rating, positive id, inherited category) of its unambiguous link
|
||||
link: tuple | None
|
||||
link: tuple | None # (hit, rating, positive id) of its unambiguous link
|
||||
|
||||
|
||||
def _current_fingerprint(path: Path) -> tuple[int, int] | None:
|
||||
@@ -1225,27 +1220,20 @@ def _admitted_entries(
|
||||
links: dict | None,
|
||||
favorites: frozenset | None = None,
|
||||
image_tags: dict | None = None,
|
||||
own_categories: dict | None = None,
|
||||
):
|
||||
"""Yield ``(position, entry, link match)`` for each entry the filters admit.
|
||||
|
||||
*links* maps normalized paths to (hit, rating, positive id, inherited
|
||||
category) for every unambiguously linked image (see
|
||||
PromptDatabase.get_image_link_matches); a tag filter admits only those,
|
||||
and ``q`` admits a filename match or a linked-positive hit. A category
|
||||
admits a file whose effective category equals it (#332 MC5): its own
|
||||
category (*own_categories*, ``{normalized path: category}``) when it has one,
|
||||
else the inherited one, so an unlinked file with its own category is
|
||||
admitted. *favorites*, when given, admits only those listed
|
||||
*links* maps normalized paths to (hit, rating, positive id) for every
|
||||
unambiguously linked image (see PromptDatabase.get_image_link_matches); a
|
||||
category or tag filter admits only those, and ``q`` admits a filename match
|
||||
or a linked-positive hit. *favorites*, when given, admits only those listed
|
||||
paths (``favorites=1``). *image_tags* maps normalized paths to their
|
||||
_ImageTagMatch: a tag filter also admits a file whose current image tags
|
||||
(with its linked positive's tags) carry every tag, and ``q`` also admits an
|
||||
image tag name hit.
|
||||
"""
|
||||
query, media, folder = filters["query"], filters["media"], filters["folder"]
|
||||
category = filters["category"]
|
||||
members_only = bool(filters["tags"])
|
||||
own = own_categories or {}
|
||||
members_only = filters["category"] is not None or bool(filters["tags"])
|
||||
native_folder = folder.replace("/", os.sep)
|
||||
nested = native_folder + os.sep
|
||||
exact = filters.get("path", "")
|
||||
@@ -1264,12 +1252,6 @@ def _admitted_entries(
|
||||
image = None if image_tags is None else image_tags.get(entry.normalized)
|
||||
if members_only and match is None and not (image and image.tags):
|
||||
continue
|
||||
if category is not None:
|
||||
effective = own.get(entry.normalized) if own else None
|
||||
if effective is None and match is not None:
|
||||
effective = match[3]
|
||||
if effective != category:
|
||||
continue
|
||||
if (
|
||||
query
|
||||
and query not in entry.folded
|
||||
@@ -1288,7 +1270,6 @@ def _filtered_page(
|
||||
links: dict | None,
|
||||
favorites: frozenset | None = None,
|
||||
image_tags: dict | None = None,
|
||||
own_categories: dict | None = None,
|
||||
) -> tuple[int, list, int | None]:
|
||||
"""Count the *entries* the filters admit and take *size* from *start* on.
|
||||
|
||||
@@ -1298,7 +1279,7 @@ def _filtered_page(
|
||||
"""
|
||||
total, page, positives = 0, [], set()
|
||||
for position, entry, match in _admitted_entries(
|
||||
entries, filters, links, favorites, image_tags, own_categories
|
||||
entries, filters, links, favorites, image_tags
|
||||
):
|
||||
total += 1
|
||||
if match is not None:
|
||||
@@ -2238,10 +2219,9 @@ class ImageRoutesMixin:
|
||||
stat'ed (never every listed file), and a candidate's tags count only
|
||||
while a byte record is verified at its present ``(size, mtime_ns)``
|
||||
(spec amendment S1). Under a Tags filter the candidates' linked
|
||||
positives are read too, so their tags combine; the category is checked
|
||||
once, on the effective category, by _admitted_entries (#332).
|
||||
positives are read too, so their tags and category combine.
|
||||
"""
|
||||
tags, query = filters["tags"], filters["query"]
|
||||
tags, query, category = filters["tags"], filters["query"], filters["category"]
|
||||
wanted = set(tags)
|
||||
located = self._workspace_locators(listings)
|
||||
# The tag index answers which files carry a relevant name (#379); no
|
||||
@@ -2271,9 +2251,11 @@ class ImageRoutesMixin:
|
||||
link = facts.get(entry.normalized)
|
||||
carried = set(names).union(link[4] if link else ())
|
||||
matches[entry.normalized] = _ImageTagMatch(
|
||||
bool(wanted) and wanted <= carried,
|
||||
bool(wanted)
|
||||
and wanted <= carried
|
||||
and (category is None or (link is not None and link[3] == category)),
|
||||
bool(query) and any(query in name.casefold() for name in names),
|
||||
tuple(link[:4]) if link else None,
|
||||
link[:3] if link else None,
|
||||
)
|
||||
return matches
|
||||
|
||||
@@ -2282,14 +2264,10 @@ class ImageRoutesMixin:
|
||||
) -> tuple:
|
||||
"""The ordered entries and what admits them (blocking; run in executor).
|
||||
|
||||
Returns ``(entries, key, links, image_tags, favorites, own)``: the
|
||||
listing in the filters' order, its keyset key, and the link, image-tag,
|
||||
favorite and own-category maps _admitted_entries reads. One SELECT
|
||||
(get_image_link_matches) reads every linked positive when *read_links*;
|
||||
under a category it keeps only links whose raw recipe or positive
|
||||
category is that value (the inherited one is decided in Python), and
|
||||
the links of files whose own category is that value are read by path,
|
||||
so their rating and positive still count (#332). A failed read raises when the
|
||||
Returns ``(entries, key, links, image_tags, favorites)``: the listing in
|
||||
the filters' order, its keyset key, and the link, image-tag and favorite
|
||||
maps _admitted_entries reads. One SELECT (get_image_link_matches) reads
|
||||
every linked positive when *read_links*. A failed read raises when the
|
||||
filters need it (category, tags, the rating sort) or *strict*; else the
|
||||
listing stays usable without links (a search then matches filenames
|
||||
only). A failed image-tag read raises under a Tags filter; a search then
|
||||
@@ -2301,10 +2279,10 @@ class ImageRoutesMixin:
|
||||
try:
|
||||
links = self.db.get_image_link_matches(
|
||||
filters["text"],
|
||||
category,
|
||||
tags,
|
||||
every_link=True,
|
||||
wanted=self._workspace_wanted(listings),
|
||||
category=category,
|
||||
)
|
||||
except Exception:
|
||||
if strict or category is not None or tags or order == "rating":
|
||||
@@ -2318,13 +2296,6 @@ class ImageRoutesMixin:
|
||||
),
|
||||
exc_info=True,
|
||||
)
|
||||
own = None
|
||||
if category is not None:
|
||||
# A failed read raises: an unfiltered page must never pass for the
|
||||
# category (DC6).
|
||||
own = self._own_category_map(self._get_all_output_dirs(), listings)
|
||||
if links is not None:
|
||||
links = self._own_category_links(links, own, filters)
|
||||
image_tags = None
|
||||
if filters["query"] or tags:
|
||||
try:
|
||||
@@ -2371,30 +2342,7 @@ class ImageRoutesMixin:
|
||||
favorites = (
|
||||
self._workspace_favorite_paths(listings) if filters["favorites"] else None
|
||||
)
|
||||
return entries, key, links, image_tags, favorites, own
|
||||
|
||||
def _own_category_links(self, links: dict, own: dict, filters: dict) -> dict:
|
||||
"""*links* plus the links of files whose own category is the filter's.
|
||||
|
||||
Their inherited category differs, so the category's link read left
|
||||
them out; read by path (get_image_link_facts), kept only when their
|
||||
positive has every filter tag, as the link read would.
|
||||
"""
|
||||
category, wanted = filters["category"], set(filters["tags"])
|
||||
missing = [
|
||||
path
|
||||
for path, value in own.items()
|
||||
if value == category and path not in links
|
||||
]
|
||||
if not missing:
|
||||
return links
|
||||
facts = self.db.get_image_link_facts(missing, filters["text"])
|
||||
extra = {
|
||||
path: tuple(fact[:4])
|
||||
for path, fact in facts.items()
|
||||
if wanted <= set(fact[4])
|
||||
}
|
||||
return {**links, **extra} if extra else links
|
||||
return entries, key, links, image_tags, favorites
|
||||
|
||||
def _select_workspace_page(
|
||||
self,
|
||||
@@ -2423,7 +2371,7 @@ class ImageRoutesMixin:
|
||||
or bool(filters["tags"])
|
||||
or order == "rating"
|
||||
)
|
||||
entries, key, links, image_tags, favorites, own = self._workspace_admission(
|
||||
entries, key, links, image_tags, favorites = self._workspace_admission(
|
||||
listings, filters, read_links=needed or after is None
|
||||
)
|
||||
start = 0
|
||||
@@ -2434,7 +2382,7 @@ class ImageRoutesMixin:
|
||||
else:
|
||||
start = bisect_right(entries, after, key=key)
|
||||
total, page, positive_count = _filtered_page(
|
||||
entries, start, limit + 1, filters, links, favorites, image_tags, own
|
||||
entries, start, limit + 1, filters, links, favorites, image_tags
|
||||
)
|
||||
if not needed and after is not None:
|
||||
positive_count = _COUNT_OMITTED
|
||||
@@ -2456,12 +2404,12 @@ class ImageRoutesMixin:
|
||||
|
||||
The links are read strictly: the positives must be the listing's own.
|
||||
"""
|
||||
entries, _key, links, image_tags, favorites, own = self._workspace_admission(
|
||||
entries, _key, links, image_tags, favorites = self._workspace_admission(
|
||||
listings, filters, read_links=True, strict=True
|
||||
)
|
||||
targets, positives, unlinked, scopes = [], {}, 0, {}
|
||||
for _position, entry, match in _admitted_entries(
|
||||
entries, filters, links, favorites, image_tags, own
|
||||
entries, filters, links, favorites, image_tags
|
||||
):
|
||||
if len(targets) == cap:
|
||||
return ListingSelection(targets, list(positives), unlinked, True)
|
||||
@@ -3780,20 +3728,13 @@ class ImageRoutesMixin:
|
||||
except Exception:
|
||||
# The link stays known; only its title is withheld.
|
||||
self.logger.warning("Inspection link title could not be read")
|
||||
# An output file's own category (#332); LoRA roots have none.
|
||||
own = (
|
||||
self._read_own_category(outputs, path)
|
||||
if index < len(outputs)
|
||||
else None
|
||||
)
|
||||
try:
|
||||
observed = inspect_current_file(
|
||||
path,
|
||||
source_label=_public_path(roots[index]),
|
||||
saved=saved,
|
||||
library=apply_own_category(
|
||||
self._inspection_library(saved, unavailable=unavailable),
|
||||
own,
|
||||
library=self._inspection_library(
|
||||
saved, unavailable=unavailable
|
||||
),
|
||||
image_tags=self._inspection_image_tags(
|
||||
roots[index], Path(relative).parts
|
||||
@@ -3861,19 +3802,6 @@ class ImageRoutesMixin:
|
||||
|
||||
return read
|
||||
|
||||
def _read_own_category(self, sources, path):
|
||||
"""One file's own category, None, or OWN_CATEGORY_UNREAD (blocking).
|
||||
|
||||
Keyed by ``image_category_key`` over the output sources, as Images
|
||||
writes it. A failed read is logged and hides the category (DC5).
|
||||
"""
|
||||
try:
|
||||
key = image_category_key(sources, path)
|
||||
return None if key is None else self.db.image_category(*key)
|
||||
except Exception:
|
||||
self.logger.warning("Image category could not be read", exc_info=True)
|
||||
return OWN_CATEGORY_UNREAD
|
||||
|
||||
def _inspection_library(self, saved: dict | None, *, unavailable: bool) -> dict:
|
||||
"""The inspector's Library facts; the route sets the title afterwards."""
|
||||
result = {
|
||||
|
||||
@@ -19,7 +19,7 @@ test.beforeEach(async ({page, browserName}) => {
|
||||
await import('/extensions/promptmanager/prompt_manager.js');
|
||||
class Node {
|
||||
constructor(){
|
||||
this.widgets=['text','category','tags','search_text'].map(name=>({name,value:'',inputEl:document.createElement('textarea')}));
|
||||
this.widgets=[...['text','category','tags','search_text'].map(name=>({name,value:'',inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true}];
|
||||
this.widgets_values=[]; this.inputs=[];
|
||||
}
|
||||
addDOMWidget(name,type,element){document.body.append(element);return {element};}
|
||||
@@ -36,6 +36,14 @@ test.afterEach(async ({page,browserName}) => {
|
||||
expect(page.classicErrors).toEqual([]);
|
||||
});
|
||||
|
||||
test('Classic loads an older workflow with Save to library on and keeps a saved off (#90)',async({page})=>{
|
||||
// A workflow saved before the switch held the search UI's slot ('') at the switch's index
|
||||
const loaded=await page.evaluate(()=>{const widget=node.widgets.find(w=>w.name==='save_to_library');widget.value='';node.configure({});return widget.value;});
|
||||
expect(loaded).toBe(true);
|
||||
const off=await page.evaluate(()=>{const widget=node.widgets.find(w=>w.name==='save_to_library');widget.value=false;node.configure({});return [widget.value,node.serialize().widgets_values[4]];});
|
||||
expect(off).toEqual([false,false]);
|
||||
});
|
||||
|
||||
test('Classic keeps immediate selection, search query and metadata serialization',async({page})=>{
|
||||
await page.route('**/prompt_manager/search?**', route=>route.fulfill({json:{success:true,results:[{text:'A new portrait',category:'Portrait',tags:['film grain'],rating:5}]}}));
|
||||
await page.evaluate(()=>{node.widgets.find(w=>w.name==='search_text').value='tag:"film grain" portrait';});
|
||||
|
||||
@@ -26,7 +26,7 @@ test.beforeEach(async({page,browserName})=>{
|
||||
window.fixtureApp={registerExtension(extension){this.extension=extension;},ui:{settings:new EventTarget()},graph:{setDirtyCanvas(){}},canvas:{}};
|
||||
await import('/extensions/promptmanager/prompt_manager.js');
|
||||
class Node {
|
||||
constructor(){this.widgets=['text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}}))];this.properties={};this.size=[550,700];}
|
||||
constructor(){this.widgets=[...['text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true,options:{}}];this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}})),{name:'save_to_library',type:'BOOLEAN',link:null,widget:{name:'save_to_library'}}];this.properties={};this.size=[550,700];}
|
||||
findInputByType(type){const index=this.inputs.findIndex(input=>input.type===type&&input.link==null);return index<0?undefined:{index,slot:this.inputs[index]};}
|
||||
addWidget(type,name,value,callback,options){const widget={type,name,value,callback,options};this.widgets.push(widget);return widget;}
|
||||
addDOMWidget(name,type,element){const host=document.createElement('div');host.style.height='650px';host.className='fixture-dom-widget';host.append(element);document.body.append(host);return {element};}
|
||||
@@ -41,6 +41,32 @@ test.beforeEach(async({page,browserName})=>{
|
||||
});
|
||||
test.afterEach(async({page,browserName})=>{await saveCoverage(page,browserName);expect(page.focusedErrors).toEqual([]);});
|
||||
|
||||
test(`${variant} Save to library switch saves its value and loads older workflows on (#90)`, async ({page}) => {
|
||||
const toggle=page.getByRole('checkbox',{name:'Save to library',exact:true});
|
||||
await expect(toggle).toBeChecked();
|
||||
expect(await page.evaluate(()=>node.widgets.map(widget=>widget.name))).toEqual(['text','category','tags','search_text','prepend_text','append_text','save_to_library']);
|
||||
// The native switch is drawn in the toolbar only: the host keeps it hidden and zero-height, like the text fallbacks
|
||||
expect(await page.evaluate(()=>{const widget=node.widgets.find(item=>item.name==='save_to_library');return {type:widget.type,hidden:widget.hidden,size:widget.computeSize()};})).toEqual({type:'hidden',hidden:true,size:[0,-4]});
|
||||
expect(await page.evaluate(()=>node.widgets.find(item=>item.name==='text').computeSize())).toEqual([0,-4]);
|
||||
await toggle.uncheck();
|
||||
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Save to library is off: runs of this node are not added to the library.');
|
||||
const saved=await page.evaluate(()=>node.serialize());
|
||||
expect(saved.widgets_values[6]).toBe(false);
|
||||
await page.evaluate(saved=>node.configure(saved),saved);
|
||||
await expect(toggle).not.toBeChecked();
|
||||
await page.getByLabel('Main prompt',{exact:true}).fill('Typed while off');
|
||||
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Save to library is off: runs of this node are not added to the library.');
|
||||
// A workflow saved before the switch held the editor's own slot ('') at its index
|
||||
await page.evaluate(()=>node.configure({widgets_values:['Old text','','','','','',''],properties:{}}));
|
||||
await expect(toggle).toBeChecked();
|
||||
expect(await page.evaluate(()=>node.widgets.find(widget=>widget.name==='save_to_library').value)).toBe(true);
|
||||
await toggle.check();
|
||||
await page.getByLabel('Main prompt',{exact:true}).fill('Typed while on');
|
||||
await expect(page.locator('.pm-focused > .pm-editor-status[role=status]')).toHaveText('Main prompt is saved when this node runs.');
|
||||
await page.evaluate(()=>{node.inputs.find(input=>input.name==='save_to_library').link=9;node.onConnectionsChange?.();});
|
||||
await expect(toggle).toBeDisabled();
|
||||
});
|
||||
|
||||
test(`${variant} library autocompletes filters and searches without changing editor metadata`, async ({page}) => {
|
||||
const requests=[];
|
||||
await page.route('**/prompt_manager/categories',route=>route.fulfill({json:{success:true,categories:['Portrait']}}));
|
||||
@@ -257,7 +283,7 @@ test(`${variant} restores workflow values and disclosures and protects connected
|
||||
test(`${variant} offers text as a link socket, keeps typed text while linked and through a reload (#343)`,async({page})=>{
|
||||
const sockets=()=>page.evaluate(()=>Object.fromEntries(node.inputs.map(input=>[input.name,input.widget?.name??null])));
|
||||
// The hidden text widget no longer owns the input, so the host draws it as a socket; the other widget inputs are unchanged.
|
||||
expect(await sockets()).toEqual({clip:null,prepend_text:null,append_text:null,text:null,category:'category',tags:'tags',search_text:'search_text'});
|
||||
expect(await sockets()).toEqual({clip:null,prepend_text:null,append_text:null,text:null,category:'category',tags:'tags',search_text:'search_text',save_to_library:'save_to_library'});
|
||||
expect(await page.evaluate(()=>node.findInputByType('STRING').slot.name)).toBe('text');
|
||||
expect(await page.evaluate(()=>node.findInputByType('CLIP').slot.name)).toBe('clip');
|
||||
const main=page.getByRole('textbox',{name:'Main prompt',exact:true});
|
||||
|
||||
@@ -17,7 +17,7 @@ test.beforeEach(async({page,browserName})=>{
|
||||
window.fixtureApp={registerExtension(extension){this.extension=extension;},ui:{settings},graph:{_nodes:[],setDirtyCanvas(){}},canvas:{},graphToPrompt:async()=>({})};
|
||||
await import('/extensions/promptmanager/prompt_manager.js');
|
||||
class Node {
|
||||
constructor(){this.id=1;this.type='PromptManagerPair';this.widgets=['text','negative_text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')}));this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}}))];this.properties={};this.size=[1100,700];}
|
||||
constructor(){this.id=1;this.type='PromptManagerPair';this.widgets=[...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'text',value:'',options:{},inputEl:document.createElement('textarea')})),{name:'save_to_library',type:'toggle',value:true,options:{}}];this.inputs=[...['clip','prepend_text','append_text'].map(name=>({name,type:name==='clip'?'CLIP':'STRING',link:null})),...['text','negative_text','category','tags','search_text'].map(name=>({name,type:'STRING',link:null,widget:{name}})),{name:'save_to_library',type:'BOOLEAN',link:null,widget:{name:'save_to_library'}}];this.properties={};this.size=[1100,700];}
|
||||
findInputByType(type){const index=this.inputs.findIndex(input=>input.type===type&&input.link==null);return index<0?undefined:{index,slot:this.inputs[index]};}
|
||||
addWidget(type,name,value,callback,options){const widget={type,name,value,callback,options};this.widgets.push(widget);return widget;}
|
||||
addDOMWidget(name,type,element){const host=document.createElement('div');host.style.cssText='height:700px;width:1100px;max-width:100%';host.className='fixture-dom-widget';host.append(element);document.body.append(host);return {element};}
|
||||
@@ -34,6 +34,18 @@ test.afterEach(async({page,browserName})=>{await saveCoverage(page,browserName);
|
||||
async function selectPair(page){await page.getByRole('button',{name:'Recent',exact:true}).click();await page.locator('.pm-editor-result').click();}
|
||||
async function draft(page,positive='Draft positive',negative='Draft negative'){await page.getByRole('textbox',{name:'Positive prompt',exact:true}).fill(positive);await page.getByRole('textbox',{name:'Negative prompt',exact:true}).fill(negative);}
|
||||
|
||||
test('paired editor Save to library switch keeps the widget order and loads older workflows on (#90)',async({page})=>{
|
||||
await expect(page.locator('.pm-pair')).toBeVisible();
|
||||
const toggle=page.getByRole('checkbox',{name:'Save to library',exact:true});
|
||||
await expect(toggle).toBeChecked();
|
||||
expect(await page.evaluate(()=>node.widgets.map(widget=>widget.name))).toEqual(['text','negative_text','category','tags','search_text','prepend_text','append_text','save_to_library']);
|
||||
await toggle.uncheck();
|
||||
expect(await page.evaluate(()=>node.serialize().widgets_values[7])).toBe(false);
|
||||
await expect(page.locator('.pm-pair > .pm-editor-status[role=status]').last()).toHaveText('Save to library is off: runs of this node are not added to the library.');
|
||||
await page.evaluate(()=>node.configure({widgets_values:['P','N','','','','','',''],properties:{}}));
|
||||
await expect(toggle).toBeChecked();
|
||||
});
|
||||
|
||||
test('paired editor previews both texts before explicitly replacing both',async({page})=>{
|
||||
await expect(page.locator('.pm-pair')).toBeVisible();
|
||||
await draft(page);
|
||||
|
||||
@@ -16,8 +16,8 @@ installHostHooks(test, ['PromptManager', 'PromptManagerText', 'PromptManagerFocu
|
||||
// `primitive`: a core string node the queue-time resolver follows. `legacy`: the frontend's virtual Primitive node,
|
||||
// whose value is queued as literal text. `preview`: PrimitiveString -> PreviewAny, the shape of the maintainer's
|
||||
// TextGenerate -> PreviewAny chain, whose text is only known when the node runs.
|
||||
function build(page, { kind, positive, negative, prefix }) {
|
||||
return page.evaluate(async ({ kind, positive, negative, prefix }) => {
|
||||
function build(page, { kind, positive, negative, prefix, saveToLibrary }) {
|
||||
return page.evaluate(async ({ kind, positive, negative, prefix, saveToLibrary }) => {
|
||||
const { app } = await import('/scripts/app.js');
|
||||
app.graph.clear();
|
||||
const add = (type, x, y) => { const node = LiteGraph.createNode(type); app.graph.add(node); node.pos = [x, y]; return node; };
|
||||
@@ -28,6 +28,7 @@ function build(page, { kind, positive, negative, prefix }) {
|
||||
const set = (name, value) => { const widget = prompt.widgets.find(item => item.name === name); widget.value = value; if (widget.inputEl) widget.inputEl.value = value; };
|
||||
set('text', positive.typed);
|
||||
if (negative) set('negative_text', negative.typed);
|
||||
if (saveToLibrary !== undefined) prompt.widgets.find(item => item.name === 'save_to_library').value = saveToLibrary;
|
||||
const drive = (name, side, y) => {
|
||||
if (!side.via) return;
|
||||
if (side.via === 'legacy') {
|
||||
@@ -73,7 +74,7 @@ function build(page, { kind, positive, negative, prefix }) {
|
||||
await prompt._pmTextSerializerReady;
|
||||
const { output, workflow } = await app.graphToPrompt();
|
||||
return { output, workflow, promptNode: String(prompt.id) };
|
||||
}, { kind, positive, negative, prefix });
|
||||
}, { kind, positive, negative, prefix, saveToLibrary });
|
||||
}
|
||||
|
||||
async function run(execution, page, testCase, built) {
|
||||
@@ -195,6 +196,47 @@ for (const [label, linked, negativeVia] of [['positive', ['text'], 'primitive'],
|
||||
});
|
||||
}
|
||||
|
||||
// #90: with Save to library off the run still produces its image, but PromptManager saves, counts and links nothing,
|
||||
// and the run's history records no prompt text for it.
|
||||
for (const kind of ['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerSplit', 'PromptManagerPair']) {
|
||||
test(`${kind} with Save to library off saves, counts and links nothing (#90)`, async ({ execution, page }) => {
|
||||
test.setTimeout(90000);
|
||||
const text = unique(`Not saved ${kind}`);
|
||||
const negative = kind === 'PromptManagerPair' ? { typed: unique('Not saved negative') } : undefined;
|
||||
const prefix = `notsaved/${randomUUID()}`;
|
||||
const { output, workflow, promptNode } = await build(page, { kind, positive: { typed: text }, negative, prefix, saveToLibrary: false });
|
||||
expect(output[promptNode].inputs.save_to_library).toBe(false);
|
||||
const id = randomUUID();
|
||||
const response = await execution.request.post('/prompt', { data: { prompt: output, prompt_id: id, extra_data: { extra_pnginfo: { workflow } } } });
|
||||
expect(response.ok(), await response.text()).toBe(true);
|
||||
let item;
|
||||
await expect.poll(async () => {
|
||||
item = (await (await execution.request.get(`/history/${id}`)).json())[id];
|
||||
return item?.status?.status_str ?? null;
|
||||
}, { timeout: 20000 }).toBe('success');
|
||||
let result;
|
||||
await expect.poll(async () => {
|
||||
result = await readExecutionRuns(execution.databasePath, id);
|
||||
const run = result.rows[0];
|
||||
return Boolean(run && result.captures[run.run_id]);
|
||||
}, { timeout: 10000, message: 'the run reaches SQLite' }).toBe(true);
|
||||
const run = result.rows[0];
|
||||
for (const context of result.captures[run.run_id].contexts || []) {
|
||||
expect(context.positive_prompt_id ?? null).toBeNull();
|
||||
expect(context.pair_id ?? null).toBeNull();
|
||||
}
|
||||
expect((result.usage[run.run_id] || []).every(entry => entry.run_count === 0)).toBe(true);
|
||||
const files = Object.values(item.outputs).flatMap(value => value.images || []).map(image => image.filename);
|
||||
expect(files.length).toBeGreaterThan(0);
|
||||
await scan(execution);
|
||||
expect(Object.values(await linkedPrompts(execution, files))).toEqual(files.map(() => null));
|
||||
for (const query of [text, negative?.typed].filter(Boolean)) {
|
||||
const found = await (await execution.request.get(`/prompt_manager/search?text=${encodeURIComponent(query)}`)).json();
|
||||
expect(found.results ?? [], query).toEqual([]);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// #347: a cached PromptManager node counts every job, as an executing one does. Jobs after the first change only
|
||||
// the sampler seed, so ComfyUI serves the PromptManager node (and its PreviewAny source) from its output cache.
|
||||
const COUNTS = `import json, sqlite3, sys, pathlib
|
||||
|
||||
@@ -370,6 +370,7 @@ test('workflows saved before #343 load unchanged and their linked text keeps wor
|
||||
const nodes = Object.fromEntries(app.graph._nodes.map(node => [node.id, node]));
|
||||
const reloaded = app.graph.serialize();
|
||||
return {
|
||||
saves: Object.fromEntries(Object.entries(output).filter(([, node]) => node.class_type.startsWith('PromptManager')).map(([id, node]) => [id, node.inputs.save_to_library])),
|
||||
values: Object.fromEntries(reloaded.nodes.map(node => [node.id, node.widgets_values])),
|
||||
links: Object.fromEntries(Object.entries(output).map(([id, node]) => [id, { text: node.inputs.text, negative_text: node.inputs.negative_text }])),
|
||||
widgetBound: Object.fromEntries(Object.values(nodes).filter(node => node.type.startsWith('PromptManager')).map(node => [node.id, node.inputs.filter(input => ['text', 'negative_text'].includes(input.name) && input.widget).map(input => input.name)])),
|
||||
@@ -377,8 +378,16 @@ test('workflows saved before #343 load unchanged and their linked text keeps wor
|
||||
}, workflow);
|
||||
for (const node of workflow.nodes) {
|
||||
// Every saved widget value comes back as it was saved, except the typed text main lost under a link (null).
|
||||
if (node.widgets_values?.[0] !== null) expect(result.values[node.id], `${node.type} ${node.id}`).toEqual(node.widgets_values);
|
||||
// #90: the last saved value is the editor's own slot (''), where the Save to library switch is now: it loads on.
|
||||
if (node.widgets_values?.[0] === null) continue;
|
||||
if (!node.type.startsWith('PromptManager')) { expect(result.values[node.id], `${node.type} ${node.id}`).toEqual(node.widgets_values); continue; }
|
||||
const saved = node.widgets_values.length - 1;
|
||||
expect(result.values[node.id].slice(0, saved), `${node.type} ${node.id}`).toEqual(node.widgets_values.slice(0, saved));
|
||||
expect(result.values[node.id][saved], `${node.type} ${node.id} save_to_library`).toBe(true);
|
||||
}
|
||||
// Every PromptManager node of the old workflow queues with Save to library on.
|
||||
for (const save of Object.values(result.saves)) expect(save).toBe(true);
|
||||
expect(Object.keys(result.saves).length).toBeGreaterThan(0);
|
||||
expect(result.links).toMatchObject({
|
||||
1: { text: 'Saved PromptManager text' }, 2: { text: 'Saved PromptManagerText text' }, 3: { text: 'Saved PromptManagerFocused text' },
|
||||
4: { text: 'Saved PromptManagerSplit text' }, 5: { text: 'Saved PromptManagerPair text', negative_text: 'Saved pair negative' },
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
// #90: the node's save_to_library switch (web/js/save-to-library.js).
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const source = import('../../web/js/save-to-library.js');
|
||||
|
||||
function classicNode(saved) {
|
||||
const names = ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text', 'save_to_library', 'prompt_manager_search_ui'];
|
||||
const widgets = names.map(name => ({name, value: name === 'save_to_library' ? true : ''}));
|
||||
const node = {widgets, widgets_values: saved ? [...saved] : undefined};
|
||||
// What a host configure does: each saved value goes to the widget at its index.
|
||||
(saved || []).forEach((value, index) => { if (widgets[index]) widgets[index].value = value; });
|
||||
return node;
|
||||
}
|
||||
|
||||
test('an older workflow stored the editor slot where the switch is now: it loads on', async () => {
|
||||
const {normalizeSaveWidget, savesToLibrary} = await source;
|
||||
const node = classicNode(['a cat', 'c', 't', '', '', '', '']);
|
||||
assert.equal(node.widgets[6].value, '', 'the host put the old DOM slot value into the switch');
|
||||
assert.equal(normalizeSaveWidget(node), true);
|
||||
assert.equal(node.widgets[6].value, true);
|
||||
assert.equal(node.widgets_values[6], true);
|
||||
assert.equal(savesToLibrary(node), true);
|
||||
assert.deepEqual(node.widgets_values.slice(0, 6), ['a cat', 'c', 't', '', '', ''], 'no earlier value moved');
|
||||
});
|
||||
|
||||
test('null, undefined and missing slots load on; saved booleans are kept', async () => {
|
||||
const {normalizeSaveWidget, savesToLibrary} = await source;
|
||||
for (const value of [null, undefined, 0, 'false']) {
|
||||
const node = classicNode(['a', '', '', '', '', '', value]);
|
||||
normalizeSaveWidget(node);
|
||||
assert.equal(savesToLibrary(node), true, String(value));
|
||||
}
|
||||
const short = classicNode(['a', '', '', '', '', '']);
|
||||
assert.equal(normalizeSaveWidget(short), false);
|
||||
assert.equal(savesToLibrary(short), true);
|
||||
const off = classicNode(['a', '', '', '', '', '', false]);
|
||||
assert.equal(normalizeSaveWidget(off), false);
|
||||
assert.equal(savesToLibrary(off), false);
|
||||
});
|
||||
|
||||
test('a node without the widget (an older backend) saves', async () => {
|
||||
const {normalizeSaveWidget, savesToLibrary} = await source;
|
||||
assert.equal(savesToLibrary({widgets: [{name: 'text', value: ''}]}), true);
|
||||
assert.equal(normalizeSaveWidget({widgets: []}), false);
|
||||
assert.equal(savesToLibrary(undefined), true);
|
||||
});
|
||||
|
||||
test('the switch moves after the editor fallbacks so every frontend has one order', async () => {
|
||||
const {placeSaveWidgetLast} = await source;
|
||||
const widgets = ['text', 'category', 'tags', 'search_text', 'save_to_library', 'prepend_text', 'append_text'].map(name => ({name}));
|
||||
const node = {widgets};
|
||||
assert.equal(placeSaveWidgetLast(node), true);
|
||||
assert.deepEqual(node.widgets.map(widget => widget.name), ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text', 'save_to_library']);
|
||||
assert.equal(placeSaveWidgetLast(node), false, 'already last');
|
||||
assert.equal(placeSaveWidgetLast({widgets: [{name: 'text'}]}), false);
|
||||
});
|
||||
|
||||
test('setting the switch updates the widget, its saved value and notifies the host', async () => {
|
||||
const {setSaveToLibrary, savesToLibrary} = await source;
|
||||
const calls = [];
|
||||
const node = classicNode(['a', '', '', '', '', '', true]);
|
||||
node.widgets[6].callback = value => calls.push(value);
|
||||
setSaveToLibrary(node, false);
|
||||
assert.equal(savesToLibrary(node), false);
|
||||
assert.equal(node.widgets_values[6], false);
|
||||
assert.deepEqual(calls, [false]);
|
||||
setSaveToLibrary({widgets: []}, false);
|
||||
});
|
||||
@@ -383,32 +383,3 @@ test('the D117 wrap rule targets only the settings summary button', async () =>
|
||||
assert.deepEqual(rules.map(line => line.trim()), ['.ca-tools #viewer-details .settings-summary { white-space: normal; /* D117 */ }']);
|
||||
assert.doesNotMatch(css, /#viewer-details > div > button/);
|
||||
});
|
||||
|
||||
test('category chip names its origin and shows an own category for any link status (#332)', async () => {
|
||||
const base = await localRecord(localPng('a.png'));
|
||||
const chipTexts = record => {
|
||||
const {f, done} = renderFull(record);
|
||||
try {
|
||||
return f.walk(f.container, n => /(^| )chip( |$)/.test(n.className ?? '')).map(n => n.textContent);
|
||||
} finally { done(); }
|
||||
};
|
||||
const cases = [
|
||||
[{status:'none', category:'Scans', category_scope:'image'}, 'Image category: Scans'],
|
||||
[{status:'ambiguous', category:'Scans', category_scope:'image'}, 'Image category: Scans'],
|
||||
[{status:'linked', category:'Portraits', category_scope:'image'}, 'Image category: Portraits'],
|
||||
[{status:'linked', category:'Landscapes', category_scope:'positive'}, 'Positive prompt category: Landscapes'],
|
||||
[{status:'linked', category:'Recipes', category_scope:'pair'}, 'Recipe category: Recipes'],
|
||||
];
|
||||
for (const [library, text] of cases) {
|
||||
const chips = chipTexts({...base, library:{...base.library, tags:[], tags_scope:null, ...library}});
|
||||
assert.ok(chips.includes(text), `${JSON.stringify(library)} -> ${chips}`);
|
||||
}
|
||||
for (const library of [
|
||||
{status:'none', category:'Inherited', category_scope:'positive'},
|
||||
{status:'linked', category:null, category_scope:null},
|
||||
{status:'none', category:null, category_scope:null},
|
||||
]) {
|
||||
const chips = chipTexts({...base, library:{...base.library, tags:[], tags_scope:null, ...library}});
|
||||
assert.equal(chips.some(chip => /category: /.test(chip)), false, JSON.stringify(library));
|
||||
}
|
||||
});
|
||||
|
||||
@@ -278,7 +278,6 @@ test('record rejects root path media and invalid versioned fields with retryable
|
||||
data => { data.resolution.sampler_id = {}; },
|
||||
data => { data.library.tags_scope = 'recipe'; },
|
||||
data => { data.library.category_scope = 'recipe'; },
|
||||
data => { data.library.tags_scope = 'image'; },
|
||||
data => { delete data.library.category_scope; },
|
||||
data => { delete data.library.tags_scope; },
|
||||
]) {
|
||||
@@ -943,14 +942,3 @@ test("names the server quotes (parentheses, '!*') still match their original and
|
||||
assert.equal(record.download.url, entry.url);
|
||||
assert.deepEqual(record.preview, { url: entry.thumbnail_url });
|
||||
});
|
||||
|
||||
test('record accepts an image\'s own category on any link status (#332)', () => {
|
||||
const toRecord = contract('toInspectionRecord');
|
||||
for (const status of ['linked', 'none', 'ambiguous', 'unavailable']) {
|
||||
const data = envelope();
|
||||
Object.assign(data.library, { status, category: 'Scans', category_scope: 'image' });
|
||||
const record = toRecord(data, image(), extractGenerationFacts);
|
||||
assert.equal(record.library.category, 'Scans');
|
||||
assert.equal(record.library.category_scope, 'image');
|
||||
}
|
||||
});
|
||||
|
||||
@@ -62,16 +62,12 @@ class TestVocabularyAPI(ImageAPITestCase):
|
||||
},
|
||||
)
|
||||
self.assertEqual(
|
||||
categories["categories"],
|
||||
[{"value": "Animal", "prompt_count": 1, "image_count": 0}],
|
||||
categories["categories"], [{"value": "Animal", "prompt_count": 1}]
|
||||
)
|
||||
_, rest = await self.get(
|
||||
"categories", limit=1, cursor=categories["next_cursor"]
|
||||
)
|
||||
self.assertEqual(
|
||||
rest["categories"],
|
||||
[{"value": "Pet", "prompt_count": 1, "image_count": 0}],
|
||||
)
|
||||
self.assertEqual(rest["categories"], [{"value": "Pet", "prompt_count": 1}])
|
||||
self.assertFalse(rest["has_more"])
|
||||
|
||||
async def test_vocabulary_cursor_is_signed_epoch_scoped(self):
|
||||
@@ -492,7 +488,6 @@ class TestVocabularyAPI(ImageAPITestCase):
|
||||
"other_prompts": 0,
|
||||
"negative_variants": 1,
|
||||
"archived_negative_variants": 0,
|
||||
"images": 0,
|
||||
}
|
||||
self.assertEqual(
|
||||
preview,
|
||||
@@ -533,73 +528,6 @@ class TestVocabularyAPI(ImageAPITestCase):
|
||||
self.assertEqual(response.status, 409)
|
||||
self.assertEqual(replay["code"], "vocabulary-conflict")
|
||||
|
||||
async def test_category_rows_report_images_and_image_only_values(self):
|
||||
self._save_prompt("Owner", category="Prompt value")
|
||||
_, options_before = await self.get_options()
|
||||
self.api.db.set_image_categories(
|
||||
[("root", "a.png"), ("root", "b.png")], "Prompt value"
|
||||
)
|
||||
self.api.db.set_image_categories([("root", "c.png")], "Image value")
|
||||
response, first = await self.get("categories", limit=1)
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(
|
||||
first["categories"],
|
||||
[{"value": "Prompt value", "prompt_count": 1, "image_count": 2}],
|
||||
)
|
||||
self.assertEqual(first["total"], 2)
|
||||
_, rest = await self.get("categories", limit=1, cursor=first["next_cursor"])
|
||||
self.assertEqual(
|
||||
rest["categories"],
|
||||
[{"value": "Image value", "prompt_count": 0, "image_count": 1}],
|
||||
)
|
||||
self.assertFalse(rest["has_more"])
|
||||
# The Prompt library's category list does not gain image values.
|
||||
_, options_after = await self.get_options()
|
||||
self.assertEqual(options_after, options_before)
|
||||
self.assertEqual(options_after["categories"], ["Prompt value"])
|
||||
|
||||
async def get_options(self):
|
||||
response = await self.client.get("/prompt_manager/library/options")
|
||||
return response, await response.json()
|
||||
|
||||
async def test_category_routes_rename_image_only_value(self):
|
||||
epoch = self.api.db.get_execution_epoch()
|
||||
self._save_prompt("Bystander", category="Kept")
|
||||
self.api.db.set_image_categories(
|
||||
[("root", "a.png"), ("second", "b.png")], "Image value"
|
||||
)
|
||||
prompts_before = self.category_rows()
|
||||
body = {"expected_epoch": epoch, "source": "Image value", "target": "Moved"}
|
||||
response, preview = await self.send("POST", "categories/preview", body)
|
||||
self.assertEqual(response.status, 200, preview)
|
||||
impact = {
|
||||
"positive_prompts": 0,
|
||||
"other_prompts": 0,
|
||||
"negative_variants": 0,
|
||||
"archived_negative_variants": 0,
|
||||
"images": 2,
|
||||
}
|
||||
self.assertEqual(preview["impact"], impact)
|
||||
stale = preview["impact_token"]
|
||||
self.api.db.set_image_categories([("root", "late.png")], "Image value")
|
||||
response, refused = await self.send(
|
||||
"POST", "categories/apply", {**body, "impact_token": stale}
|
||||
)
|
||||
self.assertEqual(response.status, 409)
|
||||
self.assertEqual(refused["code"], "vocabulary-conflict")
|
||||
self.assertEqual(self.api.db.image_category("root", "a.png"), "Image value")
|
||||
_, preview = await self.send("POST", "categories/preview", body)
|
||||
response, applied = await self.send(
|
||||
"POST",
|
||||
"categories/apply",
|
||||
{**body, "impact_token": preview["impact_token"]},
|
||||
)
|
||||
self.assertEqual(response.status, 200, applied)
|
||||
self.assertEqual(applied["impact"], {**impact, "images": 3})
|
||||
for key in (("root", "a.png"), ("second", "b.png"), ("root", "late.png")):
|
||||
self.assertEqual(self.api.db.image_category(*key), "Moved")
|
||||
self.assertEqual(self.category_rows(), prompts_before)
|
||||
|
||||
async def test_preview_validation(self):
|
||||
epoch = self.api.db.get_execution_epoch()
|
||||
self._save_prompt("Owner", category="Scenery")
|
||||
|
||||
@@ -1,262 +0,0 @@
|
||||
"""Images filters by the effective category (#332 MC4, MC5, DC6).
|
||||
|
||||
An image's effective category is its own category when it has one, else the
|
||||
category its link supplies (the recipe's when set, else the positive's), else
|
||||
none. The category filter, "select all N matching" and saved views match it.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sqlite3
|
||||
from unittest.mock import patch
|
||||
|
||||
try:
|
||||
import test_api_workspace_images as workspace
|
||||
from test_image_categories import category_checksum
|
||||
except ImportError:
|
||||
from tests import test_api_workspace_images as workspace
|
||||
from tests.test_image_categories import category_checksum
|
||||
|
||||
from py.api.images import image_category_key, parse_listing_filter
|
||||
|
||||
|
||||
class WorkspaceCategoryCase(workspace.ImageAPITestCase):
|
||||
"""Listing helpers plus own-category writes keyed as the routes key them."""
|
||||
|
||||
media = workspace.TestWorkspaceImages.media
|
||||
get = workspace.TestWorkspaceImages.get
|
||||
tag = workspace.TestWorkspaceImageTags.tag
|
||||
names = workspace.TestWorkspaceImageTags.names
|
||||
|
||||
async def tearDownAsync(self):
|
||||
self.api.db.close_all()
|
||||
workspace.assert_closed(self, self._temp_db.name, self._output_tmp.name)
|
||||
await super().tearDownAsync()
|
||||
|
||||
def own(self, path, value, sources=None):
|
||||
sources = sources or self.api._get_all_output_dirs()
|
||||
self.api.db.set_image_categories([image_category_key(sources, path)], value)
|
||||
|
||||
async def listed(self, **params):
|
||||
response, body = await self.get(sort="name", **params)
|
||||
self.assertEqual(response.status, 200, body)
|
||||
return self.names(body), body
|
||||
|
||||
|
||||
class TestEffectiveCategoryFilter(WorkspaceCategoryCase):
|
||||
def batch(self):
|
||||
"""Four images of "castle at dusk" (Landscapes); one is overridden."""
|
||||
castle = self._save_prompt("castle at dusk", category="Landscapes", rating=4)
|
||||
files = [self.media(f"castle_{n}.png", mtime=100 + n) for n in range(4)]
|
||||
for path in files:
|
||||
self.api.db.link_image_to_prompt(castle, str(path))
|
||||
self.own(files[0], "Portraits")
|
||||
return castle, files
|
||||
|
||||
async def test_category_filter_prefers_own_category(self):
|
||||
castle, _files = self.batch()
|
||||
conn = self.api.db.model.get_connection()
|
||||
before = category_checksum(conn)
|
||||
names, body = await self.listed(category="Portraits")
|
||||
self.assertEqual(names, ["castle_0.png"])
|
||||
# The caption still counts the prompt behind the image shown.
|
||||
self.assertEqual(body["positive_count"], 1)
|
||||
names, body = await self.listed(category="Landscapes")
|
||||
self.assertEqual(names, ["castle_1.png", "castle_2.png", "castle_3.png"])
|
||||
self.assertEqual(body["total"], 3)
|
||||
names, _ = await self.listed()
|
||||
self.assertEqual(len(names), 4)
|
||||
self.assertEqual(category_checksum(conn), before)
|
||||
self.assertEqual(self.api.db.get_prompt_by_id(castle)["category"], "Landscapes")
|
||||
|
||||
async def test_category_filter_lists_unlinked_with_own(self):
|
||||
scans = [self.media(f"scan_{n}.png", mtime=100 + n) for n in range(2)]
|
||||
self.media("plain.png", mtime=50)
|
||||
for path in scans:
|
||||
self.own(path, "Scans")
|
||||
names, body = await self.listed(category="Scans")
|
||||
self.assertEqual(names, ["scan_0.png", "scan_1.png"])
|
||||
self.assertEqual(body["positive_count"], 0)
|
||||
|
||||
async def test_recipe_category_is_inherited(self):
|
||||
"""MC4: a recipe's own category, not the positive's, is inherited."""
|
||||
db = self.api.db
|
||||
self._save_prompt("Pair positive", category="Night")
|
||||
pair = db.save_prompt_pair("Pair positive", "Noise", category="Recipes")
|
||||
blank = db.save_prompt_pair("Pair positive", "Other noise", category=" ")
|
||||
recipe = self.media("recipe.png", mtime=103)
|
||||
blank_recipe = self.media("blank_recipe.png", mtime=102)
|
||||
positive = db.get_prompt_pair(pair)["positive_prompt_id"]
|
||||
for path, pair_id in ((recipe, pair), (blank_recipe, blank)):
|
||||
image = db.link_image_to_prompt(positive, str(path))
|
||||
db.link_image_to_pair(pair_id, image, {"source": "fixture"})
|
||||
self.assertEqual((await self.listed(category="Recipes"))[0], ["recipe.png"])
|
||||
# A blank recipe category falls back to the positive's.
|
||||
self.assertEqual((await self.listed(category="Night"))[0], ["blank_recipe.png"])
|
||||
# An own category still wins over the recipe's.
|
||||
self.own(recipe, "Mine")
|
||||
self.assertEqual((await self.listed(category="Recipes"))[0], [])
|
||||
self.assertEqual((await self.listed(category="Mine"))[0], ["recipe.png"])
|
||||
|
||||
async def test_ambiguous_files_inherit_nothing_but_keep_their_own(self):
|
||||
first = self._save_prompt("Owner one", category="Night")
|
||||
second = self._save_prompt("Owner two", category="Night")
|
||||
ambiguous = self.media("ambiguous.png")
|
||||
for owner in (first, second):
|
||||
self.api.db.link_image_to_prompt(owner, str(ambiguous))
|
||||
self.assertEqual((await self.listed(category="Night"))[0], [])
|
||||
self.own(ambiguous, "Night")
|
||||
self.assertEqual((await self.listed(category="Night"))[0], ["ambiguous.png"])
|
||||
|
||||
async def test_combines_with_tags_search_and_rating(self):
|
||||
stone = self._save_prompt(
|
||||
"castle at dusk", category="Landscapes", tags=["stone"], rating=4
|
||||
)
|
||||
files = [self.media(f"castle_{n}.png", mtime=100 + n) for n in range(2)]
|
||||
for path in files:
|
||||
self.api.db.link_image_to_prompt(stone, str(path))
|
||||
self.own(files[0], "Portraits")
|
||||
other = self._save_prompt("harbor at noon", category="Portraits", rating=2)
|
||||
harbor = self.media("harbor.png", mtime=90)
|
||||
self.api.db.link_image_to_prompt(other, str(harbor))
|
||||
names, _ = await self.listed(category="Portraits", tags="stone")
|
||||
self.assertEqual(names, ["castle_0.png"])
|
||||
names, _ = await self.listed(category="Portraits", q="harbor")
|
||||
self.assertEqual(names, ["harbor.png"])
|
||||
names, _ = await self.listed(category="Portraits", q="castle_0")
|
||||
self.assertEqual(names, ["castle_0.png"])
|
||||
response, body = await self.get(category="Portraits", sort="rating")
|
||||
self.assertEqual(response.status, 200, body)
|
||||
self.assertEqual(self.names(body), ["castle_0.png", "harbor.png"])
|
||||
self.assertEqual(body["positive_count"], 2)
|
||||
|
||||
async def test_image_tag_admitted_file_keeps_its_inherited_category(self):
|
||||
night = self._save_prompt("Night owl", category="Night", tags=["a"])
|
||||
tagged = self.media("tagged.png", mtime=103)
|
||||
unlinked = self.media("unlinked.png", mtime=102)
|
||||
self.api.db.link_image_to_prompt(night, str(tagged))
|
||||
self.tag(tagged, ["b"])
|
||||
self.tag(unlinked, ["a", "b"])
|
||||
names, _ = await self.listed(tags="a,b", category="Night")
|
||||
self.assertEqual(names, ["tagged.png"])
|
||||
self.own(unlinked, "Night")
|
||||
names, _ = await self.listed(tags="a,b", category="Night")
|
||||
self.assertEqual(names, ["tagged.png", "unlinked.png"])
|
||||
|
||||
async def test_cursor_pages_walk_only_the_category(self):
|
||||
files = [self.media(f"img_{n:02}.png", mtime=100 + n) for n in range(7)]
|
||||
for path in files[::2]:
|
||||
self.own(path, "Even")
|
||||
seen, cursor = [], None
|
||||
while True:
|
||||
params = {"category": "Even", "limit": 2, "sort": "name"}
|
||||
if cursor:
|
||||
params["cursor"] = cursor
|
||||
response, body = await self.get(**params)
|
||||
self.assertEqual(response.status, 200, body)
|
||||
seen += self.names(body)
|
||||
self.assertEqual(body["total"], 4)
|
||||
cursor = body["next_cursor"]
|
||||
if not cursor:
|
||||
break
|
||||
self.assertEqual(seen, ["img_00.png", "img_02.png", "img_04.png", "img_06.png"])
|
||||
|
||||
async def test_an_outer_source_lists_a_nested_sources_own_category(self):
|
||||
inner = self.output_dir / "nested"
|
||||
nested = self.media("nested/a.png")
|
||||
self.media("b.png")
|
||||
self.api._get_all_output_dirs = lambda: [self.output_dir, inner]
|
||||
self.own(nested, "Scans")
|
||||
names, _ = await self.listed(category="Scans", root="0")
|
||||
self.assertEqual(names, ["nested/a.png"])
|
||||
names, _ = await self.listed(category="Scans", root="1")
|
||||
self.assertEqual(names, ["a.png"])
|
||||
# The outer source's own key for that path is not the file's key.
|
||||
from py.api.images import _favorite_root_key
|
||||
|
||||
self.api.db.clear_image_categories([image_category_key([inner], nested)])
|
||||
self.api.db.set_image_categories(
|
||||
[(_favorite_root_key(self.output_dir), "nested/a.png")], "Stale"
|
||||
)
|
||||
self.assertEqual((await self.listed(category="Stale"))[0], [])
|
||||
|
||||
async def test_a_gone_files_category_is_not_listed(self):
|
||||
gone = self.media("gone.png")
|
||||
self.own(gone, "Scans")
|
||||
os.unlink(gone)
|
||||
self.api.invalidate_gallery_cache()
|
||||
self.assertEqual((await self.listed(category="Scans"))[0], [])
|
||||
|
||||
async def test_a_failed_own_category_read_is_listing_failed(self):
|
||||
self.media("a.png")
|
||||
|
||||
def broken(*_args, **_kwargs):
|
||||
raise sqlite3.OperationalError("secret detail")
|
||||
|
||||
with patch.object(self.api.db, "own_categories_of", side_effect=broken):
|
||||
with self.assertLogs("prompt_manager", level="ERROR"):
|
||||
response, body = await self.get(category="Scans")
|
||||
self.assertEqual(response.status, 500)
|
||||
self.assertNotIn("secret", str(body))
|
||||
# Without a category nothing new is read.
|
||||
response, body = await self.get()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
|
||||
async def test_select_all_matching_uses_the_effective_category(self):
|
||||
_castle, files = self.batch()
|
||||
resolved = await self.api.resolve_listing_filter(
|
||||
parse_listing_filter({"category": "Portraits"}), 100
|
||||
)
|
||||
self.assertEqual([t[2] for t in resolved.targets], ["castle_0.png"])
|
||||
resolved = await self.api.resolve_listing_filter(
|
||||
parse_listing_filter({"category": "Landscapes"}), 100
|
||||
)
|
||||
self.assertEqual(len(resolved.targets), 3)
|
||||
self.assertEqual(len(files), 4)
|
||||
|
||||
async def test_saved_view_category_means_the_effective_category(self):
|
||||
"""CUJ 8: a saved view's category is the listing's query value."""
|
||||
_castle, files = self.batch()
|
||||
self.own(files[1], "Portraits")
|
||||
names, _ = await self.listed(category="Portraits")
|
||||
self.assertEqual(names, ["castle_0.png", "castle_1.png"])
|
||||
|
||||
|
||||
class TestFilterAndChipAgreeOnBlanks(WorkspaceCategoryCase):
|
||||
"""The filter and the inspector use one inherited-category rule."""
|
||||
|
||||
async def chip(self, path):
|
||||
response = await self.client.get(
|
||||
f"/prompt_manager/images/inspect/{path}", params={"root": "0"}
|
||||
)
|
||||
body = await response.json()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
return body["library"]["category"]
|
||||
|
||||
async def test_blank_recipe_values_agree(self):
|
||||
db = self.api.db
|
||||
positive = self._save_prompt("Pair positive", category="Night")
|
||||
blanks = {"spaces": " ", "tab": "\t", "nbsp": " ", "set": "Recipes"}
|
||||
for index, (label, value) in enumerate(blanks.items()):
|
||||
pair = db.save_prompt_pair("Pair positive", f"noise {label}")
|
||||
path = self.media(f"{label}.png", mtime=100 + index)
|
||||
image = db.link_image_to_prompt(positive, str(path))
|
||||
db.link_image_to_pair(pair, image, {"source": "fixture"})
|
||||
with db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE prompt_pairs SET category=? WHERE id=?", (value, pair)
|
||||
)
|
||||
blob = db.save_prompt_pair("Pair positive", "noise blob")
|
||||
path = self.media("blob.png", mtime=90)
|
||||
image = db.link_image_to_prompt(positive, str(path))
|
||||
db.link_image_to_pair(blob, image, {"source": "fixture"})
|
||||
with db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE prompt_pairs SET category=CAST(' ' AS BLOB) WHERE id=?",
|
||||
(blob,),
|
||||
)
|
||||
for name in ("spaces", "tab", "nbsp", "set", "blob"):
|
||||
with self.subTest(name=name):
|
||||
shown = await self.chip(f"{name}.png")
|
||||
for value in ("Night", "Recipes"):
|
||||
listed = f"{name}.png" in (await self.listed(category=value))[0]
|
||||
self.assertEqual(listed, shown == value, (name, value, shown))
|
||||
@@ -1347,7 +1347,7 @@ class TestWorkspaceImages(ImageAPITestCase):
|
||||
_, data = await self.get(q="harbor")
|
||||
self.assertEqual(self._names(data), ["night_a.png"])
|
||||
self.assertEqual(
|
||||
self.api.db.get_image_paths_matching("zebra", []),
|
||||
self.api.db.get_image_paths_matching("zebra", None, []),
|
||||
{
|
||||
normalize_image_path(str(files[name]))
|
||||
for name in ("night_ab", "day_ab", "titled")
|
||||
@@ -1407,10 +1407,8 @@ class TestWorkspaceImages(ImageAPITestCase):
|
||||
for query in statements
|
||||
if query.lstrip().upper().startswith(("SELECT", "WITH"))
|
||||
]
|
||||
# Plus the one stream of tagged image records q and tags read (B3),
|
||||
# and the category's own-category read of the one source (#332).
|
||||
self.assertEqual(len(selects), 5, selects)
|
||||
self.assertEqual(sum("image_categories" in q for q in selects), 1, selects)
|
||||
# Plus the one stream of tagged image records q and tags read (B3).
|
||||
self.assertEqual(len(selects), 4, selects)
|
||||
self.assertEqual(sum("image_favorites" in q for q in selects), 1, selects)
|
||||
self.assertEqual(sum("media_image_tags" in q for q in selects), 1, selects)
|
||||
|
||||
@@ -1530,16 +1528,15 @@ class TestWorkspaceImages(ImageAPITestCase):
|
||||
elsewhere = self.api.db.get_prompt_id_for_image("/elsewhere/7.png")
|
||||
self.assertEqual(
|
||||
self.api.db.get_image_link_matches(
|
||||
"", [], every_link=True, wanted=wanted, category="Night"
|
||||
"", "Night", [], every_link=True, wanted=wanted
|
||||
),
|
||||
{
|
||||
normalize_image_path(str(files["night_ab"])): (
|
||||
True,
|
||||
5,
|
||||
self.api.db.get_prompt_id_for_image(str(files["night_ab"])),
|
||||
"Night",
|
||||
),
|
||||
"/elsewhere/7.png": (True, 5, elsewhere, "Night"),
|
||||
"/elsewhere/7.png": (True, 5, elsewhere),
|
||||
},
|
||||
)
|
||||
|
||||
@@ -1591,24 +1588,6 @@ class TestWorkspaceImages(ImageAPITestCase):
|
||||
for i in range(links)
|
||||
],
|
||||
)
|
||||
# 2,000 images (one in 25 at 50,000 files) carry their own category
|
||||
# (#332), so a category filter also reads and matches those.
|
||||
from py.api.images import _favorite_root_key
|
||||
|
||||
root_key = _favorite_root_key(self.output_dir)
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
conn.executemany(
|
||||
"INSERT INTO image_categories VALUES(?,?,?,?)",
|
||||
[
|
||||
(
|
||||
root_key,
|
||||
listed[i][0].relative_to(self.output_dir).as_posix(),
|
||||
f"cat-{i % 10}",
|
||||
"2026-10-27T00:00:00+00:00",
|
||||
)
|
||||
for i in range(1, files, 25)[:2_000]
|
||||
],
|
||||
)
|
||||
return listed
|
||||
|
||||
_BUDGET_REQUESTS = (
|
||||
@@ -1699,13 +1678,7 @@ class TestWorkspaceImages(ImageAPITestCase):
|
||||
# q and tags also stream the tagged image records once
|
||||
# (#249 B3); none are tagged here, so nothing is stat'ed.
|
||||
streamed = "q" in params or "tags" in params
|
||||
# A category also reads the own categories, one primary
|
||||
# key range per source (#332).
|
||||
own = "category" in params
|
||||
self.assertEqual(len(found), 1 + streamed + own, found)
|
||||
self.assertEqual(
|
||||
sum("image_categories" in query for query in found), int(own)
|
||||
)
|
||||
self.assertEqual(len(found), 2 if streamed else 1, found)
|
||||
self.assertEqual(
|
||||
sum("media_image_tags" in query for query in found),
|
||||
int(streamed),
|
||||
@@ -1766,7 +1739,7 @@ class TestWorkspaceImages(ImageAPITestCase):
|
||||
self.assertEqual(self.api._workspace_order_cache, {})
|
||||
# The per-row ambiguity probe is an index search, never a scan.
|
||||
# The read runs it wrapped in one JSON aggregate (#354): same plan.
|
||||
sql, params = link_filter_query("lantern", ["tag-1"], every_link=True)
|
||||
sql, params = link_filter_query("lantern", "cat-3", ["tag-1"], every_link=True)
|
||||
for query in (sql, link_matches_json_query(sql)):
|
||||
with self.subTest(wrapped=query is not sql):
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
@@ -1782,25 +1755,6 @@ class TestWorkspaceImages(ImageAPITestCase):
|
||||
self.assertFalse(
|
||||
[step for step in plan if step.startswith("SCAN other")]
|
||||
)
|
||||
# The category's narrowing clause (#332) keeps every join an index
|
||||
# search, plain and wrapped with the raw categories.
|
||||
sql, params = link_filter_query(
|
||||
"", ["tag-1"], every_link=True, category="cat-3"
|
||||
)
|
||||
for query in (sql, link_matches_json_query(sql, categories=True)):
|
||||
with self.subTest(category_wrapped=query is not sql):
|
||||
with self.api.db.model.get_connection() as conn:
|
||||
plan = [
|
||||
row[3]
|
||||
for row in conn.execute("EXPLAIN QUERY PLAN " + query, params)
|
||||
]
|
||||
self.assertIn(
|
||||
"SEARCH other USING COVERING INDEX idx_image_file_path "
|
||||
"(file_path=?)",
|
||||
plan,
|
||||
)
|
||||
scans = [step for step in plan if step.startswith("SCAN")]
|
||||
self.assertTrue(all(step.startswith("SCAN gi") for step in scans), plan)
|
||||
|
||||
@unittest.skipUnless(
|
||||
PERF_BUDGET and not TRACED,
|
||||
|
||||
@@ -210,13 +210,24 @@ class JobCase(unittest.TestCase):
|
||||
node = self.node(kind)
|
||||
self.context = (self.context[0], index)
|
||||
clip = mock.Mock()
|
||||
# The node receives its switch as the queued graph holds it (#90)
|
||||
save = g[node_id]["inputs"].get("save_to_library", True)
|
||||
if kind == "PromptManagerText":
|
||||
return node.process_text(text, prompt=g, unique_id=node_id)
|
||||
return node.process_text(
|
||||
text, prompt=g, unique_id=node_id, save_to_library=save
|
||||
)
|
||||
if kind == "PromptManagerPair":
|
||||
return node.encode_prompt(
|
||||
clip, text, negative_text=negative, prompt=g, unique_id=node_id
|
||||
clip,
|
||||
text,
|
||||
negative_text=negative,
|
||||
prompt=g,
|
||||
unique_id=node_id,
|
||||
save_to_library=save,
|
||||
)
|
||||
return node.encode_prompt(clip, text, prompt=g, unique_id=node_id)
|
||||
return node.encode_prompt(
|
||||
clip, text, prompt=g, unique_id=node_id, save_to_library=save
|
||||
)
|
||||
|
||||
def cache(self, node_id, *results):
|
||||
"""Store executed results (one per list index) as ComfyUI's cache entry."""
|
||||
@@ -313,6 +324,35 @@ class TestCachedJobsCount(JobCase):
|
||||
self.job(g, cached=["pm"])
|
||||
self.assertEqual(self.db.search_prompts(), [])
|
||||
|
||||
def test_disabled_node_then_cached_jobs_counts_0(self):
|
||||
# #90 x #347: a node with Save to library off records no receipt and is
|
||||
# no run prompt, so its cached jobs replay nothing. The prompt exists
|
||||
# (an enabled run saved it), so any replayed count would show.
|
||||
for kind in (
|
||||
"PromptManager",
|
||||
"PromptManagerFocused",
|
||||
"PromptManagerSplit",
|
||||
"PromptManagerText",
|
||||
"PromptManagerPair",
|
||||
):
|
||||
with self.subTest(kind=kind):
|
||||
text = f"{LLM} not saved by {kind}"
|
||||
negative = "blurry" if kind == "PromptManagerPair" else ""
|
||||
self.first(graph(kind, negative_text=negative), text, negative)
|
||||
before = (self.count(text), self.pair_count(text, negative))
|
||||
|
||||
def off(seed):
|
||||
g = graph(kind, negative_text=negative, seed=seed)
|
||||
g["pm"]["inputs"]["save_to_library"] = False
|
||||
return g
|
||||
|
||||
self.first(off(1), text, negative)
|
||||
self.job(off(2), cached=["pm"])
|
||||
self.job(off(3), cached=["pm"])
|
||||
after = (self.count(text), self.pair_count(text, negative))
|
||||
self.assertEqual(before[0], 1)
|
||||
self.assertEqual(after, before)
|
||||
|
||||
def test_alternating_upstream_values_each_count_their_own_prompt(self):
|
||||
g = graph()
|
||||
self.first(g, text="value A")
|
||||
|
||||
@@ -143,13 +143,9 @@ class LinkMatchesTestCase(unittest.TestCase):
|
||||
|
||||
def streamed(self, q, category, tags, every_link=False, wanted=None):
|
||||
"""Today's read, called directly (the fallback)."""
|
||||
sql, params = link_filter_query(
|
||||
q, tags, every_link=every_link, category=category
|
||||
)
|
||||
sql, params = link_filter_query(q, category, tags, every_link=every_link)
|
||||
with self.db.model.get_connection() as conn:
|
||||
return operations._streamed_link_matches(
|
||||
conn, sql, params, wanted, category is not None
|
||||
)
|
||||
return operations._streamed_link_matches(conn, sql, params, wanted)
|
||||
|
||||
def selects(self):
|
||||
found = [s for s in self.statements if s.lstrip().upper().startswith("SELECT")]
|
||||
@@ -160,17 +156,17 @@ class LinkMatchesTestCase(unittest.TestCase):
|
||||
class TestOneStepLinkRead(LinkMatchesTestCase):
|
||||
def test_plain_read_returns_every_unambiguous_link(self):
|
||||
self.seed()
|
||||
matches = self.db.get_image_link_matches("", [], every_link=True)
|
||||
matches = self.db.get_image_link_matches("", None, [], every_link=True)
|
||||
self.assertEqual(
|
||||
matches,
|
||||
{
|
||||
"/out/a.png": (True, 5, 1, None),
|
||||
QUOTED: (True, None, 2, None),
|
||||
BACKSLASHED: (True, None, 2, None),
|
||||
UNICODE: (True, 3, 3, None),
|
||||
NEWLINE: (True, 5, 1, None),
|
||||
"/out/pair.png": (True, 4, 4, None),
|
||||
"/out/two-pairs.png": (True, 3, 3, None),
|
||||
"/out/a.png": (True, 5, 1),
|
||||
QUOTED: (True, None, 2),
|
||||
BACKSLASHED: (True, None, 2),
|
||||
UNICODE: (True, 3, 3),
|
||||
NEWLINE: (True, 5, 1),
|
||||
"/out/pair.png": (True, 4, 4),
|
||||
"/out/two-pairs.png": (True, 3, 3),
|
||||
},
|
||||
)
|
||||
self.assertIsNone(matches[QUOTED][1])
|
||||
@@ -180,21 +176,21 @@ class TestOneStepLinkRead(LinkMatchesTestCase):
|
||||
def test_search_hits_follow_text_title_and_tags(self):
|
||||
self.seed()
|
||||
self.assertEqual(
|
||||
self.db.get_image_link_matches("lantern", []),
|
||||
self.db.get_image_link_matches("lantern", None, []),
|
||||
{
|
||||
"/out/a.png": (True, 5, 1, None),
|
||||
NEWLINE: (True, 5, 1, None),
|
||||
UNICODE: (True, 3, 3, None),
|
||||
"/out/two-pairs.png": (True, 3, 3, None),
|
||||
"/out/a.png": (True, 5, 1),
|
||||
NEWLINE: (True, 5, 1),
|
||||
UNICODE: (True, 3, 3),
|
||||
"/out/two-pairs.png": (True, 3, 3),
|
||||
},
|
||||
)
|
||||
self.assertEqual(
|
||||
self.db.get_image_link_matches("lantern", [], category="cat-a"),
|
||||
self.db.get_image_link_matches("lantern", "cat-a", []),
|
||||
{
|
||||
"/out/a.png": (True, 5, 1, "cat-a"),
|
||||
NEWLINE: (True, 5, 1, "cat-a"),
|
||||
QUOTED: (False, None, 2, "cat-a"),
|
||||
BACKSLASHED: (False, None, 2, "cat-a"),
|
||||
"/out/a.png": (True, 5, 1),
|
||||
NEWLINE: (True, 5, 1),
|
||||
QUOTED: (False, None, 2),
|
||||
BACKSLASHED: (False, None, 2),
|
||||
},
|
||||
)
|
||||
|
||||
@@ -210,11 +206,7 @@ class TestOneStepLinkRead(LinkMatchesTestCase):
|
||||
expected = self.streamed(q, category, tags, every_link, wanted)
|
||||
self.statements.clear()
|
||||
actual = self.db.get_image_link_matches(
|
||||
q,
|
||||
list(tags),
|
||||
every_link=every_link,
|
||||
wanted=wanted,
|
||||
category=category,
|
||||
q, category, list(tags), every_link=every_link, wanted=wanted
|
||||
)
|
||||
self.assertEqual(actual, expected)
|
||||
self.assertEqual(
|
||||
@@ -226,8 +218,10 @@ class TestOneStepLinkRead(LinkMatchesTestCase):
|
||||
self.assertIn("json_group_array", found[0])
|
||||
|
||||
def test_empty_database_returns_nothing_in_one_statement(self):
|
||||
self.assertEqual(self.db.get_image_link_matches("", [], every_link=True), {})
|
||||
self.assertEqual(self.db.get_image_link_matches("x", ["t"], category="c"), {})
|
||||
self.assertEqual(
|
||||
self.db.get_image_link_matches("", None, [], every_link=True), {}
|
||||
)
|
||||
self.assertEqual(self.db.get_image_link_matches("x", "c", ["t"]), {})
|
||||
found = self.selects()
|
||||
self.assertEqual(len(found), 2, found)
|
||||
self.assertTrue(all("json_group_array" in query for query in found))
|
||||
@@ -236,51 +230,20 @@ class TestOneStepLinkRead(LinkMatchesTestCase):
|
||||
self.seed()
|
||||
self.assertEqual(
|
||||
self.db.get_image_link_matches(
|
||||
"", [], every_link=True, wanted={"/out/pair.png", "/x.png"}
|
||||
"", None, [], every_link=True, wanted={"/out/pair.png", "/x.png"}
|
||||
),
|
||||
{"/out/pair.png": (True, 4, 4, None)},
|
||||
{"/out/pair.png": (True, 4, 4)},
|
||||
)
|
||||
self.assertEqual(
|
||||
self.db.get_image_link_matches("", [], every_link=True, wanted=set()),
|
||||
self.db.get_image_link_matches("", None, [], every_link=True, wanted=set()),
|
||||
{},
|
||||
)
|
||||
|
||||
def test_category_read_carries_the_inherited_category_in_one_step(self):
|
||||
# A recipe's own category wins over its positive's (#332 MC4); the
|
||||
# read only narrows to links holding the value, the caller compares.
|
||||
self.seed()
|
||||
with self.db.model.get_connection() as conn:
|
||||
conn.execute("UPDATE prompt_pairs SET category='cat-a' WHERE id=1")
|
||||
self.statements.clear()
|
||||
for category, expected in (
|
||||
("cat-a", {"/out/pair.png": (True, 4, 4, "cat-a")}),
|
||||
("cat-b", {"/out/pair.png": (True, 4, 4, "cat-a")}),
|
||||
):
|
||||
with self.subTest(category=category):
|
||||
actual = self.db.get_image_link_matches(
|
||||
"", [], every_link=True, category=category, wanted={"/out/pair.png"}
|
||||
)
|
||||
self.assertEqual(actual, expected)
|
||||
self.assertEqual(
|
||||
actual, self.streamed("", category, [], True, {"/out/pair.png"})
|
||||
)
|
||||
found = [query for query in self.selects() if "json_group_array" in query]
|
||||
self.assertEqual(len(found), 2, found)
|
||||
self.assertTrue(all("pair_category" in query for query in found))
|
||||
|
||||
def test_plain_read_leaves_the_categories_out_of_the_json(self):
|
||||
self.seed()
|
||||
self.statements.clear()
|
||||
self.db.get_image_link_matches("", [], every_link=True)
|
||||
(query,) = self.selects()
|
||||
aggregate = query[query.index("json_array(") :].split(")", 1)[0]
|
||||
self.assertEqual(aggregate, "json_array(file_path, rating, matched, positive")
|
||||
|
||||
def test_read_releases_its_snapshot(self):
|
||||
# A statement left open after the read would pin its WAL snapshot, and
|
||||
# a TRUNCATE checkpoint from another connection would report busy.
|
||||
self.seed()
|
||||
self.assertTrue(self.db.get_image_link_matches("", [], every_link=True))
|
||||
self.assertTrue(self.db.get_image_link_matches("", None, [], every_link=True))
|
||||
other = sqlite3.connect(str(self.root / "prompts.db"))
|
||||
try:
|
||||
other.execute("UPDATE prompts SET rating=2 WHERE id=1")
|
||||
@@ -295,15 +258,12 @@ class TestOneStepLinkRead(LinkMatchesTestCase):
|
||||
|
||||
def test_wrapped_query_keeps_the_ambiguity_probe_an_index_search(self):
|
||||
self.seed()
|
||||
sql, params = link_filter_query(
|
||||
"lantern", ["t1"], every_link=True, category="cat-a"
|
||||
)
|
||||
sql, params = link_filter_query("lantern", "cat-a", ["t1"], every_link=True)
|
||||
with self.db.model.get_connection() as conn:
|
||||
plan = [
|
||||
row[3]
|
||||
for row in conn.execute(
|
||||
"EXPLAIN QUERY PLAN "
|
||||
+ operations.link_matches_json_query(sql, categories=True),
|
||||
"EXPLAIN QUERY PLAN " + operations.link_matches_json_query(sql),
|
||||
params,
|
||||
)
|
||||
]
|
||||
@@ -326,10 +286,10 @@ class TestStreamedFallback(LinkMatchesTestCase):
|
||||
) as attempts,
|
||||
self.assertLogs(LOGGER, "WARNING") as logged,
|
||||
):
|
||||
first = self.db.get_image_link_matches("", [], every_link=True)
|
||||
first = self.db.get_image_link_matches("", None, [], every_link=True)
|
||||
found = self.selects()
|
||||
self.assertEqual(attempts.call_count, 1)
|
||||
second = self.db.get_image_link_matches("", [], every_link=True)
|
||||
second = self.db.get_image_link_matches("", None, [], every_link=True)
|
||||
again = self.selects()
|
||||
self.assertEqual(first, expected)
|
||||
self.assertEqual(second, expected)
|
||||
@@ -348,11 +308,11 @@ class TestStreamedFallback(LinkMatchesTestCase):
|
||||
self.seed()
|
||||
self.add_image(2, b"/out/blob.png")
|
||||
expected = self.streamed("", None, [], True)
|
||||
self.assertEqual(expected[b"/out/blob.png"], (True, None, 2, None))
|
||||
self.assertEqual(expected[b"/out/blob.png"], (True, None, 2))
|
||||
self.statements.clear()
|
||||
with self.assertLogs(LOGGER, "WARNING") as logged:
|
||||
first = self.db.get_image_link_matches("", [], every_link=True)
|
||||
second = self.db.get_image_link_matches("", [], every_link=True)
|
||||
first = self.db.get_image_link_matches("", None, [], every_link=True)
|
||||
second = self.db.get_image_link_matches("", None, [], every_link=True)
|
||||
self.assertEqual(first, expected)
|
||||
self.assertEqual(second, expected)
|
||||
self.assertEqual(len(logged.records), 1, logged.output)
|
||||
@@ -370,7 +330,7 @@ class TestStreamedFallback(LinkMatchesTestCase):
|
||||
self.assertIn(b"\x00", expected)
|
||||
with self.assertLogs(LOGGER, "WARNING"):
|
||||
self.assertEqual(
|
||||
self.db.get_image_link_matches("", [], every_link=True), expected
|
||||
self.db.get_image_link_matches("", None, [], every_link=True), expected
|
||||
)
|
||||
self.assertFalse(operations._one_step_unsupported)
|
||||
|
||||
@@ -382,28 +342,12 @@ class TestStreamedFallback(LinkMatchesTestCase):
|
||||
conn.execute("UPDATE prompts SET rating=? WHERE id=3", (b"\x00",))
|
||||
conn.execute("PRAGMA ignore_check_constraints=OFF")
|
||||
expected = self.streamed("", None, [], True)
|
||||
self.assertEqual(expected[UNICODE], (True, b"\x00", 3, None))
|
||||
self.assertEqual(expected[UNICODE], (True, b"\x00", 3))
|
||||
with self.assertLogs(LOGGER, "WARNING"):
|
||||
self.assertEqual(
|
||||
self.db.get_image_link_matches("", [], every_link=True), expected
|
||||
self.db.get_image_link_matches("", None, [], every_link=True), expected
|
||||
)
|
||||
|
||||
def test_blob_category_streams_a_category_read(self):
|
||||
self.seed()
|
||||
with self.db.model.get_connection() as conn:
|
||||
# Admitted by its recipe's text category; the positive's is a BLOB.
|
||||
conn.execute("UPDATE prompts SET category=? WHERE id=4", (b"cat-b",))
|
||||
conn.execute("UPDATE prompt_pairs SET category='cat-b' WHERE id=1")
|
||||
expected = self.streamed("", "cat-b", [], True)
|
||||
self.assertEqual(expected["/out/pair.png"], (True, 4, 4, "cat-b"))
|
||||
with self.assertLogs(LOGGER, "WARNING") as logged:
|
||||
actual = self.db.get_image_link_matches(
|
||||
"", [], every_link=True, category="cat-b"
|
||||
)
|
||||
self.assertEqual(actual, expected)
|
||||
self.assertIn("BLOB", logged.output[0])
|
||||
self.assertFalse(operations._one_step_unsupported)
|
||||
|
||||
def test_json_text_over_the_length_limit_streams(self):
|
||||
self.seed()
|
||||
expected = self.streamed("", None, [], True)
|
||||
@@ -411,7 +355,7 @@ class TestStreamedFallback(LinkMatchesTestCase):
|
||||
limit = conn.setlimit(sqlite3.SQLITE_LIMIT_LENGTH, 200)
|
||||
try:
|
||||
with self.assertLogs(LOGGER, "WARNING") as logged:
|
||||
actual = self.db.get_image_link_matches("", [], every_link=True)
|
||||
actual = self.db.get_image_link_matches("", None, [], every_link=True)
|
||||
finally:
|
||||
conn.setlimit(sqlite3.SQLITE_LIMIT_LENGTH, limit)
|
||||
self.assertEqual(actual, expected)
|
||||
@@ -430,7 +374,7 @@ class TestStreamedFallback(LinkMatchesTestCase):
|
||||
self.assertNoLogs(LOGGER, "WARNING"),
|
||||
self.assertRaisesRegex(sqlite3.OperationalError, "^interrupted$"),
|
||||
):
|
||||
self.db.get_image_link_matches("", [], every_link=True)
|
||||
self.db.get_image_link_matches("", None, [], every_link=True)
|
||||
finally:
|
||||
conn.set_progress_handler(None, 1)
|
||||
self.assertFalse(operations._one_step_warned)
|
||||
@@ -449,7 +393,7 @@ class TestStreamedFallback(LinkMatchesTestCase):
|
||||
self.assertLogs(LOGGER, "WARNING"),
|
||||
):
|
||||
with self.assertRaises(sqlite3.OperationalError):
|
||||
self.db.get_image_link_matches("", [], every_link=True)
|
||||
self.db.get_image_link_matches("", None, [], every_link=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -1,277 +0,0 @@
|
||||
"""The inspector and History show an image's own category (#332 §5, DC4, DC5)."""
|
||||
|
||||
import os
|
||||
import sqlite3
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
from tests import test_api_history_outputs as history
|
||||
from tests.test_api_images import make_png
|
||||
from py.api.images import _favorite_root_key, image_category_key
|
||||
from utils.inspection_response import (
|
||||
OWN_CATEGORY_UNREAD,
|
||||
_library_projection,
|
||||
apply_own_category,
|
||||
)
|
||||
|
||||
|
||||
class InspectCase(history.ImageAPITestCase):
|
||||
async def tearDownAsync(self):
|
||||
self.api.db.close_all()
|
||||
await super().tearDownAsync()
|
||||
|
||||
def own(self, path, value, sources=None):
|
||||
sources = sources or self.api._get_all_output_dirs()
|
||||
self.api.db.set_image_categories([image_category_key(sources, path)], value)
|
||||
|
||||
async def library(self, path, root="0"):
|
||||
response = await self.client.get(
|
||||
f"/prompt_manager/images/inspect/{path}", params={"root": root}
|
||||
)
|
||||
body = await response.json()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
return body["library"]
|
||||
|
||||
|
||||
class TestInspectorOwnCategory(InspectCase):
|
||||
async def test_inspect_reports_image_scope(self):
|
||||
path = make_png(self.output_dir / "image.png")
|
||||
self.own(path, "Portraits")
|
||||
library = await self.library("image.png")
|
||||
self.assertEqual(library["status"], "none")
|
||||
self.assertEqual(
|
||||
(library["category"], library["category_scope"]), ("Portraits", "image")
|
||||
)
|
||||
|
||||
async def test_own_overrides_linked_recipe_and_ambiguous(self):
|
||||
db = self.api.db
|
||||
linked = make_png(self.output_dir / "linked.png")
|
||||
self._link_image(linked, "castle at dusk")
|
||||
prompt = db.get_prompt_id_for_image(str(linked))
|
||||
with db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"UPDATE prompts SET category='Landscapes' WHERE id=?", (prompt,)
|
||||
)
|
||||
library = await self.library("linked.png")
|
||||
self.assertEqual(
|
||||
(library["category"], library["category_scope"]),
|
||||
("Landscapes", "positive"),
|
||||
)
|
||||
self.own(linked, "Portraits")
|
||||
library = await self.library("linked.png")
|
||||
self.assertEqual(library["status"], "linked")
|
||||
self.assertEqual(
|
||||
(library["category"], library["category_scope"]), ("Portraits", "image")
|
||||
)
|
||||
recipe = make_png(self.output_dir / "recipe.png")
|
||||
self._save_prompt("Pair positive", category="Night")
|
||||
pair = db.save_prompt_pair("Pair positive", "Noise", category="Recipes")
|
||||
positive = db.get_prompt_pair(pair)["positive_prompt_id"]
|
||||
image = db.link_image_to_prompt(positive, str(recipe))
|
||||
db.link_image_to_pair(pair, image, {"source": "fixture"})
|
||||
library = await self.library("recipe.png")
|
||||
self.assertEqual(
|
||||
(library["category"], library["category_scope"]), ("Recipes", "pair")
|
||||
)
|
||||
self.own(recipe, "Mine")
|
||||
library = await self.library("recipe.png")
|
||||
self.assertEqual(
|
||||
(library["category"], library["category_scope"]), ("Mine", "image")
|
||||
)
|
||||
ambiguous = make_png(self.output_dir / "ambiguous.png")
|
||||
self._link_image(ambiguous, "owner one")
|
||||
self._link_image(ambiguous, "owner two")
|
||||
self.own(ambiguous, "Scans")
|
||||
library = await self.library("ambiguous.png")
|
||||
self.assertEqual(library["status"], "ambiguous")
|
||||
self.assertEqual(
|
||||
(library["category"], library["category_scope"]), ("Scans", "image")
|
||||
)
|
||||
|
||||
async def test_a_failed_read_hides_the_category(self):
|
||||
linked = make_png(self.output_dir / "linked.png")
|
||||
self._link_image(linked, "castle at dusk")
|
||||
|
||||
def broken(*_args, **_kwargs):
|
||||
raise sqlite3.OperationalError("secret")
|
||||
|
||||
with patch.object(self.api.db, "image_category", side_effect=broken):
|
||||
with self.assertLogs("prompt_manager", level="WARNING"):
|
||||
library = await self.library("linked.png")
|
||||
self.assertEqual(library["status"], "linked")
|
||||
self.assertIsNone(library["category"])
|
||||
self.assertIsNone(library["category_scope"])
|
||||
|
||||
async def test_a_lora_root_reads_no_own_category(self):
|
||||
lora = self.comfy_dir / "loras"
|
||||
make_png(lora / "preview.png")
|
||||
roots = [self.output_dir, lora]
|
||||
reads = []
|
||||
real = self.api.db.image_category
|
||||
|
||||
def recorded(*args):
|
||||
reads.append(args)
|
||||
return real(*args)
|
||||
|
||||
with (
|
||||
patch.object(self.api, "_media_roots", lambda outputs: roots),
|
||||
patch.object(self.api, "_linked_only_error", self._no_error),
|
||||
patch.object(self.api.db, "image_category", side_effect=recorded),
|
||||
):
|
||||
response = await self.client.get(
|
||||
"/prompt_manager/images/inspect/preview.png", params={"root": "1"}
|
||||
)
|
||||
body = await response.json()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
self.assertEqual(reads, [])
|
||||
self.assertIsNone(body["library"]["category"])
|
||||
|
||||
@staticmethod
|
||||
async def _no_error(*_args):
|
||||
return None
|
||||
|
||||
|
||||
class TestProjection(unittest.TestCase):
|
||||
def test_scope_image_is_kept_for_category_only(self):
|
||||
base = {"status": "none", "scope": "saved-library", "tags": []}
|
||||
projected = _library_projection(
|
||||
{**base, "category": "Scans", "category_scope": "image"}
|
||||
)
|
||||
self.assertEqual(projected["category_scope"], "image")
|
||||
projected = _library_projection({**base, "tags_scope": "image"})
|
||||
self.assertNotIn("tags_scope", projected)
|
||||
|
||||
def test_apply_own_category_three_cases(self):
|
||||
library = {"status": "linked", "category": "L", "category_scope": "positive"}
|
||||
self.assertEqual(
|
||||
apply_own_category(library, "Mine"),
|
||||
{"status": "linked", "category": "Mine", "category_scope": "image"},
|
||||
)
|
||||
self.assertEqual(apply_own_category(library, None), library)
|
||||
self.assertEqual(
|
||||
apply_own_category(library, OWN_CATEGORY_UNREAD),
|
||||
{"status": "linked", "category": None, "category_scope": None},
|
||||
)
|
||||
self.assertEqual(library["category"], "L") # never mutated
|
||||
|
||||
|
||||
class TestHistoryOwnCategory(history.ImageAPITestCase):
|
||||
"""Only the inspection request reads the own category, keyed as Images."""
|
||||
|
||||
# The History fixture's helpers, without inheriting its tests.
|
||||
_base = history.HistoryOutputAPITests
|
||||
ref = _base.ref
|
||||
capture = _base.capture
|
||||
outputs = _base.outputs
|
||||
current = _base.current
|
||||
media = _base.media
|
||||
|
||||
async def get_application(self):
|
||||
app = await super().get_application()
|
||||
self.temp_dir = self.comfy_dir / "temp"
|
||||
self.temp_dir.mkdir()
|
||||
self.host_output = str(self.output_dir)
|
||||
self.host_temp = str(self.temp_dir)
|
||||
self.patches = history.ExitStack()
|
||||
self.addCleanup(self.patches.close)
|
||||
for name, value in (
|
||||
("folder_paths.get_output_directory", lambda: self.host_output),
|
||||
("folder_paths.get_temp_directory", lambda: self.host_temp),
|
||||
):
|
||||
self.patches.enter_context(patch(name, value, create=True))
|
||||
self.patches.enter_context(
|
||||
patch.object(
|
||||
history.GalleryConfig,
|
||||
"allowed_gallery_parents",
|
||||
lambda: [str(self.comfy_dir)],
|
||||
)
|
||||
)
|
||||
return app
|
||||
|
||||
async def tearDownAsync(self):
|
||||
self.api.close_history_reads()
|
||||
self.patches.close()
|
||||
await super().tearDownAsync()
|
||||
|
||||
async def inspect(self, start):
|
||||
descriptor = await self.media(start)
|
||||
response = await self.current(descriptor["inspection_url"])
|
||||
body = await response.json()
|
||||
self.assertEqual(response.status, 200, body)
|
||||
return body["library"]
|
||||
|
||||
async def test_history_inspection_shows_own_category(self):
|
||||
path = make_png(self.output_dir / "image.png")
|
||||
self.api.db.set_image_categories(
|
||||
[image_category_key([self.output_dir], path)], "Portraits"
|
||||
)
|
||||
start = self.capture([self.ref()])
|
||||
library = await self.inspect(start)
|
||||
self.assertEqual(
|
||||
(library["category"], library["category_scope"]), ("Portraits", "image")
|
||||
)
|
||||
|
||||
async def test_nested_sources_give_the_images_key(self):
|
||||
inner = self.output_dir / "nested"
|
||||
path = make_png(inner / "image.png")
|
||||
self.api._get_all_output_dirs = lambda: [self.output_dir, inner]
|
||||
self.api.db.set_image_categories(
|
||||
[(_favorite_root_key(inner), "image.png")], "S"
|
||||
)
|
||||
start = self.capture([self.ref(subfolder="nested")])
|
||||
library = await self.inspect(start)
|
||||
self.assertEqual(library["category"], "S")
|
||||
self.assertEqual(
|
||||
image_category_key([self.output_dir, inner], path),
|
||||
(_favorite_root_key(inner), "image.png"),
|
||||
)
|
||||
|
||||
async def test_a_symlinked_output_folder_still_shows_it(self):
|
||||
real = self.comfy_dir / "real-output"
|
||||
real.mkdir()
|
||||
link = self.comfy_dir / "linked-output"
|
||||
self._make_symlink(real, link, target_is_directory=True)
|
||||
make_png(real / "image.png")
|
||||
self.host_output = str(link)
|
||||
self.api._get_all_output_dirs = lambda: [real]
|
||||
self.api.db.set_image_categories([(_favorite_root_key(real), "image.png")], "S")
|
||||
start = self.capture([self.ref()])
|
||||
library = await self.inspect(start)
|
||||
self.assertEqual(library["category"], "S")
|
||||
|
||||
async def test_outputs_and_media_read_no_own_category(self):
|
||||
make_png(self.output_dir / "image.png")
|
||||
start = self.capture([self.ref()])
|
||||
reads = []
|
||||
real = self.api.db.image_category
|
||||
|
||||
def recorded(*args):
|
||||
reads.append(args)
|
||||
return real(*args)
|
||||
|
||||
with patch.object(self.api.db, "image_category", side_effect=recorded):
|
||||
descriptor = await self.media(start)
|
||||
for field in ("url", "thumbnail_url"):
|
||||
response = await self.current(descriptor[field])
|
||||
await response.read()
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(reads, [])
|
||||
await self.current(descriptor["inspection_url"])
|
||||
self.assertEqual(len(reads), 1)
|
||||
|
||||
async def test_a_temp_output_has_none(self):
|
||||
reference = {"folder_type": "temp", "subfolder": "", "filename": "a.png"}
|
||||
self.assertIsNone(self.api._history_own_category(reference, None))
|
||||
|
||||
async def test_a_failed_read_hides_it(self):
|
||||
make_png(self.output_dir / "image.png")
|
||||
start = self.capture([self.ref()])
|
||||
|
||||
def broken(*_args):
|
||||
raise sqlite3.OperationalError("secret")
|
||||
|
||||
with patch.object(self.api.db, "image_category", side_effect=broken):
|
||||
with self.assertLogs("prompt_manager", level="WARNING"):
|
||||
library = await self.inspect(start)
|
||||
self.assertIsNone(library["category"])
|
||||
self.assertTrue(os.path.exists(self.output_dir / "image.png"))
|
||||
@@ -1426,32 +1426,3 @@ class TestPerRecordBudgets(unittest.TestCase):
|
||||
self.assertEqual(metadata["record_formats"], {"workflow": "limited"})
|
||||
self.assertEqual(metadata["workflow_graph"]["status"], "limited")
|
||||
self.assertEqual(metadata["prompt_graph"]["status"], "absent")
|
||||
|
||||
|
||||
class TestInheritedCategory(unittest.TestCase):
|
||||
"""#332 MC4: one inherited-category rule for the chip, filter and History."""
|
||||
|
||||
def test_recipe_set_blank_and_absent(self):
|
||||
from utils.inspection_response import inherited_category, library_metadata
|
||||
|
||||
cases = [
|
||||
(("Night", "Recipes", True), ("Recipes", "pair")),
|
||||
(("Night", " ", True), ("Night", "positive")),
|
||||
(("Night", " \t", True), ("Night", "positive")),
|
||||
(("Night", b" ", True), ("Night", "positive")),
|
||||
(("Night", None, True), ("Night", "positive")),
|
||||
(("Night", "Recipes", False), ("Night", "positive")),
|
||||
((" ", None, False), (" ", "positive")),
|
||||
((None, None, False), (None, "positive")),
|
||||
]
|
||||
for args, expected in cases:
|
||||
with self.subTest(args=args):
|
||||
self.assertEqual(inherited_category(*args), expected)
|
||||
positive, recipe, has_recipe = args
|
||||
metadata = library_metadata(
|
||||
positive, ["t"], (recipe, []) if has_recipe else None
|
||||
)
|
||||
self.assertEqual(
|
||||
(metadata["category"], metadata["category_scope"]), expected
|
||||
)
|
||||
self.assertEqual(metadata["tags_scope"], "positive")
|
||||
|
||||
@@ -0,0 +1,508 @@
|
||||
"""#90 item 3: a node's ``save_to_library`` switch.
|
||||
|
||||
Off: PromptManager ignores the run completely (no prompt saved, no run count,
|
||||
no image link), while the node still returns its normal outputs. On (the
|
||||
default, and every workflow or image made before the switch existed): as before.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import sys
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest import mock
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
try:
|
||||
from tests.test_prompt_attribution import MonitorCase, NodeCase
|
||||
from tests.test_execution_contexts import load_module
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_prompt_attribution import MonitorCase, NodeCase
|
||||
from test_execution_contexts import load_module
|
||||
|
||||
from prompt_manager import PromptManager, PromptManagerFocused, PromptManagerSplit
|
||||
from prompt_manager_pair import PromptManagerPair
|
||||
from prompt_manager_text import PromptManagerText
|
||||
from prompt_search_list import PromptSearchList
|
||||
from utils.hashing import generate_prompt_hash
|
||||
from utils.prompt_graph import (
|
||||
intrinsic_pair_nodes,
|
||||
positive_opt_outs,
|
||||
run_prompt_components,
|
||||
run_prompt_nodes,
|
||||
save_setting,
|
||||
)
|
||||
from utils.run_keys import prompt_node_key
|
||||
from utils.usage_tracking import (
|
||||
PendingFirstUse,
|
||||
QueuedRuns,
|
||||
handle_queued_prompt,
|
||||
on_dequeue,
|
||||
queued_pair_runs,
|
||||
queued_prompt_runs,
|
||||
)
|
||||
|
||||
POSITIVE = "a lighthouse at dusk"
|
||||
NEGATIVE = "blurry"
|
||||
NODE_CLASSES = (
|
||||
PromptManager,
|
||||
PromptManagerText,
|
||||
PromptManagerFocused,
|
||||
PromptManagerSplit,
|
||||
PromptManagerPair,
|
||||
)
|
||||
|
||||
|
||||
def run_graph(save=None, kind="PromptManager", negative_save=None):
|
||||
"""A PromptManager positive feeding a KSampler; ``save`` None leaves it out."""
|
||||
inputs = {"text": POSITIVE, "clip": ["4", 1]}
|
||||
if save is not None:
|
||||
inputs["save_to_library"] = save
|
||||
negative_inputs = {"text": NEGATIVE, "clip": ["4", 1]}
|
||||
if negative_save is not None:
|
||||
negative_inputs["save_to_library"] = negative_save
|
||||
return {
|
||||
"1": {"class_type": kind, "inputs": inputs},
|
||||
"2": {"class_type": "PromptManager", "inputs": negative_inputs},
|
||||
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
|
||||
"3": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"model": ["4", 0],
|
||||
"positive": ["1", 0],
|
||||
"negative": ["2", 0],
|
||||
"latent_image": ["6", 0],
|
||||
},
|
||||
},
|
||||
"6": {"class_type": "EmptyLatentImage", "inputs": {"batch_size": 1}},
|
||||
}
|
||||
|
||||
|
||||
def pair_graph(save=None):
|
||||
inputs = {"text": POSITIVE, "negative_text": NEGATIVE, "clip": ["4", 1]}
|
||||
if save is not None:
|
||||
inputs["save_to_library"] = save
|
||||
return {
|
||||
"1": {"class_type": "PromptManagerPair", "inputs": inputs},
|
||||
"4": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "m"}},
|
||||
"3": {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"model": ["4", 0],
|
||||
"positive": ["1", 0],
|
||||
"negative": ["1", 1],
|
||||
"latent_image": ["6", 0],
|
||||
},
|
||||
},
|
||||
"6": {"class_type": "EmptyLatentImage", "inputs": {"batch_size": 1}},
|
||||
}
|
||||
|
||||
|
||||
class TestInputDeclaration(unittest.TestCase):
|
||||
def test_every_prompt_manager_node_declares_a_boolean_defaulting_on(self):
|
||||
for cls in NODE_CLASSES:
|
||||
optional = cls.INPUT_TYPES()["optional"]
|
||||
kind, options = optional["save_to_library"]
|
||||
self.assertEqual(kind, "BOOLEAN", cls.__name__)
|
||||
self.assertIs(options["default"], True, cls.__name__)
|
||||
# Declared last: no saved widget value of an older workflow shifts
|
||||
self.assertEqual(list(optional)[-1], "save_to_library", cls.__name__)
|
||||
|
||||
def test_search_list_has_no_switch(self):
|
||||
inputs = PromptSearchList.INPUT_TYPES()
|
||||
self.assertNotIn("save_to_library", inputs.get("optional", {}))
|
||||
self.assertNotIn("save_to_library", inputs.get("required", {}))
|
||||
|
||||
|
||||
class TestSaveSetting(unittest.TestCase):
|
||||
def test_missing_input_is_on(self):
|
||||
self.assertIs(save_setting(run_graph(), "1"), True)
|
||||
|
||||
def test_literals_follow_comfyui_boolean_coercion(self):
|
||||
for value, expected in (
|
||||
(True, True),
|
||||
(False, False),
|
||||
("", False),
|
||||
(0, False),
|
||||
({"__value__": False}, False),
|
||||
({"__value__": True}, True),
|
||||
):
|
||||
with self.subTest(value=value):
|
||||
self.assertIs(save_setting(run_graph(save=value), "1"), expected)
|
||||
|
||||
def test_link_to_primitive_boolean_is_read(self):
|
||||
for value in (True, False):
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "PrimitiveBoolean", "inputs": {"value": value}}
|
||||
self.assertIs(save_setting(graph, "1"), value)
|
||||
|
||||
def test_link_to_an_unknown_node_is_not_knowable(self):
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
|
||||
self.assertIsNone(save_setting(graph, "1"))
|
||||
self.assertIsNone(save_setting(run_graph(save=["missing", 0]), "1"))
|
||||
|
||||
|
||||
class TestGraphRoles(unittest.TestCase):
|
||||
def test_opted_out_positive_is_not_a_run_prompt(self):
|
||||
graph = run_graph(save=False)
|
||||
self.assertEqual(run_prompt_nodes(graph), [])
|
||||
self.assertEqual(run_prompt_components(graph), [])
|
||||
self.assertEqual(positive_opt_outs(graph), ["1"])
|
||||
|
||||
def test_enabled_positive_is_unchanged(self):
|
||||
for save in (None, True):
|
||||
graph = run_graph(save=save)
|
||||
self.assertEqual(run_prompt_nodes(graph), ["1"])
|
||||
self.assertEqual(positive_opt_outs(graph), [])
|
||||
|
||||
def test_opted_out_negative_is_no_positive_opt_out(self):
|
||||
graph = run_graph(negative_save=False)
|
||||
self.assertEqual(run_prompt_nodes(graph), ["1"])
|
||||
self.assertEqual(positive_opt_outs(graph), [])
|
||||
|
||||
def test_walk_stops_at_an_opted_out_node(self):
|
||||
graph = run_graph(save=False)
|
||||
graph["1"]["inputs"]["text"] = ["7", 0]
|
||||
graph["7"] = {"class_type": "PromptManagerText", "inputs": {"text": "x"}}
|
||||
self.assertEqual(run_prompt_nodes(graph), [])
|
||||
|
||||
def test_sampler_less_fallback_skips_opted_out_nodes(self):
|
||||
graph = {
|
||||
"1": {
|
||||
"class_type": "PromptManagerText",
|
||||
"inputs": {"text": "a", "save_to_library": False},
|
||||
},
|
||||
"2": {"class_type": "PromptManagerText", "inputs": {"text": "b"}},
|
||||
}
|
||||
self.assertEqual(run_prompt_nodes(graph), ["2"])
|
||||
self.assertEqual(run_prompt_components(graph), [("2", "text")])
|
||||
self.assertEqual(positive_opt_outs(graph), ["1"])
|
||||
|
||||
def test_opted_out_pair_is_not_an_intrinsic_pair(self):
|
||||
self.assertEqual(intrinsic_pair_nodes(pair_graph()), ["1"])
|
||||
self.assertEqual(intrinsic_pair_nodes(pair_graph(save=False)), [])
|
||||
|
||||
|
||||
class TestQueueHook(unittest.TestCase):
|
||||
def test_opted_out_node_is_never_stashed(self):
|
||||
self.assertEqual(queued_prompt_runs(run_graph(save=False)), [])
|
||||
self.assertEqual(queued_pair_runs(pair_graph(save=False)), [])
|
||||
|
||||
def test_unknown_setting_is_left_to_the_node(self):
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
|
||||
self.assertEqual(queued_prompt_runs(graph), [])
|
||||
pair = pair_graph(save=["9", 0])
|
||||
pair["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
|
||||
self.assertEqual(queued_pair_runs(pair), [])
|
||||
|
||||
def test_primitive_boolean_on_still_counts(self):
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "PrimitiveBoolean", "inputs": {"value": True}}
|
||||
self.assertEqual(
|
||||
queued_prompt_runs(graph), [(generate_prompt_hash(POSITIVE), POSITIVE)]
|
||||
)
|
||||
|
||||
def test_dequeue_counts_nothing_for_a_cached_opted_out_node(self):
|
||||
"""A cached node never executes: only the queued graph can say it is off."""
|
||||
db = mock.Mock()
|
||||
db.get_prompt_by_hash.return_value = {"id": 5}
|
||||
queued, pending = QueuedRuns(), PendingFirstUse()
|
||||
request = {"prompt": run_graph(save=False), "prompt_id": "job-1"}
|
||||
handle_queued_prompt(request, queued, pair_queued=QueuedRuns())
|
||||
on_dequeue("job-1", queued, lambda: db, pending, pair_queued=QueuedRuns())
|
||||
db.record_prompt_use.assert_not_called()
|
||||
self.assertEqual(pending.consume(generate_prompt_hash(POSITIVE)), 0)
|
||||
|
||||
|
||||
class TestNodesSaveNothing(NodeCase):
|
||||
def run_node(self, cls, graph, save):
|
||||
node = self.node(cls)
|
||||
node.record_pair = mock.Mock()
|
||||
kwargs = {"prompt": graph, "unique_id": "1", "save_to_library": save}
|
||||
if cls is PromptManagerText:
|
||||
return node, node.process_text(POSITIVE, **kwargs)
|
||||
if cls is PromptManagerPair:
|
||||
return node, node.encode_prompt(
|
||||
self.clip(), POSITIVE, negative_text=NEGATIVE, **kwargs
|
||||
)
|
||||
return node, node.encode_prompt(self.clip(), POSITIVE, **kwargs)
|
||||
|
||||
def test_off_saves_counts_and_tracks_nothing_but_returns_outputs(self):
|
||||
for cls in NODE_CLASSES:
|
||||
with self.subTest(node=cls.__name__):
|
||||
graph = (
|
||||
pair_graph(save=False)
|
||||
if cls is PromptManagerPair
|
||||
else run_graph(save=False, kind=cls.__name__)
|
||||
)
|
||||
node, result = self.run_node(cls, graph, False)
|
||||
self.assertEqual(self.count_rows("prompts"), 0)
|
||||
self.assertEqual(self.count_rows("prompt_pairs"), 0)
|
||||
self.assertEqual(self.tracker.keyed_snapshot(), {})
|
||||
self.assertEqual(self.tracker._prompt_queue, [])
|
||||
node.comfyui_integration.register_prompt.assert_not_called()
|
||||
if cls is PromptManagerText:
|
||||
self.assertEqual(result, (POSITIVE,))
|
||||
elif cls is PromptManagerPair:
|
||||
self.assertEqual(result[0], ("cond", POSITIVE))
|
||||
self.assertEqual(result[1], ("cond", NEGATIVE))
|
||||
self.assertEqual(result[2:], (POSITIVE, NEGATIVE))
|
||||
else:
|
||||
self.assertEqual(result, (("cond", POSITIVE), POSITIVE))
|
||||
|
||||
def test_off_without_graph_context_still_saves_nothing(self):
|
||||
node = self.node(PromptManager)
|
||||
node.encode_prompt(self.clip(), POSITIVE, save_to_library=False)
|
||||
self.assertEqual(self.count_rows("prompts"), 0)
|
||||
self.assertEqual(self.tracker._prompt_queue, [])
|
||||
|
||||
def test_off_leaves_an_existing_prompt_uncounted(self):
|
||||
prompt_id = self.save(POSITIVE)
|
||||
self.run_node(PromptManager, run_graph(save=False), False)
|
||||
row = self.db.get_prompt_by_id(prompt_id)
|
||||
self.assertFalse(row.get("run_count") or 0)
|
||||
|
||||
def test_on_is_unchanged(self):
|
||||
self.run_node(PromptManager, run_graph(save=True), True)
|
||||
self.assertIsNotNone(self.db.get_prompt_by_hash(generate_prompt_hash(POSITIVE)))
|
||||
self.assertIn(
|
||||
prompt_node_key(run_graph(save=True), "1"), self.tracker.keyed_snapshot()
|
||||
)
|
||||
|
||||
def test_unknown_setting_on_at_run_time_counts_at_execution(self):
|
||||
"""The hook could not read a wired switch, so the node counts its run."""
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
|
||||
prompt_id = self.save(POSITIVE)
|
||||
self.run_node(PromptManager, graph, True)
|
||||
self.assertEqual(self.db.get_prompt_by_id(prompt_id)["run_count"], 1)
|
||||
|
||||
|
||||
class TestWiredSwitchOffMarksItsRun(NodeCase):
|
||||
"""Review finding: a wired switch off at run time keeps the node a run prompt."""
|
||||
|
||||
def wired_graph(self):
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
|
||||
return graph
|
||||
|
||||
def test_an_earlier_entry_of_the_key_never_stands_in(self):
|
||||
graph = self.wired_graph()
|
||||
key = prompt_node_key(graph, "1")
|
||||
node = self.node(PromptManager)
|
||||
node.encode_prompt(
|
||||
self.clip(), POSITIVE, prompt=graph, unique_id="1", save_to_library=True
|
||||
)
|
||||
node.encode_prompt(
|
||||
self.clip(), POSITIVE, prompt=graph, unique_id="1", save_to_library=False
|
||||
)
|
||||
latest, _run, _count = self.tracker.keyed_snapshot()[key]
|
||||
self.assertTrue(latest.get("placeholder"))
|
||||
|
||||
def test_literal_off_marks_nothing(self):
|
||||
node = self.node(PromptManager)
|
||||
node.encode_prompt(
|
||||
self.clip(),
|
||||
POSITIVE,
|
||||
prompt=run_graph(save=False),
|
||||
unique_id="1",
|
||||
save_to_library=False,
|
||||
)
|
||||
self.assertEqual(self.tracker.keyed_snapshot(), {})
|
||||
|
||||
|
||||
class TestMonitorLeavesOptedOutFilesUnlinked(MonitorCase):
|
||||
def test_existing_prompt_is_not_linked_from_metadata(self):
|
||||
self.save(POSITIVE)
|
||||
self.assertIsNone(self.process({"prompt": run_graph(save=False)}))
|
||||
|
||||
def test_find_prompt_reports_opted_out(self):
|
||||
self.save(POSITIVE)
|
||||
state, prompt = self.handler._find_prompt_from_metadata(
|
||||
{"prompt": run_graph(save=False)}
|
||||
)
|
||||
self.assertEqual((state, prompt), ("opted_out", None))
|
||||
|
||||
def test_mixed_positive_settings_leave_the_file_unlinked(self):
|
||||
self.save(POSITIVE)
|
||||
graph = run_graph(save=True)
|
||||
graph["8"] = {
|
||||
"class_type": "PromptManager",
|
||||
"inputs": {"text": "other", "save_to_library": False},
|
||||
}
|
||||
graph["5"] = {
|
||||
"class_type": "KSampler",
|
||||
"inputs": {
|
||||
"model": ["4", 0],
|
||||
"positive": ["8", 0],
|
||||
"negative": ["2", 0],
|
||||
"latent_image": ["6", 0],
|
||||
},
|
||||
}
|
||||
self.assertIsNone(self.process({"prompt": graph}))
|
||||
|
||||
def test_workflow_only_metadata_with_switch_off_is_opted_out(self):
|
||||
self.save(POSITIVE)
|
||||
metadata = {
|
||||
"text_encoder_nodes": [
|
||||
{
|
||||
"type": "PromptManager",
|
||||
"inputs": [],
|
||||
"widgets_values": [POSITIVE, "", "", "", "", "", False],
|
||||
}
|
||||
]
|
||||
}
|
||||
state, _prompt = self.handler._find_prompt_from_metadata(metadata)
|
||||
self.assertEqual(state, "opted_out")
|
||||
|
||||
def test_workflow_only_ignores_muted_nodes_and_reads_subgraphs(self):
|
||||
from utils.prompt_graph import metadata_opted_out
|
||||
|
||||
node = {
|
||||
"type": "PromptManager",
|
||||
"widgets_values": [POSITIVE, "", "", "", "", "", False],
|
||||
}
|
||||
self.assertFalse(
|
||||
metadata_opted_out({"workflow": {"nodes": [{**node, "mode": 4}]}})
|
||||
)
|
||||
self.assertTrue(
|
||||
metadata_opted_out(
|
||||
{
|
||||
"workflow": {
|
||||
"nodes": [],
|
||||
"definitions": {"subgraphs": [{"nodes": [node]}]},
|
||||
}
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
def test_unknown_setting_is_left_to_the_run(self):
|
||||
self.save(POSITIVE)
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
|
||||
state, _prompt = self.handler._find_prompt_from_metadata({"prompt": graph})
|
||||
self.assertEqual(state, "unknown")
|
||||
|
||||
def test_enabled_positive_still_links(self):
|
||||
prompt_id = self.save(POSITIVE)
|
||||
self.assertEqual(self.process({"prompt": run_graph(save=True)}), prompt_id)
|
||||
|
||||
|
||||
class TestLinkKeepsOptedOutNegativeOut(MonitorCase):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
# The real link, not the recording stub of MonitorCase
|
||||
del self.handler.link_image_to_prompt
|
||||
|
||||
def test_opted_out_negative_is_never_stored_as_a_pair(self):
|
||||
prompt_id = self.save(POSITIVE)
|
||||
metadata = {"prompt": run_graph(save=True, negative_save=False)}
|
||||
self.handler.link_image_to_prompt(self.image, {"id": prompt_id}, metadata)
|
||||
self.assertIsNone(self.db.get_prompt_by_hash(generate_prompt_hash(NEGATIVE)))
|
||||
self.assertEqual(self.count_pairs(), 0)
|
||||
linked = (
|
||||
self.db.model.get_connection()
|
||||
.execute("SELECT prompt_id FROM generated_images")
|
||||
.fetchall()
|
||||
)
|
||||
self.assertEqual([row[0] for row in linked], [prompt_id])
|
||||
|
||||
def test_opted_out_positive_links_nothing(self):
|
||||
prompt_id = self.save(POSITIVE)
|
||||
metadata = {"prompt": run_graph(save=False)}
|
||||
self.handler.link_image_to_prompt(self.image, {"id": prompt_id}, metadata)
|
||||
count = (
|
||||
self.db.model.get_connection()
|
||||
.execute("SELECT COUNT(*) FROM generated_images")
|
||||
.fetchone()[0]
|
||||
)
|
||||
self.assertEqual(count, 0)
|
||||
|
||||
def count_pairs(self):
|
||||
conn = self.db.model.get_connection()
|
||||
return conn.execute("SELECT COUNT(*) FROM prompt_pairs").fetchone()[0]
|
||||
|
||||
|
||||
class TestScanSkipsOptedOutFiles(NodeCase):
|
||||
def setUp(self):
|
||||
super().setUp()
|
||||
from py.api import admin
|
||||
|
||||
self.mixin = admin.AdminRoutesMixin.__new__(admin.AdminRoutesMixin)
|
||||
self.mixin.db = self.db
|
||||
self.mixin.logger = logging.getLogger("test.save_to_library.scan")
|
||||
self.file = Path(self.tmp.name) / "out.png"
|
||||
|
||||
def test_opted_out_file_stores_nothing(self):
|
||||
status = self.mixin._ingest_scanned_file(
|
||||
self.file, {"prompt": run_graph(save=False)}
|
||||
)
|
||||
self.assertEqual(status, "not_saved")
|
||||
self.assertEqual(self.count_rows("prompts"), 0)
|
||||
self.assertEqual(self.count_rows("generated_images"), 0)
|
||||
|
||||
def test_opted_out_file_is_never_queued_for_review(self):
|
||||
ambiguous = []
|
||||
graph = run_graph(save=False)
|
||||
graph["5"] = dict(graph["3"]) # a second sampler: ambiguous otherwise
|
||||
graph["5"]["inputs"] = {**graph["3"]["inputs"], "positive": ["2", 0]}
|
||||
status = self.mixin._ingest_scanned_file(
|
||||
self.file, {"prompt": graph}, ambiguous=ambiguous
|
||||
)
|
||||
self.assertEqual(status, "not_saved")
|
||||
self.assertEqual(ambiguous, [])
|
||||
|
||||
def test_opted_out_negative_stores_the_positive_only(self):
|
||||
status = self.mixin._ingest_scanned_file(
|
||||
self.file, {"prompt": run_graph(save=True, negative_save=False)}
|
||||
)
|
||||
self.assertEqual(status, "added")
|
||||
self.assertIsNotNone(self.db.get_prompt_by_hash(generate_prompt_hash(POSITIVE)))
|
||||
self.assertIsNone(self.db.get_prompt_by_hash(generate_prompt_hash(NEGATIVE)))
|
||||
self.assertEqual(self.count_rows("prompt_pairs"), 0)
|
||||
|
||||
def test_wired_switch_is_left_to_the_live_run(self):
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
|
||||
status = self.mixin._ingest_scanned_file(self.file, {"prompt": graph})
|
||||
self.assertEqual(status, "not_saved_wired")
|
||||
self.assertEqual(self.count_rows("prompts"), 0)
|
||||
|
||||
def test_status_has_a_scan_detail(self):
|
||||
from py.api import admin
|
||||
|
||||
group, row, _line = admin._scan_outcome(self.file, [], "not_saved")
|
||||
self.assertEqual(group, "no_metadata")
|
||||
self.assertIn("does not save", row["detail"])
|
||||
|
||||
|
||||
class TestHistoryRecordsNoText(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.m = load_module("execution_context_graph")
|
||||
|
||||
def test_opted_out_role_has_no_sources_and_no_text(self):
|
||||
recipe = self.m.project_contexts(run_graph(save=False), None).recipes[0]
|
||||
self.assertEqual(recipe.positive.sources, ())
|
||||
self.assertIsNone(recipe.positive.inferred)
|
||||
self.assertEqual(recipe.positive.reason, "not_saved")
|
||||
# The enabled negative is recorded as before
|
||||
self.assertEqual(recipe.negative.inferred.raw.value, NEGATIVE)
|
||||
|
||||
def test_unknown_setting_records_no_text_either(self):
|
||||
graph = run_graph(save=["9", 0])
|
||||
graph["9"] = {"class_type": "SomeCustomSwitch", "inputs": {}}
|
||||
recipe = self.m.project_contexts(graph, None).recipes[0]
|
||||
self.assertIsNone(recipe.positive.inferred)
|
||||
self.assertEqual(recipe.positive.reason, "not_saved")
|
||||
|
||||
def test_enabled_role_is_unchanged(self):
|
||||
recipe = self.m.project_contexts(run_graph(save=True), None).recipes[0]
|
||||
self.assertEqual(recipe.positive.inferred.raw.value, POSITIVE)
|
||||
self.assertEqual(len(recipe.positive.sources), 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+1
-291
@@ -13,11 +13,6 @@ from database.library_commands import LibraryCommandError
|
||||
from database.operations import PromptDatabase
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
try:
|
||||
from test_image_categories import category_checksum
|
||||
except ImportError:
|
||||
from tests.test_image_categories import category_checksum
|
||||
|
||||
|
||||
class TestVocabularyReads(unittest.TestCase):
|
||||
def setUp(self):
|
||||
@@ -173,8 +168,7 @@ class TestVocabularyReads(unittest.TestCase):
|
||||
self.assertEqual(page["total"], 4)
|
||||
self.assertFalse(page["has_more"])
|
||||
for row in page["categories"]:
|
||||
self.assertEqual(set(row), {"value", "prompt_count", "image_count"})
|
||||
self.assertEqual(row["image_count"], 0)
|
||||
self.assertEqual(set(row), {"value", "prompt_count"})
|
||||
self.assertEqual(
|
||||
row["prompt_count"],
|
||||
self.db.list_library(category=row["value"])["total"],
|
||||
@@ -244,111 +238,6 @@ class TestVocabularyReads(unittest.TestCase):
|
||||
with self.assertRaises(ValueError):
|
||||
self.read("list_vocabulary_tags", **options)
|
||||
|
||||
def own(self, value, *paths, root="root"):
|
||||
"""Give each relative path under ``root`` the own category ``value``."""
|
||||
self.db.set_image_categories([(root, path) for path in paths], value)
|
||||
|
||||
def all_categories(self, limit=200):
|
||||
rows, pages, after = [], [], None
|
||||
while True:
|
||||
page = self.read("list_vocabulary_categories", limit=limit, after=after)
|
||||
pages.append(page)
|
||||
rows.extend(page["categories"])
|
||||
if not page["has_more"]:
|
||||
return rows, pages
|
||||
after = page["next_key"]
|
||||
|
||||
def test_category_list_includes_image_only_values_last(self):
|
||||
self.save("Oldest", category="Zeta")
|
||||
self.save("Newer", category="Alpha")
|
||||
self.own("Zeta", "a.png", "b.png")
|
||||
self.own("Only images B", "c.png")
|
||||
self.own("Only images A", "d.png", "e.png", "gone.png")
|
||||
rows, _pages = self.all_categories()
|
||||
self.assertEqual(
|
||||
rows,
|
||||
[
|
||||
{"value": "Zeta", "prompt_count": 1, "image_count": 2},
|
||||
{"value": "Alpha", "prompt_count": 1, "image_count": 0},
|
||||
{"value": "Only images A", "prompt_count": 0, "image_count": 3},
|
||||
{"value": "Only images B", "prompt_count": 0, "image_count": 1},
|
||||
],
|
||||
)
|
||||
page = self.read("list_vocabulary_categories")
|
||||
self.assertEqual(page["total"], 4)
|
||||
|
||||
def test_category_cursor_crosses_into_image_only_values(self):
|
||||
for index in range(3):
|
||||
self.save(f"Owner {index}", category=f"P{index}")
|
||||
self.own("P1", "shared.png")
|
||||
self.own("I-b", "x.png")
|
||||
self.own("I-a", "y.png")
|
||||
self.own("I-c", "z.png", root="other")
|
||||
for limit in (1, 2, 3, 4):
|
||||
with self.subTest(limit=limit):
|
||||
rows, pages = self.all_categories(limit=limit)
|
||||
self.assertEqual(
|
||||
[row["value"] for row in rows],
|
||||
["P0", "P1", "P2", "I-a", "I-b", "I-c"],
|
||||
)
|
||||
self.assertEqual({page["total"] for page in pages}, {6})
|
||||
self.assertEqual(
|
||||
[row["image_count"] for row in rows], [0, 1, 0, 1, 1, 1]
|
||||
)
|
||||
first = self.read("list_vocabulary_categories", limit=3)
|
||||
self.assertEqual(first["next_key"], [3, "P2"])
|
||||
second = self.read("list_vocabulary_categories", limit=1, after=[3, "P2"])
|
||||
self.assertEqual(second["next_key"], [2**63 - 1, "I-a"])
|
||||
|
||||
def test_image_rows_on_a_prompt_value_count_once(self):
|
||||
self.save("Owner", category="Shared")
|
||||
self.db.save_prompt_pair("Owner", "Noise", category="Shared")
|
||||
self.own("Shared", *[f"{n}.png" for n in range(5)])
|
||||
self.own("Shared", "elsewhere.png", root="second")
|
||||
rows, _pages = self.all_categories()
|
||||
self.assertEqual(
|
||||
rows, [{"value": "Shared", "prompt_count": 1, "image_count": 6}]
|
||||
)
|
||||
|
||||
def test_library_options_ignore_image_categories(self):
|
||||
self.save("Owner", category="Prompt value")
|
||||
before = self.db.get_library_options()
|
||||
self.own("Image value", "a.png")
|
||||
self.own("Prompt value", "b.png")
|
||||
self.assertEqual(self.db.get_library_options(), before)
|
||||
self.assertEqual(before["categories"], ["Prompt value"])
|
||||
|
||||
def test_category_reads_use_the_image_category_index(self):
|
||||
from database import vocabulary
|
||||
|
||||
self.own("Value", "a.png")
|
||||
statements = [
|
||||
vocabulary._CATEGORY_ROWS + "SELECT COUNT(*) FROM counted",
|
||||
vocabulary._CATEGORY_ROWS
|
||||
+ "SELECT value,first_pid,prompt_count,image_count FROM counted "
|
||||
"WHERE first_pid>? OR (first_pid=? AND value>?) "
|
||||
"ORDER BY first_pid,value LIMIT ?",
|
||||
]
|
||||
plans = []
|
||||
with self.db.model.get_connection() as conn:
|
||||
for sql in statements:
|
||||
params = [1] * sql.count("?")
|
||||
plans.extend(
|
||||
row[-1] for row in conn.execute("EXPLAIN QUERY PLAN " + sql, params)
|
||||
)
|
||||
for sql in (
|
||||
vocabulary._IMAGE_CATEGORY_ROWIDS,
|
||||
*vocabulary._CATEGORY_UPDATES,
|
||||
):
|
||||
params = [1] * sql.count("?")
|
||||
plans.extend(
|
||||
row[-1] for row in conn.execute("EXPLAIN QUERY PLAN " + sql, params)
|
||||
)
|
||||
image_steps = [step for step in plans if "image_categories" in step]
|
||||
self.assertTrue(image_steps)
|
||||
for step in image_steps:
|
||||
self.assertIn("idx_image_categories_category", step, step)
|
||||
|
||||
def test_reads_share_one_snapshot_and_report_epoch(self):
|
||||
self.save("Cat", category="Animal", tags=["cat"])
|
||||
epoch = self.db.get_execution_epoch()
|
||||
@@ -852,7 +741,6 @@ class TestVocabularyCategoryCommands(unittest.TestCase):
|
||||
"other_prompts": 1,
|
||||
"negative_variants": 2,
|
||||
"archived_negative_variants": 1,
|
||||
"images": 0,
|
||||
},
|
||||
)
|
||||
self.assertFalse(result["target_exists"])
|
||||
@@ -922,7 +810,6 @@ class TestVocabularyCategoryCommands(unittest.TestCase):
|
||||
"other_prompts": 0,
|
||||
"negative_variants": 0,
|
||||
"archived_negative_variants": 0,
|
||||
"images": 0,
|
||||
},
|
||||
)
|
||||
self.assertEqual(self.rows("prompts")[variant]["category"], "Other")
|
||||
@@ -1015,171 +902,6 @@ class TestVocabularyCategoryCommands(unittest.TestCase):
|
||||
self.apply(bad)
|
||||
self.assertEqual(self.snapshot(), before)
|
||||
|
||||
def own(self, value, *paths, root="root"):
|
||||
self.db.set_image_categories([(root, path) for path in paths], value)
|
||||
|
||||
def image_rows(self):
|
||||
with self.db.model.get_connection() as conn:
|
||||
return {
|
||||
(row[0], row[1]): (row[2], row[3], row[4])
|
||||
for row in conn.execute(
|
||||
"SELECT root_key,relative_path,category,updated_at,rowid "
|
||||
"FROM image_categories ORDER BY rowid"
|
||||
)
|
||||
}
|
||||
|
||||
def checksum(self):
|
||||
with self.db.model.get_connection() as conn:
|
||||
return category_checksum(conn)
|
||||
|
||||
def test_preview_counts_image_rows(self):
|
||||
self.owners()
|
||||
self.own("Scenery", "a.png", "b.png", "missing.png")
|
||||
self.own("Scenery", "c.png", root="second")
|
||||
self.own("Other", "d.png")
|
||||
before = self.image_rows()
|
||||
result = self.preview()
|
||||
self.assertEqual(result["impact"]["images"], 4)
|
||||
self.assertEqual(self.image_rows(), before)
|
||||
self.assertTrue(self.preview(target="Other")["target_exists"])
|
||||
|
||||
def test_apply_renames_image_categories(self):
|
||||
self.owners()
|
||||
bystander = self.save("Bystander", category="Other")
|
||||
self.own("Scenery", "a.png", "b.png")
|
||||
self.own("Scenery", "c.png", root="second")
|
||||
self.own(" Scenery ", "padded.png")
|
||||
self.own("Other", "d.png")
|
||||
self.age_rows()
|
||||
with self.db.model.get_connection() as conn:
|
||||
conn.execute("UPDATE image_categories SET updated_at=?", (_OLD_STAMP,))
|
||||
images_before = self.image_rows()
|
||||
prompts_before, pairs_before = self.snapshot()
|
||||
preview = self.preview()
|
||||
receipt = self.apply(preview["impact_token"])
|
||||
self.assertEqual(receipt["impact"]["images"], 3)
|
||||
images_after = self.image_rows()
|
||||
self.assertEqual(set(images_after), set(images_before))
|
||||
renamed = {("root", "a.png"), ("root", "b.png"), ("second", "c.png")}
|
||||
for key, (value, stamp, rowid) in images_after.items():
|
||||
old_value, old_stamp, old_rowid = images_before[key]
|
||||
self.assertEqual(rowid, old_rowid, key)
|
||||
if key in renamed:
|
||||
self.assertEqual(value, "Landscape")
|
||||
self.assertNotEqual(stamp, _OLD_STAMP)
|
||||
else:
|
||||
self.assertEqual((value, stamp), (old_value, old_stamp))
|
||||
prompts_after, pairs_after = self.snapshot()
|
||||
self.assertEqual(prompts_after[bystander], prompts_before[bystander])
|
||||
for before, after in (
|
||||
(prompts_before, prompts_after),
|
||||
(pairs_before, pairs_after),
|
||||
):
|
||||
for row_id, row in after.items():
|
||||
if before[row_id]["category"] != "Scenery":
|
||||
self.assertEqual(row, before[row_id])
|
||||
# Merging into a value images already hold keeps every file key.
|
||||
merge = self.preview(source="Landscape", target="Other")
|
||||
self.assertTrue(merge["target_exists"])
|
||||
self.assertEqual(merge["impact"]["images"], 3)
|
||||
receipt = self.apply(merge["impact_token"], source="Landscape", target="Other")
|
||||
self.assertTrue(receipt["consolidated"])
|
||||
merged = self.image_rows()
|
||||
self.assertEqual(set(merged), set(images_before))
|
||||
self.assertEqual(
|
||||
sorted(key for key, row in merged.items() if row[0] == "Other"),
|
||||
sorted([*renamed, ("root", "d.png")]),
|
||||
)
|
||||
|
||||
def test_image_only_value_can_be_renamed(self):
|
||||
self.save("Bystander", category="Kept")
|
||||
self.db.save_prompt_pair("Pair", "Noise", category="Kept")
|
||||
self.own("Image only", "a.png", "b.png")
|
||||
self.age_rows()
|
||||
before = self.checksum()
|
||||
preview = self.preview(source="Image only", target="Renamed")
|
||||
self.assertEqual(
|
||||
preview["impact"],
|
||||
{
|
||||
"positive_prompts": 0,
|
||||
"other_prompts": 0,
|
||||
"negative_variants": 0,
|
||||
"archived_negative_variants": 0,
|
||||
"images": 2,
|
||||
},
|
||||
)
|
||||
self.assertFalse(preview["target_exists"])
|
||||
receipt = self.apply(
|
||||
preview["impact_token"], source="Image only", target="Renamed"
|
||||
)
|
||||
self.assertFalse(receipt["consolidated"])
|
||||
self.assertEqual({row[0] for row in self.image_rows().values()}, {"Renamed"})
|
||||
self.assertEqual(self.checksum(), before)
|
||||
# An image-only target counts as existing.
|
||||
self.own("Image target", "c.png")
|
||||
self.assertTrue(
|
||||
self.preview(source="Renamed", target="Image target")["target_exists"]
|
||||
)
|
||||
# Nothing holds the value any more: refused.
|
||||
with self.assertRaises(LibraryCommandError) as caught:
|
||||
self.preview(source="Image only", target="Again")
|
||||
self.assertEqual(caught.exception.code, "vocabulary-conflict")
|
||||
|
||||
def test_token_refused_after_an_image_row_gains_or_loses_the_value(self):
|
||||
self.owners()
|
||||
self.own("Scenery", "a.png", "b.png")
|
||||
|
||||
def gain():
|
||||
self.own("Scenery", "late.png")
|
||||
|
||||
def lose():
|
||||
self.db.clear_image_categories([("root", "a.png")])
|
||||
|
||||
def move_away():
|
||||
self.own("Elsewhere", "b.png")
|
||||
|
||||
def target_on_image():
|
||||
self.own("Landscape", "target.png")
|
||||
|
||||
for mutate in (gain, lose, move_away, target_on_image):
|
||||
with self.subTest(mutate.__name__):
|
||||
token = self.preview()["impact_token"]
|
||||
mutate()
|
||||
before = self.snapshot(), self.image_rows()
|
||||
with self.assertRaises(LibraryCommandError) as caught:
|
||||
self.apply(token)
|
||||
self.assertEqual(caught.exception.code, "vocabulary-conflict")
|
||||
self.assertEqual((self.snapshot(), self.image_rows()), before)
|
||||
|
||||
def test_image_category_failure_rolls_back_prompts(self):
|
||||
self.owners()
|
||||
self.own("Scenery", "a.png")
|
||||
token = self.preview()["impact_token"]
|
||||
with self.db.model.get_connection() as conn:
|
||||
conn.execute(
|
||||
"CREATE TRIGGER abort_images BEFORE UPDATE ON image_categories "
|
||||
"BEGIN SELECT RAISE(ABORT, 'boom'); END"
|
||||
)
|
||||
before = self.snapshot(), self.image_rows()
|
||||
with self.assertRaises(sqlite3.Error):
|
||||
self.apply(token)
|
||||
self.assertEqual((self.snapshot(), self.image_rows()), before)
|
||||
|
||||
def test_category_exists_reads_the_image_index(self):
|
||||
from database import vocabulary
|
||||
|
||||
with self.db.model.get_connection() as conn:
|
||||
plan = [
|
||||
row[-1]
|
||||
for row in conn.execute(
|
||||
"EXPLAIN QUERY PLAN " + vocabulary._CATEGORY_EXISTS, {"v": "x"}
|
||||
)
|
||||
]
|
||||
steps = [step for step in plan if "image_categories" in step]
|
||||
self.assertTrue(steps, plan)
|
||||
for step in steps:
|
||||
self.assertIn("idx_image_categories_category", step)
|
||||
|
||||
def test_category_apply_failure_rolls_back(self):
|
||||
self.owners()
|
||||
token = self.preview()["impact_token"]
|
||||
@@ -1220,18 +942,6 @@ class TestSharedTokenHashing(unittest.TestCase):
|
||||
),
|
||||
"cefe7f6a5544b7f6a203f5527810899c6af568aa7abc1b75adffbfb5ae6251ae",
|
||||
)
|
||||
with_images = category_impact_token(
|
||||
"e", "S", "T", True, [(1, True)], [], images=[9, 4]
|
||||
)
|
||||
self.assertRegex(with_images, r"\A[0-9a-f]{64}\Z")
|
||||
self.assertEqual(
|
||||
with_images,
|
||||
category_impact_token("e", "S", "T", True, [(1, True)], [], images=[4, 9]),
|
||||
)
|
||||
self.assertNotEqual(
|
||||
with_images,
|
||||
category_impact_token("e", "S", "T", True, [(1, True)], [], images=[4]),
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+2
-58
@@ -2,14 +2,12 @@ import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { createServer } from 'node:net';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { EventEmitter, once } from 'node:events';
|
||||
import { once } from 'node:events';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import { isAbsolute } from 'node:path';
|
||||
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||
import { setTimeout as delay } from 'node:timers/promises';
|
||||
import { startBrowserServer, createServerSupervisor, attachServerStderr, forwardPrefixedLines, SLOW_RESET_PREFIX } from '../../tools/vendor/start-browser-server.mjs';
|
||||
import { reporters, workflowWorkers } from '../../tools/vendor/test-workers.mjs';
|
||||
import RunningTestsReporter from '../../tools/vendor/running-tests-reporter.mjs';
|
||||
import { workflowWorkers } from '../../tools/vendor/test-workers.mjs';
|
||||
|
||||
const fakeServer = fileURLToPath(new URL('./fake-browser-server.mjs', import.meta.url));
|
||||
const fake = (...extra) => ({ command: process.execPath, args: port => [fakeServer, '--port', String(port), ...extra] });
|
||||
@@ -257,57 +255,3 @@ test('worker count follows PM_TEST_WORKERS, then one in CI, then a quarter of th
|
||||
assert.throws(() => workflowWorkers({ env: { PM_TEST_WORKERS: 'many' }, cpus: 8 }), /PM_TEST_WORKERS must be a positive integer/);
|
||||
assert.throws(() => workflowWorkers({ env: { PM_TEST_WORKERS: '0' }, cpus: 8 }), /PM_TEST_WORKERS must be a positive integer/);
|
||||
});
|
||||
|
||||
test('CI adds the running-tests reporter and a blob report under test-results/blob/, which the trace upload covers; local runs keep list', () => {
|
||||
assert.equal(reporters({ env: {}, name: 'workflows' }), 'list');
|
||||
const [list, running, blob] = reporters({ env: { CI: '1' }, name: 'workflows' });
|
||||
assert.deepEqual(list, ['list']);
|
||||
// Absolute: a config imported from another directory (workflow-artifacts.test.mjs) still finds it.
|
||||
assert.ok(isAbsolute(running[0]) && running[0].endsWith('running-tests-reporter.mjs'), running[0]);
|
||||
assert.deepEqual(blob, ['blob', { outputDir: './test-results/blob/workflows' }]);
|
||||
});
|
||||
|
||||
function fakeHost() {
|
||||
const host = new EventEmitter();
|
||||
Object.assign(host, { pid: 4242, signals: [], cwd: () => '/repo/tools/vendor' });
|
||||
host.kill = (pid, signal) => host.signals.push([pid, signal]);
|
||||
return host;
|
||||
}
|
||||
|
||||
const spec = (id, title, line) => ({ id, title, location: { file: '/repo/tests/browser/a.spec.mjs', line },
|
||||
parent: { project: () => ({ name: 'firefox' }) } });
|
||||
|
||||
test('the running-tests reporter names only the tests that never ended, relative to rootDir', () => {
|
||||
let clock = 1_000;
|
||||
const output = [];
|
||||
const host = fakeHost();
|
||||
const reporter = new RunningTestsReporter({ now: () => clock, write: text => output.push(text), host });
|
||||
reporter.onBegin({ rootDir: '/repo/tests/browser' });
|
||||
const done = spec('1', 'finishes', 10);
|
||||
const stuck = spec('2', 'hangs on a fetch', 20);
|
||||
reporter.onTestBegin(done, { retry: 0 });
|
||||
reporter.onTestBegin(stuck, { retry: 0 });
|
||||
clock += 500;
|
||||
reporter.onTestEnd(done, { retry: 0 });
|
||||
clock += 41_500;
|
||||
reporter.onEnd({ status: 'interrupted' });
|
||||
assert.equal(output.join(''), '\nStill running when the run ended (interrupted): 1\n'
|
||||
+ ' [firefox] › a.spec.mjs:20 › hangs on a fetch (running 42.0s)\n');
|
||||
assert.equal(host.listenerCount('SIGTERM'), 0, 'the run is over: SIGTERM gets its default again');
|
||||
output.length = 0;
|
||||
reporter.onTestEnd(stuck, { retry: 0 });
|
||||
reporter.onEnd({ status: 'passed' });
|
||||
assert.deepEqual(output, [], 'a run where every test ended prints nothing');
|
||||
});
|
||||
|
||||
test('on the first SIGTERM the reporter prints the running tests and hands over to Playwright\'s SIGINT', () => {
|
||||
const output = [];
|
||||
const host = fakeHost();
|
||||
const reporter = new RunningTestsReporter({ now: () => 0, write: text => output.push(text), host });
|
||||
reporter.onBegin({ rootDir: '/repo/tests/browser' });
|
||||
reporter.onTestBegin(spec('2', 'hangs on a fetch', 20), { retry: 0 });
|
||||
host.emit('SIGTERM');
|
||||
assert.match(output.join(''), /Still running when the run ended \(SIGTERM\): 1\n {2}\[firefox\] › a\.spec\.mjs:20 › hangs on a fetch/);
|
||||
assert.deepEqual(host.signals, [[4242, 'SIGINT']], 'Playwright\'s own interrupt ends the run and writes the blob');
|
||||
assert.equal(host.listenerCount('SIGTERM'), 0, 'a second SIGTERM kills the process as usual');
|
||||
});
|
||||
|
||||
Vendored
-20
@@ -247,26 +247,6 @@ test('browser job timeouts keep a real margin over measured runs', () => {
|
||||
}
|
||||
});
|
||||
|
||||
// A hang (#425) must end inside the test step, before the job limit, so the log and the always()
|
||||
// uploads survive: the step's timeout-minutes < the job's.
|
||||
const BROWSER_TEST_STEP = /npm run --prefix tools\/vendor test:(workflows|coverage)/;
|
||||
|
||||
test('each browser test step stops itself before its job limit and its traces upload anyway', () => {
|
||||
for (const id of browserJobIds) {
|
||||
const body = allJobs.get(id);
|
||||
const jobMinutes = Number(/^ {4}timeout-minutes: (\d+)$/m.exec(body)[1]);
|
||||
const jobSteps = steps(body);
|
||||
const testAt = jobSteps.findIndex(step => BROWSER_TEST_STEP.test(step));
|
||||
const testStep = jobSteps[testAt];
|
||||
const stepMinutes = Number(/^ {8}timeout-minutes: (\d+)$/m.exec(testStep)?.[1]);
|
||||
assert.ok(stepMinutes > 0, `${id}: the test step sets its own timeout-minutes`);
|
||||
assert.ok(jobMinutes - stepMinutes >= 2, `${id}: the step limit leaves time for setup and the uploads`);
|
||||
const upload = jobSteps.slice(testAt + 1).find(step => /^ {10}path: tools\/vendor\/test-results\/$/m.test(step));
|
||||
assert.ok(upload, `${id} uploads tools/vendor/test-results/ (traces and blob report) after the tests`);
|
||||
assert.match(upload, /^ {8}if: always\(\)$/m, `${id} uploads them when the step failed or timed out`);
|
||||
}
|
||||
});
|
||||
|
||||
// Every job is bounded (#350): a hang ends at a measured limit instead of GitHub's 6-hour default.
|
||||
const GATING_WORKFLOWS = ['test.yml', 'code-quality.yml', 'viewerjs-check.yml'];
|
||||
const BOUNDED_WORKFLOWS = [...GATING_WORKFLOWS, 'portability.yml'];
|
||||
|
||||
Vendored
+4
-3
@@ -35,15 +35,16 @@ test('execution script discovers only the new matrix and preserves all 78 origin
|
||||
assert.match(executions, /\[nodes2-start\]/);
|
||||
// + the fresh-host library location case, + #342 image attribution, + #343 linked text,
|
||||
// + #332 image category routes on a generated output (image-attribution.spec.mjs, 2 projects),
|
||||
// + #347 cached runtime-linked runs: five node kinds and eight one-renderer cases (execution-text-link.spec.mjs, 2 projects)
|
||||
assert.match(executions, /Total: 118 tests in 5 files/);
|
||||
// + #347 cached runtime-linked runs: five node kinds and eight one-renderer cases (execution-text-link.spec.mjs, 2 projects),
|
||||
// + #90 Save to library off (five node types)
|
||||
assert.match(executions, /Total: 128 tests in 5 files/);
|
||||
for (const project of ['classic-start', 'nodes2-start']) {
|
||||
const cases = executions.split('\n').filter(line => line.includes(`[${project}]`));
|
||||
assert.equal(cases.filter(line => line.includes('execution-history.spec.mjs')).length, 15);
|
||||
assert.equal(cases.filter(line => line.includes('execution-outputs.spec.mjs')).length, 8);
|
||||
assert.equal(cases.filter(line => line.includes('library-location.spec.mjs')).length, 1);
|
||||
assert.equal(cases.filter(line => line.includes('image-attribution.spec.mjs')).length, 3);
|
||||
assert.equal(cases.filter(line => line.includes('execution-text-link.spec.mjs')).length, 32);
|
||||
assert.equal(cases.filter(line => line.includes('execution-text-link.spec.mjs')).length, 37);
|
||||
}
|
||||
assert.doesNotMatch(executions, /prompt-manager.spec/);
|
||||
});
|
||||
|
||||
+2
-47
@@ -1,11 +1,9 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { existsSync, mkdtempSync, mkdirSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { EventEmitter } from 'node:events';
|
||||
import { setTimeout as delay } from 'node:timers/promises';
|
||||
import { existsSync, mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { plan, runCoverage, runNode } from '../../tools/vendor/frontend-coverage.mjs';
|
||||
import { plan, runCoverage } from '../../tools/vendor/frontend-coverage.mjs';
|
||||
|
||||
const steps = argv => plan(argv).map(({ step, ...options }) => (Object.keys(options).length ? { step, ...options } : step));
|
||||
|
||||
@@ -93,46 +91,3 @@ test('shard 1 runs the node units into the shared raw dir, then the viewer suite
|
||||
assert.ok(calls[1].args.includes(join(paths.tools, 'playwright.config.mjs')));
|
||||
assert.deepEqual(calls[2].args.slice(-2), ['--shard', '1/3']);
|
||||
});
|
||||
|
||||
// #425: npm and a CI step timeout signal only frontend-coverage.mjs; its running child must get it.
|
||||
function hostWithSignals() {
|
||||
return new EventEmitter();
|
||||
}
|
||||
|
||||
test('runNode resolves on exit 0, rejects with the exit code otherwise and leaves no signal listener', async () => {
|
||||
const host = hostWithSignals();
|
||||
await runNode(['-e', 'process.exit(0)'], {}, { host });
|
||||
await assert.rejects(runNode(['-e', 'process.exit(3)'], {}, { host }), /Command failed \(3\): -e process\.exit\(3\)/);
|
||||
assert.equal(host.listenerCount('SIGTERM') + host.listenerCount('SIGINT'), 0);
|
||||
});
|
||||
|
||||
test('runNode forwards SIGTERM to the running child, which can report before it exits', async t => {
|
||||
const root = mkdtempSync(join(tmpdir(), 'pm-coverage-signal-'));
|
||||
t.after(() => rmSync(root, { recursive: true, force: true }));
|
||||
const ready = join(root, 'ready');
|
||||
const seen = join(root, 'seen');
|
||||
const child = `const fs = require('node:fs');
|
||||
process.on('SIGTERM', () => { fs.writeFileSync(${JSON.stringify(seen)}, 'SIGTERM'); process.exit(7); });
|
||||
fs.writeFileSync(${JSON.stringify(ready)}, ''); setInterval(() => {}, 1000);`;
|
||||
const host = hostWithSignals();
|
||||
const running = runNode(['-e', child], {}, { host });
|
||||
while (!existsSync(ready)) await delay(20);
|
||||
host.emit('SIGTERM');
|
||||
await assert.rejects(running, /Command failed \(7\)/);
|
||||
assert.equal(readFileSync(seen, 'utf8'), 'SIGTERM', 'the child handled the forwarded signal');
|
||||
assert.equal(host.listenerCount('SIGTERM'), 0);
|
||||
});
|
||||
|
||||
test('runNode forwards SIGINT and starts the child in its own process group', async () => {
|
||||
const host = hostWithSignals();
|
||||
const kills = [];
|
||||
let options;
|
||||
const fake = new EventEmitter();
|
||||
fake.kill = signal => { kills.push(signal); queueMicrotask(() => fake.emit('exit', null, signal)); };
|
||||
const running = runNode(['x.mjs'], {}, { host, spawnChild: (command, args, spawnOptions) => { options = spawnOptions; return fake; } });
|
||||
assert.equal(options.detached, true, 'a group-wide signal reaches the child only through forwarding, once');
|
||||
host.emit('SIGINT');
|
||||
await assert.rejects(running, /Command failed \(null\): x\.mjs/);
|
||||
assert.deepEqual(kills, ['SIGINT']);
|
||||
assert.equal(host.listenerCount('SIGINT'), 0);
|
||||
});
|
||||
|
||||
@@ -107,6 +107,12 @@
|
||||
"functions": 98,
|
||||
"statements": 98
|
||||
},
|
||||
"save-to-library.js": {
|
||||
"lines": 98,
|
||||
"branches": 98,
|
||||
"functions": 98,
|
||||
"statements": 98
|
||||
},
|
||||
"scan-progress.js": {
|
||||
"lines": 98,
|
||||
"branches": 98,
|
||||
|
||||
Vendored
+9
-26
@@ -1,7 +1,7 @@
|
||||
import { existsSync, readFileSync, readdirSync, mkdirSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { dirname, join, relative } from 'node:path';
|
||||
import { fileURLToPath, pathToFileURL } from 'node:url';
|
||||
import { spawn } from 'node:child_process';
|
||||
import { spawnSync } from 'node:child_process';
|
||||
import { repositoryRoot } from './vendor-viewerjs.mjs';
|
||||
|
||||
import { discoverSources, checkCoverage } from '@comfyassets/toolkit/testing/coverage';
|
||||
@@ -10,27 +10,10 @@ const metrics = ['lines', 'branches', 'functions', 'statements'];
|
||||
const SHARD = /^([1-9]\d*)\/([1-9]\d*)$/;
|
||||
const USAGE = 'usage: frontend-coverage.mjs [--update-baseline] | --shard I/N | --merge-only [--update-baseline]';
|
||||
|
||||
const FORWARDED_SIGNALS = ['SIGTERM', 'SIGINT'];
|
||||
|
||||
// Runs one step's node child and waits for it. npm and a CI step timeout signal only this process,
|
||||
// so SIGTERM and SIGINT are forwarded to the running child (#425): Playwright then reports the tests
|
||||
// still running and writes its blob. The child gets its own process group, so a signal sent to the
|
||||
// whole group (Ctrl-C, `timeout`) still reaches it exactly once, through this forwarding.
|
||||
export function runNode(args, env = {}, { spawnChild = spawn, host = process } = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawnChild(process.execPath, args, { cwd: repositoryRoot, stdio: 'inherit',
|
||||
env: { ...process.env, ...env }, detached: true });
|
||||
const forward = signal => () => child.kill(signal);
|
||||
const handlers = FORWARDED_SIGNALS.map(signal => [signal, forward(signal)]);
|
||||
for (const [signal, handler] of handlers) host.on(signal, handler);
|
||||
const done = () => { for (const [signal, handler] of handlers) host.removeListener(signal, handler); };
|
||||
child.once('error', error => { done(); reject(error); });
|
||||
child.once('exit', code => {
|
||||
done();
|
||||
if (code === 0) resolve();
|
||||
else reject(new Error(`Command failed (${code}): ${args.join(' ')}`));
|
||||
});
|
||||
});
|
||||
function runNode(args, env = {}) {
|
||||
const result = spawnSync(process.execPath, args, { cwd: repositoryRoot, stdio: 'inherit',
|
||||
env: { ...process.env, ...env } });
|
||||
if (result.status !== 0) throw new Error(`Command failed (${result.status}): ${args.join(' ')}`);
|
||||
}
|
||||
|
||||
function defaultPaths() {
|
||||
@@ -129,17 +112,17 @@ export async function runCoverage({ argv = process.argv.slice(2), run = runNode,
|
||||
'node-units'() {
|
||||
const tests = readdirSync(join(root, 'tests/js')).filter(file => file.endsWith('.test.js'))
|
||||
.map(file => `tests/js/${file}`);
|
||||
return run(['--test', ...tests], { NODE_V8_COVERAGE: raw });
|
||||
run(['--test', ...tests], { NODE_V8_COVERAGE: raw });
|
||||
},
|
||||
viewer() {
|
||||
return run([playwright, 'test', '--config', join(tools, 'playwright.config.mjs')]);
|
||||
run([playwright, 'test', '--config', join(tools, 'playwright.config.mjs')]);
|
||||
},
|
||||
workflows({ shard }) {
|
||||
return run([playwright, 'test', '--config', join(tools, 'playwright.workflows.config.mjs'),
|
||||
run([playwright, 'test', '--config', join(tools, 'playwright.workflows.config.mjs'),
|
||||
'--project', 'chromium', '--project', 'touch', ...(shard ? ['--shard', shard] : [])], { PM_JS_COVERAGE: raw });
|
||||
},
|
||||
report() {
|
||||
return run([join(tools, 'node_modules/c8/bin/c8.js'), 'report', '--all', '--src', sourceRoot,
|
||||
run([join(tools, 'node_modules/c8/bin/c8.js'), 'report', '--all', '--src', sourceRoot,
|
||||
'--include', 'web/js/**/*.js', '--include', 'web/js/**/*.mjs', '--temp-directory', raw, '--reports-dir', reportRoot,
|
||||
'--reporter', 'text', '--reporter', 'json-summary', '--reporter', 'html']);
|
||||
},
|
||||
|
||||
Vendored
+1
-2
@@ -1,5 +1,4 @@
|
||||
import { defineConfig } from '@playwright/test';
|
||||
import { reporters } from './test-workers.mjs';
|
||||
|
||||
export default defineConfig({
|
||||
testDir: './tests',
|
||||
@@ -9,7 +8,7 @@ export default defineConfig({
|
||||
workers: 2,
|
||||
forbidOnly: Boolean(process.env.CI),
|
||||
retries: process.env.CI ? 1 : 0,
|
||||
reporter: reporters({ name: 'viewer' }),
|
||||
reporter: 'list',
|
||||
use: { browserName: 'chromium', headless: true, trace: 'retain-on-failure' },
|
||||
projects: [
|
||||
{ name: 'desktop', use: { viewport: { width: 1280, height: 800 } } },
|
||||
|
||||
+2
-2
@@ -1,5 +1,5 @@
|
||||
import { defineConfig, devices } from '@playwright/test';
|
||||
import { reporters, workflowWorkers } from './test-workers.mjs';
|
||||
import { workflowWorkers } from './test-workers.mjs';
|
||||
|
||||
// No global webServer: tests/browser/fixtures.mjs starts one server per worker
|
||||
// on an OS-chosen port. Files run in parallel; tests inside a file keep their order.
|
||||
@@ -11,7 +11,7 @@ export default defineConfig({
|
||||
fullyParallel: false,
|
||||
forbidOnly: Boolean(process.env.CI),
|
||||
retries: 0,
|
||||
reporter: reporters({ name: 'workflows' }),
|
||||
reporter: 'list',
|
||||
use: { trace: 'retain-on-failure', screenshot: 'only-on-failure' },
|
||||
projects: [
|
||||
{ name: 'chromium', outputDir: './test-results/workflows/chromium', use: { browserName: 'chromium' } },
|
||||
|
||||
-58
@@ -1,58 +0,0 @@
|
||||
import { writeSync } from 'node:fs';
|
||||
import { relative } from 'node:path';
|
||||
|
||||
// Names the tests still running when a run is cut off (#425). A browser test step that reaches its
|
||||
// timeout-minutes gets SIGTERM, which ends Playwright at once: no reporter's onEnd runs, so neither
|
||||
// `list` nor the blob report says what was running. On the first SIGTERM this reporter prints every
|
||||
// started test that has not ended, then turns the signal into SIGINT, Playwright's own interrupt,
|
||||
// which stops the workers, ends the run as interrupted (non-zero exit) and writes the blob report.
|
||||
// A second SIGTERM gets the default behaviour: the process dies.
|
||||
export default class RunningTestsReporter {
|
||||
constructor({ now = () => Date.now(), write = text => writeSync(1, text), host = process } = {}) {
|
||||
this.now = now;
|
||||
this.write = write;
|
||||
this.host = host;
|
||||
this.rootDir = host.cwd();
|
||||
this.running = new Map();
|
||||
this.onSigterm = () => {
|
||||
this.host.removeListener('SIGTERM', this.onSigterm);
|
||||
this.report('SIGTERM');
|
||||
this.host.kill(this.host.pid, 'SIGINT');
|
||||
};
|
||||
}
|
||||
|
||||
printsToStdio() {
|
||||
return false;
|
||||
}
|
||||
|
||||
onBegin(config) {
|
||||
this.rootDir = config.rootDir;
|
||||
this.host.on('SIGTERM', this.onSigterm);
|
||||
}
|
||||
|
||||
onTestBegin(test, result) {
|
||||
this.running.set(`${test.id}#${result.retry}`, { test, startedAt: this.now() });
|
||||
}
|
||||
|
||||
onTestEnd(test, result) {
|
||||
this.running.delete(`${test.id}#${result.retry}`);
|
||||
}
|
||||
|
||||
describe({ test, startedAt }) {
|
||||
const project = test.parent?.project()?.name;
|
||||
const where = `${relative(this.rootDir, test.location.file)}:${test.location.line}`;
|
||||
const seconds = ((this.now() - startedAt) / 1000).toFixed(1);
|
||||
return `${project ? `[${project}] › ` : ''}${where} › ${test.title} (running ${seconds}s)`;
|
||||
}
|
||||
|
||||
report(reason) {
|
||||
if (!this.running.size) return;
|
||||
const lines = [...this.running.values()].map(entry => ` ${this.describe(entry)}`);
|
||||
this.write(`\nStill running when the run ended (${reason}): ${this.running.size}\n${lines.join('\n')}\n`);
|
||||
}
|
||||
|
||||
onEnd(result) {
|
||||
this.host.removeListener('SIGTERM', this.onSigterm);
|
||||
this.report(result.status);
|
||||
}
|
||||
}
|
||||
Vendored
-11
@@ -1,8 +1,4 @@
|
||||
import { availableParallelism } from 'node:os';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
// Absolute, so a config that imports this one from another directory still finds it.
|
||||
const RUNNING_TESTS_REPORTER = fileURLToPath(new URL('./running-tests-reporter.mjs', import.meta.url));
|
||||
|
||||
const MAX_LOCAL_WORKERS = 8;
|
||||
const CPUS_PER_WORKER = 4;
|
||||
@@ -20,10 +16,3 @@ export function workflowWorkers({ env = process.env, cpus = availableParallelism
|
||||
if (env.CI) return 1;
|
||||
return Math.max(1, Math.min(MAX_LOCAL_WORKERS, Math.floor(cpus / CPUS_PER_WORKER)));
|
||||
}
|
||||
|
||||
// Under CI, running-tests-reporter.mjs names the tests a step timeout's signal cut off (`list`
|
||||
// cannot: they never end), and a blob report beside the traces is uploaded with them.
|
||||
export function reporters({ env = process.env, name } = {}) {
|
||||
if (!env.CI) return 'list';
|
||||
return [['list'], [RUNNING_TESTS_REPORTER], ['blob', { outputDir: `./test-results/blob/${name}` }]];
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ import weakref
|
||||
|
||||
from .execution_context_budget import ContextBudget, PayloadLease, SealedContext
|
||||
from .execution_context_graph import (
|
||||
NOT_SAVED,
|
||||
GraphProjection,
|
||||
GraphRole,
|
||||
SourceRef,
|
||||
@@ -325,6 +326,8 @@ class ContextCapture:
|
||||
nonlocal ambiguous
|
||||
if role.reason == "source-transformed":
|
||||
return _unknown("unsupported")
|
||||
if role.reason == NOT_SAVED:
|
||||
return _unknown("unavailable")
|
||||
if not role.sources:
|
||||
return role.inferred or _unknown("unsupported")
|
||||
values = [
|
||||
|
||||
@@ -3,8 +3,13 @@
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from .execution_contexts import PromptSnapshot, TextEvidence, text_size
|
||||
from .prompt_graph import save_setting
|
||||
from .prompt_pair_resolver import _DEFAULT_ADAPTERS
|
||||
|
||||
# A role whose PromptManager source has save_to_library off, or wired from a
|
||||
# node the graph cannot read: no text is recorded for it (#90)
|
||||
NOT_SAVED = "not_saved"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceRef:
|
||||
@@ -332,6 +337,11 @@ class _Projector:
|
||||
self.transformed = False
|
||||
# Only the capture-source-to-consumer path determines transforms.
|
||||
sources = self.discover_sources(value, "conditioning")
|
||||
if any(
|
||||
save_setting(self.graph, source.node_id) is not True
|
||||
for source in sources
|
||||
):
|
||||
return GraphRole((), None, NOT_SAVED)
|
||||
snapshot, _ = self.text(value, "conditioning")
|
||||
return GraphRole(
|
||||
sources, snapshot, "source-transformed" if self.transformed else ""
|
||||
|
||||
+36
-3
@@ -37,7 +37,13 @@ from .hashing import generate_prompt_hash
|
||||
from .metadata_extractor import ComfyUIMetadataExtractor
|
||||
from .parallel import note_refusal, refused_at_shutdown
|
||||
from .logging_config import get_logger
|
||||
from .prompt_graph import resolve_text, run_prompt_nodes
|
||||
from .prompt_graph import (
|
||||
metadata_opted_out,
|
||||
resolve_text,
|
||||
run_prompt_nodes,
|
||||
save_setting,
|
||||
side_opted_out,
|
||||
)
|
||||
from .prompt_pair_resolver import resolve_prompt_pair
|
||||
from .run_keys import image_run_keys
|
||||
from .usage_tracking import survive_switch
|
||||
@@ -450,6 +456,12 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
# that produced this image, independent of node caching and of any
|
||||
# negative-prompt PromptManager nodes in the workflow.
|
||||
state, current_prompt = self._find_prompt_from_metadata(metadata)
|
||||
if state == "opted_out":
|
||||
self.logger.info(
|
||||
"Save to library is off for this run's prompt, skipping image: "
|
||||
f"{image_path}"
|
||||
)
|
||||
return
|
||||
if state == "foreign":
|
||||
# Another workflow saved this image: it belongs to no prompt of
|
||||
# ours, and must not consume a batch queue entry.
|
||||
@@ -715,16 +727,27 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
``("linked", prompt)`` with a context dict holding 'id' and 'text';
|
||||
``("foreign", None)`` when the workflow has no positive PromptManager
|
||||
node, so the image belongs to no prompt of ours;
|
||||
``("opted_out", None)`` when a PromptManager node that would be the
|
||||
run's prompt has its ``save_to_library`` switch off (#90): the image
|
||||
is linked to no prompt, by no fallback;
|
||||
``("unknown", None)`` when metadata can't decide (missing, a batch
|
||||
item whose text came from another node, or a lookup failure) and the
|
||||
caller's queue and tracker fallbacks apply.
|
||||
"""
|
||||
if not metadata:
|
||||
return "unknown", None
|
||||
if metadata_opted_out(metadata):
|
||||
return "opted_out", None
|
||||
graph = metadata.get("prompt")
|
||||
if isinstance(graph, dict) and any(
|
||||
save_setting(graph, node_id) is None for node_id in run_prompt_nodes(graph)
|
||||
):
|
||||
# A switch wired from a node the graph cannot read: only the node's
|
||||
# own run (it saw the real value) can link the image (#90)
|
||||
return "unknown", None
|
||||
|
||||
prompt_text = None
|
||||
resolution = resolve_prompt_pair(metadata)
|
||||
graph = metadata.get("prompt")
|
||||
if resolution["state"] == "resolved" and isinstance(graph, dict):
|
||||
positive = resolution["positive"]
|
||||
path = positive["provenance"].get("path", [])
|
||||
@@ -865,13 +888,23 @@ class ImageGenerationHandler(FileSystemEventHandler):
|
||||
metadata: Extracted metadata from the image file (workflow, parameters,
|
||||
etc.)
|
||||
"""
|
||||
if metadata_opted_out(metadata):
|
||||
# Reached from a run or history: the file's own graph says no (#90)
|
||||
self.logger.info(
|
||||
f"Save to library is off for {os.path.basename(image_path)}; "
|
||||
"left unlinked"
|
||||
)
|
||||
return
|
||||
try:
|
||||
resolution = resolve_prompt_pair(metadata)
|
||||
graph = metadata.get("prompt") if isinstance(metadata, dict) else None
|
||||
# An opted-out negative is never stored: link the positive alone
|
||||
pair_allowed = not side_opted_out(graph, resolution.get("negative"))
|
||||
|
||||
def link():
|
||||
# One write; run again on the moved library when a Move
|
||||
# library switched while it waited (#249 S6b).
|
||||
if resolution["state"] == "resolved":
|
||||
if resolution["state"] == "resolved" and pair_allowed:
|
||||
positive = resolution["positive"]
|
||||
expected = self.db_manager.get_prompt_by_hash(
|
||||
generate_prompt_hash(
|
||||
|
||||
@@ -300,49 +300,6 @@ def _blank(value: str | bytes | None) -> bool:
|
||||
return value is None or not value.strip()
|
||||
|
||||
|
||||
def inherited_category(
|
||||
positive_category: str | bytes | None,
|
||||
recipe_category: str | bytes | None,
|
||||
has_recipe: bool,
|
||||
) -> tuple:
|
||||
"""``(value, scope)`` of the category an image's link supplies (#332 MC4).
|
||||
|
||||
The one rule for the inspector chip, the Images category filter and
|
||||
History (D39): a linked recipe's own category when it is not blank,
|
||||
``(recipe category, "pair")``; otherwise the positive's value unchanged,
|
||||
blank or not, ``(positive category, "positive")``.
|
||||
"""
|
||||
if has_recipe and not _blank(recipe_category):
|
||||
return recipe_category, "pair"
|
||||
return positive_category, "positive"
|
||||
|
||||
|
||||
class _OwnCategoryUnread:
|
||||
"""The own-category read failed: hide the category (#332 DC5)."""
|
||||
|
||||
def __repr__(self) -> str:
|
||||
return "OWN_CATEGORY_UNREAD"
|
||||
|
||||
|
||||
OWN_CATEGORY_UNREAD = _OwnCategoryUnread()
|
||||
|
||||
|
||||
def apply_own_category(library: dict, own) -> dict:
|
||||
"""*library* with an image's own category applied (#332 §5).
|
||||
|
||||
``own`` is the own category (a string, shown with scope ``"image"``
|
||||
whatever the link status), None (the inherited values stay) or
|
||||
OWN_CATEGORY_UNREAD (category and scope hidden rather than possibly
|
||||
wrong). Returns a new dict.
|
||||
"""
|
||||
result = dict(library)
|
||||
if own is OWN_CATEGORY_UNREAD:
|
||||
result.update(category=None, category_scope=None)
|
||||
elif isinstance(own, str):
|
||||
result.update(category=own, category_scope="image")
|
||||
return result
|
||||
|
||||
|
||||
def library_metadata(
|
||||
positive_category: str | bytes | None,
|
||||
positive_tags: list,
|
||||
@@ -355,14 +312,13 @@ def library_metadata(
|
||||
the linked positive prompt's, the values Set category writes and the Images
|
||||
filters match. Tag sets are never merged. ``recipe`` is (category, tags).
|
||||
"""
|
||||
category, category_scope = inherited_category(
|
||||
positive_category,
|
||||
recipe[0] if recipe is not None else None,
|
||||
recipe is not None,
|
||||
)
|
||||
category, category_scope = positive_category, "positive"
|
||||
tags, tags_scope = positive_tags, "positive"
|
||||
if recipe is not None and recipe[1]:
|
||||
tags, tags_scope = recipe[1], "pair"
|
||||
if recipe is not None:
|
||||
if not _blank(recipe[0]):
|
||||
category, category_scope = recipe[0], "pair"
|
||||
if recipe[1]:
|
||||
tags, tags_scope = recipe[1], "pair"
|
||||
return {
|
||||
"category": category,
|
||||
"category_scope": category_scope,
|
||||
@@ -389,13 +345,9 @@ def _library_projection(library: dict) -> dict:
|
||||
value = library.get(key)
|
||||
if value is None or isinstance(value, str):
|
||||
result[key] = value
|
||||
# An image's own category (#332) has the scope "image"; tags never do.
|
||||
for key, scopes in (
|
||||
("category_scope", ("pair", "positive", "image", None)),
|
||||
("tags_scope", ("pair", "positive", None)),
|
||||
):
|
||||
for key in ("category_scope", "tags_scope"):
|
||||
scope = library.get(key)
|
||||
if scope in scopes:
|
||||
if scope in ("pair", "positive", None):
|
||||
result[key] = scope
|
||||
tags = library.get("tags")
|
||||
result["tags"] = (
|
||||
|
||||
+184
-18
@@ -21,6 +21,11 @@ PROMPT_MANAGER_TYPES = frozenset(
|
||||
}
|
||||
)
|
||||
|
||||
# A node's switch (#90): False means PromptManager ignores that node's runs
|
||||
SAVE_INPUT = "save_to_library"
|
||||
# Core nodes whose output is a known boolean: (class_type) -> input holding it
|
||||
_BOOLEAN_SOURCES = {"PrimitiveBoolean": "value"}
|
||||
|
||||
# Upper bound on nodes expanded per graph; queued and embedded graphs are untrusted
|
||||
MAX_VISITS = 500
|
||||
# Longest text resolve_text will build; StringConcatenate chains can double
|
||||
@@ -59,6 +64,116 @@ def _inputs(node: Any) -> Dict[str, Any]:
|
||||
return inputs if isinstance(inputs, dict) else {}
|
||||
|
||||
|
||||
def _as_boolean(value: Any) -> bool:
|
||||
"""ComfyUI's coercion of a BOOLEAN input before the node runs."""
|
||||
if isinstance(value, dict) and "__value__" in value:
|
||||
value = value["__value__"]
|
||||
return bool(value)
|
||||
|
||||
|
||||
def save_setting(graph: Any, node_id: Any) -> Optional[bool]:
|
||||
"""A PromptManager node's ``save_to_library`` as the node will receive it.
|
||||
|
||||
True when the input is missing (every workflow and image from before the
|
||||
switch), the literal coerced like ComfyUI does, the value of a linked core
|
||||
``PrimitiveBoolean``, and None when the switch is wired from a node whose
|
||||
output cannot be known from the graph.
|
||||
"""
|
||||
node = graph.get(str(node_id)) if isinstance(graph, dict) else None
|
||||
inputs = _inputs(node)
|
||||
if SAVE_INPUT not in inputs:
|
||||
return True
|
||||
value = inputs[SAVE_INPUT]
|
||||
if not _is_link(value):
|
||||
return _as_boolean(value)
|
||||
source = graph.get(str(value[0]))
|
||||
if not isinstance(source, dict) or value[1] != 0:
|
||||
return None
|
||||
key = _BOOLEAN_SOURCES.get(str(source.get("class_type", "")))
|
||||
source_inputs = _inputs(source)
|
||||
if key is None or key not in source_inputs or _is_link(source_inputs[key]):
|
||||
return None
|
||||
return _as_boolean(source_inputs[key])
|
||||
|
||||
|
||||
def opted_out(graph: Any, node_id: Any) -> bool:
|
||||
"""Whether a PromptManager node's switch is off: PromptManager ignores it."""
|
||||
return save_setting(graph, node_id) is False
|
||||
|
||||
|
||||
def side_opted_out(graph: Any, side: Any) -> bool:
|
||||
"""Whether a resolved side's path passes a PromptManager node that is off.
|
||||
|
||||
``side`` is a resolver side (``positive`` or ``negative`` of a resolved
|
||||
pair): its text must not be stored when any PromptManager node on its
|
||||
provenance path has its switch off or wired from an unreadable node (#90).
|
||||
"""
|
||||
if not isinstance(graph, dict) or not isinstance(side, dict):
|
||||
return False
|
||||
provenance = side.get("provenance")
|
||||
path = provenance.get("path") if isinstance(provenance, dict) else None
|
||||
for step in path if isinstance(path, list) else []:
|
||||
node_id = step.get("node_id") if isinstance(step, dict) else None
|
||||
node = graph.get(str(node_id)) if node_id is not None else None
|
||||
if (
|
||||
isinstance(node, dict)
|
||||
and node.get("class_type") in PROMPT_MANAGER_TYPES
|
||||
and save_setting(graph, node_id) is not True
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _workflow_node_opted_out(node: Any) -> bool:
|
||||
"""A saved-workflow PromptManager node whose widget values hold ``False``.
|
||||
|
||||
No other PromptManager widget holds a boolean, so the value is the switch.
|
||||
"""
|
||||
if not isinstance(node, dict):
|
||||
return False
|
||||
if (node.get("type") or node.get("class_type")) not in PROMPT_MANAGER_TYPES:
|
||||
return False
|
||||
if node.get("mode") in (2, 4):
|
||||
return False # muted or bypassed: it did not run
|
||||
values = node.get("widgets_values")
|
||||
return isinstance(values, list) and any(value is False for value in values)
|
||||
|
||||
|
||||
def metadata_opted_out(metadata: Any, *, unreadable: bool = False) -> bool:
|
||||
"""Whether a file's metadata says its run's prompt does not save (#90).
|
||||
|
||||
The executed graph (``prompt``) decides when present: a node of
|
||||
``positive_opt_outs``, or with ``unreadable`` also a run prompt whose
|
||||
switch is wired from a node the graph cannot read. Without one, the
|
||||
saved workflow (``workflow`` nodes or the extractor's
|
||||
``text_encoder_nodes``) is opted out when a PromptManager node stores
|
||||
``False``.
|
||||
"""
|
||||
if not isinstance(metadata, dict):
|
||||
return False
|
||||
graph = metadata.get("prompt")
|
||||
if isinstance(graph, dict):
|
||||
if positive_opt_outs(graph):
|
||||
return True
|
||||
return unreadable and any(
|
||||
save_setting(graph, node_id) is None for node_id in run_prompt_nodes(graph)
|
||||
)
|
||||
workflow = metadata.get("workflow")
|
||||
nodes = list(metadata.get("text_encoder_nodes") or [])
|
||||
if isinstance(workflow, dict) and isinstance(workflow.get("nodes"), list):
|
||||
nodes.extend(workflow["nodes"])
|
||||
definitions = workflow.get("definitions")
|
||||
subgraphs = (
|
||||
definitions.get("subgraphs") if isinstance(definitions, dict) else None
|
||||
)
|
||||
for subgraph in subgraphs if isinstance(subgraphs, list) else []:
|
||||
inner = subgraph.get("nodes") if isinstance(subgraph, dict) else None
|
||||
nodes.extend(inner if isinstance(inner, list) else [])
|
||||
# Without the executed graph the node's role is unknown: any opted-out
|
||||
# PromptManager node opts the file out (never a guessed link).
|
||||
return any(_workflow_node_opted_out(node) for node in nodes[:MAX_VISITS])
|
||||
|
||||
|
||||
def _positive_roots(graph: Dict[str, Any]) -> List[Any]:
|
||||
"""Links into samplers' positive inputs (KSampler, CFGGuider, BasicGuider, ...).
|
||||
|
||||
@@ -100,11 +215,21 @@ def has_sampler(graph: Any) -> bool:
|
||||
|
||||
|
||||
def positive_prompt_components(graph: Any) -> List[Tuple[str, str]]:
|
||||
"""Node/input identities reaching actual sampler positive wires."""
|
||||
"""Node/input identities reaching actual sampler positive wires.
|
||||
|
||||
A node whose switch is off is not one, and nothing upstream of it is
|
||||
followed (#90): see ``positive_opt_outs``.
|
||||
"""
|
||||
return _walk_positive(graph)[0]
|
||||
|
||||
|
||||
def _walk_positive(graph: Any) -> Tuple[List[Tuple[str, str]], List[str]]:
|
||||
"""(components of enabled nodes, opted-out nodes) on sampler positive wires."""
|
||||
if not isinstance(graph, dict):
|
||||
return []
|
||||
return [], []
|
||||
|
||||
found: List[Tuple[str, str]] = []
|
||||
skipped: List[str] = []
|
||||
seen = set()
|
||||
stack = list(reversed(_positive_roots(graph)))
|
||||
while stack and len(seen) < MAX_VISITS:
|
||||
@@ -117,6 +242,10 @@ def positive_prompt_components(graph: Any) -> List[Tuple[str, str]]:
|
||||
if not isinstance(node, dict):
|
||||
continue
|
||||
if node.get("class_type") in PROMPT_MANAGER_TYPES:
|
||||
if opted_out(graph, node_id):
|
||||
if node_id not in skipped:
|
||||
skipped.append(node_id)
|
||||
continue
|
||||
field = "text"
|
||||
if node.get("class_type") == "PromptManagerPair":
|
||||
field = {
|
||||
@@ -132,7 +261,33 @@ def positive_prompt_components(graph: Any) -> List[Tuple[str, str]]:
|
||||
found.append(identity)
|
||||
continue
|
||||
stack.extend(reversed(_upstream_links(_inputs(node))))
|
||||
return found
|
||||
return found, skipped
|
||||
|
||||
|
||||
def _sampler_less_nodes(graph: Dict[str, Any]) -> List[str]:
|
||||
"""Every PromptManager node of a graph without a recognisable sampler."""
|
||||
return [
|
||||
str(node_id)
|
||||
for node_id, node in graph.items()
|
||||
if isinstance(node, dict) and node.get("class_type") in PROMPT_MANAGER_TYPES
|
||||
]
|
||||
|
||||
|
||||
def positive_opt_outs(graph: Any) -> List[str]:
|
||||
"""PromptManager nodes with their switch off that would be a run's prompt.
|
||||
|
||||
Those on a sampler's positive wires, or, in a graph without a recognisable
|
||||
sampler, any. A run with one of them links no file to any prompt (#90):
|
||||
the file's graph cannot always say which sampler wrote it.
|
||||
"""
|
||||
if not isinstance(graph, dict):
|
||||
return []
|
||||
found, skipped = _walk_positive(graph)
|
||||
if found or skipped or has_sampler(graph):
|
||||
return skipped
|
||||
return [
|
||||
node_id for node_id in _sampler_less_nodes(graph) if opted_out(graph, node_id)
|
||||
]
|
||||
|
||||
|
||||
def positive_prompt_nodes(graph: Any) -> List[str]:
|
||||
@@ -148,13 +303,13 @@ def positive_prompt_nodes(graph: Any) -> List[str]:
|
||||
|
||||
def run_prompt_components(graph: Any) -> List[Tuple[str, str]]:
|
||||
"""Actual positive components, retaining standalone main-editor fallback."""
|
||||
components = positive_prompt_components(graph)
|
||||
if components or not isinstance(graph, dict) or has_sampler(graph):
|
||||
components, skipped = _walk_positive(graph)
|
||||
if components or skipped or not isinstance(graph, dict) or has_sampler(graph):
|
||||
return components
|
||||
return [
|
||||
(str(node_id), "text")
|
||||
for node_id, node in graph.items()
|
||||
if isinstance(node, dict) and node.get("class_type") in PROMPT_MANAGER_TYPES
|
||||
(node_id, "text")
|
||||
for node_id in _sampler_less_nodes(graph)
|
||||
if not opted_out(graph, node_id)
|
||||
]
|
||||
|
||||
|
||||
@@ -165,16 +320,16 @@ def run_prompt_nodes(graph: Any) -> List[str]:
|
||||
say), every PromptManager node counts, as before 3.2.4, so an exotic workflow
|
||||
never silently loses usage counting or image linking. When a sampler exists
|
||||
and no PromptManager node feeds its positive input, nothing counts: the only
|
||||
PromptManager nodes are negative prompts.
|
||||
PromptManager nodes are negative prompts. A node whose switch is off never
|
||||
counts (#90).
|
||||
"""
|
||||
positive = positive_prompt_nodes(graph)
|
||||
if positive or not isinstance(graph, dict) or has_sampler(graph):
|
||||
return positive
|
||||
return [
|
||||
str(node_id)
|
||||
for node_id, node in graph.items()
|
||||
if isinstance(node, dict) and node.get("class_type") in PROMPT_MANAGER_TYPES
|
||||
]
|
||||
return list(
|
||||
dict.fromkeys(
|
||||
node_id
|
||||
for node_id, field in run_prompt_components(graph)
|
||||
if field == "text"
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _join_text_inputs(
|
||||
@@ -270,7 +425,18 @@ def resolve_text(graph: Any, node_id: str) -> Optional[str]:
|
||||
|
||||
|
||||
def intrinsic_pair_nodes(graph: Any) -> List[str]:
|
||||
"""Paired nodes whose matching outputs feed the same sampler, or standalone."""
|
||||
"""Paired nodes whose matching outputs feed the same sampler, or standalone.
|
||||
|
||||
A pair whose switch is off is never one (#90).
|
||||
"""
|
||||
return [
|
||||
node_id
|
||||
for node_id in _intrinsic_pair_nodes(graph)
|
||||
if not opted_out(graph, node_id)
|
||||
]
|
||||
|
||||
|
||||
def _intrinsic_pair_nodes(graph: Any) -> List[str]:
|
||||
if not isinstance(graph, dict) or len(graph) > MAX_VISITS:
|
||||
return []
|
||||
if not has_sampler(graph):
|
||||
|
||||
+13
-1
@@ -30,7 +30,12 @@ from typing import Any, Callable, List, Optional, Tuple
|
||||
try:
|
||||
from .hashing import generate_prompt_hash
|
||||
from .logging_config import get_logger
|
||||
from .prompt_graph import intrinsic_pair_nodes, resolve_text, run_prompt_components
|
||||
from .prompt_graph import (
|
||||
intrinsic_pair_nodes,
|
||||
resolve_text,
|
||||
run_prompt_components,
|
||||
save_setting,
|
||||
)
|
||||
from .prompt_pair_resolver import resolve_text_input
|
||||
from .pair_execution import clear_pair_execution
|
||||
from .queue_observers import subscribe_dequeue
|
||||
@@ -41,6 +46,7 @@ except ImportError:
|
||||
intrinsic_pair_nodes,
|
||||
resolve_text,
|
||||
run_prompt_components,
|
||||
save_setting,
|
||||
)
|
||||
from utils.prompt_pair_resolver import resolve_text_input
|
||||
from utils.pair_execution import clear_pair_execution
|
||||
@@ -127,10 +133,14 @@ def queued_prompt_runs(graph: Any) -> List[Run]:
|
||||
"""(hash, text) of each positive PromptManager node whose text is knowable.
|
||||
|
||||
Each hash once: the same text in two positive nodes is one run of the prompt.
|
||||
A node whose switch is off is never counted; one whose switch is wired from
|
||||
a node the graph cannot read counts itself when it executes (#90).
|
||||
"""
|
||||
runs: List[Run] = []
|
||||
seen = set()
|
||||
for node_id, field in run_prompt_components(graph):
|
||||
if save_setting(graph, node_id) is not True:
|
||||
continue
|
||||
node = graph.get(node_id, {}) if isinstance(graph, dict) else {}
|
||||
text = (
|
||||
resolve_text_input(graph, node_id, field)
|
||||
@@ -156,6 +166,8 @@ def queued_pair_runs(graph: Any) -> List[Tuple[str, str, str]]:
|
||||
"""Resolve raw paired recipes only when both outputs form the used pair."""
|
||||
found, seen = [], set()
|
||||
for node_id in intrinsic_pair_nodes(graph):
|
||||
if save_setting(graph, node_id) is not True:
|
||||
continue # wired switch: the node counts itself (#90)
|
||||
positive = resolve_text_input(graph, node_id, "text")
|
||||
negative = resolve_text_input(graph, node_id, "negative_text")
|
||||
if positive is None or not positive.strip() or negative is None:
|
||||
|
||||
@@ -6,6 +6,7 @@ const { installClassicControls } = await import(api.fileURL('/prompt_manager/js/
|
||||
const { installFocusedNode, installSplitNode, installPairNode } = await import(api.fileURL('/prompt_manager/js/focused-node.js'));
|
||||
const { installPromptBridge } = await import(api.fileURL('/prompt_manager/js/prompt-bridge-host.js'));
|
||||
const { collapseLinkedWidget, isLinked, linkNoteText, primitiveTextValue, storedWidgetValue } = await import(api.fileURL('/prompt_manager/js/text-links.js'));
|
||||
const { normalizeSaveWidget } = await import(api.fileURL('/prompt_manager/js/save-to-library.js'));
|
||||
const promptManagerTypes = new Set(['PromptManager', 'PromptManagerText', 'PromptManagerFocused', 'PromptManagerSplit', 'PromptManagerPair']);
|
||||
|
||||
// ============================================================================
|
||||
@@ -525,6 +526,8 @@ app.registerExtension({
|
||||
if (originalConfigure) {
|
||||
originalConfigure.call(this, data);
|
||||
}
|
||||
// #90: an older workflow loads with Save to library on
|
||||
normalizeSaveWidget(this);
|
||||
if (data._userHasResized) {
|
||||
this._userHasResized = data._userHasResized;
|
||||
}
|
||||
|
||||
+31
-4
@@ -2,9 +2,13 @@ import {createScope, createButton, createField, mountTheme} from './shared/toolk
|
||||
import {createPromptLibrary} from './prompt-library.js';
|
||||
import {attachMetadataAutocomplete} from './metadata-autocomplete.js';
|
||||
import {exposeTextSockets, linkNoteText, preferTextInputs, textInputAtPoint} from './text-links.js';
|
||||
import {SAVE_WIDGET, normalizeSaveWidget, placeSaveWidgetLast, saveWidget, savesToLibrary, setSaveToLibrary} from './save-to-library.js';
|
||||
|
||||
const names = ['text', 'category', 'tags', 'search_text', 'prepend_text', 'append_text'];
|
||||
let nextEditorId = 0;
|
||||
// The editor draws these widgets itself; the host keeps them as hidden, zero-height values.
|
||||
function hiddenWidgetSize() { return [0, -4]; }
|
||||
function hideNativeWidget(widget) { widget.hidden = true; widget.type = 'hidden'; widget.computeSize = hiddenWidgetSize; }
|
||||
|
||||
function loadStyles() {
|
||||
const href = new URL('./prompt-editor.css', import.meta.url).href;
|
||||
@@ -43,10 +47,15 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
for (const name of pair ? [...names,'negative_text'] : names) {
|
||||
let widget = this.widgets?.find(item => item.name === name);
|
||||
if (!widget) widget = this.addWidget('text', name, '', () => {}, {});
|
||||
widget.hidden = true; widget.type = 'hidden'; widget.computeSize = () => [0, -4];
|
||||
hideNativeWidget(widget);
|
||||
widgets.set(name, widget);
|
||||
}
|
||||
exposeTextSockets(this, textInputs);
|
||||
// #90: the switch follows the hidden fallbacks above (one widget order on every frontend) and is drawn in the
|
||||
// toolbar, like the other hidden widgets are drawn by the editor.
|
||||
placeSaveWidgetLast(this);
|
||||
const nativeSave = saveWidget(this);
|
||||
if (nativeSave) hideNativeWidget(nativeSave);
|
||||
const element = document.createElement('div'); element.className = `ca-tools pm-focused${split ? ' pm-split' : ''}${pair ? ' pm-pair' : ''}`;
|
||||
element.setAttribute('aria-label',pair ? 'Paired prompt editor' : split ? 'Split prompt editor' : 'Focused prompt editor');
|
||||
loadStyles(); mountTheme(element, app.ui?.settings, scope);
|
||||
@@ -58,6 +67,11 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
toolbar.append(web);
|
||||
const save = pair ? createButton('Save pair','ca-primary') : null;
|
||||
if (save) toolbar.append(save);
|
||||
const saveSwitch = document.createElement('input'); saveSwitch.type = 'checkbox'; saveSwitch.dataset.saveToLibrary = '';
|
||||
const saveLabel = document.createElement('label'); saveLabel.className = 'pm-editor-save';
|
||||
saveLabel.title = 'Off: runs of this node are not saved, counted or linked to their images. The output is unchanged.';
|
||||
saveLabel.append(saveSwitch, document.createTextNode('Save to library'));
|
||||
if (nativeSave) toolbar.append(saveLabel);
|
||||
let saving = false;
|
||||
const workspace = document.createElement('div'); workspace.className = 'pm-editor-workspace';
|
||||
const editor = document.createElement('div'); editor.className = 'pm-editor-compose';
|
||||
@@ -138,7 +152,11 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
}
|
||||
const composed = disclosure('preview','Composed text preview');
|
||||
const preview = document.createElement('p'); preview.className = 'pm-editor-preview'; preview.dataset.composedPreview = ''; composed.append(preview);
|
||||
const footer = document.createElement('p'); footer.className = 'pm-editor-status'; footer.setAttribute('role','status'); footer.textContent = pair ? 'The prompt pair is saved when this node runs, or use Save pair.' : 'Main prompt is saved when this node runs.';
|
||||
const footer = document.createElement('p'); footer.className = 'pm-editor-status'; footer.setAttribute('role','status');
|
||||
const SAVE_OFF = 'Save to library is off: runs of this node are not added to the library.';
|
||||
const SAVE_LINKED = 'Save to library comes from a link: runs are added to the library only while it is on.';
|
||||
function runNote(saved) { return isConnected(SAVE_WIDGET) ? SAVE_LINKED : savesToLibrary(thisNode) ? saved : SAVE_OFF; }
|
||||
footer.textContent = runNote(pair ? 'The prompt pair is saved when this node runs, or use Save pair.' : 'Main prompt is saved when this node runs.');
|
||||
const saveConnectionNotice = pair ? document.createElement('p') : null;
|
||||
if (saveConnectionNotice) {
|
||||
saveConnectionNotice.className = 'pm-editor-status'; saveConnectionNotice.setAttribute('role','status');
|
||||
@@ -166,6 +184,7 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
mainField.hidden = Boolean(isConnected('text')); connected.hidden = !isConnected('text');
|
||||
if (negative) { negative.field.hidden = Boolean(isConnected('negative_text')); negative.note.hidden = !isConnected('negative_text'); }
|
||||
if (save) { save.disabled = saving || hasConnectedPairValues(); saveConnectionNotice.hidden = !hasConnectedPairValues(); }
|
||||
saveSwitch.checked = savesToLibrary(thisNode); saveSwitch.disabled = isConnected(SAVE_WIDGET);
|
||||
controls.forEach((input,name) => {
|
||||
input.value = value(name);
|
||||
if (name !== 'text' && name !== 'negative_text') {
|
||||
@@ -185,7 +204,7 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
write('category',prompt.category || '');
|
||||
write('tags',Array.isArray(prompt.tags) ? prompt.tags.join(', ') : prompt.tags || '');
|
||||
}
|
||||
render(); footer.setAttribute('role','status'); footer.textContent = pair ? 'Loaded pair. Saved when this node runs.' : 'Loaded prompt. Saved when this node runs.'; main.focus();
|
||||
render(); footer.setAttribute('role','status'); footer.textContent = runNote(pair ? 'Loaded pair. Saved when this node runs.' : 'Loaded prompt. Saved when this node runs.'); main.focus();
|
||||
}});
|
||||
workspace.append(library.element); element.append(footer);
|
||||
function snapshot() {
|
||||
@@ -197,7 +216,13 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
library.element.hidden = !split && !state.libraryOpen; browse.textContent = state.libraryOpen ? 'Hide library' : 'Browse library'; browse.setAttribute('aria-expanded', String(Boolean(state.libraryOpen)));
|
||||
library.setQuery(value('search_text')); library.restoreFilters(state.libraryFilters); render();
|
||||
}
|
||||
controls.forEach((input,name) => scope.listen(input,'input',() => {write(name,input.value);render();footer.setAttribute('role','status');footer.textContent=pair?'The pair is saved when this node runs.':'Main prompt is saved when this node runs.';}));
|
||||
controls.forEach((input,name) => scope.listen(input,'input',() => {write(name,input.value);render();footer.setAttribute('role','status');footer.textContent=runNote(pair?'The pair is saved when this node runs.':'Main prompt is saved when this node runs.');}));
|
||||
scope.listen(saveSwitch,'change',() => {
|
||||
if (isConnected(SAVE_WIDGET)) return;
|
||||
setSaveToLibrary(thisNode, saveSwitch.checked); render();
|
||||
footer.setAttribute('role','status'); footer.textContent = runNote(pair ? 'The pair is saved when this node runs.' : 'Main prompt is saved when this node runs.');
|
||||
thisNode.setDirtyCanvas(true,true); app.graph?.setDirtyCanvas(true,true);
|
||||
});
|
||||
scope.listen(browse,'click',() => {
|
||||
library.element.hidden = !library.element.hidden;
|
||||
library.closeCompletions();
|
||||
@@ -248,6 +273,8 @@ export function installFocusedNode(nodeType, app, api, parseQuery, {layout = 'fo
|
||||
const result = configure?.apply(this,args);
|
||||
// A saved workflow restores its inputs with their widget bindings; keep the text inputs as sockets.
|
||||
exposeTextSockets(this, textInputs);
|
||||
// #90: an older workflow loads with Save to library on
|
||||
normalizeSaveWidget(this);
|
||||
this._focusedView?.restore();
|
||||
return result;
|
||||
};
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
.pm-focused [hidden] { display: none !important; }
|
||||
.pm-editor-row { display: flex; align-items: center; gap: 8px; flex-wrap: wrap; }
|
||||
.pm-editor-toolbar > strong { margin-inline-end: auto; font-weight: 500; }
|
||||
.pm-editor-save { display: inline-flex; align-items: center; gap: 6px; color: var(--ca-muted); cursor: pointer; }
|
||||
.pm-editor-save:has(input:checked) { color: inherit; }
|
||||
.pm-focused .ca-field-label { align-self: flex-start; margin-inline: 0; text-align: start; }
|
||||
.pm-editor-workspace, .pm-editor-compose { display: contents; }
|
||||
.pm-focused .pm-editor-main { display: flex; flex: 1; min-block-size: 180px; flex-direction: column; align-items: stretch; gap: 5px; text-align: start; }
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
// #90: a PromptManager node's `save_to_library` switch. Off: PromptManager ignores the node's runs (nothing saved,
|
||||
// counted or linked); the node's output is unchanged.
|
||||
//
|
||||
// The input is declared last, so it takes the widget index an older workflow used for the editor's own
|
||||
// (non-serialized) DOM widget, where it stored `''` or nothing. ComfyUI coerces a BOOLEAN with `bool(value)`, so that
|
||||
// `''` would turn saving off: after a workflow loads, anything that is not a boolean is the default, on.
|
||||
|
||||
export const SAVE_WIDGET = 'save_to_library';
|
||||
|
||||
export function saveWidget(node) {
|
||||
return node?.widgets?.find(widget => widget?.name === SAVE_WIDGET);
|
||||
}
|
||||
|
||||
/** Whether the node saves its runs (a missing widget, as on an older backend, is on). */
|
||||
export function savesToLibrary(node) {
|
||||
return saveWidget(node)?.value !== false;
|
||||
}
|
||||
|
||||
/** Set a non-boolean switch value (from an older workflow) to on. Returns whether it changed. */
|
||||
export function normalizeSaveWidget(node) {
|
||||
const widget = saveWidget(node);
|
||||
if (!widget || typeof widget.value === 'boolean') return false;
|
||||
widget.value = true;
|
||||
const index = node.widgets.indexOf(widget);
|
||||
if (Array.isArray(node.widgets_values) && index >= 0) node.widgets_values[index] = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Move the switch after every other widget the node has now, so its index is the same on every frontend. */
|
||||
export function placeSaveWidgetLast(node) {
|
||||
const widget = saveWidget(node);
|
||||
const widgets = node?.widgets;
|
||||
if (!widget || widgets.at(-1) === widget) return false;
|
||||
widgets.splice(widgets.indexOf(widget), 1);
|
||||
widgets.push(widget);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Turn the switch on or off the way a native widget edit does. */
|
||||
export function setSaveToLibrary(node, value) {
|
||||
const widget = saveWidget(node);
|
||||
if (!widget) return;
|
||||
widget.value = Boolean(value);
|
||||
const index = node.widgets.indexOf(widget);
|
||||
if (Array.isArray(node.widgets_values) && index >= 0) node.widgets_values[index] = widget.value;
|
||||
widget.callback?.(widget.value, undefined, node);
|
||||
}
|
||||
@@ -20,8 +20,6 @@ const REPRESENTATIONS = {
|
||||
limited: 'Record exceeds inspection limits; text is unavailable',
|
||||
};
|
||||
const TEXTLESS = new Set(['invalid', 'limited']);
|
||||
// #332: the hidden prefix that names where a category chip's value came from.
|
||||
const CATEGORY_PREFIX = {image: 'Image category: ', positive: 'Positive prompt category: ', pair: 'Recipe category: '};
|
||||
const REVEAL_URL = '/prompt_manager/images/reveal';
|
||||
// D38 Show in folder: the route's answers, in the maintainer's words.
|
||||
const REVEAL_OPENED = 'Opened the folder on this computer.';
|
||||
@@ -267,10 +265,7 @@ export function mountFileInspector(container, {api, sender, parentScope, variant
|
||||
inner.append(heading, el('p', filename, 'small'));
|
||||
const badges = el('div', undefined, 'tags'), badge = evidenceBadge(record.prompts);
|
||||
if (badge) badges.append(el('span', badge, 'chip blue'));
|
||||
// #332: an image's own category shows for any link status; an inherited one only when linked.
|
||||
if (library?.category && (library.category_scope === 'image' || linked)) {
|
||||
badges.append(prefixedChip(CATEGORY_PREFIX[library.category_scope] ?? CATEGORY_PREFIX.positive, library.category));
|
||||
}
|
||||
if (linked && library.category) badges.append(prefixedChip(library.category_scope === 'pair' ? 'Recipe category: ' : 'Positive prompt category: ', library.category));
|
||||
if (badges.childNodes.length) inner.append(badges);
|
||||
const prompts = el('div'); inner.append(prompts);
|
||||
const blocks = mountPromptBlocks(prompts, {sender, parentScope:contentScope, includePairAction:false, contextLabel:promptContext, title});
|
||||
|
||||
@@ -201,9 +201,7 @@ function resolutionFor(value) {
|
||||
function libraryFor(value) {
|
||||
if (!object(value) || !['linked', 'none', 'ambiguous', 'unavailable', 'limited'].includes(value.status) || value.scope !== 'saved-library' ||
|
||||
!Array.isArray(value.tags) || value.tags.some(tag => typeof tag !== 'string') ||
|
||||
![null, 'pair', 'positive', 'image'].includes(value.category_scope) ||
|
||||
![null, 'pair', 'positive'].includes(value.tags_scope)) throw new TypeError('Invalid inspection library');
|
||||
// #332: an image's own category (scope "image") may sit on any link status; tags never have that scope.
|
||||
![value.category_scope, value.tags_scope].every(scope => [null, 'pair', 'positive'].includes(scope))) throw new TypeError('Invalid inspection library');
|
||||
// D39: each field names its own source, so a recipe's category can sit beside its positive's tags.
|
||||
const library = { status: value.status, scope: value.scope, tags: [...value.tags], category_scope: value.category_scope, tags_scope: value.tags_scope };
|
||||
for (const key of ['positive_id', 'pair_id', 'title', 'category']) {
|
||||
|
||||
Reference in New Issue
Block a user