Compare commits
14
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
bac05711b9 | ||
|
|
d7ccd9b4df | ||
|
|
e2525a9d79 | ||
|
|
c47be3731c | ||
|
|
f65de3fea4 | ||
|
|
dead4a7aa3 | ||
|
|
9d1e5a0e22 | ||
|
|
5123654b0d | ||
|
|
ed4bd97005 | ||
|
|
ebebf0d69c | ||
|
|
9193373eac | ||
|
|
fba5903049 | ||
|
|
a8aff3a39d | ||
|
|
9dcad31441 |
+140
-3
@@ -861,9 +861,10 @@ Recorded deviations on this page (spec §8):
|
||||
- **D7**: the star is a real persisted favorite with "Added to favorites." /
|
||||
"Removed from favorites." and a rollback toast on failure.
|
||||
- **D8**: every Images card has the concept's select checkbox before the star; the
|
||||
`#bulkbar` shows "N selected" and only the real actions in the concept's order (Set
|
||||
category, Export metadata, Clear). Add tags and Add to collection wait for image tags
|
||||
and collections; "Remove links…" is dropped. The selection survives Load more and
|
||||
`#bulkbar` shows "N selected" and only the real actions in the concept's order (Add
|
||||
tags, then D85's Auto-tag selected, Set category, Export metadata, Clear). Add to
|
||||
collection waits for collections; "Remove links…" is dropped. (Amended by #249 PR D:
|
||||
Add tags ships, see "Bulk tagging from the selection bar".) The selection survives Load more and
|
||||
bounded re-mounting and is cleared whenever the collection is replaced (search,
|
||||
category, sort, Filters, source change, rescan, reload). A return to Images restores
|
||||
the saved collection unreplaced and keeps its selection; a full page reload starts
|
||||
@@ -1201,6 +1202,142 @@ production-versus-concept difference of PR C and its record:
|
||||
| Retry leads the actions when the controller allows it | D116 (ruling T5) |
|
||||
| States the concept lacks (busy, conflict, changed storage or file, unconfirmed save, unavailable model, video) | Global Constraints: `#tag-generation-status` text only |
|
||||
|
||||
### Bulk tagging from the selection bar (#249 PR D)
|
||||
|
||||
The Images selection bar shows the concept's **Add tags** after the count, then
|
||||
**Auto-tag selected** (D85), before Set category. Both take the checked files when
|
||||
pressed, in listing order, as their reveal identity (`{root_index, root_scope, path}`);
|
||||
more than 500 toasts "Select at most 500 images." and opens nothing. The selection
|
||||
stays after every start, refusal and review. The job's start body is bounded so 500
|
||||
targets of the longest legal path (1024 code points, even escaped as `\uXXXX`) fit:
|
||||
`500 × (1024 × 6 + 256) + 64 KiB`; a larger body answers 413
|
||||
(`tests/test_image_tag_jobs.py` "test_start_body_bounds").
|
||||
|
||||
- **Add tags** opens the concept's `simpleInput` (app.js a1378–1383) as Set category
|
||||
does: "Tag selected images", "Tags, separated by commas" (empty: the concept's
|
||||
"cinematic, concept" is sample text), Cancel / Apply. The names are parsed as the Tag
|
||||
image dialog's manual field parses them (trimmed, case-insensitive repeats dropped,
|
||||
first spelling wins); a field that parses to no names keeps the dialog open with the
|
||||
typed text and reports "Enter at least one tag." on the field (its native validity,
|
||||
cleared on the next edit), and nothing is sent. Apply starts the
|
||||
`image-tags` job (`POST /prompt_manager/jobs/image-tags`, `{mode: "add", targets,
|
||||
tags}`) and toasts "Tagging started."; Activity shows its progress.
|
||||
- A refused start toasts why and changes nothing: another record-changing job (a scan)
|
||||
gives "Another job is running: <title>." with a link to it (D57's toast), a held model
|
||||
gives the server's `model-busy` sentence, any other coded refusal its own sentence,
|
||||
and anything else "Tagging could not start. Try again." / "Auto-tagging could not
|
||||
start. Try again.".
|
||||
- When a job this page started ends having tagged files, those files are read again
|
||||
once: the selected file's inspection (its chips, D88) and, under a Tags filter or a
|
||||
search, the loaded range, as after a Tag save.
|
||||
|
||||
- **D85** (plan D62, ruling L1, beyond the concept): **Auto-tag selected** opens
|
||||
"Auto-tag selected images" in the Tag image dialog's shell (`tagging-modal`) with
|
||||
its model panel, headed "Auto-tag <N> image(s)" (the panel's "Auto-tag this image"
|
||||
for N files), without the preview, the manual field, the suggestions or Generate.
|
||||
Its fields show the server's defaults (`GET /prompt_manager/autotag/models`, the
|
||||
D91 Suggestions count included) once that read lands; Start waits for it. A model
|
||||
the status lists as not downloaded keeps Start `aria-disabled` and
|
||||
`#tag-generation-status` says "<model> is not downloaded. Download it from the
|
||||
legacy Auto-tag page." (the server's own sentence), so no job starts only to fail.
|
||||
Start validates the fields as Generate does (native validity; nothing is sent).
|
||||
The concept comparison (`images-concept.spec.mjs`) uses the concept page itself as
|
||||
the oracle, with Add to collection and Remove links… removed and a copy of its Add
|
||||
tags button reading "Auto-tag selected" inserted after it.
|
||||
- **D90** (plan D67, ruling T3-bulk): the dialog's **How to apply** radio group
|
||||
(`.field[role=radiogroup]`, radios beside their labels): **Apply automatically**
|
||||
(checked; primary "Start auto-tagging") starts the `image-tags` job
|
||||
(`{mode: "auto", targets, model_type, options}`) and toasts "Auto-tagging started.";
|
||||
**Review each image** (primary "Start review") walks the selection in the browser
|
||||
(`images-tagging-review.mjs`). Each image opens the Tag image dialog titled "Tag
|
||||
image · Image <i> of <M>" on the chosen model and options with Generate already
|
||||
running; its actions are Stop reviewing, Skip and Approve and next (Approve and
|
||||
finish on the last). Approve saves the manual field plus the remaining chips as Save
|
||||
does, without its per-image toast, and opens the next image; Skip writes nothing and
|
||||
advances; Stop reviewing, Escape and × end the session. While Approve's save is in
|
||||
flight, Skip and Stop reviewing are `aria-disabled` and Escape and × do nothing, so the
|
||||
closing count includes that save. When no suggestions are on screen (a model change
|
||||
cleared them, or Generate was refused) the primary saves the manual field only and
|
||||
reads "Save and next" ("Save and finish" on the last image). Each image is one gated
|
||||
`suggest` call, so the session holds no model between images; a `model-busy` answer
|
||||
stays on that image with its sentence in `#tag-generation-status` and D116's Retry
|
||||
until the user retries or skips. Ending toasts "<n> of <M> images tagged." and
|
||||
returns focus to Auto-tag selected.
|
||||
- **D91** extends to the bulk dialog: its Suggestions field is the job's and the
|
||||
review's `suggestion_limit`.
|
||||
- **D118** (copy the concept has no line for; the concept's `bulk-apply` only toasts a
|
||||
demo sentence): an Add tags field that parses to no names reports "Enter at least one
|
||||
tag." on the field; a start refused for no coded reason toasts "Tagging could not start. Try
|
||||
again." or "Auto-tagging could not start. Try again." (as D55's "The scan could not
|
||||
start. Try again."); and the bulk dialog's model panel is headed "Auto-tag <N>
|
||||
image(s)" where the Tag image dialog's reads "Auto-tag this image". Everything else
|
||||
the selection bar's tagging shows is the concept's or the plan's (D85, D90, D91).
|
||||
- **D89** (plan D66, Activity for the `image-tags` job): the job's Activity item and its
|
||||
view are the scan's (D54, D56, D57, D59) with the job kind's own counters, from one
|
||||
per-kind table (`JOB_VIEWS` in `activity-store.mjs`; a kind without one keeps the
|
||||
scan's tiles under "Job log" and "Cancel job"). The titles are the server's: "Add
|
||||
tags to images" and "Auto-tag images".
|
||||
- The pill's verb is "Tagging" ("Tagging · 41% · 8,240 / 19,622").
|
||||
- The item line and the view's heading line read "Started HH:MM · N images · closing
|
||||
this view does not cancel it" (the job's image count, grouped "en-US"), "Finished
|
||||
HH:MM · N tagged · M errors", "Cancelled HH:MM at a file boundary · N tagged ·
|
||||
completed work is kept" or "Stopped on an error at HH:MM · N tagged · M errors".
|
||||
- The view's actions are Download report and, while running, "Cancel tagging"
|
||||
(`.danger`); the progress bar is "<title> progress".
|
||||
- Four `dl.stat-row` tiles in the mockup's tones: Tagged (`.added`, green),
|
||||
Unchanged, Skipped (`.warn`, warning), Errors (`.bad`, red). They keep the
|
||||
mockup's five-column row (two columns on a phone), so a tile is the scan's width
|
||||
and the fifth cell stays empty.
|
||||
- The outcome list reads Tagged, Errors, Skipped, Unchanged (what got tagged and what
|
||||
went wrong first, as the scan's), and the view opens on the first with files.
|
||||
- Rows under Tagged and Unchanged are D57's `button.file-row`s that open the file in
|
||||
the viewer by its listing identity; Skipped and Errors rows are plain text.
|
||||
- The log is "● Tagging log — raw detail, click to show".
|
||||
- The row and log wording is the job's (D1), since neither the concept nor the
|
||||
mockup has a line for it: "Added N tag(s)", "No new tags", and the skip reasons
|
||||
"Video: manual tags only", "Too large to verify", "File changed", "File not found",
|
||||
"Not a supported still image" and "The file could not be read now. Run the job again
|
||||
for it." (the capture store stayed at capacity, for example with Tag dialogs open in
|
||||
other tabs, after a short backoff of retries). Log prefixes are `+` tagged, `=`
|
||||
unchanged, `-` skipped and `!` error.
|
||||
- The progress line is the scan's: "Processing file N of M · about X min left", the
|
||||
estimate (D56's `_time_left`) shown once two files are done, each file one batch.
|
||||
- Download report saves `prompt-manager-image-tags-<YYYYMMDD-HHMM>.json`.
|
||||
- The start toasts are "Tagging started." / "Auto-tagging started." (D85); the end
|
||||
toasts follow the design's pattern with the job's counts, as the scan's: "<title>
|
||||
finished · N tagged · M errors", "<title> cancelled · N tagged" or "<title> failed ·
|
||||
N tagged · M errors".
|
||||
- Tests: `tests/js/workspace-activity-store.test.js` ("image-tags jobs get their own
|
||||
tiles outcomes and pill verb", "unknown kinds keep the generic view");
|
||||
`tests/browser/workspace-activity.spec.mjs` ("an image tagging job shows Tagged
|
||||
Unchanged Skipped Errors and opens tagged rows", "cancel tagging stops at a file
|
||||
boundary", "the report download names the kind", on fake `image-tags` jobs from
|
||||
`workspace_activity_fixture.py`); `tests/browser/activity-concept.spec.mjs` ("image
|
||||
tagging job view follows the approved mockup", dark and light, with a mutation
|
||||
check). The scan's view is unchanged.
|
||||
|
||||
Browser tests (`tests/browser/workspace-tagging-bulk.spec.mjs`, on the tagging
|
||||
fixture): "selection bar shows Add tags then Auto-tag selected before Set category",
|
||||
"Add tags runs a job and the cards show the tags after it finishes", "Auto-tag
|
||||
selected with Apply automatically runs a job with the chosen model, thresholds and
|
||||
suggestion count", "a model that is not downloaded keeps Start disabled and says
|
||||
why", "Suggestions field prefills from the setting and caps chips at the entered
|
||||
count", "Review each image walks the selection one dialog at a time and only approved
|
||||
images gain tags", "Review holds Stop reviewing, Escape and × while a save is in
|
||||
flight, and names a manual-only save" and "a running scan makes the start report
|
||||
busy". Units:
|
||||
`tests/js/workspace-images-tagging-bulk.test.js`,
|
||||
`tests/js/workspace-images-tagging-review.test.js`.
|
||||
|
||||
| Difference | Record |
|
||||
| --- | --- |
|
||||
| Add tags ships (Add to collection and Remove links… still omitted) | D8 (amended) |
|
||||
| Auto-tag selected and its dialog | D85 (ruling L1) |
|
||||
| How to apply and the Review each image session | D90 (ruling T3-bulk) |
|
||||
| Suggestions count in the bulk dialog | D91 |
|
||||
| Copy without a concept line: no tags, start fallbacks, the panel heading | D118 |
|
||||
| Activity item and job view for image tagging jobs | D89 (plan D66) |
|
||||
|
||||
Native image tagging (#249 PR B routes, PR C UI): `POST
|
||||
/prompt_manager/image-annotations/observe` (capture one listed file, its saved set and
|
||||
`tag_revision`), `GET`/`DELETE /prompt_manager/image-annotations/capture/<token>`
|
||||
|
||||
@@ -7,6 +7,7 @@ Split into domain-specific mixins for maintainability:
|
||||
- LoggingRoutesMixin: log management endpoints
|
||||
- AutotagRoutesMixin: auto-tagging model management and tagging
|
||||
- JobRoutesMixin: the session job registry behind Activity (py/api/jobs.py)
|
||||
- ImageTagJobsMixin: bulk Add tags / Auto-tag as image-tags jobs
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
@@ -31,7 +32,9 @@ from .saved_views import SavedViewRoutesMixin
|
||||
from .history import HistoryRoutesMixin
|
||||
from .images import ImageRoutesMixin
|
||||
from .admin import AdminRoutesMixin
|
||||
from .settings_routes import SettingsRoutesMixin
|
||||
from .job_routes import JobRoutesMixin
|
||||
from .image_tag_jobs import ImageTagJobsMixin
|
||||
from .jobs import JobRegistry
|
||||
from .logging_routes import LoggingRoutesMixin
|
||||
from .autotag_routes import AutotagRoutesMixin
|
||||
@@ -259,7 +262,9 @@ class PromptManagerAPI(
|
||||
PromptRoutesMixin,
|
||||
ImageRoutesMixin,
|
||||
AdminRoutesMixin,
|
||||
SettingsRoutesMixin,
|
||||
JobRoutesMixin,
|
||||
ImageTagJobsMixin,
|
||||
LoggingRoutesMixin,
|
||||
AutotagRoutesMixin,
|
||||
LoraIntegrationMixin,
|
||||
@@ -358,6 +363,24 @@ class PromptManagerAPI(
|
||||
"""409 envelope for a long job that is already running."""
|
||||
return _json_error(f"{label} already running", 409)
|
||||
|
||||
async def _run_held(self, kind, title, admission, operation):
|
||||
"""Run a legacy-page operation under a registry hold (D80).
|
||||
|
||||
While ``operation()`` runs, the registry lists it as a running,
|
||||
non-cancellable job, so workspace jobs its admission class conflicts
|
||||
with are refused, naming it. When the registry refuses the hold, the
|
||||
legacy 409 envelope names the running job and nothing is done. A
|
||||
response of 400 or above ends the hold ``failed``.
|
||||
"""
|
||||
busy = self._jobs.blocker(kind, admission=admission)
|
||||
if busy is not None:
|
||||
return self._long_job_busy_response(busy.title)
|
||||
async with self._jobs.hold(kind, title, admission=admission) as job:
|
||||
response = await operation()
|
||||
if getattr(response, "status", 200) >= 400:
|
||||
job.finish("failed")
|
||||
return response
|
||||
|
||||
async def _run_in_executor(self, func, *args, **kwargs):
|
||||
"""Run a blocking function in the default thread pool executor.
|
||||
|
||||
@@ -453,6 +476,7 @@ class PromptManagerAPI(
|
||||
self._register_prompt_routes(routes)
|
||||
self._register_image_routes(routes)
|
||||
self._register_admin_routes(routes)
|
||||
self._register_settings_routes(routes)
|
||||
self._register_job_routes(routes)
|
||||
self._register_logging_routes(routes)
|
||||
self._register_autotag_routes(routes)
|
||||
|
||||
+91
-235
@@ -22,16 +22,17 @@ from .jobs import (
|
||||
SCAN_GROUP_NAMES,
|
||||
SCAN_GROUPS,
|
||||
SCAN_SCOPES,
|
||||
DUPLICATE_CLEANUP_JOB_KIND,
|
||||
MAINTENANCE_JOB_KIND,
|
||||
RESTORE_JOB_KIND,
|
||||
Job,
|
||||
JobBusy,
|
||||
)
|
||||
|
||||
try:
|
||||
from ...utils.parallel import map_parallel
|
||||
from ...utils.validators import validate_result_timeout, validate_worker_threads
|
||||
except ImportError:
|
||||
from utils.parallel import map_parallel
|
||||
from utils.validators import validate_result_timeout, validate_worker_threads
|
||||
|
||||
try:
|
||||
from ...database.operations import PromptDatabase
|
||||
@@ -198,13 +199,22 @@ def _plural(count, word):
|
||||
return f"{count} {word}" if count == 1 else f"{count} {word}s"
|
||||
|
||||
|
||||
def _time_left(done, total, batches, elapsed):
|
||||
""" " · about X min left" once enough batches ran, else "" (D56).
|
||||
|
||||
Shared by every Activity job that reports "Processing file N of M": the
|
||||
scan counts its batches, the image tagging job each file.
|
||||
"""
|
||||
if batches < MIN_BATCHES_FOR_ESTIMATE or done <= 0 or done >= total:
|
||||
return ""
|
||||
minutes = max(1, math.ceil(elapsed / done * (total - done) / 60))
|
||||
return f" · about {minutes} min left"
|
||||
|
||||
|
||||
def _scan_status_line(done, total, batches, elapsed):
|
||||
""" "Processing file N of M", with an estimate once enough batches ran."""
|
||||
line = f"Processing file {done:,} of {total:,}"
|
||||
if batches < MIN_BATCHES_FOR_ESTIMATE or done <= 0 or done >= total:
|
||||
return line
|
||||
minutes = max(1, math.ceil(elapsed / done * (total - done) / 60))
|
||||
return f"{line} · about {minutes} min left"
|
||||
return line + _time_left(done, total, batches, elapsed)
|
||||
|
||||
|
||||
def _file_size(path):
|
||||
@@ -493,14 +503,6 @@ class AdminRoutesMixin:
|
||||
async def get_stats_route(request):
|
||||
return await self.get_statistics(request)
|
||||
|
||||
@routes.get("/prompt_manager/settings")
|
||||
async def get_settings_route(request):
|
||||
return await self.get_settings(request)
|
||||
|
||||
@routes.post("/prompt_manager/settings")
|
||||
async def save_settings_route(request):
|
||||
return await self.save_settings(request)
|
||||
|
||||
@routes.get("/prompt_manager/backup")
|
||||
async def backup_database_route(request):
|
||||
return await self.backup_database(request)
|
||||
@@ -642,7 +644,15 @@ class AdminRoutesMixin:
|
||||
job.finish()
|
||||
|
||||
async def cleanup_duplicates_endpoint(self, request):
|
||||
"""Cleanup duplicate prompts endpoint."""
|
||||
"""Cleanup duplicate prompts endpoint (holds the registry: records)."""
|
||||
return await self._run_held(
|
||||
MAINTENANCE_JOB_KIND,
|
||||
"Prompt cleanup (admin page)",
|
||||
"records",
|
||||
self._cleanup_duplicates,
|
||||
)
|
||||
|
||||
async def _cleanup_duplicates(self):
|
||||
try:
|
||||
removed_count = await self._run_in_executor(self.db.cleanup_duplicates)
|
||||
|
||||
@@ -757,26 +767,39 @@ class AdminRoutesMixin:
|
||||
return hash_sha256.hexdigest()
|
||||
|
||||
async def delete_duplicate_images_endpoint(self, request):
|
||||
"""Delete duplicate image files from disk (single flight with the scan)."""
|
||||
job = self._acquire_long_job(self.LONG_JOB_DUPLICATES)
|
||||
if job is None:
|
||||
"""Delete duplicate image files from disk.
|
||||
|
||||
Single flight with the duplicate scan, and a records hold on the
|
||||
registry, so no scan, restore or tagging job runs meanwhile. A body
|
||||
without paths is refused before the hold (it is no job).
|
||||
"""
|
||||
data, error_response = await _read_json(request)
|
||||
if error_response is not None:
|
||||
return error_response
|
||||
image_paths = data.get("image_paths", [])
|
||||
if not image_paths or not isinstance(image_paths, list):
|
||||
return web.json_response(
|
||||
{"success": False, "error": "No image paths provided"},
|
||||
status=400,
|
||||
)
|
||||
guard = self._acquire_long_job(self.LONG_JOB_DUPLICATES)
|
||||
if guard is None:
|
||||
return self._long_job_busy_response("Duplicate scan or delete")
|
||||
async with job:
|
||||
return await self._delete_duplicate_images(request)
|
||||
|
||||
async def _delete_duplicate_images(self, request):
|
||||
async def delete():
|
||||
# No await since the check above: the lock is still free.
|
||||
async with guard:
|
||||
return await self._delete_duplicate_images(image_paths)
|
||||
|
||||
return await self._run_held(
|
||||
DUPLICATE_CLEANUP_JOB_KIND,
|
||||
"Duplicate image delete (gallery page)",
|
||||
"records",
|
||||
delete,
|
||||
)
|
||||
|
||||
async def _delete_duplicate_images(self, image_paths):
|
||||
try:
|
||||
data, error_response = await _read_json(request)
|
||||
if error_response is not None:
|
||||
return error_response
|
||||
image_paths = data.get("image_paths", [])
|
||||
|
||||
if not image_paths or not isinstance(image_paths, list):
|
||||
return web.json_response(
|
||||
{"success": False, "error": "No image paths provided"},
|
||||
status=400,
|
||||
)
|
||||
|
||||
output_dir = await self._run_in_executor(self._find_comfyui_output_dir)
|
||||
result = await self._run_in_executor(
|
||||
self._delete_duplicate_images_sync, image_paths, output_dir
|
||||
@@ -873,208 +896,6 @@ class AdminRoutesMixin:
|
||||
status=500,
|
||||
)
|
||||
|
||||
async def get_settings(self, request):
|
||||
"""Get current settings."""
|
||||
try:
|
||||
from ..config import PromptManagerConfig, GalleryConfig
|
||||
|
||||
root_paths = [
|
||||
self._public_path(d) for d in GalleryConfig.MONITORING_DIRECTORIES
|
||||
]
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
"settings": {
|
||||
"result_timeout": PromptManagerConfig.RESULT_TIMEOUT,
|
||||
"webui_display_mode": PromptManagerConfig.WEBUI_DISPLAY_MODE,
|
||||
"infinite_scroll": PromptManagerConfig.INFINITE_SCROLL,
|
||||
"worker_threads": PromptManagerConfig.WORKER_THREADS,
|
||||
"cpu_count": PromptManagerConfig.max_worker_threads(),
|
||||
"gallery_root_paths": root_paths,
|
||||
"gallery_root_path": root_paths[0] if root_paths else "",
|
||||
"monitored_directories": [
|
||||
self._public_path(d) for d in self._monitored_directories()
|
||||
],
|
||||
},
|
||||
}
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Get settings error: {e}", exc_info=True)
|
||||
return web.json_response(
|
||||
{"success": False, "error": "Failed to get settings"},
|
||||
status=500,
|
||||
)
|
||||
|
||||
def _monitored_directories(self):
|
||||
"""Directories the running image monitor watches, else the configured roots."""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
monitor = _current_image_monitor()
|
||||
if monitor is not None:
|
||||
return list(getattr(monitor, "monitored_directories", []))
|
||||
return list(GalleryConfig.MONITORING_DIRECTORIES)
|
||||
|
||||
async def save_settings(self, request):
|
||||
"""Save settings: the whole payload is validated before anything changes."""
|
||||
try:
|
||||
from ..config import PromptManagerConfig
|
||||
|
||||
data, error_response = await _read_json(request)
|
||||
if error_response is not None:
|
||||
return error_response
|
||||
|
||||
try:
|
||||
updates = self._validate_settings(data)
|
||||
except ValueError as ve:
|
||||
return web.json_response(
|
||||
{"success": False, "error": str(ve)}, status=400
|
||||
)
|
||||
|
||||
new_roots, error = self._parse_gallery_roots(data)
|
||||
if error:
|
||||
return web.json_response({"success": False, "error": error}, status=400)
|
||||
|
||||
for name, value in updates.items():
|
||||
setattr(PromptManagerConfig, name, value)
|
||||
restart_required = False
|
||||
if new_roots is not None:
|
||||
restart_required = self._apply_gallery_roots(new_roots)
|
||||
|
||||
self._persist_settings()
|
||||
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
"message": "Settings saved successfully",
|
||||
"restart_required": restart_required,
|
||||
}
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Save settings error: {e}", exc_info=True)
|
||||
return web.json_response(
|
||||
{"success": False, "error": "Failed to save settings"},
|
||||
status=500,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _validate_settings(data):
|
||||
"""Scalar settings of a payload as ``{CONFIG_ATTR: value}``.
|
||||
|
||||
Raises ValueError with a user-facing message on the first bad value;
|
||||
nothing is applied until every value passed.
|
||||
"""
|
||||
from ..config import PromptManagerConfig
|
||||
|
||||
updates = {}
|
||||
if "result_timeout" in data:
|
||||
updates["RESULT_TIMEOUT"] = validate_result_timeout(data["result_timeout"])
|
||||
if "webui_display_mode" in data:
|
||||
mode = data["webui_display_mode"]
|
||||
if mode not in ("popup", "newtab"):
|
||||
raise ValueError("webui_display_mode must be 'popup' or 'newtab'")
|
||||
updates["WEBUI_DISPLAY_MODE"] = mode
|
||||
if "infinite_scroll" in data:
|
||||
if not isinstance(data["infinite_scroll"], bool):
|
||||
raise ValueError("infinite_scroll must be true or false")
|
||||
updates["INFINITE_SCROLL"] = data["infinite_scroll"]
|
||||
if "worker_threads" in data:
|
||||
updates["WORKER_THREADS"] = validate_worker_threads(
|
||||
data["worker_threads"], PromptManagerConfig.max_worker_threads()
|
||||
)
|
||||
return updates
|
||||
|
||||
def _parse_gallery_roots(self, data):
|
||||
"""Extract and validate gallery roots from a settings payload.
|
||||
|
||||
Accepts ``gallery_root_paths`` (list, preferred) or the legacy single
|
||||
``gallery_root_path`` string. Blank entries are dropped.
|
||||
|
||||
Returns:
|
||||
(roots, error): ``roots`` is the validated list, or ``None`` when
|
||||
the payload carries no gallery root key; ``error`` is a message
|
||||
when validation failed (and ``roots`` is then ``None``).
|
||||
"""
|
||||
if "gallery_root_paths" in data:
|
||||
raw = data["gallery_root_paths"]
|
||||
if not isinstance(raw, list):
|
||||
return None, "gallery_root_paths must be a list"
|
||||
elif "gallery_root_path" in data:
|
||||
raw = [data["gallery_root_path"]]
|
||||
else:
|
||||
return None, None
|
||||
|
||||
from ..config import GalleryConfig
|
||||
|
||||
roots = []
|
||||
for entry in raw:
|
||||
if not isinstance(entry, str):
|
||||
return None, "Gallery root paths must be strings"
|
||||
entry = entry.strip()
|
||||
if not entry:
|
||||
continue
|
||||
ok, reason = GalleryConfig.validate_gallery_root(entry)
|
||||
if not ok:
|
||||
return (
|
||||
None,
|
||||
f"Invalid gallery root '{os.path.basename(entry)}': {reason}",
|
||||
)
|
||||
roots.append(GalleryConfig.resolve_gallery_root(entry))
|
||||
return roots, None
|
||||
|
||||
def _apply_gallery_roots(self, roots):
|
||||
"""Install validated gallery roots; returns True when they changed."""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
if roots == list(GalleryConfig.MONITORING_DIRECTORIES):
|
||||
return False
|
||||
GalleryConfig.MONITORING_DIRECTORIES = roots
|
||||
self._cached_output_dir = None
|
||||
self.reset_gallery_roots_cache()
|
||||
return True
|
||||
|
||||
def _persist_settings(self):
|
||||
"""Merge the user-editable settings into the configured config.json.
|
||||
|
||||
Only the keys this endpoint owns are rewritten; every other section
|
||||
(integrations, database, performance, ...) and every other key in
|
||||
``web_ui``/``gallery`` is kept as it was. A config file that cannot
|
||||
be parsed is replaced.
|
||||
"""
|
||||
from ..config import GalleryConfig, PromptManagerConfig, write_private_json
|
||||
|
||||
config_file = PromptManagerConfig.get_config_path()
|
||||
existing = self._read_config_file(config_file)
|
||||
web_ui = dict(existing.get("web_ui") or {})
|
||||
web_ui["result_timeout"] = PromptManagerConfig.RESULT_TIMEOUT
|
||||
web_ui["webui_display_mode"] = PromptManagerConfig.WEBUI_DISPLAY_MODE
|
||||
web_ui["infinite_scroll"] = PromptManagerConfig.INFINITE_SCROLL
|
||||
gallery = dict(existing.get("gallery") or {})
|
||||
monitoring = dict(gallery.get("monitoring") or {})
|
||||
monitoring["directories"] = list(GalleryConfig.MONITORING_DIRECTORIES)
|
||||
gallery["monitoring"] = monitoring
|
||||
performance = dict(existing.get("performance") or {})
|
||||
performance["worker_threads"] = PromptManagerConfig.WORKER_THREADS
|
||||
config_data = {
|
||||
**existing,
|
||||
"web_ui": web_ui,
|
||||
"gallery": gallery,
|
||||
"performance": performance,
|
||||
}
|
||||
try:
|
||||
write_private_json(config_file, config_data)
|
||||
self.logger.info(f"Settings saved to {config_file}")
|
||||
except OSError as save_err:
|
||||
self.logger.warning(f"Could not save config file: {save_err}")
|
||||
|
||||
def _read_config_file(self, config_file):
|
||||
"""Current contents of config.json as a dict, or {} when unusable."""
|
||||
try:
|
||||
with open(config_file, "r", encoding="utf-8") as f:
|
||||
data = json.load(f)
|
||||
except (OSError, ValueError):
|
||||
return {}
|
||||
return data if isinstance(data, dict) else {}
|
||||
|
||||
async def run_diagnostics(self, request):
|
||||
"""Run comprehensive system diagnostics and health checks."""
|
||||
try:
|
||||
@@ -1167,6 +988,11 @@ class AdminRoutesMixin:
|
||||
"vacuum",
|
||||
"cleanup_orphaned_images",
|
||||
)
|
||||
# Operations that only read; a run of only these holds the registry as
|
||||
# "read", every other run as "records".
|
||||
READ_ONLY_MAINTENANCE_OPERATIONS = frozenset(
|
||||
{"check_hash_duplicates", "statistics", "check_consistency"}
|
||||
)
|
||||
|
||||
async def run_maintenance(self, request):
|
||||
"""Perform database maintenance operations and report each outcome."""
|
||||
@@ -1179,7 +1005,27 @@ class AdminRoutesMixin:
|
||||
operations = data.get(
|
||||
"operations", list(self.DEFAULT_MAINTENANCE_OPERATIONS)
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Maintenance error: {e}", exc_info=True)
|
||||
return web.json_response(
|
||||
{"success": False, "error": "Maintenance failed"}, status=500
|
||||
)
|
||||
writes = not (
|
||||
isinstance(operations, list)
|
||||
and all(
|
||||
isinstance(name, str) and name in self.READ_ONLY_MAINTENANCE_OPERATIONS
|
||||
for name in operations
|
||||
)
|
||||
)
|
||||
return await self._run_held(
|
||||
MAINTENANCE_JOB_KIND,
|
||||
"Maintenance (admin page)",
|
||||
"records" if writes else "read",
|
||||
lambda: self._run_maintenance(operations),
|
||||
)
|
||||
|
||||
async def _run_maintenance(self, operations):
|
||||
try:
|
||||
results = await self._run_in_executor(
|
||||
self._run_maintenance_operations, operations
|
||||
)
|
||||
@@ -1333,8 +1179,18 @@ class AdminRoutesMixin:
|
||||
|
||||
The upload is streamed to a temp file (capped at restore_max_bytes),
|
||||
verified with PromptModel.verify_database_file and only then copied
|
||||
over the live database, which is backed up first.
|
||||
over the live database, which is backed up first. The whole request
|
||||
holds the registry (records): it is refused while a scan or another
|
||||
record-changing job runs, and those are refused while it runs.
|
||||
"""
|
||||
return await self._run_held(
|
||||
RESTORE_JOB_KIND,
|
||||
"Database restore (admin page)",
|
||||
"records",
|
||||
lambda: self._restore_database(request),
|
||||
)
|
||||
|
||||
async def _restore_database(self, request):
|
||||
max_bytes = getattr(self, "restore_max_bytes", 100 * 1024 * 1024)
|
||||
temp_path = None
|
||||
try:
|
||||
|
||||
@@ -8,6 +8,7 @@ from pathlib import Path
|
||||
from typing import Optional
|
||||
from urllib.parse import quote
|
||||
|
||||
from .jobs import AUTOTAG_BATCH_JOB_KIND
|
||||
from .prompts import thumbnail_url_for
|
||||
from .. import model_gate
|
||||
|
||||
@@ -63,6 +64,12 @@ def path_is_within(candidate, root) -> bool:
|
||||
return child != parent and child.is_relative_to(parent)
|
||||
|
||||
|
||||
async def _refusal(title):
|
||||
"""The single error event of an auto-tag batch refused by another job."""
|
||||
message = f"{title} already running"
|
||||
yield {"type": "error", "status": "error", "message": message}
|
||||
|
||||
|
||||
class AutotagRoutesMixin:
|
||||
"""Mixin providing auto-tagging API endpoints."""
|
||||
|
||||
@@ -425,6 +432,8 @@ class AutotagRoutesMixin:
|
||||
"general_threshold and character_threshold must be numeric"
|
||||
)
|
||||
|
||||
errors = [] # an error event ends the run; Activity lists it failed
|
||||
|
||||
async def events():
|
||||
try:
|
||||
from ..autotag import get_autotag_service
|
||||
@@ -432,14 +441,40 @@ class AutotagRoutesMixin:
|
||||
async for event in self._autotag_batch_events(
|
||||
get_autotag_service(), opts
|
||||
):
|
||||
if event.get("type") == "error":
|
||||
errors.append(event)
|
||||
yield event
|
||||
except Exception:
|
||||
self.logger.exception("AutoTag error")
|
||||
errors.append(_INTERNAL_ERROR)
|
||||
yield {
|
||||
"type": "error",
|
||||
"message": _INTERNAL_ERROR,
|
||||
}
|
||||
|
||||
# The batch writes prompt tags: a records hold on the registry, so it
|
||||
# never runs beside a scan, restore or the workspace tagging job.
|
||||
busy = self._jobs.blocker(AUTOTAG_BATCH_JOB_KIND, admission="records")
|
||||
if busy is not None:
|
||||
if busy.kind == AUTOTAG_BATCH_JOB_KIND: # unchanged payload
|
||||
return web.json_response(
|
||||
{"success": False, "error": _BUSY_ERROR}, status=409
|
||||
)
|
||||
# Both legacy pages read this route as a stream, where a 409 reads
|
||||
# as a lost connection: refuse with one error frame instead (the
|
||||
# admin page keys on ``type``, the gallery on ``status``).
|
||||
return await self._stream_sse(request, _refusal(busy.title))
|
||||
async with self._jobs.hold(
|
||||
AUTOTAG_BATCH_JOB_KIND,
|
||||
"Auto-tag images (legacy page)",
|
||||
admission="records",
|
||||
) as job:
|
||||
response = await self._start_autotag_batch(request, opts, events)
|
||||
if response.status >= 400 or errors:
|
||||
job.finish("failed")
|
||||
return response
|
||||
|
||||
async def _start_autotag_batch(self, request, opts, events):
|
||||
if not _autotag_batch_lock.acquire(blocking=False):
|
||||
return web.json_response(
|
||||
{"success": False, "error": _BUSY_ERROR}, status=409
|
||||
|
||||
@@ -0,0 +1,509 @@
|
||||
"""Bulk image tagging as an Activity job: Add tags and Auto-tag (#249 D1).
|
||||
|
||||
``POST /prompt_manager/jobs/image-tags`` starts one ``image-tags`` job over
|
||||
1..500 listed images, named by the listing identity ``{root_index,
|
||||
root_scope, path}``. The job is record-changing (it writes image tag sets and
|
||||
new vocabulary names), so it never runs beside the metadata scan.
|
||||
|
||||
- **Add tags** (``mode: "add"``) hashes each current file and unions a fixed
|
||||
tag set into the record of exactly those bytes.
|
||||
- **Auto-tag** (``mode: "auto"``) copies each still image into a private
|
||||
capture, asks one legacy engine for its top N tags on the capture, re-hashes
|
||||
the current file and unions the suggestions only when the bytes are still
|
||||
the captured ones. It holds the shared model gate for the whole job, loads
|
||||
the model once and never downloads one.
|
||||
|
||||
Every step reuses the Tag dialog's pieces: the capture store, the model gate,
|
||||
the suggestion worker and the storage call ``add_image_tags``. Files are
|
||||
handled one at a time and the job stops at a file boundary when cancelled,
|
||||
keeping what it already wrote. Rows and log lines carry root-relative paths
|
||||
and public root labels only.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import functools
|
||||
import os
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from .. import model_gate
|
||||
from .admin import _plural, _time_left
|
||||
from .annotation_suggest import (
|
||||
OptionError,
|
||||
unavailable_message,
|
||||
validate_options,
|
||||
)
|
||||
from .annotations import (
|
||||
_MESSAGES,
|
||||
MAX_TARGET_PATH,
|
||||
_coded,
|
||||
_error,
|
||||
_pinned_suggestions,
|
||||
_settled,
|
||||
_submit,
|
||||
_target,
|
||||
)
|
||||
from .autotag_routes import model_busy_response
|
||||
from .images import _public_path
|
||||
from .job_routes import busy_response
|
||||
from .jobs import IMAGE_TAG_GROUP_NAMES, IMAGE_TAG_JOB_KIND, JobBusy
|
||||
from .prompts import bad_request
|
||||
|
||||
try:
|
||||
from ...database.annotations import ImageTagLimitError, normalize_image_tags
|
||||
from ...database.library_commands import LibraryCommandError
|
||||
from ...utils.media_capture import (
|
||||
CaptureCapacity,
|
||||
CaptureChanged,
|
||||
CaptureDenied,
|
||||
CaptureError,
|
||||
CaptureTooLarge,
|
||||
)
|
||||
from ...utils.video_metadata import is_video_path
|
||||
except ImportError:
|
||||
from database.annotations import ImageTagLimitError, normalize_image_tags
|
||||
from database.library_commands import LibraryCommandError
|
||||
from utils.media_capture import (
|
||||
CaptureCapacity,
|
||||
CaptureChanged,
|
||||
CaptureDenied,
|
||||
CaptureError,
|
||||
CaptureTooLarge,
|
||||
)
|
||||
from utils.video_metadata import is_video_path
|
||||
|
||||
MAX_JOB_TARGETS = 500
|
||||
# 500 maximal targets plus the options: a path of MAX_TARGET_PATH code points
|
||||
# takes at most 6 bytes each on the wire (a \uXXXX escape; raw UTF-8 is at most
|
||||
# 4), and the root index and scope fit in 256. The annotation routes' 64 KiB
|
||||
# bound is for one target, too small for a full selection.
|
||||
MAX_JOB_BODY_BYTES = MAX_JOB_TARGETS * (MAX_TARGET_PATH * 6 + 256) + 64 * 1024
|
||||
# A busy capture worker (a Tag dialog in another tab) is waited for this long,
|
||||
# try by try, before the file is left for a rerun.
|
||||
CAPACITY_BACKOFF = (0.1, 0.2, 0.4, 0.8)
|
||||
|
||||
ADD_TITLE = "Add tags to images"
|
||||
AUTO_TITLE = "Auto-tag images"
|
||||
GATE_OWNER_KIND = "image-tag-job"
|
||||
|
||||
_FIELDS = {
|
||||
"add": frozenset({"mode", "targets", "tags"}),
|
||||
"auto": frozenset({"mode", "targets", "model_type", "options"}),
|
||||
}
|
||||
_VERBS = {"add": "Add tags", "auto": "Auto-tag"}
|
||||
_LOG_PREFIXES = {"tagged": "+", "unchanged": "=", "skipped": "-", "errors": "!"}
|
||||
|
||||
NO_NEW_TAGS = "No new tags"
|
||||
VIDEO_SKIPPED = "Video: manual tags only"
|
||||
TOO_LARGE = "Too large to verify"
|
||||
FILE_CHANGED = "File changed"
|
||||
FILE_NOT_FOUND = "File not found"
|
||||
NOT_A_STILL = "Not a supported still image"
|
||||
INFERENCE_FAILED = "Auto-tagging failed. Check server logs for details."
|
||||
SAVE_FAILED = "The tags could not be saved. Check server logs for details."
|
||||
READ_FAILED = "The file could not be read."
|
||||
SOURCE_CHANGED = "The source changed. Refresh the image collection."
|
||||
STORAGE_RESTORED = "Prompt storage was restored; start the job again."
|
||||
BUSY_RETRY = "The file could not be read now. Run the job again for it."
|
||||
|
||||
_monotonic = time.monotonic # the job's clock (a test can fake it)
|
||||
|
||||
|
||||
class _JobStopped(Exception):
|
||||
"""No later file can be written (prompt storage was restored)."""
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Run:
|
||||
"""One running job: its request, the gate owner and the resolved roots.
|
||||
|
||||
``roots`` maps each target's ``(root_index, root_scope)`` to ``(root,
|
||||
label)``, read once per job off the event loop; ``root`` is None when the
|
||||
scope no longer names a current root.
|
||||
"""
|
||||
|
||||
job: object
|
||||
spec: object
|
||||
epoch: str
|
||||
owner: object
|
||||
service: object = None
|
||||
roots: dict = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _Spec:
|
||||
"""A validated start request."""
|
||||
|
||||
mode: str
|
||||
targets: tuple
|
||||
tags: tuple = ()
|
||||
model_type: str | None = None
|
||||
options: dict | None = None
|
||||
|
||||
@property
|
||||
def title(self) -> str:
|
||||
return ADD_TITLE if self.mode == "add" else AUTO_TITLE
|
||||
|
||||
def scope(self) -> dict:
|
||||
scope = {"mode": self.mode, "images": len(self.targets)}
|
||||
if self.mode == "auto":
|
||||
scope["model_type"] = self.model_type
|
||||
return scope
|
||||
|
||||
|
||||
def _parse_targets(raw):
|
||||
"""Unique validated targets in request order, or ``(None, error)``."""
|
||||
if not isinstance(raw, list) or not 1 <= len(raw) <= MAX_JOB_TARGETS:
|
||||
return None, bad_request(f"targets must list 1 to {MAX_JOB_TARGETS} images")
|
||||
seen, targets = set(), []
|
||||
for item in raw:
|
||||
target, error = _target(item)
|
||||
if error is not None:
|
||||
return None, error
|
||||
if target not in seen:
|
||||
seen.add(target)
|
||||
targets.append(target)
|
||||
return tuple(targets), None
|
||||
|
||||
|
||||
def _parse_add(targets, data):
|
||||
try:
|
||||
tags = normalize_image_tags(data["tags"])
|
||||
except ImageTagLimitError as exc:
|
||||
return None, _error(400, "tag-limit", str(exc))
|
||||
except ValueError as exc:
|
||||
return None, bad_request(str(exc))
|
||||
if not tags:
|
||||
return None, bad_request("tags must name at least one tag")
|
||||
return _Spec("add", targets, tags=tuple(tags)), None
|
||||
|
||||
|
||||
def _parse_auto(targets, data):
|
||||
from ..config import MAX_SUGGESTION_LIMIT, AutotagConfig
|
||||
|
||||
try:
|
||||
model_type, options = validate_options(
|
||||
data["model_type"],
|
||||
data["options"],
|
||||
default_limit=AutotagConfig.suggestion_limit(),
|
||||
max_limit=MAX_SUGGESTION_LIMIT,
|
||||
)
|
||||
except OptionError as exc:
|
||||
return None, bad_request(str(exc))
|
||||
return _Spec("auto", targets, model_type=model_type, options=options), None
|
||||
|
||||
|
||||
def parse_image_tag_request(data: dict):
|
||||
"""A :class:`_Spec` for a start body, or ``(None, error response)``.
|
||||
|
||||
``{mode:"add", targets, tags}`` or ``{mode:"auto", targets, model_type,
|
||||
options}`` exactly; options follow the Tag dialog's Generate rules.
|
||||
"""
|
||||
fields = _FIELDS.get(data.get("mode"))
|
||||
if fields is None:
|
||||
return None, bad_request('mode must be "add" or "auto"')
|
||||
if set(data) != fields:
|
||||
return None, bad_request(
|
||||
"Request must contain exactly " + ", ".join(sorted(fields))
|
||||
)
|
||||
targets, error = _parse_targets(data["targets"])
|
||||
if error is not None:
|
||||
return None, error
|
||||
if data["mode"] == "add":
|
||||
return _parse_add(targets, data)
|
||||
return _parse_auto(targets, data)
|
||||
|
||||
|
||||
def _capture_outcome(exc: CaptureError, root, relative: str):
|
||||
"""The (group, detail) of a file the capture store refused to read."""
|
||||
if isinstance(exc, CaptureTooLarge):
|
||||
return "skipped", TOO_LARGE
|
||||
if isinstance(exc, CaptureDenied):
|
||||
return "skipped", FILE_NOT_FOUND
|
||||
if isinstance(exc, CaptureChanged):
|
||||
path = os.path.join(root, *relative.split("/"))
|
||||
return "skipped", FILE_CHANGED if os.path.lexists(path) else FILE_NOT_FOUND
|
||||
if isinstance(exc, CaptureCapacity):
|
||||
return "skipped", BUSY_RETRY
|
||||
return "errors", READ_FAILED
|
||||
|
||||
|
||||
async def _capacity_retried(job, attempt):
|
||||
"""``await attempt()``, tried again while the capture store is at capacity.
|
||||
|
||||
Raises the last ``CaptureCapacity`` once ``CAPACITY_BACKOFF`` is spent or the
|
||||
job was cancelled meanwhile.
|
||||
"""
|
||||
for delay in CAPACITY_BACKOFF:
|
||||
try:
|
||||
return await attempt()
|
||||
except CaptureCapacity:
|
||||
if job.cancelled:
|
||||
raise
|
||||
await asyncio.sleep(delay)
|
||||
return await attempt()
|
||||
|
||||
|
||||
def _load_model(service, model_type: str, use_gpu: bool) -> None:
|
||||
"""Worker thread: load (or swap to) *model_type*; never downloads."""
|
||||
if service.is_model_loaded() and service.get_loaded_model_type() == model_type:
|
||||
return
|
||||
if service.load_model(model_type, use_gpu) is False:
|
||||
raise RuntimeError(f"{model_type} could not be loaded")
|
||||
|
||||
|
||||
class ImageTagJobsMixin:
|
||||
"""The ``image-tags`` job kind: start route and runner."""
|
||||
|
||||
async def start_image_tag_job(self, request: web.Request) -> web.Response:
|
||||
"""Start bulk tagging (202), or 409 ``busy`` / ``model-busy``.
|
||||
|
||||
Every target's root scope is checked before the job starts
|
||||
(``source-changed``). Auto-tag takes the model gate here, so a start
|
||||
while anything else holds it is refused at once.
|
||||
"""
|
||||
data, error = await self._read_json_body(request, max_bytes=MAX_JOB_BODY_BYTES)
|
||||
if error is not None:
|
||||
return error
|
||||
spec, error = parse_image_tag_request(data)
|
||||
if error is not None:
|
||||
return error
|
||||
roots = await self._run_in_executor(self._job_roots, spec.targets)
|
||||
if any(root is None for root, _label in roots.values()):
|
||||
return _coded("source-changed")
|
||||
epoch = await self._run_in_executor(self.db.get_execution_epoch)
|
||||
# A running job in the way answers first: it says which job to wait for.
|
||||
blocker = self._jobs.blocker(IMAGE_TAG_JOB_KIND, record_changing=True)
|
||||
if blocker is not None:
|
||||
return busy_response(JobBusy(blocker))
|
||||
owner = None
|
||||
if spec.mode == "auto":
|
||||
owner = model_gate.new_owner(GATE_OWNER_KIND)
|
||||
if not model_gate.try_acquire(owner):
|
||||
return model_busy_response()
|
||||
try:
|
||||
job = self._jobs.start(
|
||||
IMAGE_TAG_JOB_KIND,
|
||||
spec.title,
|
||||
record_changing=True,
|
||||
scope=spec.scope(),
|
||||
runner=functools.partial(self._run_image_tag_job, spec, epoch, owner),
|
||||
groups=IMAGE_TAG_GROUP_NAMES,
|
||||
)
|
||||
except JobBusy as busy:
|
||||
if owner is not None:
|
||||
model_gate.release(owner)
|
||||
return busy_response(busy)
|
||||
if owner is not None:
|
||||
# Also when the task is cancelled before its runner ever starts.
|
||||
job.task.add_done_callback(lambda _task: model_gate.release(owner))
|
||||
return web.json_response({"success": True, "job": job.summary()}, status=202)
|
||||
|
||||
# -- runner ----------------------------------------------------------------
|
||||
|
||||
def _job_roots(self, targets) -> dict:
|
||||
"""Worker thread: ``{(index, scope): (root or None, label)}`` (see _Run)."""
|
||||
roots = list(self._get_all_output_dirs())
|
||||
resolved = {}
|
||||
for index, scope in dict.fromkeys((t[0], t[1]) for t in targets):
|
||||
label = _public_path(roots[index]) if index < len(roots) else ""
|
||||
error = self._gallery_scope_error(
|
||||
{"root_scope": scope, "root": str(index)}, roots
|
||||
)
|
||||
resolved[(index, scope)] = (roots[index] if error is None else None, label)
|
||||
return resolved
|
||||
|
||||
async def _run_image_tag_job(self, spec: _Spec, epoch: str, owner, job) -> None:
|
||||
verb = _VERBS[spec.mode]
|
||||
total = len(spec.targets)
|
||||
job.update(total=total, status_line=f"Processing file 0 of {total:,}")
|
||||
started = f"{verb} started · {_plural(total, 'image')}"
|
||||
if spec.mode == "auto":
|
||||
started += f" · {spec.model_type}"
|
||||
job.log_line(started)
|
||||
run = _Run(job, spec, epoch, owner)
|
||||
try:
|
||||
if spec.mode == "auto":
|
||||
run.service = await self._prepare_tagging_model(job, spec, owner)
|
||||
if run.service is None:
|
||||
job.finish("failed")
|
||||
return
|
||||
if not await self._tag_each_file(run):
|
||||
job.log_line(f"{verb} stopped · {STORAGE_RESTORED}")
|
||||
job.finish("failed")
|
||||
return
|
||||
state = "cancelled" if job.cancelled else "finished"
|
||||
job.log_line(f"{verb} {state} · {_plural(job.processed, 'file')} processed")
|
||||
job.publish({"type": state, "progress": job.progress})
|
||||
job.finish(state)
|
||||
finally:
|
||||
if owner is not None:
|
||||
model_gate.release(owner)
|
||||
|
||||
async def _tag_each_file(self, run: _Run) -> bool:
|
||||
"""Tag the targets in order; False when the job had to stop."""
|
||||
job, total = run.job, len(run.spec.targets)
|
||||
run.roots = await self._run_in_executor(self._job_roots, run.spec.targets)
|
||||
started = _monotonic()
|
||||
for number, target in enumerate(run.spec.targets, 1):
|
||||
if job.cancelled:
|
||||
break
|
||||
# The scan's estimate (D56), with each finished file as one batch.
|
||||
left = _time_left(number - 1, total, number - 1, _monotonic() - started)
|
||||
job.update(status_line=f"Processing file {number:,} of {total:,}{left}")
|
||||
job.publish({"type": "progress", "progress": job.progress})
|
||||
stopped = False
|
||||
try:
|
||||
group, detail = await self._tag_one_file(run, target)
|
||||
except _JobStopped:
|
||||
stopped, group, detail = True, "errors", STORAGE_RESTORED
|
||||
except Exception:
|
||||
self.logger.exception("Image tagging failed for one file")
|
||||
group, detail = "errors", SAVE_FAILED
|
||||
self._record_tag_outcome(run, target, group, detail)
|
||||
job.update(processed=number, progress=number * 100 // total)
|
||||
if stopped:
|
||||
return False
|
||||
return True
|
||||
|
||||
async def _prepare_tagging_model(self, job, spec: _Spec, owner):
|
||||
"""The loaded service, or None (logged) when the job cannot run."""
|
||||
from ..autotag import MODELS, get_autotag_service
|
||||
|
||||
service = get_autotag_service()
|
||||
status = await self._run_in_executor(service.get_models_status)
|
||||
if not status.get(spec.model_type, {}).get("downloaded"):
|
||||
name = MODELS.get(spec.model_type, {}).get("name", spec.model_type)
|
||||
message = unavailable_message(name)
|
||||
job.update(status_line=message)
|
||||
job.log_line(f"! {message}")
|
||||
return None
|
||||
try:
|
||||
await model_gate.run(
|
||||
owner,
|
||||
_load_model,
|
||||
service,
|
||||
spec.model_type,
|
||||
spec.options["use_gpu"],
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception("The auto-tag job could not load its model")
|
||||
job.update(status_line=INFERENCE_FAILED)
|
||||
job.log_line(f"! {INFERENCE_FAILED}")
|
||||
return None
|
||||
return service
|
||||
|
||||
async def _verified(self, run: _Run, store, root, relative):
|
||||
"""The current file's facts, waiting out a busy capture store."""
|
||||
return await _capacity_retried(
|
||||
run.job, lambda: self._run_in_executor(store.verify, root, relative)
|
||||
)
|
||||
|
||||
async def _tag_one_file(self, run: _Run, target):
|
||||
"""The (group, detail) of one target; raises _JobStopped on a restore."""
|
||||
index, scope, relative = target
|
||||
root, _label = run.roots[(index, scope)]
|
||||
if root is None:
|
||||
return "errors", SOURCE_CHANGED
|
||||
store = self._capture_store()
|
||||
if run.spec.mode == "add":
|
||||
try:
|
||||
facts = await self._verified(run, store, root, relative)
|
||||
except CaptureError as exc:
|
||||
return _capture_outcome(exc, root, relative)
|
||||
return await self._union_image_tags(facts, list(run.spec.tags), run.epoch)
|
||||
if is_video_path(relative):
|
||||
return "skipped", VIDEO_SKIPPED
|
||||
try:
|
||||
capture = await _capacity_retried(
|
||||
run.job,
|
||||
lambda: _settled(
|
||||
_submit(
|
||||
store.capture,
|
||||
root,
|
||||
relative,
|
||||
epoch=run.epoch,
|
||||
scope=scope,
|
||||
root_index=index,
|
||||
),
|
||||
lambda made: made is not None and store.release(made.token),
|
||||
),
|
||||
)
|
||||
except CaptureError as exc:
|
||||
return _capture_outcome(exc, root, relative)
|
||||
try:
|
||||
return await self._auto_tag_capture(run, store, capture, root, relative)
|
||||
finally:
|
||||
store.release(capture.token)
|
||||
|
||||
async def _auto_tag_capture(self, run: _Run, store, capture, root, relative):
|
||||
"""Suggest on the capture, then union only onto the same current bytes."""
|
||||
if not capture.inferable:
|
||||
return "skipped", NOT_A_STILL
|
||||
if store.acquire(capture.token) is None:
|
||||
raise _JobStopped # every capture was invalidated (restore)
|
||||
spec = run.spec
|
||||
try:
|
||||
pairs = await model_gate.run(
|
||||
run.owner,
|
||||
_pinned_suggestions,
|
||||
store,
|
||||
capture.token,
|
||||
run.service,
|
||||
capture,
|
||||
spec.model_type,
|
||||
spec.options,
|
||||
on_skipped=functools.partial(store.release_use, capture.token),
|
||||
)
|
||||
except Exception:
|
||||
self.logger.exception("Auto-tagging failed for one file")
|
||||
return "errors", INFERENCE_FAILED
|
||||
try:
|
||||
facts = await self._verified(run, store, root, relative)
|
||||
except CaptureError as exc:
|
||||
return _capture_outcome(exc, root, relative)
|
||||
if facts.locator != capture.locator or facts.sha256 != capture.sha256:
|
||||
return "skipped", FILE_CHANGED
|
||||
names = [str(name) for name, _score in pairs]
|
||||
if not names:
|
||||
return "unchanged", NO_NEW_TAGS
|
||||
return await self._union_image_tags(facts, names, run.epoch)
|
||||
|
||||
async def _union_image_tags(self, facts, tags: list, epoch: str):
|
||||
"""``add_image_tags`` on the verified bytes, as a (group, detail)."""
|
||||
try:
|
||||
receipt = await self._run_in_executor(
|
||||
self.db.add_image_tags,
|
||||
locator=facts.locator,
|
||||
sha256=facts.sha256,
|
||||
size=facts.size,
|
||||
mtime_ns=facts.mtime_ns,
|
||||
expected_epoch=epoch,
|
||||
tags=tags,
|
||||
)
|
||||
except LibraryCommandError as exc:
|
||||
if exc.code == "storage-changed":
|
||||
raise _JobStopped from None
|
||||
if exc.code == "tag-limit":
|
||||
return "errors", _MESSAGES["tag-limit"]
|
||||
self.logger.warning(f"Image tags not saved ({exc.code})")
|
||||
return "errors", SAVE_FAILED
|
||||
except ValueError:
|
||||
self.logger.exception("Image tags were refused")
|
||||
return "errors", SAVE_FAILED
|
||||
added = receipt.get("added", 0)
|
||||
if added:
|
||||
return "tagged", f"Added {_plural(added, 'tag')}"
|
||||
return "unchanged", NO_NEW_TAGS
|
||||
|
||||
def _record_tag_outcome(self, run: _Run, target, group: str, detail: str) -> None:
|
||||
index, scope, relative = target
|
||||
_root, label = run.roots[(index, scope)]
|
||||
where = f"{label}/{relative}" if label else relative
|
||||
run.job.record(
|
||||
group,
|
||||
{"path": relative, "root": label, "root_index": index, "detail": detail},
|
||||
f"{_LOG_PREFIXES[group]} {where} · {detail}",
|
||||
)
|
||||
+54
-8
@@ -19,6 +19,7 @@ from typing import Callable, NamedTuple
|
||||
from aiohttp import web
|
||||
from PIL import Image, UnidentifiedImageError
|
||||
|
||||
from .jobs import THUMBNAILS_JOB_KIND
|
||||
from .prompts import bad_request, parse_page_params, publish_image_paths
|
||||
from .prompts import thumbnail_url_for
|
||||
from .prompts import safe_error_message as _safe_error
|
||||
@@ -104,6 +105,10 @@ def _card_header(path: Path) -> tuple:
|
||||
return inspection.width, inspection.height, records
|
||||
|
||||
|
||||
# The legacy gallery page's thumbnail generation, as Activity lists it (D80).
|
||||
THUMBNAIL_GENERATION_TITLE = "Thumbnail generation (gallery page)"
|
||||
|
||||
|
||||
class _CardFacts(NamedTuple):
|
||||
"""One image file's card facts, derived from its header alone.
|
||||
|
||||
@@ -1736,11 +1741,24 @@ class ImageRoutesMixin:
|
||||
return cls.THUMBNAIL_SIZES.get(quality, cls.THUMBNAIL_SIZES["medium"])
|
||||
|
||||
async def generate_thumbnails(self, request):
|
||||
"""Generate every thumbnail in one executor job (POST, blocking)."""
|
||||
"""Generate every thumbnail in one executor job (POST, blocking).
|
||||
|
||||
Holds the registry as guarded work: refused while a record-changing
|
||||
job runs, and such jobs are refused while it runs. A body that cannot
|
||||
be read is refused before the hold, so it is never listed as a job.
|
||||
"""
|
||||
data, error = await _json_body_or_error(request)
|
||||
if error is not None:
|
||||
return error
|
||||
return await self._run_held(
|
||||
THUMBNAILS_JOB_KIND,
|
||||
THUMBNAIL_GENERATION_TITLE,
|
||||
"guarded",
|
||||
lambda: self._generate_thumbnails(data),
|
||||
)
|
||||
|
||||
async def _generate_thumbnails(self, data):
|
||||
try:
|
||||
data, error = await _json_body_or_error(request)
|
||||
if error is not None:
|
||||
return error
|
||||
thumbnail_size = self._thumbnail_size(data.get("quality", "medium"))
|
||||
|
||||
output_dirs = self._get_all_output_dirs()
|
||||
@@ -1935,14 +1953,32 @@ class ImageRoutesMixin:
|
||||
"Connection": "keep-alive",
|
||||
},
|
||||
)
|
||||
await response.prepare(request)
|
||||
busy = self._jobs.blocker(THUMBNAILS_JOB_KIND, admission="guarded")
|
||||
if busy is not None:
|
||||
# The gallery reads this stream: refuse with its error event.
|
||||
await response.prepare(request)
|
||||
message = f"{busy.title} already running"
|
||||
await self._emit_sse(
|
||||
response, "error", {"error": message, "message": message}
|
||||
)
|
||||
return response
|
||||
async with self._jobs.hold(
|
||||
THUMBNAILS_JOB_KIND, THUMBNAIL_GENERATION_TITLE, admission="guarded"
|
||||
) as job:
|
||||
await response.prepare(request)
|
||||
if not await self._thumbnail_progress_stream(response, thumbnail_size):
|
||||
job.finish("failed") # an error event ended the stream
|
||||
return response
|
||||
|
||||
async def _thumbnail_progress_stream(self, response, thumbnail_size) -> bool:
|
||||
"""Stream one generation run; False when it ended with an error event."""
|
||||
try:
|
||||
output_dirs = self._get_all_output_dirs()
|
||||
if not output_dirs:
|
||||
await self._emit_sse(
|
||||
response, "error", {"error": "ComfyUI output directory not found"}
|
||||
)
|
||||
return response
|
||||
return False
|
||||
|
||||
await self._emit_sse(
|
||||
response,
|
||||
@@ -1960,6 +1996,7 @@ class ImageRoutesMixin:
|
||||
remaining=plan["remaining"],
|
||||
)
|
||||
self.invalidate_gallery_cache()
|
||||
return True
|
||||
except Exception:
|
||||
self.logger.exception("Thumbnail generation failed")
|
||||
await self._emit_sse(
|
||||
@@ -1970,7 +2007,7 @@ class ImageRoutesMixin:
|
||||
"message": "Thumbnail generation failed. Check server logs.",
|
||||
},
|
||||
)
|
||||
return response
|
||||
return False
|
||||
|
||||
async def _stream_thumbnails(
|
||||
self, response, targets, thumbnail_size, skipped=0, remaining=0
|
||||
@@ -2179,8 +2216,17 @@ class ImageRoutesMixin:
|
||||
"""Clear generated thumbnails across gallery roots, preserving originals.
|
||||
|
||||
Invalid thumbnail directories return 400; unavailable gallery roots
|
||||
return 404, and unexpected filesystem failures return 500.
|
||||
return 404, and unexpected filesystem failures return 500. Holds the
|
||||
registry as guarded work, like generation.
|
||||
"""
|
||||
return await self._run_held(
|
||||
THUMBNAILS_JOB_KIND,
|
||||
"Thumbnail clear (gallery page)",
|
||||
"guarded",
|
||||
self._clear_thumbnails,
|
||||
)
|
||||
|
||||
async def _clear_thumbnails(self) -> web.Response:
|
||||
try:
|
||||
output_dirs = self._get_all_output_dirs()
|
||||
if not output_dirs:
|
||||
|
||||
+23
-13
@@ -1,8 +1,9 @@
|
||||
"""Session job routes (Activity): list, inspect, follow and cancel jobs.
|
||||
|
||||
The registry itself is ``py/api/jobs.py``; the scan job's runner is in
|
||||
``py/api/admin.py``. Every response leaves out absolute server paths: job
|
||||
rows already carry root-relative paths and public root labels.
|
||||
The registry itself is ``py/api/jobs.py``. The scan job's runner is in
|
||||
``py/api/admin.py``, the image tagging job's in ``py/api/image_tag_jobs.py``.
|
||||
Every response leaves out absolute server paths: job rows already carry
|
||||
root-relative paths and public root labels.
|
||||
"""
|
||||
|
||||
import contextlib
|
||||
@@ -42,6 +43,20 @@ async def _scan_scope(request):
|
||||
return scope if scope in SCAN_SCOPES else None
|
||||
|
||||
|
||||
def busy_response(busy: JobBusy):
|
||||
"""``409 busy``, naming the running job that holds the slot."""
|
||||
running = busy.running
|
||||
return web.json_response(
|
||||
{
|
||||
"success": False,
|
||||
"code": "busy",
|
||||
"error": str(busy),
|
||||
"running": {"id": running.id, "title": running.title},
|
||||
},
|
||||
status=409,
|
||||
)
|
||||
|
||||
|
||||
def _summary_event(job):
|
||||
return f"data: {json.dumps(job.summary())}\n\n".encode("utf-8")
|
||||
|
||||
@@ -58,6 +73,10 @@ class JobRoutesMixin:
|
||||
async def start_scan_job_route(request):
|
||||
return await self.start_scan_job(request)
|
||||
|
||||
@routes.post("/prompt_manager/jobs/image-tags")
|
||||
async def start_image_tag_job_route(request):
|
||||
return await self.start_image_tag_job(request)
|
||||
|
||||
@routes.get("/prompt_manager/jobs/{job_id}")
|
||||
async def job_detail_route(request):
|
||||
return await self.job_detail(request)
|
||||
@@ -121,16 +140,7 @@ class JobRoutesMixin:
|
||||
try:
|
||||
job = self._start_scan_job(files)
|
||||
except JobBusy as busy:
|
||||
running = busy.running
|
||||
return web.json_response(
|
||||
{
|
||||
"success": False,
|
||||
"code": "busy",
|
||||
"error": str(busy),
|
||||
"running": {"id": running.id, "title": running.title},
|
||||
},
|
||||
status=409,
|
||||
)
|
||||
return busy_response(busy)
|
||||
return web.json_response({"success": True, "job": job.summary()}, status=202)
|
||||
|
||||
async def job_events(self, request):
|
||||
|
||||
+172
-23
@@ -9,8 +9,21 @@ as a snapshot and then every event published after it, until the job ends.
|
||||
|
||||
Jobs live only in server memory and are gone after a restart. The registry
|
||||
keeps the running jobs and this session's last ``MAX_FINISHED_JOBS`` finished
|
||||
ones. Record-changing jobs (the metadata scan, and later maintenance, cleanup
|
||||
and restore) never run together, and one job of a kind runs at a time.
|
||||
ones. One job of a kind runs at a time, and each job has an admission class
|
||||
(``ADMISSIONS``) that decides what may run beside it:
|
||||
|
||||
- ``records`` changes library records or files (the metadata scan, bulk image
|
||||
tagging, and the legacy restore, maintenance, cleanup, duplicate delete,
|
||||
LoRA import and batch auto-tag routes). It is refused while any
|
||||
``records`` or ``guarded`` job runs.
|
||||
- ``guarded`` must not overlap record changes but may overlap other guarded
|
||||
work (thumbnail generation and clearing, later backup). It is refused while
|
||||
a ``records`` job runs.
|
||||
- ``read`` only reads; it is refused only by a running job of its own kind.
|
||||
|
||||
A short legacy-page operation takes a ``hold``: a running, non-cancellable
|
||||
job that lasts as long as the request, so the Activity page lists it and a
|
||||
workspace job is refused while it runs (and the other way round).
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
@@ -54,6 +67,46 @@ SCAN_GROUPS = {
|
||||
"failed": "errors",
|
||||
}
|
||||
|
||||
# Bulk image tagging (#249 D1): Add tags and Auto-tag over selected images.
|
||||
# Record-changing: it writes image tag sets and new vocabulary names.
|
||||
IMAGE_TAG_JOB_KIND = "image-tags"
|
||||
IMAGE_TAG_GROUP_NAMES = ("tagged", "unchanged", "skipped", "errors")
|
||||
|
||||
ADMISSIONS = ("read", "guarded", "records")
|
||||
# What each admission class is refused beside (besides its own kind).
|
||||
_REFUSED_BESIDE = {
|
||||
"read": frozenset(),
|
||||
"guarded": frozenset({"records"}),
|
||||
"records": frozenset({"records", "guarded"}),
|
||||
}
|
||||
|
||||
# Kinds of the legacy-page operations that hold the registry (D80; titles in
|
||||
# their handlers end in "(admin page)", "(gallery page)" or "(legacy page)").
|
||||
RESTORE_JOB_KIND = "restore"
|
||||
MAINTENANCE_JOB_KIND = "maintenance"
|
||||
DUPLICATE_CLEANUP_JOB_KIND = "duplicate_cleanup"
|
||||
THUMBNAILS_JOB_KIND = "thumbnails"
|
||||
LORA_IMPORT_JOB_KIND = "lora_import"
|
||||
AUTOTAG_BATCH_JOB_KIND = "autotag"
|
||||
|
||||
|
||||
def resolve_admission(admission=None, record_changing=None) -> str:
|
||||
"""The admission class for a start call; ``record_changing`` is an alias.
|
||||
|
||||
``record_changing=True`` means ``records`` and ``False`` means ``read``;
|
||||
with neither given the class is ``read``. Raises ValueError for an
|
||||
unknown class or one that contradicts ``record_changing``.
|
||||
"""
|
||||
if admission is None:
|
||||
return "records" if record_changing else "read"
|
||||
if admission not in ADMISSIONS:
|
||||
raise ValueError(f"Unknown job admission {admission!r}")
|
||||
if record_changing is not None and record_changing != (admission == "records"):
|
||||
raise ValueError(
|
||||
f"record_changing={record_changing!r} contradicts admission {admission!r}"
|
||||
)
|
||||
return admission
|
||||
|
||||
|
||||
def _relative_path(path) -> str:
|
||||
"""``path`` as a root-relative POSIX path; an absolute one keeps its name."""
|
||||
@@ -79,13 +132,23 @@ class Job:
|
||||
"""One tracked operation: state, counters, outcomes, log and subscribers."""
|
||||
|
||||
def __init__(
|
||||
self, kind, title, *, record_changing=False, scope=None, groups=(), clock=None
|
||||
self,
|
||||
kind,
|
||||
title,
|
||||
*,
|
||||
admission=None,
|
||||
record_changing=None,
|
||||
cancellable=True,
|
||||
scope=None,
|
||||
groups=(),
|
||||
clock=None,
|
||||
):
|
||||
self._clock = clock or time.time
|
||||
self.id = uuid.uuid4().hex
|
||||
self.kind = kind
|
||||
self.title = title
|
||||
self.record_changing = record_changing
|
||||
self.admission = resolve_admission(admission, record_changing)
|
||||
self.cancellable = bool(cancellable)
|
||||
self.scope = dict(scope or {})
|
||||
self.state = "running"
|
||||
self.started_at = self._clock()
|
||||
@@ -104,6 +167,10 @@ class Job:
|
||||
self.task = None
|
||||
self._subscribers = set()
|
||||
|
||||
@property
|
||||
def record_changing(self):
|
||||
return self.admission == "records"
|
||||
|
||||
@property
|
||||
def running(self):
|
||||
return self.state == "running"
|
||||
@@ -234,6 +301,9 @@ class Job:
|
||||
# What detail() lists, so a view re-reads it only when it changed.
|
||||
"listed": {group: len(rows) for group, rows in self.outcomes.items()},
|
||||
"log_length": len(self.log),
|
||||
# A view offers Cancel only when the server can stop the job.
|
||||
"cancellable": self.cancellable,
|
||||
"admission": self.admission,
|
||||
}
|
||||
|
||||
def detail(self):
|
||||
@@ -285,27 +355,89 @@ class JobRegistry:
|
||||
self._logger = logger_ or logger
|
||||
self._jobs = {}
|
||||
|
||||
def start(self, kind, title, *, record_changing, scope, runner, groups=()) -> Job:
|
||||
def start(
|
||||
self,
|
||||
kind,
|
||||
title,
|
||||
*,
|
||||
admission=None,
|
||||
record_changing=None,
|
||||
cancellable=True,
|
||||
scope=None,
|
||||
runner,
|
||||
groups=(),
|
||||
) -> Job:
|
||||
"""Create a job and run ``runner(job)`` as a task.
|
||||
|
||||
Raises JobBusy, naming the job in the way, when a job of ``kind`` is
|
||||
running or when ``record_changing`` and another record-changing job
|
||||
runs. Must be called from the event loop.
|
||||
``admission`` is one of ``ADMISSIONS``; ``record_changing=True`` is
|
||||
the older spelling of ``records`` and ``False`` of ``read``. One of
|
||||
the two is required (TypeError otherwise), so no new job runs as
|
||||
``read`` beside a scan by omission. Raises JobBusy, naming the job in
|
||||
the way, when the admission class refuses it (see the module
|
||||
docstring). Must be called from the event loop.
|
||||
"""
|
||||
if admission is None and record_changing is None:
|
||||
raise TypeError(
|
||||
f"JobRegistry.start({kind!r}) needs an admission class "
|
||||
"(admission= or record_changing=)"
|
||||
)
|
||||
job = self._admit(
|
||||
kind,
|
||||
title,
|
||||
admission=admission,
|
||||
record_changing=record_changing,
|
||||
cancellable=cancellable,
|
||||
scope=scope,
|
||||
groups=groups,
|
||||
)
|
||||
job.task = asyncio.ensure_future(self._run(job, runner))
|
||||
return job
|
||||
|
||||
def hold(self, kind, title, *, admission, scope=None):
|
||||
"""A running, non-cancellable job for a legacy-page operation.
|
||||
|
||||
Returns an async context manager. Entering it registers the job and
|
||||
yields it, or raises JobBusy at once, without waiting, when the
|
||||
admission class refuses it; nothing is registered until then, so a
|
||||
hold that is never entered leaves no job behind. The job ends when
|
||||
the block exits: ``finished``, or ``failed`` when it raised (unless
|
||||
the body already ended it). A caller that answers a refusal in its
|
||||
own way checks ``blocker(kind, admission=...)`` first, with no await
|
||||
before the ``async with``.
|
||||
"""
|
||||
resolve_admission(admission) # an unknown class fails at the call
|
||||
return _Hold(
|
||||
lambda: self._admit(
|
||||
kind, title, admission=admission, cancellable=False, scope=scope
|
||||
)
|
||||
)
|
||||
|
||||
def _admit(self, kind, title, *, admission, record_changing=None, **job_args):
|
||||
resolved = resolve_admission(admission, record_changing)
|
||||
blocker = self.blocker(kind, admission=resolved)
|
||||
if blocker is not None:
|
||||
raise JobBusy(blocker)
|
||||
self._prune()
|
||||
job = Job(kind, title, admission=resolved, **job_args)
|
||||
self._jobs[job.id] = job
|
||||
return job
|
||||
|
||||
def blocker(self, kind, *, admission=None, record_changing=None):
|
||||
"""The running job a new ``kind`` job would be refused for, or None.
|
||||
|
||||
The same kind refuses first; then the oldest running job whose class
|
||||
the new job's admission class may not run beside.
|
||||
"""
|
||||
same_kind = self.active(kind)
|
||||
if same_kind is not None:
|
||||
raise JobBusy(same_kind)
|
||||
if record_changing:
|
||||
for job in self._jobs.values():
|
||||
if job.running and job.record_changing:
|
||||
raise JobBusy(job)
|
||||
self._prune()
|
||||
job = Job(
|
||||
kind, title, record_changing=record_changing, scope=scope, groups=groups
|
||||
)
|
||||
self._jobs[job.id] = job
|
||||
job.task = asyncio.ensure_future(self._run(job, runner))
|
||||
return job
|
||||
return same_kind
|
||||
refused = _REFUSED_BESIDE[resolve_admission(admission, record_changing)]
|
||||
running = [job for job in self._jobs.values() if job.running]
|
||||
running.sort(key=lambda job: job._started_seq)
|
||||
for job in running:
|
||||
if job.admission in refused:
|
||||
return job
|
||||
return None
|
||||
|
||||
async def _run(self, job, runner):
|
||||
try:
|
||||
@@ -349,11 +481,28 @@ class JobRegistry:
|
||||
def cancel(self, job_id) -> bool:
|
||||
"""Ask a running job to stop at its next safe boundary.
|
||||
|
||||
Returns whether a cancellation was requested: False for a finished or
|
||||
unknown job, so a cancel after the end is a no-op.
|
||||
Returns whether a cancellation was requested: False for a finished,
|
||||
unknown or non-cancellable job, so a cancel after the end is a no-op.
|
||||
"""
|
||||
job = self._jobs.get(job_id)
|
||||
if job is None or job.finished:
|
||||
if job is None or job.finished or not job.cancellable:
|
||||
return False
|
||||
job.cancelled = True
|
||||
return True
|
||||
|
||||
|
||||
class _Hold:
|
||||
"""The async context manager ``JobRegistry.hold`` returns."""
|
||||
|
||||
def __init__(self, admit):
|
||||
self._admit = admit
|
||||
self._job = None
|
||||
|
||||
async def __aenter__(self):
|
||||
self._job = self._admit()
|
||||
return self._job
|
||||
|
||||
async def __aexit__(self, exc_type, exc, tb):
|
||||
if self._job.running:
|
||||
self._job.finish("finished" if exc_type is None else "failed")
|
||||
return False
|
||||
|
||||
+104
-20
@@ -5,12 +5,47 @@ from pathlib import Path
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..config_store import ConfigUnreadable, RevisionConflict, get_config_store
|
||||
from .jobs import LORA_IMPORT_JOB_KIND
|
||||
|
||||
# Flags of POST /lora/enable; each must be a JSON boolean when present.
|
||||
_LORA_FLAGS = ("enabled", "trigger_words_enabled", "clear_civitai_api_key")
|
||||
|
||||
_LORA_PATH_ERROR = (
|
||||
"LoraManager path must be a directory under ComfyUI's custom_nodes "
|
||||
"whose name contains 'lora'"
|
||||
)
|
||||
|
||||
|
||||
def _lora_settings_mutation(values):
|
||||
"""The config commit for a LoRA save: only ``integrations.lora_manager``.
|
||||
|
||||
The submitted keys replace their values; other LoRA keys, other
|
||||
integrations and every other section stay as they are on disk.
|
||||
"""
|
||||
|
||||
def mutate(config):
|
||||
integrations = config.get("integrations")
|
||||
integrations = dict(integrations) if isinstance(integrations, dict) else {}
|
||||
lora = integrations.get("lora_manager")
|
||||
lora = dict(lora) if isinstance(lora, dict) else {}
|
||||
lora.update(values)
|
||||
integrations["lora_manager"] = lora
|
||||
config["integrations"] = integrations
|
||||
return config
|
||||
|
||||
return mutate
|
||||
|
||||
|
||||
def _lora_error_frame(message):
|
||||
"""An import stream's error event, as the admin page's modal reads it.
|
||||
|
||||
The modal shows ``status`` and ``progress`` from every frame and closes
|
||||
on ``type: error``, naming ``message``.
|
||||
"""
|
||||
return {"type": "error", "status": message, "progress": 0, "message": message}
|
||||
|
||||
|
||||
class LoraIntegrationMixin:
|
||||
"""Mixin providing LoraManager detection, scanning, and trigger word endpoints."""
|
||||
|
||||
@@ -131,14 +166,25 @@ class LoraIntegrationMixin:
|
||||
return allowed, None
|
||||
|
||||
async def lora_enable(self, request):
|
||||
"""Enable or disable LoraManager integration and save to config.json."""
|
||||
"""Enable or disable LoraManager integration and save it to config.json.
|
||||
|
||||
The values are committed through the config store first (#220) and
|
||||
published at runtime only once config.json holds them; a save that
|
||||
cannot be written reports ``persisted: false`` and changes nothing.
|
||||
The CivitAI key is never echoed back.
|
||||
"""
|
||||
try:
|
||||
data = await request.json()
|
||||
data, error = await self._read_json_body(request)
|
||||
if error is not None:
|
||||
return error
|
||||
for flag in _LORA_FLAGS:
|
||||
if flag in data and not isinstance(data[flag], bool):
|
||||
return self._bad_lora_request(f"{flag} must be true or false")
|
||||
enabled = data.get("enabled", False)
|
||||
path = data.get("path", "")
|
||||
trigger_words = data.get("trigger_words_enabled", False)
|
||||
|
||||
from ..config import IntegrationConfig, PromptManagerConfig
|
||||
from ..config import IntegrationConfig
|
||||
from ..lora_utils import get_trigger_cache
|
||||
|
||||
# The key is never echoed back to the UI, so a blank or absent
|
||||
@@ -161,20 +207,40 @@ class LoraIntegrationMixin:
|
||||
if error is not None:
|
||||
return error
|
||||
|
||||
# Update in-memory config
|
||||
IntegrationConfig.LORA_MANAGER_ENABLED = enabled
|
||||
IntegrationConfig.LORA_MANAGER_PATH = path
|
||||
IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED = trigger_words
|
||||
IntegrationConfig.CIVITAI_API_KEY = civitai_key
|
||||
values = {
|
||||
"enabled": enabled,
|
||||
"path": path,
|
||||
"trigger_words_enabled": trigger_words,
|
||||
"civitai_api_key": civitai_key,
|
||||
}
|
||||
|
||||
# Persist to the configured config.json
|
||||
PromptManagerConfig.save_to_file()
|
||||
def publish(_config):
|
||||
# Under the commit lock, so the last save written is the one
|
||||
# the runtime keeps.
|
||||
IntegrationConfig.LORA_MANAGER_ENABLED = enabled
|
||||
IntegrationConfig.LORA_MANAGER_PATH = path
|
||||
IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED = trigger_words
|
||||
IntegrationConfig.CIVITAI_API_KEY = civitai_key
|
||||
|
||||
try:
|
||||
_saved, revision = await self._run_in_executor(
|
||||
get_config_store().commit,
|
||||
_lora_settings_mutation(values),
|
||||
expected_revision=None,
|
||||
publish=publish,
|
||||
)
|
||||
except (ConfigUnreadable, RevisionConflict, OSError) as exc:
|
||||
return self._config_commit_failure(exc)
|
||||
|
||||
# Load trigger word cache if enabling
|
||||
cache = get_trigger_cache()
|
||||
if enabled and trigger_words and path:
|
||||
count = await self._run_in_executor(cache.load, path)
|
||||
self.logger.info(f"Trigger word cache loaded: {count} LoRAs")
|
||||
try:
|
||||
count = await self._run_in_executor(cache.load, path)
|
||||
self.logger.info(f"Trigger word cache loaded: {count} LoRAs")
|
||||
except Exception as e:
|
||||
# Saved and enabled; only the trigger words are missing.
|
||||
self.logger.warning(f"Trigger word cache could not be loaded: {e}")
|
||||
cache.clear()
|
||||
elif not enabled:
|
||||
cache.clear()
|
||||
|
||||
@@ -184,6 +250,10 @@ class LoraIntegrationMixin:
|
||||
"enabled": enabled,
|
||||
"path": self._public_path(path),
|
||||
"trigger_words_enabled": trigger_words,
|
||||
"persisted": True,
|
||||
"revision": revision,
|
||||
"runtime_applied": True,
|
||||
"cache_loaded": cache.is_loaded,
|
||||
}
|
||||
)
|
||||
except Exception as e:
|
||||
@@ -218,19 +288,33 @@ class LoraIntegrationMixin:
|
||||
reason="OK",
|
||||
headers={"Content-Type": "text/event-stream"},
|
||||
)
|
||||
await response.prepare(request)
|
||||
|
||||
async def send_progress(data):
|
||||
line = f"data: {json.dumps(data)}\n\n"
|
||||
await response.write(line.encode("utf-8"))
|
||||
|
||||
try:
|
||||
await self._lora_scan_stream(allowed, send_progress)
|
||||
except Exception as e:
|
||||
self.logger.exception("LoRA scan failed")
|
||||
await send_progress({"type": "error", "message": self._public_error(e)})
|
||||
# The import replaces prompts and images: a records hold, so it
|
||||
# never runs beside a scan, restore or tagging job.
|
||||
busy = self._jobs.blocker(LORA_IMPORT_JOB_KIND, admission="records")
|
||||
if busy is not None:
|
||||
await response.prepare(request)
|
||||
message = f"{busy.title} already running"
|
||||
await send_progress(_lora_error_frame(message))
|
||||
await response.write_eof()
|
||||
return response
|
||||
|
||||
await response.write_eof()
|
||||
async with self._jobs.hold(
|
||||
LORA_IMPORT_JOB_KIND, "LoRA import (admin page)", admission="records"
|
||||
) as job:
|
||||
await response.prepare(request)
|
||||
try:
|
||||
await self._lora_scan_stream(allowed, send_progress)
|
||||
except Exception as e:
|
||||
self.logger.exception("LoRA scan failed")
|
||||
job.finish("failed")
|
||||
await send_progress(_lora_error_frame(self._public_error(e)))
|
||||
|
||||
await response.write_eof()
|
||||
return response
|
||||
|
||||
except Exception as e:
|
||||
|
||||
@@ -0,0 +1,339 @@
|
||||
"""Settings API routes: GET and POST /prompt_manager/settings (#220).
|
||||
|
||||
A save validates the payload, commits it to config.json through the config
|
||||
store and publishes it at runtime inside that commit; a save that cannot be
|
||||
written changes nothing at runtime and says so. ``_config_commit_failure``
|
||||
is shared with the LoRA integration's save.
|
||||
"""
|
||||
|
||||
import os
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
from ..config_store import (
|
||||
ConfigUnreadable,
|
||||
RevisionConflict,
|
||||
get_config_store,
|
||||
)
|
||||
from .admin import _current_image_monitor
|
||||
|
||||
try:
|
||||
from ...utils.validators import validate_result_timeout, validate_worker_threads
|
||||
except ImportError:
|
||||
from utils.validators import validate_result_timeout, validate_worker_threads
|
||||
|
||||
|
||||
# Settings owned by POST /settings: config attribute -> (section, key).
|
||||
_SETTINGS_FILE_KEYS = {
|
||||
"RESULT_TIMEOUT": ("web_ui", "result_timeout"),
|
||||
"WEBUI_DISPLAY_MODE": ("web_ui", "webui_display_mode"),
|
||||
"INFINITE_SCROLL": ("web_ui", "infinite_scroll"),
|
||||
"WORKER_THREADS": ("performance", "worker_threads"),
|
||||
}
|
||||
|
||||
|
||||
def _section(config, name):
|
||||
"""``config[name]`` as a fresh dict (a non-object section starts empty)."""
|
||||
value = config.get(name)
|
||||
return dict(value) if isinstance(value, dict) else {}
|
||||
|
||||
|
||||
def _settings_mutation(updates, new_roots):
|
||||
"""The config commit for a settings save: only the submitted keys change.
|
||||
|
||||
Every other section and key (integrations and the CivitAI key, database,
|
||||
unknown sections) is kept as it is on disk.
|
||||
"""
|
||||
|
||||
def mutate(config):
|
||||
for name, value in updates.items():
|
||||
section, key = _SETTINGS_FILE_KEYS[name]
|
||||
merged = _section(config, section)
|
||||
merged[key] = value
|
||||
config[section] = merged
|
||||
if new_roots is not None:
|
||||
gallery = _section(config, "gallery")
|
||||
monitoring = _section(gallery, "monitoring")
|
||||
monitoring["directories"] = list(new_roots)
|
||||
gallery["monitoring"] = monitoring
|
||||
config["gallery"] = gallery
|
||||
return config
|
||||
|
||||
return mutate
|
||||
|
||||
|
||||
async def _read_json(request):
|
||||
"""Parse a JSON object body through the package's capped reader."""
|
||||
from . import _read_json_body
|
||||
|
||||
return await _read_json_body(request)
|
||||
|
||||
|
||||
class SettingsRoutesMixin:
|
||||
"""Mixin providing the settings read and save endpoints."""
|
||||
|
||||
def _register_settings_routes(self, routes):
|
||||
@routes.get("/prompt_manager/settings")
|
||||
async def get_settings_route(request):
|
||||
return await self.get_settings(request)
|
||||
|
||||
@routes.post("/prompt_manager/settings")
|
||||
async def save_settings_route(request):
|
||||
return await self.save_settings(request)
|
||||
|
||||
async def get_settings(self, request):
|
||||
"""Get current settings."""
|
||||
try:
|
||||
from ..config import PromptManagerConfig, GalleryConfig
|
||||
|
||||
root_paths = [
|
||||
self._public_path(d) for d in GalleryConfig.MONITORING_DIRECTORIES
|
||||
]
|
||||
revision = await self._run_in_executor(self._config_revision)
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
**revision,
|
||||
"settings": {
|
||||
"result_timeout": PromptManagerConfig.RESULT_TIMEOUT,
|
||||
"webui_display_mode": PromptManagerConfig.WEBUI_DISPLAY_MODE,
|
||||
"infinite_scroll": PromptManagerConfig.INFINITE_SCROLL,
|
||||
"worker_threads": PromptManagerConfig.WORKER_THREADS,
|
||||
"cpu_count": PromptManagerConfig.max_worker_threads(),
|
||||
"gallery_root_paths": root_paths,
|
||||
"gallery_root_path": root_paths[0] if root_paths else "",
|
||||
"monitored_directories": [
|
||||
self._public_path(d) for d in self._monitored_directories()
|
||||
],
|
||||
},
|
||||
}
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Get settings error: {e}", exc_info=True)
|
||||
return web.json_response(
|
||||
{"success": False, "error": "Failed to get settings"},
|
||||
status=500,
|
||||
)
|
||||
|
||||
def _monitored_directories(self):
|
||||
"""Directories the running image monitor watches, else the configured roots."""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
monitor = _current_image_monitor()
|
||||
if monitor is not None:
|
||||
return list(getattr(monitor, "monitored_directories", []))
|
||||
return list(GalleryConfig.MONITORING_DIRECTORIES)
|
||||
|
||||
async def save_settings(self, request):
|
||||
"""Save settings: validate, commit to config.json, then publish (#220).
|
||||
|
||||
Nothing changes at runtime until config.json holds the new values, so
|
||||
a save that cannot be written reports ``persisted: false`` and leaves
|
||||
the running settings as they were. ``expected_revision`` (the
|
||||
``revision`` from GET) refuses to overwrite a file changed meanwhile;
|
||||
without it the payload merges into what is on disk (legacy page).
|
||||
"""
|
||||
try:
|
||||
data, error_response = await _read_json(request)
|
||||
if error_response is not None:
|
||||
return error_response
|
||||
|
||||
expected_revision = data.get("expected_revision")
|
||||
if expected_revision is not None and not isinstance(expected_revision, str):
|
||||
return web.json_response(
|
||||
{"success": False, "error": "expected_revision must be a string"},
|
||||
status=400,
|
||||
)
|
||||
try:
|
||||
updates = self._validate_settings(data)
|
||||
except ValueError as ve:
|
||||
return web.json_response(
|
||||
{"success": False, "error": str(ve)}, status=400
|
||||
)
|
||||
|
||||
new_roots, error = self._parse_gallery_roots(data)
|
||||
if error:
|
||||
return web.json_response({"success": False, "error": error}, status=400)
|
||||
|
||||
published = {}
|
||||
|
||||
def publish(_config):
|
||||
# Under the commit lock: saves reach the runtime in the order
|
||||
# they were written, so the runtime ends on the last one.
|
||||
published.update(self._publish_settings(updates, new_roots))
|
||||
|
||||
try:
|
||||
_saved, revision = await self._run_in_executor(
|
||||
get_config_store().commit,
|
||||
_settings_mutation(updates, new_roots),
|
||||
expected_revision=expected_revision,
|
||||
publish=publish,
|
||||
)
|
||||
except (ConfigUnreadable, RevisionConflict, OSError) as exc:
|
||||
return self._config_commit_failure(exc)
|
||||
|
||||
return web.json_response(
|
||||
{
|
||||
"success": True,
|
||||
"message": "Settings saved successfully",
|
||||
"restart_required": bool(published["restart_required_fields"]),
|
||||
"persisted": True,
|
||||
"revision": revision,
|
||||
**published,
|
||||
}
|
||||
)
|
||||
except Exception as e:
|
||||
self.logger.error(f"Save settings error: {e}", exc_info=True)
|
||||
return web.json_response(
|
||||
{"success": False, "error": "Failed to save settings"},
|
||||
status=500,
|
||||
)
|
||||
|
||||
def _publish_settings(self, updates, new_roots):
|
||||
"""Install committed settings at runtime (inside the config commit).
|
||||
|
||||
The values are already durable. A publication that fails is reported
|
||||
as ``runtime_applied: false`` with the fields that need a restart;
|
||||
nothing is retried or rolled back. Never raises.
|
||||
"""
|
||||
from ..config import PromptManagerConfig
|
||||
|
||||
for name, value in updates.items():
|
||||
setattr(PromptManagerConfig, name, value)
|
||||
restart_fields = []
|
||||
runtime_applied = True
|
||||
if new_roots is not None:
|
||||
try:
|
||||
if self._apply_gallery_roots(new_roots):
|
||||
restart_fields.append("gallery_root_paths")
|
||||
except Exception as e:
|
||||
self.logger.error(
|
||||
f"Saved gallery roots could not be applied: {e}", exc_info=True
|
||||
)
|
||||
runtime_applied = False
|
||||
restart_fields.append("gallery_root_paths")
|
||||
return {
|
||||
"runtime_applied": runtime_applied,
|
||||
"restart_required_fields": restart_fields,
|
||||
}
|
||||
|
||||
def _config_commit_failure(self, exc):
|
||||
"""The response for a config commit that wrote nothing."""
|
||||
not_saved = {"success": False, "persisted": False, "runtime_applied": False}
|
||||
if isinstance(exc, ConfigUnreadable):
|
||||
self.logger.warning(f"Settings not saved: {exc}")
|
||||
return web.json_response(
|
||||
{
|
||||
**not_saved,
|
||||
"code": "config-unreadable",
|
||||
"error": (
|
||||
"config.json cannot be read, so nothing was saved. "
|
||||
"Fix or remove the file, then save again."
|
||||
),
|
||||
},
|
||||
status=409,
|
||||
)
|
||||
if isinstance(exc, RevisionConflict):
|
||||
return web.json_response(
|
||||
{
|
||||
**not_saved,
|
||||
"code": "settings-changed",
|
||||
"error": "Settings changed since they were loaded. Reload them.",
|
||||
"revision": exc.current,
|
||||
},
|
||||
status=409,
|
||||
)
|
||||
self.logger.error(f"Could not write config.json: {exc}")
|
||||
return web.json_response(
|
||||
{
|
||||
**not_saved,
|
||||
"code": "config-write-failed",
|
||||
"error": "Settings could not be written to config.json.",
|
||||
},
|
||||
status=500,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _config_revision():
|
||||
"""``{"revision": ...}``, or a null revision with ``config_error``."""
|
||||
try:
|
||||
_data, revision = get_config_store().read()
|
||||
except ConfigUnreadable:
|
||||
return {"revision": None, "config_error": "config-unreadable"}
|
||||
return {"revision": revision}
|
||||
|
||||
@staticmethod
|
||||
def _validate_settings(data):
|
||||
"""Scalar settings of a payload as ``{CONFIG_ATTR: value}``.
|
||||
|
||||
Raises ValueError with a user-facing message on the first bad value;
|
||||
nothing is applied until every value passed.
|
||||
"""
|
||||
from ..config import PromptManagerConfig
|
||||
|
||||
updates = {}
|
||||
if "result_timeout" in data:
|
||||
updates["RESULT_TIMEOUT"] = validate_result_timeout(data["result_timeout"])
|
||||
if "webui_display_mode" in data:
|
||||
mode = data["webui_display_mode"]
|
||||
if mode not in ("popup", "newtab"):
|
||||
raise ValueError("webui_display_mode must be 'popup' or 'newtab'")
|
||||
updates["WEBUI_DISPLAY_MODE"] = mode
|
||||
if "infinite_scroll" in data:
|
||||
if not isinstance(data["infinite_scroll"], bool):
|
||||
raise ValueError("infinite_scroll must be true or false")
|
||||
updates["INFINITE_SCROLL"] = data["infinite_scroll"]
|
||||
if "worker_threads" in data:
|
||||
updates["WORKER_THREADS"] = validate_worker_threads(
|
||||
data["worker_threads"], PromptManagerConfig.max_worker_threads()
|
||||
)
|
||||
return updates
|
||||
|
||||
def _parse_gallery_roots(self, data):
|
||||
"""Extract and validate gallery roots from a settings payload.
|
||||
|
||||
Accepts ``gallery_root_paths`` (list, preferred) or the legacy single
|
||||
``gallery_root_path`` string. Blank entries are dropped.
|
||||
|
||||
Returns:
|
||||
(roots, error): ``roots`` is the validated list, or ``None`` when
|
||||
the payload carries no gallery root key; ``error`` is a message
|
||||
when validation failed (and ``roots`` is then ``None``).
|
||||
"""
|
||||
if "gallery_root_paths" in data:
|
||||
raw = data["gallery_root_paths"]
|
||||
if not isinstance(raw, list):
|
||||
return None, "gallery_root_paths must be a list"
|
||||
elif "gallery_root_path" in data:
|
||||
raw = [data["gallery_root_path"]]
|
||||
else:
|
||||
return None, None
|
||||
|
||||
from ..config import GalleryConfig
|
||||
|
||||
roots = []
|
||||
for entry in raw:
|
||||
if not isinstance(entry, str):
|
||||
return None, "Gallery root paths must be strings"
|
||||
entry = entry.strip()
|
||||
if not entry:
|
||||
continue
|
||||
ok, reason = GalleryConfig.validate_gallery_root(entry)
|
||||
if not ok:
|
||||
return (
|
||||
None,
|
||||
f"Invalid gallery root '{os.path.basename(entry)}': {reason}",
|
||||
)
|
||||
roots.append(GalleryConfig.resolve_gallery_root(entry))
|
||||
return roots, None
|
||||
|
||||
def _apply_gallery_roots(self, roots):
|
||||
"""Install validated gallery roots; returns True when they changed."""
|
||||
from ..config import GalleryConfig
|
||||
|
||||
if roots == list(GalleryConfig.MONITORING_DIRECTORIES):
|
||||
return False
|
||||
GalleryConfig.MONITORING_DIRECTORIES = roots
|
||||
self._cached_output_dir = None
|
||||
self.reset_gallery_roots_cache()
|
||||
return True
|
||||
@@ -0,0 +1,197 @@
|
||||
"""The one serialized writer of config.json (#220).
|
||||
|
||||
Every save that changes config.json goes through ``ConfigStore.commit``: the
|
||||
settings page, the LoRA integration and, later, every System section. A
|
||||
commit runs under one process-wide lock, re-reads the file inside it, applies
|
||||
the caller's change to a copy and replaces the file atomically. So two saves
|
||||
never overwrite each other's keys, sections this process does not know are
|
||||
kept byte-for-byte in meaning, and a write that fails leaves the old file in
|
||||
place and raises instead of being logged and reported as saved.
|
||||
|
||||
A revision names the file content a client last saw: ``"sha256:<hex>"`` of
|
||||
the file bytes, or ``"absent"`` when there is no file. A commit that names a
|
||||
revision refuses to write over different content (``RevisionConflict``).
|
||||
|
||||
Reads take the same lock, so a read handle is never open while the file is
|
||||
replaced (Windows refuses that), and a caller's ``publish`` runs inside the
|
||||
lock right after its write, so runtimes publish saves in the order they were
|
||||
written. A symlinked config.json is written beside its real target.
|
||||
|
||||
config.json can hold the CivitAI key, so the file is written owner-only.
|
||||
"""
|
||||
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
from typing import Callable, Optional, Tuple
|
||||
|
||||
ABSENT_REVISION = "absent"
|
||||
|
||||
# One lock for the process: every ConfigStore writes the same file. It is
|
||||
# re-entrant so a commit's ``publish`` callback may read the config again.
|
||||
_COMMIT_LOCK = threading.RLock()
|
||||
|
||||
# Windows: an antivirus, indexer or backup handle on config.json makes
|
||||
# os.replace fail with PermissionError for a moment, so it is retried briefly.
|
||||
_REPLACE_ATTEMPTS = 5 if os.name == "nt" else 1
|
||||
_REPLACE_BACKOFF_SECONDS = 0.05
|
||||
|
||||
|
||||
class ConfigUnreadable(Exception):
|
||||
"""config.json exists but is not a readable JSON object; it is left alone."""
|
||||
|
||||
|
||||
class RevisionConflict(Exception):
|
||||
"""config.json changed since the revision the caller saw."""
|
||||
|
||||
def __init__(self, current: str):
|
||||
super().__init__("config.json changed since it was read")
|
||||
self.current = current
|
||||
|
||||
|
||||
def revision_of(data: Optional[bytes]) -> str:
|
||||
"""The revision of a file's bytes; ``None`` means the file is absent."""
|
||||
if data is None:
|
||||
return ABSENT_REVISION
|
||||
return "sha256:" + hashlib.sha256(data).hexdigest()
|
||||
|
||||
|
||||
def _default_path() -> str:
|
||||
from .config import PromptManagerConfig
|
||||
|
||||
return PromptManagerConfig.get_config_path()
|
||||
|
||||
|
||||
def _replace(source: str, target: str) -> None:
|
||||
"""``os.replace`` with a short retry on a transient Windows PermissionError."""
|
||||
for attempt in range(1, _REPLACE_ATTEMPTS + 1):
|
||||
try:
|
||||
os.replace(source, target)
|
||||
return
|
||||
except PermissionError:
|
||||
if attempt == _REPLACE_ATTEMPTS:
|
||||
raise
|
||||
time.sleep(_REPLACE_BACKOFF_SECONDS)
|
||||
|
||||
|
||||
def _write_atomically(path: str, payload: bytes) -> None:
|
||||
"""Replace ``path`` with ``payload``: temp file beside it, fsync, replace.
|
||||
|
||||
The descriptor is closed before ``os.replace`` (Windows refuses to replace
|
||||
over an open handle), and on any failure the temp file is closed and
|
||||
removed, so a failed write leaves the old file and no stray temp file.
|
||||
``path`` is resolved first: a symlink keeps pointing at the updated
|
||||
target. A single-file bind mount cannot be replaced (EBUSY) and fails as
|
||||
an OSError, which callers report as not saved.
|
||||
"""
|
||||
path = os.path.realpath(path)
|
||||
parent = os.path.dirname(path) or "."
|
||||
os.makedirs(parent, exist_ok=True)
|
||||
fd, temp_path = tempfile.mkstemp(dir=parent, prefix=".config-", suffix=".tmp")
|
||||
try:
|
||||
try:
|
||||
handle = os.fdopen(fd, "wb")
|
||||
except BaseException:
|
||||
os.close(fd)
|
||||
raise
|
||||
with handle: # closed before the replace, also when a write fails
|
||||
handle.write(payload)
|
||||
handle.flush()
|
||||
os.fsync(handle.fileno())
|
||||
if os.name == "posix":
|
||||
os.chmod(temp_path, 0o600)
|
||||
_replace(temp_path, path)
|
||||
except BaseException:
|
||||
try:
|
||||
os.unlink(temp_path)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
class ConfigStore:
|
||||
"""Serialized, atomic reads and commits of config.json."""
|
||||
|
||||
def __init__(self, path_getter: Optional[Callable[[], str]] = None):
|
||||
self._path_getter = path_getter or _default_path
|
||||
|
||||
@property
|
||||
def path(self) -> str:
|
||||
"""The config file this store reads and writes (resolved per call)."""
|
||||
return self._path_getter()
|
||||
|
||||
def read(self) -> Tuple[dict, str]:
|
||||
"""``(config, revision)``; a missing file is ``({}, "absent")``.
|
||||
|
||||
Raises ConfigUnreadable when the file cannot be read, is not JSON, or
|
||||
is not a JSON object. Waits for a running commit (see the module
|
||||
docstring).
|
||||
"""
|
||||
with _COMMIT_LOCK:
|
||||
return self._read(self.path)
|
||||
|
||||
@staticmethod
|
||||
def _read(path: str) -> Tuple[dict, str]:
|
||||
try:
|
||||
with open(path, "rb") as handle:
|
||||
raw = handle.read()
|
||||
except (FileNotFoundError, NotADirectoryError):
|
||||
# No file there (a parent that is a file cannot hold one either);
|
||||
# a commit then fails on the write, as config-write-failed.
|
||||
return {}, ABSENT_REVISION
|
||||
except OSError as exc:
|
||||
raise ConfigUnreadable(f"config.json cannot be read: {exc}") from exc
|
||||
try:
|
||||
data = json.loads(raw.decode("utf-8"))
|
||||
except (UnicodeDecodeError, ValueError) as exc:
|
||||
raise ConfigUnreadable("config.json is not valid JSON") from exc
|
||||
if not isinstance(data, dict):
|
||||
raise ConfigUnreadable("config.json is not a JSON object")
|
||||
return data, revision_of(raw)
|
||||
|
||||
def commit(
|
||||
self,
|
||||
mutate: Callable[[dict], dict],
|
||||
*,
|
||||
expected_revision: Optional[str],
|
||||
publish: Optional[Callable[[dict], None]] = None,
|
||||
) -> Tuple[dict, str]:
|
||||
"""Apply ``mutate`` to the current config and write the result.
|
||||
|
||||
``mutate`` receives a deep copy of the file's content and returns the
|
||||
whole new config (an object). ``expected_revision`` None merges into
|
||||
whatever is on disk (legacy partial updates); a revision refuses to
|
||||
write over different content. Returns ``(config, revision)`` as
|
||||
written. Raises ConfigUnreadable, RevisionConflict, TypeError (a
|
||||
non-object result) or OSError (the write failed; the old file stays).
|
||||
|
||||
``publish(config)``, when given, runs under the lock once the file
|
||||
holds the new config, so concurrent saves reach the runtime in the
|
||||
order they were written. It must handle its own errors: the values
|
||||
are already durable when it runs.
|
||||
"""
|
||||
with _COMMIT_LOCK:
|
||||
path = self.path
|
||||
current, revision = self._read(path)
|
||||
if expected_revision is not None and expected_revision != revision:
|
||||
raise RevisionConflict(revision)
|
||||
candidate = mutate(copy.deepcopy(current))
|
||||
if not isinstance(candidate, dict):
|
||||
raise TypeError("A config commit must produce a JSON object")
|
||||
payload = json.dumps(candidate, indent=2).encode("utf-8")
|
||||
_write_atomically(path, payload)
|
||||
if publish is not None:
|
||||
publish(candidate)
|
||||
return candidate, revision_of(payload)
|
||||
|
||||
|
||||
_store = ConfigStore()
|
||||
|
||||
|
||||
def get_config_store() -> ConfigStore:
|
||||
"""The process's config store (on ``PromptManagerConfig.get_config_path``)."""
|
||||
return _store
|
||||
@@ -249,14 +249,16 @@ const EXPECTED_JOB_VIEW = {
|
||||
pressed: { border: true, background: true, added: true, bad: true, warn: true },
|
||||
};
|
||||
|
||||
async function openRunningJob(page, request, theme, viewport = { width: 1600, height: 1100 }) {
|
||||
const SCAN_START = { start: { total: 19622 }, step: { processed: 8240,
|
||||
counters: { known: 6511, review: 37, no_metadata: 254, errors: 18 },
|
||||
records: [{ group: 'added', path: 'CCC/mabes9sans-dataset-v1/IMG_GEN_00369_.png', detail: 'Added to the Library as “Mabes portrait”' }] } };
|
||||
|
||||
async function openRunningJob(page, request, theme, viewport = { width: 1600, height: 1100 }, data = SCAN_START) {
|
||||
await page.setViewportSize(viewport);
|
||||
const started = await request.post('/__test__/jobs/start', { data: { total: 19622 } });
|
||||
const started = await request.post('/__test__/jobs/start', { data: data.start });
|
||||
await expect(started).toBeOK();
|
||||
const { job } = await started.json();
|
||||
await expect(await request.post(`/__test__/jobs/${job.id}/step`, { data: { processed: 8240,
|
||||
counters: { known: 6511, review: 37, no_metadata: 254, errors: 18 },
|
||||
records: [{ group: 'added', path: 'CCC/mabes9sans-dataset-v1/IMG_GEN_00369_.png', detail: 'Added to the Library as “Mabes portrait”' }] } })).toBeOK();
|
||||
await expect(await request.post(`/__test__/jobs/${job.id}/step`, { data: data.step })).toBeOK();
|
||||
await openProduction(page, `activity/${job.id}`, theme);
|
||||
await expect(page.locator('#workspace-main .file-row')).toHaveCount(1);
|
||||
return job;
|
||||
@@ -278,6 +280,30 @@ for (const theme of THEMES) test(`job view follows the approved mockup ${theme}`
|
||||
expect(await page.evaluate(measureJobView)).toEqual(EXPECTED_JOB_VIEW);
|
||||
});
|
||||
|
||||
// D89 (plan D66): an image tagging job's view is the same mockup with its own four tiles (Tagged green, Unchanged,
|
||||
// Skipped warning, Errors red) in the mockup's five-column row, its own outcome list and "Tagging log".
|
||||
const TAG_START = { start: { kind: 'image-tags', title: 'Auto-tag images', mode: 'auto', images: 4, total: 4 }, step: { processed: 2,
|
||||
counters: { unchanged: 1, errors: 1 }, records: [{ group: 'tagged', path: 'CCC/mabes9sans-dataset-v1/IMG_GEN_00369_.png', detail: 'Added 3 tags' }] } };
|
||||
const EXPECTED_TAG_VIEW = {
|
||||
...EXPECTED_JOB_VIEW,
|
||||
progress: [MOCKUP_PROGRESS[0], MOCKUP_PROGRESS[1], 'dl.stat-row(div.added.stat,div.stat,div.stat.warn,div.bad.stat)'],
|
||||
summary: '● Tagging log — raw detail, click to show',
|
||||
};
|
||||
|
||||
for (const theme of THEMES) test(`image tagging job view follows the approved mockup ${theme}`, async ({ page, request }) => {
|
||||
await seedConcept(request);
|
||||
await openRunningJob(page, request, theme, undefined, TAG_START);
|
||||
expect(await page.evaluate(measureJobView)).toEqual(EXPECTED_TAG_VIEW);
|
||||
await expect(page.locator('#workspace-main .outcome-list > button')).toHaveText(['Tagged1', 'Errors1', 'Skipped0', 'Unchanged1']);
|
||||
// Mutation check: the scan's five tiles and the scan's log label are both noticed.
|
||||
expect(diffInventory(EXPECTED_JOB_VIEW, await page.evaluate(measureJobView)).map(item => item.path).sort())
|
||||
.toEqual(['progress.2', 'summary']);
|
||||
const undoCss = await mutate(page, { css: '#workspace-main .stat-row .stat.warn dd { color: red; }' });
|
||||
expect(diffInventory(EXPECTED_TAG_VIEW, await page.evaluate(measureJobView)).map(item => item.path)).toEqual(['pressed.warn']);
|
||||
await undoCss();
|
||||
expect(await page.evaluate(measureJobView)).toEqual(EXPECTED_TAG_VIEW);
|
||||
});
|
||||
|
||||
test('job view fits a phone without page scroll', async ({ page, request }) => {
|
||||
await seedConcept(request);
|
||||
await openRunningJob(page, request, 'dark', { width: 390, height: 844 });
|
||||
|
||||
@@ -138,10 +138,12 @@ export async function hoverStateOf(page,locator) {
|
||||
/**
|
||||
* Mutate the open page so a comparison can be shown to notice a change (the A7 mutation check): `css` is appended as
|
||||
* the last stylesheet, so it wins over equally specific concept rules, and every element matching a `remove` selector
|
||||
* is detached. Returns an async undo that restores the stylesheet and the removed elements in place.
|
||||
* is detached; each `clone` ({selector, text}) inserts a copy of the first match after it, holding `text` and no
|
||||
* `data-action` (a control production adds beside a concept one, D85). Returns an async undo that restores the
|
||||
* stylesheet and the removed elements in place and drops the copies.
|
||||
*/
|
||||
export async function mutate(page,{css=null,remove=[]}={}) {
|
||||
const token=await page.evaluate(({css,remove})=>{
|
||||
export async function mutate(page,{css=null,remove=[],clone=[]}={}) {
|
||||
const token=await page.evaluate(({css,remove,clone})=>{
|
||||
const token=`mutation-${Math.random().toString(36).slice(2)}`;
|
||||
const undo=[];
|
||||
if(css) {
|
||||
@@ -156,9 +158,15 @@ export async function mutate(page,{css=null,remove=[]}={}) {
|
||||
node.remove();undo.push(()=>parent.insertBefore(node,next));
|
||||
}
|
||||
}
|
||||
for(const {selector,text} of clone) {
|
||||
const node=document.querySelector(selector);
|
||||
if(!node)throw new Error(`mutate: nothing matches ${selector}`);
|
||||
const copy=node.cloneNode(true);copy.textContent=text;copy.removeAttribute('data-action');
|
||||
node.after(copy);undo.push(()=>copy.remove());
|
||||
}
|
||||
(window.__conceptMutations??=new Map()).set(token,undo);
|
||||
return token;
|
||||
},{css,remove});
|
||||
},{css,remove,clone});
|
||||
return ()=>page.evaluate(token=>{
|
||||
const undo=window.__conceptMutations?.get(token);
|
||||
if(!undo)throw new Error('mutate: this mutation was already undone or the page reloaded');
|
||||
|
||||
@@ -30,9 +30,11 @@ function captionDay(seconds,now=new Date()) {
|
||||
return `${name} · ${date.toLocaleDateString('en-US',options)}`;
|
||||
}
|
||||
|
||||
/** D8: the selection bar's real actions only; Add tags, Add to collection and Remove links… are left out. */
|
||||
const OMITTED_BULK=Object.freeze(['Add tags','Add to collection','Remove links…']);
|
||||
const OMITTED_BULK_SELECTORS=Object.freeze(['bulk-tags','bulk-collection','bulk-remove'].map(action=>`#bulkbar [data-action="${action}"]`));
|
||||
/** D8 (amended by #249 PR D): the selection bar's real actions only; Add to collection and Remove links… are left out. */
|
||||
const OMITTED_BULK=Object.freeze(['Add to collection','Remove links…']);
|
||||
const OMITTED_BULK_SELECTORS=Object.freeze(['bulk-collection','bulk-remove'].map(action=>`#bulkbar [data-action="${action}"]`));
|
||||
/** D85: Auto-tag selected follows the concept's Add tags, a concept button of its own text. */
|
||||
const AUTO_TAG_CLONE=Object.freeze([{selector:'#bulkbar [data-action="bulk-tags"]',text:'Auto-tag selected'}]);
|
||||
|
||||
/**
|
||||
* `withoutItems` for a row with padding and a border (the `.bulkbar`): its items flow again inside the content box,
|
||||
@@ -443,6 +445,10 @@ for(const [width,height] of SIZES)for(const theme of THEMES)test(`images selecti
|
||||
const restore=await mutate(page,{remove:[...OMITTED_BULK_SELECTORS]});
|
||||
const removed=(await inventoryOf(page,'#main')).bulkbar;
|
||||
await restore();
|
||||
// The oracle for production's bar: the concept page itself without the omitted actions and with D85's button.
|
||||
const restoreProduction=await mutate(page,{remove:[...OMITTED_BULK_SELECTORS],clone:AUTO_TAG_CLONE});
|
||||
const oracle=await inventoryOf(page,'#main');
|
||||
await restoreProduction();
|
||||
await captureCanonicalState(page,testInfo,`concept-images-selected-${width}-${theme}`);
|
||||
await openProductionLoaded(page,theme,'no-preference');
|
||||
const emptyActual=await inventoryOf(page,'#workspace-main');
|
||||
@@ -454,12 +460,13 @@ for(const [width,height] of SIZES)for(const theme of THEMES)test(`images selecti
|
||||
expect.soft(reference.bulkbar.bar.items.map(item=>item.text),'concept bar').toEqual(['1 selected','Add tags','Set category',
|
||||
'Add to collection','Export metadata','Remove links…','Clear']);
|
||||
expect.soft(diffInventory(stripExact(mapBulkbar(reference.bulkbar,0).bulkbar.bar),stripExact(removed.bar)),'bar without the omitted actions').toEqual([]);
|
||||
expect.soft(actual.bulkbar.bar.items.map(item=>item.text),'production bar').toEqual(['1 selected','Set category','Export metadata','Clear']);
|
||||
expect.soft(actual.bulkbar.bar.items.map(item=>item.text),'production bar').toEqual(['1 selected','Add tags','Auto-tag selected',
|
||||
'Set category','Export metadata','Clear']);
|
||||
expect.soft(compare(empty,emptyActual,manifest,['bulkbar','cards','panel']),'nothing checked').toEqual([]);
|
||||
expect.soft(compare(reference,actual,manifest,['bulkbar','cards','panel']),'one card checked').toEqual([]);
|
||||
expect.soft(compare(oracle,actual,manifest,['bulkbar','cards','panel']),'one card checked').toEqual([]);
|
||||
// The bar comparison notices a change to the production bar.
|
||||
await mutate(page,{css:'.bulkbar { padding: 4px }'});
|
||||
expect(compare(reference,await inventoryOf(page,'#workspace-main'),manifest,['bulkbar']).map(item=>item.path)).toContain('bulkbar.bar.paddingTop');
|
||||
expect(compare(oracle,await inventoryOf(page,'#workspace-main'),manifest,['bulkbar']).map(item=>item.path)).toContain('bulkbar.bar.paddingTop');
|
||||
});
|
||||
|
||||
// The comparison is only worth something if it notices a change (spec §7 mutation check): one CSS mutation (the
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Hold real requests at this test server, so a page can leave while one is in flight.
|
||||
|
||||
A Playwright ``page.route`` hold answers inside the browser's interception layer,
|
||||
which WebKit cancels differently from a request the server has not answered yet.
|
||||
These commands hold the request at the server itself:
|
||||
|
||||
- ``POST /__test__/hold`` with ``{"path": "/prompt_manager/..."}`` holds every
|
||||
later request for that path until it is released. With ``"stage": "body"``
|
||||
the server answers first and holds after sending the headers and the first
|
||||
half of the body, so the browser has a response whose body is still loading.
|
||||
- ``GET /__test__/hold`` answers ``{"held": {path: count}}``: how many requests
|
||||
for each held path have arrived and are waiting.
|
||||
- ``POST /__test__/release`` lets every held request go on, and stops holding.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from aiohttp import web
|
||||
|
||||
HOLDABLE_PREFIX = "/prompt_manager/"
|
||||
|
||||
|
||||
class RequestHolds:
|
||||
def __init__(self) -> None:
|
||||
self._gates: dict[str, asyncio.Event] = {}
|
||||
self._waiting: dict[str, int] = {}
|
||||
self._stages: dict[str, str] = {}
|
||||
|
||||
def hold(self, path: str, stage: str = "request") -> None:
|
||||
self._gates.setdefault(path, asyncio.Event())
|
||||
self._waiting.setdefault(path, 0)
|
||||
self._stages[path] = stage
|
||||
|
||||
def stage(self, path: str) -> str | None:
|
||||
return self._stages.get(path) if path in self._gates else None
|
||||
|
||||
def held(self) -> dict[str, int]:
|
||||
return dict(self._waiting)
|
||||
|
||||
def release(self) -> None:
|
||||
for gate in self._gates.values():
|
||||
gate.set()
|
||||
self._gates.clear()
|
||||
self._waiting.clear()
|
||||
self._stages.clear()
|
||||
|
||||
async def wait(self, path: str) -> None:
|
||||
gate = self._gates.get(path)
|
||||
if gate is None:
|
||||
return
|
||||
self._waiting[path] = self._waiting.get(path, 0) + 1
|
||||
try:
|
||||
await gate.wait()
|
||||
finally:
|
||||
if path in self._waiting:
|
||||
self._waiting[path] -= 1
|
||||
|
||||
|
||||
def register_request_holds(app: web.Application, routes) -> RequestHolds:
|
||||
holds = RequestHolds()
|
||||
|
||||
@web.middleware
|
||||
async def hold_middleware(request: web.Request, handler):
|
||||
stage = holds.stage(request.path)
|
||||
if stage == "request":
|
||||
await holds.wait(request.path)
|
||||
response = await handler(request)
|
||||
if stage != "body" or not isinstance(response, web.Response):
|
||||
return response
|
||||
body = response.body if isinstance(response.body, bytes) else b""
|
||||
streamed = web.StreamResponse(status=response.status, headers=response.headers)
|
||||
streamed.content_length = len(body)
|
||||
await streamed.prepare(request)
|
||||
await streamed.write(body[: len(body) // 2])
|
||||
await holds.wait(request.path)
|
||||
await streamed.write(body[len(body) // 2 :])
|
||||
await streamed.write_eof()
|
||||
return streamed
|
||||
|
||||
app.middlewares.append(hold_middleware)
|
||||
|
||||
@routes.post("/__test__/hold")
|
||||
async def hold(request: web.Request) -> web.Response:
|
||||
data = await request.json() if request.can_read_body else {}
|
||||
path = data.get("path") if isinstance(data, dict) else None
|
||||
stage = data.get("stage", "request") if isinstance(data, dict) else None
|
||||
if (
|
||||
not isinstance(path, str)
|
||||
or not path.startswith(HOLDABLE_PREFIX)
|
||||
or set(data) - {"path", "stage"}
|
||||
or stage not in ("request", "body")
|
||||
):
|
||||
raise web.HTTPBadRequest()
|
||||
holds.hold(path, stage)
|
||||
return web.json_response({"success": True})
|
||||
|
||||
@routes.get("/__test__/hold")
|
||||
async def held(request: web.Request) -> web.Response:
|
||||
return web.json_response({"success": True, "held": holds.held()})
|
||||
|
||||
@routes.post("/__test__/release")
|
||||
async def release(request: web.Request) -> web.Response:
|
||||
holds.release()
|
||||
return web.json_response({"success": True})
|
||||
|
||||
return holds
|
||||
@@ -18,6 +18,7 @@ from aiohttp.web_runner import GracefulExit
|
||||
from PIL import Image
|
||||
from PIL.PngImagePlugin import PngInfo
|
||||
|
||||
from tests.browser.request_hold import register_request_holds
|
||||
from tests.browser.workspace_activity_fixture import register_activity_commands
|
||||
from tests.browser.workspace_concept_fixture import (
|
||||
MAX_EXTRA_IMAGES as CONCEPT_MAX_EXTRA_IMAGES,
|
||||
@@ -195,6 +196,7 @@ def create_app(root: Path) -> web.Application:
|
||||
register_tagging_commands(routes, api, root, model)
|
||||
reveal_stub = register_reveal_stub(routes, api, root)
|
||||
register_activity_commands(app, routes, api)
|
||||
request_holds = register_request_holds(app, routes)
|
||||
|
||||
async def canonical_asset(request: web.Request) -> web.Response:
|
||||
filename, content_type = CANONICAL_ASSETS[request.path]
|
||||
@@ -374,6 +376,7 @@ def create_app(root: Path) -> web.Application:
|
||||
@routes.post("/__test__/reset")
|
||||
async def reset(request: web.Request) -> web.Response:
|
||||
nonlocal history_fixture
|
||||
request_holds.release()
|
||||
library_editor_commands.reset()
|
||||
data = await request.json() if request.can_read_body else {}
|
||||
if not isinstance(data, dict):
|
||||
|
||||
@@ -43,6 +43,55 @@ test('actual gallery form saves a prompt that the admin page can edit and reload
|
||||
expect((await saved.json()).prompt_id).not.toBe(Number(id));
|
||||
});
|
||||
|
||||
// WebKit fails a fetch that a leaving page starts ("Fetch API cannot load … due to access control checks."), and
|
||||
// Playwright reports that console error as a page error. A save's follow-up reads must not start once the page hides:
|
||||
// here the statistics read after an in-place edit is held at the server while the page reloads.
|
||||
test('an in-place edit whose follow-up reads outlive the page starts no read while it leaves', async ({ page, request }) => {
|
||||
await page.goto('/prompt_manager/admin');
|
||||
const text = page.locator('.prompt-text').filter({ hasText: 'Cat portrait' });
|
||||
const id = await text.getAttribute('data-id');
|
||||
await expect(await request.post('/__test__/hold', { data: { path: '/prompt_manager/stats' } })).toBeOK();
|
||||
await page.locator(`[data-action="edit"][data-prompt-id="${id}"]`).click();
|
||||
const editor = page.locator(`.prompt-text[data-id="${id}"]`);
|
||||
await editor.fill('Cat portrait at dusk');
|
||||
await editor.press('Enter');
|
||||
const held = async () => (await (await request.get('/__test__/hold')).json()).held['/prompt_manager/stats'];
|
||||
await expect.poll(held).toBe(1);
|
||||
await page.reload({ waitUntil: 'commit' });
|
||||
await expect(await request.post('/__test__/release')).toBeOK();
|
||||
await expect(page.locator('.prompt-text').filter({ hasText: 'Cat portrait at dusk' })).toBeVisible();
|
||||
});
|
||||
|
||||
// A leave that never happens (a stopped navigation, or one answered with a download) must not leave the list dead.
|
||||
test('a page whose leave was called off reads its list again on the next input', async ({ page }) => {
|
||||
await page.goto('/prompt_manager/admin');
|
||||
await expect(page.locator('.prompt-text').first()).toBeVisible();
|
||||
await page.evaluate(() => window.dispatchEvent(new Event('beforeunload')));
|
||||
await page.mouse.click(1, 1);
|
||||
const reread = page.waitForResponse(response => new URL(response.url()).pathname === '/prompt_manager/recent');
|
||||
await page.locator('#sortBy').selectOption('created_asc');
|
||||
expect(new URL((await reread).url()).searchParams.get('sort')).toBe('created_asc');
|
||||
await expect(page.locator('#resultsTitle')).toHaveText('Recent Prompts');
|
||||
});
|
||||
|
||||
// The gallery's startup thumbnail check runs a second after load. When that second ends while the page is leaving,
|
||||
// the check must start no read: WebKit fails it with "Fetch API cannot load … due to access control checks." and
|
||||
// Playwright reports a page error (seen in CI on PR #289). The page clock makes the second end after the leave began.
|
||||
for (const leave of ['beforeunload', 'pagehide']) {
|
||||
test(`the gallery startup check starts no read after ${leave}`, async ({ page }) => {
|
||||
await page.clock.install();
|
||||
const statusReads = [];
|
||||
page.on('request', req => { if (req.url().includes('/images/thumbnails/status')) statusReads.push(req.url()); });
|
||||
await page.goto('/prompt_manager/gallery');
|
||||
await expect(page.locator('#galleryGrid .image-item')).toHaveCount(2);
|
||||
await page.evaluate(type => window.dispatchEvent(new Event(type)), leave);
|
||||
await page.clock.runFor(1500);
|
||||
// A read made after the clock ran is listed, so the listener saw every status read.
|
||||
await page.evaluate(() => fetch('/prompt_manager/images/thumbnails/status?probe=1'));
|
||||
expect(statusReads.map(url => new URL(url).search)).toEqual(['?probe=1']);
|
||||
});
|
||||
}
|
||||
|
||||
test('actual gallery review generates and persists tags for the selected root', async ({ page, request }) => {
|
||||
await page.goto('/prompt_manager/gallery');
|
||||
await expect(page.locator('#galleryGrid .image-item')).toHaveCount(2);
|
||||
|
||||
@@ -678,6 +678,31 @@ test('cancel scan ends cancelled with partial counts', async ({ page, request })
|
||||
await expect(heading(page).getByRole('button', { name: 'Cancel scan', exact: true })).not.toHaveAttribute('aria-disabled', 'true');
|
||||
});
|
||||
|
||||
// Truth rule: a held legacy-page job (a restore from the admin page) cannot be cancelled, so neither the Activity
|
||||
// list nor the job's view offers Cancel; the cancellable scan beside it keeps its own (review I3).
|
||||
test('a held legacy job shows no Cancel in the list or its view', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const response = await request.post('/__test__/jobs/hold', { data: { kind: 'restore', title: 'Database restore (admin page)' } });
|
||||
await expect(response).toBeOK();
|
||||
const held = (await response.json()).job;
|
||||
expect(held.cancellable).toBe(false);
|
||||
await page.goto(`${workspace}#/activity`);
|
||||
const item = items(page).filter({ has: page.getByRole('link', { name: /Database restore \(admin page\)$/ }) });
|
||||
await expect(item.locator('.chip')).toHaveText('Running');
|
||||
await expect(item.locator('progress')).toHaveCount(1);
|
||||
await expect(main(page).getByRole('button', { name: 'Cancel job', exact: true })).toHaveCount(0);
|
||||
|
||||
await item.getByRole('link').click();
|
||||
await expect(heading(page).locator('h1')).toHaveText('Database restore (admin page)');
|
||||
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
|
||||
await expect(heading(page).getByRole('button', { name: /^Cancel/ })).toHaveCount(0);
|
||||
|
||||
// When the hold ends the view shows the end, still without a Cancel.
|
||||
await finish(request, held.id);
|
||||
await expect(heading(page).locator('p.muted')).toHaveText(/^Finished/);
|
||||
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
|
||||
});
|
||||
|
||||
test('busy start names the running job', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const running = await startJob(request, { title: 'Database maintenance', kind: 'maintenance' });
|
||||
@@ -799,3 +824,114 @@ test('a failed report download says so and leaves the button usable', async ({ p
|
||||
await expect(toast(page)).toHaveText('The report could not be downloaded. Try again.');
|
||||
await expect(button).not.toHaveAttribute('aria-disabled', 'true');
|
||||
});
|
||||
|
||||
// ── #249 PR D Task D3 (D89, plan D66): image tagging jobs in Activity ─────────────────────────
|
||||
|
||||
const TAG_JOB = { kind: 'image-tags', title: 'Auto-tag images', mode: 'auto', images: 4, total: 4 };
|
||||
|
||||
test('an image tagging job shows Tagged Unchanged Skipped Errors and opens tagged rows', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const job = await startJob(request, TAG_JOB);
|
||||
await step(request, job.id, { processed: 3, records: [
|
||||
{ group: 'tagged', path: 'shared/same.png', root: 'output', detail: 'Added 2 tags' },
|
||||
{ group: 'unchanged', path: 'shared/same.png', root: 'input', detail: 'No new tags' },
|
||||
{ group: 'skipped', path: 'video/clip_0004.webm', root: 'output', detail: 'Video: manual tags only' },
|
||||
] });
|
||||
await page.goto(`${workspace}#/images`);
|
||||
// The pill names the kind's verb.
|
||||
await expect(pill(page)).toHaveText('Tagging · 75% · 3 / 4');
|
||||
await openJob(page, job.id);
|
||||
await expect(heading(page).locator('h1')).toHaveText('Auto-tag images');
|
||||
await expect(page).toHaveTitle('Auto-tag images · Prompt Manager');
|
||||
// The header's scope is the number of images, and the job cancels as "Cancel tagging" (danger).
|
||||
await expect(heading(page).locator('p.muted')).toHaveText(/^Started \d\d:\d\d · 4 images · closing this view does not cancel it$/);
|
||||
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report', 'Cancel tagging']);
|
||||
await expect(heading(page).getByRole('button', { name: 'Cancel tagging', exact: true })).toHaveClass(/danger/);
|
||||
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Processing file 3 of 4');
|
||||
await expect(progressSection(page).locator('progress')).toHaveAttribute('aria-label', 'Auto-tag images progress');
|
||||
// Four tiles in the scan's tones: Tagged green, Unchanged plain, Skipped warning, Errors red.
|
||||
expect(await tileValues(page)).toEqual([
|
||||
['stat added', 'Tagged', '1'], ['stat', 'Unchanged', '1'], ['stat warn', 'Skipped', '1'], ['stat bad', 'Errors', '0'],
|
||||
]);
|
||||
// What got tagged and what went wrong first, as the scan's list does.
|
||||
await expect(outcomeButtons(page)).toHaveText(['Tagged1', 'Errors0', 'Skipped1', 'Unchanged1']);
|
||||
await expect(outcomeButtons(page).first()).toHaveAttribute('aria-pressed', 'true');
|
||||
|
||||
// Tagged rows open the viewer on that file (D57's identity).
|
||||
const tagged = fileRows(page).first();
|
||||
expect(await tagged.evaluate(node => node.localName)).toBe('button');
|
||||
await expect(tagged.locator('.path')).toHaveText('shared/same.png');
|
||||
await expect(tagged.locator('.why')).toHaveText('Added 2 tags');
|
||||
await expect(resultsNote(page)).toHaveText('Each row opens the image in the viewer.');
|
||||
await tagged.click();
|
||||
const viewer = page.locator('dialog#viewer-dialog[open]');
|
||||
await expect(viewer.locator('#viewer-stage img').first()).toHaveAttribute('src', /[?&]root=0(&|$)/);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(viewer).toHaveCount(0);
|
||||
await expect(tagged).toBeFocused();
|
||||
// Unchanged rows open too, here the input root's file.
|
||||
await outcomeButtons(page).filter({ hasText: 'Unchanged' }).click();
|
||||
await fileRows(page).first().click();
|
||||
await expect(page.locator('dialog#viewer-dialog[open] #viewer-stage img').first()).toHaveAttribute('src', /[?&]root=1(&|$)/);
|
||||
await page.keyboard.press('Escape');
|
||||
// Skipped rows are plain text with their reason.
|
||||
await outcomeButtons(page).filter({ hasText: 'Skipped' }).click();
|
||||
expect(await fileRows(page).first().evaluate(node => node.localName)).toBe('div');
|
||||
await expect(fileRows(page).first().locator('.why')).toHaveText('Video: manual tags only');
|
||||
await expect(resultsNote(page)).toHaveCount(0);
|
||||
// The log is the job's own: "Tagging log", with the runner's prefixes.
|
||||
await expect(scanLog(page).locator('summary')).toHaveText('● Tagging log — raw detail, click to show');
|
||||
await scanLog(page).locator('summary').click();
|
||||
await expect(scanLog(page).locator('pre')).toContainText('+ shared/same.png · Added 2 tags');
|
||||
|
||||
// An error, then the end: the header and the Activity item read "N tagged · M errors".
|
||||
await step(request, job.id, { processed: 4, records: [
|
||||
{ group: 'errors', path: 'broken/truncated.png', root: 'output', detail: 'Auto-tagging failed. Check server logs for details.' },
|
||||
] });
|
||||
await expect(tiles(page).last().locator('dd')).toHaveText('1');
|
||||
await finish(request, job.id);
|
||||
await expect(toast(page)).toHaveText('Auto-tag images finished · 1 tagged · 1 error');
|
||||
await expect(heading(page).locator('p.muted')).toHaveText(/^Finished \d\d:\d\d · 1 tagged · 1 error$/);
|
||||
await expect(heading(page).locator('.heading-actions > button')).toHaveText(['Download report']);
|
||||
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('4 of 4 files processed');
|
||||
await page.goto(`${workspace}#/activity`);
|
||||
await expect(items(page).first().locator('h3')).toHaveText('✓ Auto-tag images');
|
||||
await expect(items(page).first().locator('p')).toHaveText(/^Finished \d\d:\d\d · 1 tagged · 1 error$/);
|
||||
});
|
||||
|
||||
test('cancel tagging stops at a file boundary', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const job = await startJob(request, { ...TAG_JOB, title: 'Add tags to images', mode: 'add', images: 1250, total: 1250 });
|
||||
await step(request, job.id, { processed: 2, counters: { tagged: 2 } });
|
||||
await page.goto(`${workspace}#/activity`);
|
||||
await expect(items(page).first().locator('p')).toHaveText(/^Started \d\d:\d\d · 1,250 images · closing this view does not cancel it$/);
|
||||
await openJob(page, job.id);
|
||||
await heading(page).getByRole('button', { name: 'Cancel tagging', exact: true }).focus();
|
||||
await page.keyboard.press('Enter');
|
||||
await expect(toast(page)).toHaveText('Add tags to images cancelled · 2 tagged');
|
||||
await expect(heading(page).locator('h1')).toBeFocused();
|
||||
await expect(heading(page).locator('p.muted')).toHaveText(/^Cancelled \d\d:\d\d at a file boundary · 2 tagged · completed work is kept$/);
|
||||
await expect(heading(page).getByRole('button', { name: 'Cancel tagging', exact: true })).toHaveCount(0);
|
||||
await expect(progressSection(page).locator('.panel-heading > span.small')).toHaveText('Cancelled after 2 of 1,250 files');
|
||||
expect((await tileValues(page)).map(([, , value]) => value)).toEqual(['2', '0', '0', '0']);
|
||||
expect((await listed(request))[0].state).toBe('cancelled');
|
||||
});
|
||||
|
||||
test('the report download names the kind', async ({ page, request }) => {
|
||||
await page.setViewportSize({ width: 1600, height: 1100 });
|
||||
const job = await startJob(request, TAG_JOB);
|
||||
await step(request, job.id, { processed: 1, records: [
|
||||
{ group: 'tagged', path: 'sub/a.png', root: 'output', detail: 'Added 1 tag' },
|
||||
] });
|
||||
await finish(request, job.id);
|
||||
await openJob(page, job.id);
|
||||
const button = heading(page).getByRole('button', { name: 'Download report', exact: true });
|
||||
const [download] = await Promise.all([page.waitForEvent('download'), button.click()]);
|
||||
expect(download.suggestedFilename()).toMatch(/^prompt-manager-image-tags-\d{8}-\d{4}\.json$/);
|
||||
const report = JSON.parse(await (await import('node:fs/promises')).readFile(await download.path(), 'utf8'));
|
||||
expect(report.job.kind).toBe('image-tags');
|
||||
expect(report.job.scope).toEqual({ mode: 'auto', images: 4 });
|
||||
expect(Object.keys(report.outcomes)).toEqual(['tagged', 'unchanged', 'skipped', 'errors']);
|
||||
expect(report.outcomes.tagged.rows.map(row => [row.path, row.detail])).toEqual([['sub/a.png', 'Added 1 tag']]);
|
||||
await expect(toast(page)).toHaveText('Job report downloaded.');
|
||||
});
|
||||
|
||||
@@ -1702,7 +1702,8 @@ test('card checkboxes select images into the concept bar and Clear empties it',
|
||||
const selectedBefore = await cards(page).evaluateAll(nodes => nodes.map(node => node.classList.contains('selected')));
|
||||
await checkbox(first, 'same.png').check();
|
||||
await expect(bulkbar(page).locator('.bulkbar > strong')).toHaveText('1 selected');
|
||||
await expect(barButtons(page)).toHaveText(['Set category', 'Export metadata', 'Clear']);
|
||||
// D85: Add tags (the concept's) and Auto-tag selected lead the real actions.
|
||||
await expect(barButtons(page)).toHaveText(['Add tags', 'Auto-tag selected', 'Set category', 'Export metadata', 'Clear']);
|
||||
// Toggling neither opens the viewer, nor inspects, nor moves focus.
|
||||
await expect(page.locator('dialog[open]')).toHaveCount(0);
|
||||
await expect(checkbox(first, 'same.png')).toBeFocused();
|
||||
|
||||
@@ -0,0 +1,323 @@
|
||||
// The Images selection bar's Add tags and Auto-tag selected (#249 PR D, plan Task D2, Mock inventory M3; D85, D90,
|
||||
// D91) against the real `image-tags` job route and the injected fake model (tests/browser/workspace_tagging_fixture.py).
|
||||
// No model is ever downloaded or loaded for real: the fake answers, and every test checks no download request was sent.
|
||||
import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs';
|
||||
import { test, expect } from './fixtures.mjs';
|
||||
|
||||
const images = '/prompt_manager/workspace#/images';
|
||||
const FIRST = 'tagging/dunes_00001.png', SECOND = 'tagging/dunes_00002.png', UNLINKED = 'loose/unlinked.png';
|
||||
const card = (page, path) => page.locator(`article[data-image-key="0:${path}"]`);
|
||||
const bar = page => page.locator('#workspace-main > #bulkbar > .bulkbar');
|
||||
const barButton = (page, name) => bar(page).getByRole('button', { name, exact: true });
|
||||
const modal = page => page.locator('dialog#modal[open]');
|
||||
const title = page => modal(page).locator('#modal-title');
|
||||
const toast = page => page.locator('#workspace-toast');
|
||||
const toasts = page => page.evaluate(() => window.__toasts);
|
||||
const chips = page => modal(page).locator('#tag-suggestions .chip.tag-suggestion');
|
||||
const actionButtons = page => modal(page).locator('.dialog-actions button');
|
||||
const panel = page => page.getByRole('complementary', { name: 'Selected image details', exact: true });
|
||||
const tagRow = page => panel(page).locator('.detail-inner > .tags').last();
|
||||
const NOT_DOWNLOADED = 'WD14 SwinV2 is not downloaded. Download it from the legacy Auto-tag page.';
|
||||
|
||||
test.beforeEach(async ({ page, request, browserName }) => {
|
||||
await startCoverage(page, browserName);
|
||||
// Every toast text in order: a fast job's finish toast may replace its start toast before an assertion reads it.
|
||||
await page.addInitScript(() => {
|
||||
window.__toasts = [];
|
||||
new MutationObserver(() => {
|
||||
const text = document.getElementById('workspace-toast')?.textContent ?? '';
|
||||
if (text && window.__toasts.at(-1) !== text) window.__toasts.push(text);
|
||||
}).observe(document, { subtree: true, childList: true, characterData: true });
|
||||
});
|
||||
await expect(await request.post('/__test__/tagging/reset', { data: {} })).toBeOK();
|
||||
page.taggingErrors = [];
|
||||
page.on('pageerror', error => page.taggingErrors.push(error.message));
|
||||
page.downloads = [];
|
||||
page.on('request', item => { if (/\/autotag\/download\//.test(item.url())) page.downloads.push(item.url()); });
|
||||
});
|
||||
test.afterEach(async ({ page, browserName }) => {
|
||||
await saveCoverage(page, browserName);
|
||||
expect(page.taggingErrors).toEqual([]);
|
||||
expect(page.downloads).toEqual([]);
|
||||
});
|
||||
|
||||
/** Images with `paths` checked, in that order. */
|
||||
async function select(page, ...paths) {
|
||||
await page.goto(images);
|
||||
for (const path of paths) await card(page, path).getByRole('checkbox').check();
|
||||
await expect(bar(page).locator('strong')).toHaveText(`${paths.length} selected`);
|
||||
}
|
||||
async function scopeOf(request) {
|
||||
const listing = await (await request.get('/prompt_manager/images/output?mode=workspace&limit=40')).json();
|
||||
return listing.images[0].root_scope;
|
||||
}
|
||||
/** The saved tags of `path` through the real observe route, then that capture's release. */
|
||||
async function savedTags(request, path) {
|
||||
const target = { root_index: 0, root_scope: await scopeOf(request), path };
|
||||
const observed = await (await request.post('/prompt_manager/image-annotations/observe', { data: { target } })).json();
|
||||
expect(observed.success).toBe(true);
|
||||
await request.delete(`/prompt_manager/image-annotations/capture/${observed.capture.token}`);
|
||||
return observed.tags;
|
||||
}
|
||||
/** Wait until the session's job `id` has ended; returns its summary. */
|
||||
async function ended(request, id) {
|
||||
let job;
|
||||
await expect.poll(async () => {
|
||||
const { jobs } = await (await request.get('/prompt_manager/jobs')).json();
|
||||
job = jobs.find(item => item.id === id);
|
||||
return job?.state;
|
||||
}).toBe('finished');
|
||||
return job;
|
||||
}
|
||||
const jobStart = page => page.waitForRequest(item => item.method() === 'POST' && new URL(item.url()).pathname === '/prompt_manager/jobs/image-tags');
|
||||
|
||||
test('selection bar shows Add tags then Auto-tag selected before Set category', async ({ page }) => {
|
||||
await select(page, FIRST);
|
||||
await expect(bar(page).locator(':scope > *')).toHaveText(['1 selected', 'Add tags', 'Auto-tag selected', 'Set category', 'Export metadata', 'Clear']);
|
||||
});
|
||||
|
||||
test('Add tags runs a job and the cards show the tags after it finishes', async ({ page, request }) => {
|
||||
await select(page, SECOND, FIRST);
|
||||
await expect(card(page, FIRST)).toHaveClass(/\bselected\b/);
|
||||
await barButton(page, 'Add tags').click();
|
||||
await expect(title(page)).toHaveText('Tag selected images');
|
||||
const field = modal(page).getByRole('textbox', { name: 'Tags, separated by commas', exact: true });
|
||||
await expect(field).toHaveValue('');
|
||||
await expect(actionButtons(page)).toHaveText(['Cancel', 'Apply']);
|
||||
let posted = 0;
|
||||
page.on('request', item => { if (new URL(item.url()).pathname === '/prompt_manager/jobs/image-tags') posted += 1; });
|
||||
// A field that parses to no names keeps the dialog and its text, and reports why on the field (D118).
|
||||
await field.fill(' , ,');
|
||||
await actionButtons(page).last().click();
|
||||
await expect.poll(() => field.evaluate(node => node.validationMessage)).toBe('Enter at least one tag.');
|
||||
await expect(title(page)).toHaveText('Tag selected images');
|
||||
await expect(field).toHaveValue(' , ,');
|
||||
await field.fill('sunset, Sunset , blue_hour,,');
|
||||
expect(await field.evaluate(node => node.validity.valid)).toBe(true);
|
||||
const started = jobStart(page);
|
||||
const answer = page.waitForResponse(item => new URL(item.url()).pathname === '/prompt_manager/jobs/image-tags');
|
||||
await actionButtons(page).last().click();
|
||||
const scope = await scopeOf(request);
|
||||
// Listing order (the newer FIRST before SECOND), the reveal identity, the dialog's dedupe.
|
||||
expect((await started).postDataJSON()).toEqual({ mode: 'add', tags: ['sunset', 'blue_hour'], targets: [
|
||||
{ root_index: 0, root_scope: scope, path: FIRST }, { root_index: 0, root_scope: scope, path: SECOND }] });
|
||||
const { job } = await (await answer).json();
|
||||
// The refused Apply sent nothing: only the valid one was posted.
|
||||
expect(posted).toBe(1);
|
||||
await expect.poll(() => toasts(page)).toContain('Tagging started.');
|
||||
expect(await toasts(page)).not.toContain('Enter at least one tag.');
|
||||
await expect(modal(page)).toHaveCount(0);
|
||||
await expect(bar(page).locator('strong')).toHaveText('2 selected');
|
||||
expect((await ended(request, job.id)).counters).toEqual({ tagged: 2, unchanged: 0, skipped: 0, errors: 0 });
|
||||
// The finished job's files are read again: the selected card's panel shows its new image tags.
|
||||
await expect(tagRow(page).locator('button.chip')).toHaveText(['sunset', 'blue_hour']);
|
||||
expect(await savedTags(request, SECOND)).toEqual(['sunset', 'blue_hour']);
|
||||
expect(await savedTags(request, UNLINKED)).toEqual([]);
|
||||
});
|
||||
|
||||
test('Auto-tag selected with Apply automatically runs a job with the chosen model, thresholds and suggestion count', async ({ page, request }) => {
|
||||
await select(page, FIRST, UNLINKED);
|
||||
await barButton(page, 'Auto-tag selected').click();
|
||||
await expect(title(page)).toHaveText('Auto-tag selected images');
|
||||
await expect(modal(page)).toHaveClass(/(^| )tagging-modal( |$)/);
|
||||
const body = modal(page).locator('#modal-body');
|
||||
expect(await body.evaluate(node => [...node.children].map(child => child.className))).toEqual(['tag-model-panel', 'field', 'small', 'dialog-actions']);
|
||||
await expect(body.locator('.tag-model-panel h3')).toHaveText('Auto-tag 2 images');
|
||||
// No preview, manual field, suggestions or Generate: only the model panel's fields.
|
||||
await expect(body.getByRole('button', { name: 'Generate suggestions', exact: true })).toHaveCount(0);
|
||||
await expect(body.locator('#manual-image-tags, .tag-suggestion-panel, .tag-image-preview')).toHaveCount(0);
|
||||
const group = body.getByRole('radiogroup', { name: 'How to apply', exact: true });
|
||||
await expect(group.getByRole('radio', { name: 'Apply automatically', exact: true })).toBeChecked();
|
||||
await expect(group.getByRole('radio', { name: 'Review each image', exact: true })).not.toBeChecked();
|
||||
await expect(actionButtons(page)).toHaveText(['Cancel', 'Start auto-tagging']);
|
||||
await body.getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('wd14-vit');
|
||||
await body.getByRole('spinbutton', { name: 'General confidence', exact: true }).fill('0.5');
|
||||
await body.getByRole('spinbutton', { name: 'Character confidence', exact: true }).fill('0.9');
|
||||
const limit = body.getByRole('spinbutton', { name: 'Suggestions', exact: true });
|
||||
// An invalid count reports its validity and sends nothing.
|
||||
await limit.fill('0');
|
||||
let posted = 0;
|
||||
page.on('request', item => { if (new URL(item.url()).pathname === '/prompt_manager/jobs/image-tags') posted += 1; });
|
||||
await actionButtons(page).last().click();
|
||||
expect(await limit.evaluate(node => node.validity.valid)).toBe(false);
|
||||
await expect(title(page)).toHaveText('Auto-tag selected images');
|
||||
await limit.fill('2');
|
||||
const started = jobStart(page);
|
||||
const answer = page.waitForResponse(item => new URL(item.url()).pathname === '/prompt_manager/jobs/image-tags');
|
||||
await actionButtons(page).last().click();
|
||||
const scope = await scopeOf(request);
|
||||
expect((await started).postDataJSON()).toEqual({ mode: 'auto', model_type: 'wd14-vit',
|
||||
options: { general_threshold: 0.5, character_threshold: 0.9, suggestion_limit: 2 },
|
||||
targets: [{ root_index: 0, root_scope: scope, path: FIRST }, { root_index: 0, root_scope: scope, path: UNLINKED }] });
|
||||
const { job } = await (await answer).json();
|
||||
// The invalid press sent nothing: only this Start was posted.
|
||||
expect(posted).toBe(1);
|
||||
await expect.poll(() => toasts(page)).toContain('Auto-tagging started.');
|
||||
await expect(modal(page)).toHaveCount(0);
|
||||
await expect(bar(page).locator('strong')).toHaveText('2 selected');
|
||||
expect((await ended(request, job.id)).counters).toEqual({ tagged: 2, unchanged: 0, skipped: 0, errors: 0 });
|
||||
expect(await savedTags(request, FIRST)).toEqual(['sandstone', 'desert']);
|
||||
expect(await savedTags(request, UNLINKED)).toEqual(['sandstone', 'desert']);
|
||||
expect(await savedTags(request, SECOND)).toEqual([]);
|
||||
});
|
||||
|
||||
test('a model that is not downloaded keeps Start disabled and says why', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/tagging/models', { data: { unavailable: ['wd14-swinv2'] } })).toBeOK();
|
||||
await select(page, FIRST);
|
||||
await barButton(page, 'Auto-tag selected').click();
|
||||
const status = modal(page).locator('#tag-generation-status');
|
||||
await expect(status).toHaveAttribute('role', 'status');
|
||||
await expect(status).toHaveText(NOT_DOWNLOADED);
|
||||
const start = actionButtons(page).last();
|
||||
await expect(start).toHaveAttribute('aria-disabled', 'true');
|
||||
let posted = 0;
|
||||
page.on('request', item => { if (new URL(item.url()).pathname === '/prompt_manager/jobs/image-tags') posted += 1; });
|
||||
// aria-disabled: a press does nothing.
|
||||
await start.dispatchEvent('click');
|
||||
await expect(title(page)).toHaveText('Auto-tag selected images');
|
||||
await modal(page).getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('wd14-vit');
|
||||
await expect(status).toHaveText('');
|
||||
await expect(start).not.toHaveAttribute('aria-disabled', 'true');
|
||||
await modal(page).getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('wd14-swinv2');
|
||||
await expect(start).toHaveAttribute('aria-disabled', 'true');
|
||||
await actionButtons(page).first().click();
|
||||
await expect(modal(page)).toHaveCount(0);
|
||||
await expect(barButton(page, 'Auto-tag selected')).toBeFocused();
|
||||
expect(posted).toBe(0);
|
||||
});
|
||||
|
||||
test('Suggestions field prefills from the setting and caps chips at the entered count', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/tagging/defaults', { data: { general: 0.5, character: 0.9, limit: 7 } })).toBeOK();
|
||||
await select(page, FIRST);
|
||||
await barButton(page, 'Auto-tag selected').click();
|
||||
const limit = modal(page).getByRole('spinbutton', { name: 'Suggestions', exact: true });
|
||||
await expect(limit).toHaveValue('7');
|
||||
await expect(modal(page).getByRole('spinbutton', { name: 'General confidence', exact: true })).toHaveValue('0.5');
|
||||
await expect(modal(page).getByRole('spinbutton', { name: 'Character confidence', exact: true })).toHaveValue('0.9');
|
||||
await modal(page).getByRole('radio', { name: 'Review each image', exact: true }).check();
|
||||
await expect(actionButtons(page)).toHaveText(['Cancel', 'Start review']);
|
||||
await limit.fill('3');
|
||||
const suggest = page.waitForRequest(item => item.url().endsWith('/image-annotations/suggest'));
|
||||
await actionButtons(page).last().click();
|
||||
await expect(title(page)).toHaveText('Tag image · Image 1 of 1');
|
||||
expect((await suggest).postDataJSON().options).toEqual({ general_threshold: 0.5, character_threshold: 0.9, suggestion_limit: 3 });
|
||||
await expect(chips(page)).toHaveCount(3);
|
||||
await expect(modal(page).getByRole('spinbutton', { name: 'Suggestions', exact: true })).toHaveValue('3');
|
||||
await expect(actionButtons(page)).toHaveText(['Stop reviewing', 'Skip', 'Approve and finish']);
|
||||
await page.keyboard.press('Escape');
|
||||
await expect(modal(page)).toHaveCount(0);
|
||||
await expect(toast(page)).toHaveText('0 of 1 images tagged.');
|
||||
expect(await savedTags(request, FIRST)).toEqual([]);
|
||||
});
|
||||
|
||||
test('Review each image walks the selection one dialog at a time and only approved images gain tags', async ({ page, request }) => {
|
||||
await select(page, FIRST, SECOND, UNLINKED);
|
||||
// The second image's first Generate finds the model busy (a legacy batch): the dialog waits for Retry or Skip.
|
||||
let suggests = 0;
|
||||
await page.route('**/prompt_manager/image-annotations/suggest', async route => {
|
||||
suggests += 1;
|
||||
if (suggests === 2) {
|
||||
await route.fulfill({ status: 409, json: { success: false, code: 'model-busy', error: 'The tagging model is busy. Try again when it finishes.' } });
|
||||
return;
|
||||
}
|
||||
await route.continue();
|
||||
});
|
||||
const puts = [];
|
||||
page.on('request', item => { if (item.method() === 'PUT' && item.url().endsWith('/image-annotations/tags')) puts.push(item.postDataJSON()); });
|
||||
await barButton(page, 'Auto-tag selected').click();
|
||||
await modal(page).getByRole('radio', { name: 'Review each image', exact: true }).check();
|
||||
await modal(page).getByRole('spinbutton', { name: 'Suggestions', exact: true }).fill('2');
|
||||
await actionButtons(page).last().click();
|
||||
await expect(title(page)).toHaveText('Tag image · Image 1 of 3');
|
||||
await expect(modal(page).getByRole('combobox', { name: 'Tagging model', exact: true })).toHaveValue('wd14-swinv2');
|
||||
await expect(chips(page)).toHaveCount(2);
|
||||
await expect(actionButtons(page)).toHaveText(['Stop reviewing', 'Skip', 'Approve and next']);
|
||||
await expect(actionButtons(page).last()).toHaveClass(/ca-primary/);
|
||||
// Approve saves the dialog's manual field plus its remaining chips.
|
||||
await modal(page).getByRole('textbox', { name: 'Image tags · edit manually', exact: true }).fill('mine');
|
||||
await modal(page).getByRole('button', { name: 'Remove desert', exact: true }).click();
|
||||
await actionButtons(page).last().click();
|
||||
await expect(title(page)).toHaveText('Tag image · Image 2 of 3');
|
||||
await expect(modal(page).locator('#tag-generation-status')).toHaveText('The tagging model is busy. Try again when it finishes.');
|
||||
// No suggestions on screen: the primary would save the manual field only (D90).
|
||||
await expect(actionButtons(page)).toHaveText(['Retry', 'Stop reviewing', 'Skip', 'Save and next']);
|
||||
await actionButtons(page).nth(2).click();
|
||||
await expect(title(page)).toHaveText('Tag image · Image 3 of 3');
|
||||
await expect(chips(page)).toHaveCount(2);
|
||||
await expect(actionButtons(page)).toHaveText(['Stop reviewing', 'Skip', 'Approve and finish']);
|
||||
await actionButtons(page).last().click();
|
||||
await expect(modal(page)).toHaveCount(0);
|
||||
await expect(toast(page)).toHaveText('2 of 3 images tagged.');
|
||||
await expect(barButton(page, 'Auto-tag selected')).toBeFocused();
|
||||
await expect(bar(page).locator('strong')).toHaveText('3 selected');
|
||||
expect(puts.map(body => [body.target.path, body.tags])).toEqual([[FIRST, ['mine', 'sandstone']], [UNLINKED, ['sandstone', 'desert']]]);
|
||||
expect(await savedTags(request, FIRST)).toEqual(['mine', 'sandstone']);
|
||||
expect(await savedTags(request, SECOND)).toEqual([]);
|
||||
expect(await savedTags(request, UNLINKED)).toEqual(['sandstone', 'desert']);
|
||||
// The reviewed file's panel shows its saved tags without reselecting.
|
||||
await expect(tagRow(page).locator('button.chip')).toHaveText(['mine', 'sandstone']);
|
||||
await expect.poll(async () => (await (await request.get('/__test__/tagging/captures')).json()).captures).toBe(0);
|
||||
});
|
||||
|
||||
test('Review holds Stop reviewing, Escape and × while a save is in flight, and names a manual-only save', async ({ page, request }) => {
|
||||
await select(page, FIRST, SECOND);
|
||||
let release;
|
||||
const held = new Promise(resolve => { release = resolve; });
|
||||
await page.route('**/prompt_manager/image-annotations/tags', async route => {
|
||||
if (route.request().method() === 'PUT') await held;
|
||||
await route.continue();
|
||||
});
|
||||
await barButton(page, 'Auto-tag selected').click();
|
||||
await modal(page).getByRole('radio', { name: 'Review each image', exact: true }).check();
|
||||
await actionButtons(page).last().click();
|
||||
await expect(title(page)).toHaveText('Tag image · Image 1 of 2');
|
||||
await expect(chips(page)).not.toHaveCount(0);
|
||||
await expect(actionButtons(page)).toHaveText(['Stop reviewing', 'Skip', 'Approve and next']);
|
||||
// D90: a model change clears the suggestions, so the primary saves the manual field only and says so.
|
||||
await modal(page).getByRole('combobox', { name: 'Tagging model', exact: true }).selectOption('wd14-vit');
|
||||
await expect(chips(page)).toHaveCount(0);
|
||||
await expect(actionButtons(page)).toHaveText(['Stop reviewing', 'Skip', 'Save and next']);
|
||||
await modal(page).getByRole('textbox', { name: 'Image tags · edit manually', exact: true }).fill('mine');
|
||||
const put = page.waitForRequest(item => item.method() === 'PUT' && item.url().endsWith('/image-annotations/tags'));
|
||||
await actionButtons(page).last().click();
|
||||
await put;
|
||||
const stop = actionButtons(page).first();
|
||||
await expect(stop).toHaveAttribute('aria-disabled', 'true');
|
||||
await expect(actionButtons(page).nth(1)).toHaveAttribute('aria-disabled', 'true');
|
||||
// None of these ends the session while the save is in flight; the count then includes it.
|
||||
await page.keyboard.press('Escape');
|
||||
await modal(page).locator('#modal-close').click();
|
||||
await stop.dispatchEvent('click'); // aria-disabled: Playwright's click would wait for it
|
||||
release();
|
||||
await expect(title(page)).toHaveText('Tag image · Image 2 of 2');
|
||||
await expect(chips(page)).not.toHaveCount(0);
|
||||
await expect(actionButtons(page)).toHaveText(['Stop reviewing', 'Skip', 'Approve and finish']);
|
||||
await expect(actionButtons(page).first()).not.toHaveAttribute('aria-disabled', 'true');
|
||||
await actionButtons(page).first().click();
|
||||
await expect(modal(page)).toHaveCount(0);
|
||||
await expect(toast(page)).toHaveText('1 of 2 images tagged.');
|
||||
expect(await savedTags(request, FIRST)).toEqual(['mine']);
|
||||
expect(await savedTags(request, SECOND)).toEqual([]);
|
||||
});
|
||||
|
||||
test('a running scan makes the start report busy', async ({ page, request }) => {
|
||||
const scan = await (await request.post('/__test__/jobs/start', { data: { kind: 'scan', title: 'Scan for metadata' } })).json();
|
||||
try {
|
||||
await select(page, FIRST, SECOND);
|
||||
await barButton(page, 'Add tags').click();
|
||||
await modal(page).getByRole('textbox', { name: 'Tags, separated by commas', exact: true }).fill('sunset');
|
||||
await actionButtons(page).last().click();
|
||||
await expect(toast(page)).toHaveText('Another job is running: Scan for metadata.');
|
||||
await expect(toast(page).getByRole('link', { name: 'Scan for metadata', exact: true })).toHaveAttribute('href', `#/activity/${scan.job.id}`);
|
||||
await expect(bar(page).locator('strong')).toHaveText('2 selected');
|
||||
// Auto-tag is refused the same way.
|
||||
await barButton(page, 'Auto-tag selected').click();
|
||||
await expect(actionButtons(page).last()).not.toHaveAttribute('aria-disabled', 'true');
|
||||
await actionButtons(page).last().click();
|
||||
await expect(toast(page)).toHaveText('Another job is running: Scan for metadata.');
|
||||
await expect(bar(page).locator('strong')).toHaveText('2 selected');
|
||||
expect(await savedTags(request, FIRST)).toEqual([]);
|
||||
} finally {
|
||||
await request.post(`/__test__/jobs/${scan.job.id}/finish`, { data: { state: 'finished' } });
|
||||
}
|
||||
});
|
||||
@@ -4,7 +4,9 @@ A fake job is a real registry job whose runner waits for commands instead of
|
||||
reading files, so a test sets its progress and counters exactly:
|
||||
|
||||
- ``POST /__test__/jobs/start`` starts one (``kind``, ``title``, ``total``,
|
||||
``output_folders``) and returns its summary, or 409 while one runs;
|
||||
``output_folders``; for ``kind: "image-tags"`` the image tagging job's
|
||||
groups and its ``{mode, images}`` scope instead) and returns its summary,
|
||||
or 409 while one runs;
|
||||
- ``POST /__test__/jobs/{id}/step`` records ``records`` (``{group, path,
|
||||
root, root_index, detail}`` file outcomes, counted, listed and logged as the scan
|
||||
records them) and ``bulk`` (``{group, count, root}``: ``count`` numbered
|
||||
@@ -12,6 +14,9 @@ reading files, so a test sets its progress and counters exactly:
|
||||
the progress to every subscriber;
|
||||
- ``POST /__test__/jobs/{id}/finish`` ends it as ``finished``, ``cancelled``
|
||||
or ``failed``; a cancel through the real route ends it ``cancelled``;
|
||||
- ``POST /__test__/jobs/hold`` opens a legacy-page hold (``kind``, ``title``):
|
||||
a running, non-cancellable ``records`` job, as a legacy restore or
|
||||
maintenance request takes, until ``finish`` ends it;
|
||||
- ``POST /__test__/jobs/restart`` acts out a ComfyUI restart: the open job
|
||||
streams drop without a final event and the registry starts empty.
|
||||
"""
|
||||
@@ -28,6 +33,8 @@ POLL_SECONDS = 0.02
|
||||
|
||||
PREFIXES = {"added": "+", "known": "=", "review": "?", "errors": "!"}
|
||||
PREFIXES["no_metadata"] = "-"
|
||||
# The image tagging job's groups (#249 PR D), logged with the runner's prefixes.
|
||||
PREFIXES.update({"tagged": "+", "unchanged": "=", "skipped": "-"})
|
||||
ROOT_INDEXES = {"output": 0, "input": 1}
|
||||
|
||||
|
||||
@@ -72,10 +79,17 @@ def outcome_records(data):
|
||||
|
||||
|
||||
def register_activity_commands(app, routes, api):
|
||||
from py.api.jobs import SCAN_GROUP_NAMES, JobBusy, JobRegistry
|
||||
from py.api.jobs import (
|
||||
IMAGE_TAG_GROUP_NAMES,
|
||||
IMAGE_TAG_JOB_KIND,
|
||||
SCAN_GROUP_NAMES,
|
||||
JobBusy,
|
||||
JobRegistry,
|
||||
)
|
||||
|
||||
streams = set()
|
||||
endings = {}
|
||||
holders = set()
|
||||
|
||||
@web.middleware
|
||||
async def track_streams(request, handler):
|
||||
@@ -123,17 +137,29 @@ def register_activity_commands(app, routes, api):
|
||||
async def start_job(request):
|
||||
data = await body(request)
|
||||
ending = asyncio.get_running_loop().create_future()
|
||||
kind = str(data.get("kind", "scan"))
|
||||
if kind == IMAGE_TAG_JOB_KIND:
|
||||
title = "Add tags to images"
|
||||
scope = {
|
||||
"mode": str(data.get("mode", "add")),
|
||||
"images": count(data.get("images"), 0),
|
||||
}
|
||||
groups = IMAGE_TAG_GROUP_NAMES
|
||||
else:
|
||||
title = "Scan for metadata"
|
||||
scope = {
|
||||
"files": str(data.get("files", "new")),
|
||||
"output_folders": count(data.get("output_folders"), 2),
|
||||
}
|
||||
groups = SCAN_GROUP_NAMES
|
||||
try:
|
||||
job = api._jobs.start(
|
||||
str(data.get("kind", "scan")),
|
||||
str(data.get("title", "Scan for metadata")),
|
||||
kind,
|
||||
str(data.get("title", title)),
|
||||
record_changing=True,
|
||||
scope={
|
||||
"files": str(data.get("files", "new")),
|
||||
"output_folders": count(data.get("output_folders"), 2),
|
||||
},
|
||||
scope=scope,
|
||||
runner=fake_runner(ending),
|
||||
groups=SCAN_GROUP_NAMES,
|
||||
groups=groups,
|
||||
)
|
||||
except JobBusy:
|
||||
raise web.HTTPConflict()
|
||||
@@ -141,6 +167,36 @@ def register_activity_commands(app, routes, api):
|
||||
endings[job.id] = ending
|
||||
return web.json_response({"success": True, "job": job.summary()})
|
||||
|
||||
@routes.post("/__test__/jobs/hold")
|
||||
async def hold_job(request):
|
||||
data = await body(request)
|
||||
loop = asyncio.get_running_loop()
|
||||
ending = loop.create_future()
|
||||
entered = loop.create_future()
|
||||
|
||||
async def hold():
|
||||
async with api._jobs.hold(
|
||||
str(data.get("kind", "restore")),
|
||||
str(data.get("title", "Database restore (admin page)")),
|
||||
admission="records",
|
||||
) as job:
|
||||
entered.set_result(job)
|
||||
state = await ending
|
||||
if state != "finished":
|
||||
job.finish(state)
|
||||
|
||||
holder = asyncio.ensure_future(hold())
|
||||
holders.add(holder)
|
||||
holder.add_done_callback(holders.discard)
|
||||
await asyncio.wait({entered, holder}, return_when=asyncio.FIRST_COMPLETED)
|
||||
if not entered.done():
|
||||
if isinstance(holder.exception(), JobBusy):
|
||||
raise web.HTTPConflict()
|
||||
raise holder.exception()
|
||||
job = entered.result()
|
||||
endings[job.id] = ending
|
||||
return web.json_response({"success": True, "job": job.summary()})
|
||||
|
||||
@routes.post("/__test__/jobs/{job_id}/step")
|
||||
async def step_job(request):
|
||||
job = job_or_404(request)
|
||||
@@ -188,6 +244,7 @@ def register_activity_commands(app, routes, api):
|
||||
for transport in list(streams):
|
||||
transport.close()
|
||||
tasks = [job.task for job in old.list() if job.task and not job.task.done()]
|
||||
tasks += [holder for holder in holders if not holder.done()]
|
||||
for task in tasks:
|
||||
task.cancel()
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
@@ -236,6 +236,35 @@ test("latestOnly: errors from the current call propagate, errors from superseded
|
||||
assert.equal(await first, STALE);
|
||||
});
|
||||
|
||||
test("latestOnly with a lifetime: once it ends, no request starts and the call is STALE", async () => {
|
||||
const fetch = fakeFetch();
|
||||
const api = createApiClient(fetch);
|
||||
const page = new AbortController();
|
||||
const latest = latestOnly({ lifetime: page.signal });
|
||||
page.abort();
|
||||
assert.equal(await latest((signal) => api.get("/recent", { signal })), STALE);
|
||||
assert.equal(fetch.calls.length, 0, "a leaving page starts no read");
|
||||
});
|
||||
|
||||
test("latestOnly with a lifetime: its end aborts the request in flight, which is STALE, not an error", async () => {
|
||||
const fetch = fakeFetch();
|
||||
const api = createApiClient(fetch);
|
||||
const page = new AbortController();
|
||||
const latest = latestOnly({ lifetime: page.signal });
|
||||
const call = latest((signal) => api.get("/recent", { signal }));
|
||||
page.abort();
|
||||
assert.equal(fetch.calls[0].aborted, true, "the read in flight is aborted");
|
||||
fetch.calls[0].reject(new DOMException("The operation was aborted.", "AbortError"));
|
||||
assert.equal(await call, STALE);
|
||||
});
|
||||
|
||||
test("latestOnly with a lifetime: real errors still propagate while the page lives", async () => {
|
||||
const page = new AbortController();
|
||||
const latest = latestOnly({ lifetime: page.signal });
|
||||
await assert.rejects(latest(async () => { throw new Error("server down"); }), /server down/);
|
||||
assert.equal(await latest(async () => "fine"), "fine");
|
||||
});
|
||||
|
||||
test("ApiError exposes only validated history recovery actions on all error paths", async () => {
|
||||
const permitted = [null, { action: "refresh-history" }, { action: "reload-outputs" },
|
||||
{ action: "retry-read" }, { action: "smaller-limit", limit: 1 },
|
||||
|
||||
@@ -73,6 +73,28 @@ test('Not Now persists the collection state and suppresses the unchanged prompt'
|
||||
assert.equal(f.prompts.length, 1);
|
||||
});
|
||||
|
||||
// WebKit fails a fetch a leaving page starts ("Fetch API cannot load … due to access control checks.") and logs it
|
||||
// as a page error: the delayed startup check starts no read once the page has begun to leave (as admin.js, #290).
|
||||
test('the startup check starts no read once the page is leaving', async () => {
|
||||
const f = fixture({ success: true, total_images: 100, pending: 30, collection_id: 'a' });
|
||||
f.manager.pageLifetime = new AbortController();
|
||||
f.manager.pageLifetime.abort();
|
||||
await f.check();
|
||||
assert.equal(f.requests.length, 0);
|
||||
assert.equal(f.prompts.length, 0);
|
||||
});
|
||||
|
||||
test('the startup check passes the page lifetime to its read', async () => {
|
||||
const f = fixture({ success: true, total_images: 100, pending: 30, collection_id: 'a' });
|
||||
const signals = [];
|
||||
f.manager.pageLifetime = new AbortController();
|
||||
const get = f.manager.api.get;
|
||||
f.manager.api.get = async (url, options) => { signals.push(options?.signal); return get(url); };
|
||||
await f.check();
|
||||
assert.deepEqual(signals, [f.manager.pageLifetime.signal]);
|
||||
assert.equal(f.prompts.length, 1);
|
||||
});
|
||||
|
||||
test('disabled checks do not request collection status', async () => {
|
||||
const f = fixture({}, { checkThumbnailsAtStartup: false });
|
||||
await f.check();
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
// The legacy admin and gallery pages show a server refusal's reason (#220, PR #289 review I1 and M8): a job that
|
||||
// runs in the workspace refuses their record-changing requests, and a corrupt config.json refuses a settings save.
|
||||
// Each page must name the reason, not report a lost connection, a bare HTTP status or a success.
|
||||
const test = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const vm = require('node:vm');
|
||||
|
||||
function elements() {
|
||||
const nodes = {};
|
||||
return {
|
||||
nodes,
|
||||
getElementById: id => nodes[id] ||= { id, textContent: '', innerHTML: '', disabled: false, style: {},
|
||||
value: '', checked: false, addEventListener() {}, querySelectorAll: () => [] },
|
||||
};
|
||||
}
|
||||
|
||||
function jsonResponse(status, body) {
|
||||
return { ok: status >= 200 && status < 300, status, statusText: status === 409 ? 'Conflict' : 'OK',
|
||||
json: async () => body };
|
||||
}
|
||||
|
||||
function sseResponse(frames) {
|
||||
const bytes = new TextEncoder().encode(frames.map(frame => `data: ${JSON.stringify(frame)}\n\n`).join(''));
|
||||
let sent = false;
|
||||
return { ok: true, status: 200, body: { getReader: () => ({
|
||||
read: async () => (sent ? { done: true } : (sent = true, { done: false, value: bytes })),
|
||||
}) } };
|
||||
}
|
||||
|
||||
function load(file, className, fetchUrl, extra = {}, dom = elements()) {
|
||||
const timers = [];
|
||||
const notifications = [];
|
||||
const storage = new Map([['pm_cached', '1'], ['userData', 'keep']]);
|
||||
const localStorage = {
|
||||
getItem: key => storage.get(key) ?? null,
|
||||
setItem: (key, value) => storage.set(key, String(value)),
|
||||
removeItem: key => storage.delete(key),
|
||||
};
|
||||
const context = vm.createContext({
|
||||
document: { readyState: 'loading', addEventListener() {}, getElementById: dom.getElementById,
|
||||
querySelectorAll: () => [] },
|
||||
window: {}, console: { error() {}, warn() {}, debug() {} },
|
||||
setTimeout: callback => timers.push(callback),
|
||||
confirm: () => true,
|
||||
localStorage,
|
||||
Object: { ...Object, keys: target => (target === localStorage ? [...storage.keys()] : Object.keys(target)) },
|
||||
TextDecoder, TextEncoder, JSON,
|
||||
ApiClient: { fetchUrl },
|
||||
...extra,
|
||||
});
|
||||
const source = fs.readFileSync(path.join(__dirname, `../../web/js/${file}`), 'utf8');
|
||||
const Page = vm.runInContext(`${source}\n${className};`, context, { filename: path.join(__dirname, `../../web/js/${file}`) });
|
||||
const page = Object.create(Page.prototype);
|
||||
page.showNotification = (message, type) => notifications.push([type, message]);
|
||||
page.showModal = id => notifications.push(['show', id]);
|
||||
page.hideModal = id => notifications.push(['hide', id]);
|
||||
return { page, dom, timers, notifications, storage };
|
||||
}
|
||||
|
||||
const BUSY = 'Scan for metadata already running';
|
||||
|
||||
test('gallery cache clear reports a refused server clear and never claims success', async () => {
|
||||
const calls = [];
|
||||
const { page, dom, notifications, storage } = load('gallery.js', 'GalleryManager', async url => {
|
||||
calls.push(url);
|
||||
return jsonResponse(409, { success: false, error: BUSY });
|
||||
});
|
||||
page.getSettings = () => ({ thumbnailsGenerated: true });
|
||||
dom.getElementById('thumbnailStatus').textContent = 'Generated';
|
||||
|
||||
await page.clearCache();
|
||||
|
||||
assert.deepEqual(calls, ['/prompt_manager/images/clear-thumbnails']);
|
||||
assert.deepEqual(notifications, [['error', `Thumbnails were not cleared: ${BUSY}`]]);
|
||||
assert.equal(dom.nodes.thumbnailStatus.textContent, 'Generated');
|
||||
assert.equal(storage.get('gallerySettings'), undefined); // thumbnailsGenerated is not reset
|
||||
});
|
||||
|
||||
test('gallery cache clear still says so when the server clears', async () => {
|
||||
const { page, dom, notifications, storage } = load('gallery.js', 'GalleryManager',
|
||||
async () => jsonResponse(200, { success: true, cleared_count: 3 }));
|
||||
page.getSettings = () => ({ thumbnailsGenerated: true });
|
||||
|
||||
await page.clearCache();
|
||||
|
||||
assert.deepEqual(notifications, [['success', 'PromptManager cache cleared successfully! Original images untouched.']]);
|
||||
assert.equal(dom.nodes.thumbnailStatus.textContent, 'Not generated');
|
||||
assert.equal(JSON.parse(storage.get('gallerySettings')).thumbnailsGenerated, false);
|
||||
assert.equal(storage.get('userData'), 'keep');
|
||||
assert.equal(storage.has('pm_cached'), false);
|
||||
});
|
||||
|
||||
test('admin LoRA import shows a refusal frame and closes its modal', async () => {
|
||||
const { page, dom, timers, notifications } = load('admin.js', 'PromptAdmin',
|
||||
async () => sseResponse([{ type: 'error', status: BUSY, progress: 0, message: BUSY }]));
|
||||
page.saveLoraSettings = async () => true;
|
||||
page.loadStatistics = () => assert.fail('a refused import must not reload statistics');
|
||||
|
||||
await page.runLoraImport();
|
||||
|
||||
assert.equal(dom.nodes.loraImportStatus.textContent, `Error: ${BUSY}`);
|
||||
assert.deepEqual(notifications.filter(([type]) => type === 'error'), [['error', `LoRA import failed: ${BUSY}`]]);
|
||||
assert.equal(timers.length, 1);
|
||||
timers[0]();
|
||||
assert.deepEqual(notifications.at(-1), ['hide', 'loraImportModal']);
|
||||
});
|
||||
|
||||
test('admin settings save shows why config.json refused it', async () => {
|
||||
const message = 'config.json cannot be read, so nothing was saved. Fix or remove the file, then save again.';
|
||||
const { page, notifications } = load('admin.js', 'PromptAdmin',
|
||||
async () => jsonResponse(409, { success: false, persisted: false, code: 'config-unreadable', error: message }));
|
||||
page.settings = {};
|
||||
page._collectScanPaths = () => [];
|
||||
page.saveLoraSettings = async () => assert.fail('nothing else is saved after a refusal');
|
||||
|
||||
await page.saveSettings();
|
||||
|
||||
assert.deepEqual(notifications, [['error', `Failed to save settings: ${message}`]]);
|
||||
});
|
||||
|
||||
test('admin settings save keeps its plain message when the reply has no reason', async () => {
|
||||
const { page, notifications } = load('admin.js', 'PromptAdmin', async () => ({ ok: false, status: 500,
|
||||
json: async () => { throw new SyntaxError('not json'); } }));
|
||||
page.settings = {};
|
||||
page._collectScanPaths = () => [];
|
||||
|
||||
await page.saveSettings();
|
||||
|
||||
assert.deepEqual(notifications, [['error', 'Failed to save settings']]);
|
||||
});
|
||||
|
||||
test('admin maintenance names the job that refused it', async () => {
|
||||
const { page, dom, notifications } = load('admin.js', 'PromptAdmin',
|
||||
async () => jsonResponse(409, { success: false, error: BUSY }), {
|
||||
escapeHtml: text => String(text),
|
||||
document: { readyState: 'loading', addEventListener() {}, getElementById: id => dom.getElementById(id),
|
||||
querySelectorAll: selector => (selector === '.maintenance-option:checked' ? [{ id: 'vacuum' }] : []) },
|
||||
});
|
||||
|
||||
await page.runMaintenance();
|
||||
|
||||
const results = dom.nodes.maintenanceResults.innerHTML;
|
||||
assert.match(results, /Maintenance Failed/);
|
||||
assert.match(results, new RegExp(BUSY));
|
||||
assert.doesNotMatch(results, /HTTP 409/);
|
||||
assert.deepEqual(notifications, [['error', '❌ Maintenance failed']]);
|
||||
assert.equal(dom.nodes.runMaintenanceBtn.disabled, false);
|
||||
});
|
||||
@@ -261,8 +261,13 @@ test('summaries are validated and the copy follows the spec', async () => {
|
||||
const loose = parseJob({ id: ID_A, state: 'running', progress: 140.4, processed: -1, total: 1.5, counters: { added: 'x', errors: 2 },
|
||||
title: ' ', started_at: 'x', scope: [] });
|
||||
assert.deepEqual(loose, { id: ID_A, kind: '', title: 'Background job', state: 'running', started_at: null, finished_at: null,
|
||||
progress: 100, status_line: '', processed: 0, total: 0, counters: { added: 0, errors: 2 }, scope: {}, listed: {}, log_length: 0 });
|
||||
progress: 100, status_line: '', processed: 0, total: 0, counters: { added: 0, errors: 2 }, scope: {}, listed: {}, log_length: 0,
|
||||
cancellable: true });
|
||||
assert.equal(parseJob(summary({ progress: -3 })).progress, 0);
|
||||
// A held legacy job cannot be cancelled; an older server sends no key, which keeps Cancel.
|
||||
assert.equal(parseJob(summary({ cancellable: false })).cancellable, false);
|
||||
assert.equal(parseJob(summary({ cancellable: true })).cancellable, true);
|
||||
assert.equal(parseJob(summary()).cancellable, true);
|
||||
|
||||
const scan = parseJob(summary({ progress: 42, processed: 8240, total: 19622, counters: { added: 1, errors: 1 } }));
|
||||
assert.deepEqual(pillParts(scan), { label: 'Scanning · 42%', files: ' · 8,240 / 19,622', text: 'Scanning · 42% · 8,240 / 19,622' });
|
||||
@@ -283,6 +288,48 @@ test('summaries are validated and the copy follows the spec', async () => {
|
||||
assert.deepEqual([-1, 0, 1, 4, 5, 6, 1e9, 0.5].map(backoffDelay), [1000, 1000, 2000, 16000, 30000, 30000, 30000, 1000]);
|
||||
});
|
||||
|
||||
// D89 (plan D66): an image tagging job's view has its own tiles, outcomes, openable rows, log and cancel labels.
|
||||
test('image-tags jobs get their own tiles outcomes and pill verb', async () => {
|
||||
const { parseJob, jobView, pillParts, outcomeLine, startToast } = await source;
|
||||
assert.equal(typeof jobView, 'function');
|
||||
const job = parseJob(summary({ kind: 'image-tags', title: 'Auto-tag images', progress: 40, processed: 2, total: 5,
|
||||
counters: { tagged: 1420, unchanged: 3, skipped: 2, errors: 1 }, scope: { mode: 'auto', images: 5, model_type: 'wd14-vit' } }));
|
||||
const view = jobView(job.kind);
|
||||
assert.deepEqual(view.tiles, [['tagged', 'Tagged', 'added'], ['unchanged', 'Unchanged', ''], ['skipped', 'Skipped', 'warn'],
|
||||
['errors', 'Errors', 'bad']]);
|
||||
assert.deepEqual(view.outcomes, [['tagged', 'Tagged'], ['errors', 'Errors'], ['skipped', 'Skipped'], ['unchanged', 'Unchanged']]);
|
||||
assert.deepEqual([...view.openable].sort(), ['tagged', 'unchanged']);
|
||||
assert.equal(view.log, 'Tagging log');
|
||||
assert.equal(view.cancel, 'Cancel tagging');
|
||||
assert.deepEqual(pillParts(job), { label: 'Tagging · 40%', files: ' · 2 / 5', text: 'Tagging · 40% · 2 / 5' });
|
||||
assert.equal(outcomeLine(job), '1,420 tagged · 1 error');
|
||||
assert.equal(outcomeLine({ ...job, counters: {} }), '0 tagged · 0 errors');
|
||||
assert.equal(startToast(job), 'Auto-tagging started.');
|
||||
// The design's toast pattern, as the scan's: "<Job> finished · N tagged · M errors".
|
||||
const { finishToast } = await source;
|
||||
assert.equal(finishToast({ ...job, state: 'finished' }), 'Auto-tag images finished · 1,420 tagged · 1 error');
|
||||
assert.equal(finishToast({ ...job, state: 'cancelled' }), 'Auto-tag images cancelled · 1,420 tagged');
|
||||
assert.equal(finishToast({ ...job, state: 'failed' }), 'Auto-tag images failed · 1,420 tagged · 1 error');
|
||||
assert.equal(finishToast({ ...job, title: 'Add tags to images', counters: {}, state: 'finished' }),
|
||||
'Add tags to images finished · 0 tagged · 0 errors');
|
||||
});
|
||||
|
||||
test('unknown kinds keep the generic view', async () => {
|
||||
const { jobView } = await source;
|
||||
const scan = jobView('scan');
|
||||
assert.deepEqual(scan.tiles, [['added', 'Added to Library', 'added'], ['known', 'Already known', ''],
|
||||
['review', 'Needs review', 'warn'], ['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad']]);
|
||||
assert.deepEqual(scan.outcomes, [['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
|
||||
['no_metadata', 'No usable metadata'], ['known', 'Already known']]);
|
||||
assert.deepEqual([...scan.openable].sort(), ['added', 'known']);
|
||||
assert.deepEqual([scan.log, scan.cancel], ['Scan log', 'Cancel scan']);
|
||||
for (const kind of ['maintenance', '', 'constructor', '__proto__']) {
|
||||
const view = jobView(kind);
|
||||
assert.deepEqual([view.tiles, view.outcomes, [...view.openable]], [scan.tiles, scan.outcomes, [...scan.openable]]);
|
||||
assert.deepEqual([view.log, view.cancel], ['Job log', 'Cancel job']);
|
||||
}
|
||||
});
|
||||
|
||||
test('pagehide stops polling, closes streams and aborts the in-flight read; pageshow resumes', async () => {
|
||||
const { createActivityStore } = await source;
|
||||
const { world, api, ...deps } = harness();
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
|
||||
// Guarded: before the module exists every test fails on an assertion, not on the import.
|
||||
const loaded = await import('../../web/js/workspace/images-tagging-bulk.mjs').catch(() => null);
|
||||
const module = () => { assert.ok(loaded, 'images-tagging-bulk.mjs is not implemented'); return loaded; };
|
||||
const state = await import('../../web/js/workspace/image-tagging-state.mjs');
|
||||
const { startToast, parseJob } = await import('../../web/js/workspace/activity-store.mjs');
|
||||
const SCOPE = 'b'.repeat(64);
|
||||
const JOB_ID = 'c'.repeat(32);
|
||||
const image = (path, extra = {}) => ({ root_index: 0, root_scope: SCOPE, path, filename: path.split('/').pop(), ...extra });
|
||||
const pageScope = () => {
|
||||
const controller = new AbortController();
|
||||
const cleanups = [];
|
||||
return { signal: controller.signal, cleanup: fn => cleanups.push(fn), abort: () => controller.abort(), cleanups };
|
||||
};
|
||||
|
||||
test('targets keep listing order and identity', () => {
|
||||
const { bulkTargets } = module();
|
||||
const listing = [image('b/2.png', { width: 9 }), image('a/1.png'), { ...image('x.png'), root_scope: 'nope' }, image('c/3.webm')];
|
||||
assert.deepEqual(bulkTargets(listing), [
|
||||
{ root_index: 0, root_scope: SCOPE, path: 'b/2.png' },
|
||||
{ root_index: 0, root_scope: SCOPE, path: 'a/1.png' },
|
||||
{ root_index: 0, root_scope: SCOPE, path: 'c/3.webm' },
|
||||
]);
|
||||
assert.deepEqual(bulkTargets([]), []);
|
||||
});
|
||||
|
||||
test('over 500 selected refuses with the toast', () => {
|
||||
const { bulkTargets, openAddTags, openAutoTag, MAX_BULK_IMAGES, TOO_MANY } = module();
|
||||
assert.equal(MAX_BULK_IMAGES, 500);
|
||||
assert.equal(TOO_MANY, 'Select at most 500 images.');
|
||||
const many = Array.from({ length: 501 }, (_, index) => image(`bulk/${index}.png`));
|
||||
assert.equal(bulkTargets(many), null);
|
||||
assert.equal(bulkTargets(many.slice(0, 500)).length, 500);
|
||||
const toasts = [];
|
||||
const store = { start: () => assert.fail('no job starts') };
|
||||
// Refused before any dialog opens (no DOM here).
|
||||
openAddTags({ scope: pageScope(), store, images: many, notify: text => toasts.push(text) });
|
||||
openAutoTag({ api: {}, scope: pageScope(), store, images: many, notify: text => toasts.push(text) });
|
||||
assert.deepEqual(toasts, [TOO_MANY, TOO_MANY]);
|
||||
});
|
||||
|
||||
test('add parses comma tags with the same dedupe as the dialog', () => {
|
||||
const { parseBulkTags, ADD_DIALOG } = module();
|
||||
assert.deepEqual(ADD_DIALOG, { title: 'Tag selected images', label: 'Tags, separated by commas', action: 'Apply' });
|
||||
assert.deepEqual(parseBulkTags(' Sunset, sunset ,, blue_hour,SUNSET, Blue_Hour '), ['Sunset', 'blue_hour']);
|
||||
assert.deepEqual(parseBulkTags(' , ,'), []);
|
||||
assert.equal(parseBulkTags, state.parseManualTags);
|
||||
});
|
||||
|
||||
test('Add tags keeps its dialog and the typed text when the field parses to no names', async () => {
|
||||
const { openAddTags, NO_TAGS } = module();
|
||||
const toasts = [];
|
||||
const posted = [];
|
||||
const store = { start: async (kind, body) => { posted.push(body); return { job: { id: JOB_ID, state: 'running' } }; },
|
||||
find: () => null, subscribe: () => () => {} };
|
||||
let dialog = null;
|
||||
openAddTags({ scope: pageScope(), store, images: [image('a.png')], trigger: null, notify: text => toasts.push(text),
|
||||
openInput: args => { dialog = args; } });
|
||||
assert.equal(typeof dialog?.validate, 'function');
|
||||
// The dialog reports the message on its field and stays open: nothing toasts, nothing is sent.
|
||||
assert.equal(dialog.validate(' , ,'), NO_TAGS);
|
||||
assert.equal(dialog.validate('sunset'), null);
|
||||
assert.deepEqual(toasts, []);
|
||||
await dialog.onApply('sunset, Sunset');
|
||||
assert.deepEqual(posted.map(body => body.tags), [['sunset']]);
|
||||
});
|
||||
|
||||
test('busy start keeps the selection and names the running job', async () => {
|
||||
const { startTagJob } = module();
|
||||
const toasts = [];
|
||||
const posted = [];
|
||||
const running = { id: JOB_ID, title: 'Scan for metadata' };
|
||||
const store = { start: async (kind, body) => { posted.push([kind, body]); return { busy: running, message: 'busy' }; } };
|
||||
const body = { mode: 'add', targets: [{ root_index: 0, root_scope: SCOPE, path: 'a.png' }], tags: ['x'] };
|
||||
assert.equal(await startTagJob({ store, scope: pageScope(), body, notify: text => toasts.push(text) }), null);
|
||||
assert.deepEqual(posted, [['image-tags', body]]);
|
||||
assert.deepEqual(toasts, [['Another job is running: ', { text: 'Scan for metadata', href: `#/activity/${JOB_ID}` }, '.']]);
|
||||
});
|
||||
|
||||
test('a started job is returned without a toast of its own; refusals toast their sentence', async () => {
|
||||
const { startTagJob } = module();
|
||||
const toasts = [];
|
||||
const notify = text => toasts.push(text);
|
||||
const job = { id: JOB_ID, kind: 'image-tags' };
|
||||
assert.equal(await startTagJob({ store: { start: async () => ({ job }) }, scope: pageScope(), body: { mode: 'add' }, notify }), job);
|
||||
const refused = Object.assign(new Error('The tagging model is busy. Try again when it finishes.'), { name: 'ApiError', status: 409, code: 'model-busy' });
|
||||
assert.equal(await startTagJob({ store: { start: async () => { throw refused; } }, scope: pageScope(), body: { mode: 'auto' }, notify }), null);
|
||||
assert.equal(await startTagJob({ store: { start: async () => { throw new TypeError('down'); } }, scope: pageScope(), body: { mode: 'auto' }, notify }), null);
|
||||
assert.equal(await startTagJob({ store: { start: async () => { throw new TypeError('down'); } }, scope: pageScope(), body: { mode: 'add' }, notify }), null);
|
||||
assert.deepEqual(toasts, ['The tagging model is busy. Try again when it finishes.', 'Auto-tagging could not start. Try again.',
|
||||
'Tagging could not start. Try again.']);
|
||||
// A page left meanwhile says nothing.
|
||||
const left = pageScope();
|
||||
left.abort();
|
||||
assert.equal(await startTagJob({ store: { start: async () => { throw refused; } }, scope: left, body: { mode: 'auto' }, notify }), null);
|
||||
assert.equal(toasts.length, 3);
|
||||
});
|
||||
|
||||
test('the start toast names the tagging mode', () => {
|
||||
const job = mode => parseJob({ id: JOB_ID, state: 'running', kind: 'image-tags', title: 'Add tags to images', scope: { mode, images: 2 } });
|
||||
assert.equal(startToast(job('add')), 'Tagging started.');
|
||||
assert.equal(startToast(job('auto')), 'Auto-tagging started.');
|
||||
assert.equal(startToast(job('other')), 'Add tags to images started.');
|
||||
assert.equal(startToast(parseJob({ id: JOB_ID, state: 'running', kind: 'scan', title: 'Scan for metadata' })), 'Scan started.');
|
||||
});
|
||||
|
||||
test('a followed job reports its end once', () => {
|
||||
const { followJobEnd } = module();
|
||||
let jobs = [{ id: JOB_ID, state: 'running', counters: {} }];
|
||||
const listeners = new Set();
|
||||
const store = { find: id => jobs.find(job => job.id === id), subscribe: fn => { listeners.add(fn); return () => listeners.delete(fn); } };
|
||||
const ended = [];
|
||||
const scope = pageScope();
|
||||
followJobEnd({ store, scope, id: JOB_ID, onEnd: job => ended.push(job.state) });
|
||||
assert.equal(listeners.size, 1);
|
||||
for (const fn of [...listeners]) fn(store);
|
||||
assert.deepEqual(ended, []);
|
||||
jobs = [{ id: JOB_ID, state: 'finished', counters: { tagged: 1 } }];
|
||||
for (const fn of [...listeners]) fn(store);
|
||||
for (const fn of [...listeners]) fn(store);
|
||||
assert.deepEqual(ended, ['finished']);
|
||||
assert.equal(listeners.size, 0);
|
||||
assert.equal(scope.cleanups.length, 1);
|
||||
});
|
||||
|
||||
test('a model that is not downloaded is named with the server sentence', () => {
|
||||
const { modelUnavailable } = state;
|
||||
assert.equal(modelUnavailable(null, 'wd14-vit'), '');
|
||||
assert.equal(modelUnavailable({ 'wd14-vit': { downloaded: true } }, 'wd14-vit'), '');
|
||||
assert.equal(modelUnavailable({ 'wd14-vit': { downloaded: false } }, 'wd14-vit'), 'WD14 ViT is not downloaded. Download it from the legacy Auto-tag page.');
|
||||
assert.equal(modelUnavailable({ gguf: { downloaded: false, name: 'GGUF (Recommended)' } }, 'gguf'),
|
||||
'GGUF (Recommended) is not downloaded. Download it from the legacy Auto-tag page.');
|
||||
assert.throws(() => modelUnavailable({}, 'nope'), TypeError);
|
||||
});
|
||||
|
||||
test('the bulk options start from the server defaults and wait for them before Start', async () => {
|
||||
const { createTaggingOptionsController } = state;
|
||||
let answer;
|
||||
const api = { get: () => new Promise(resolve => { answer = resolve; }) };
|
||||
const options = createTaggingOptionsController({ api });
|
||||
const seen = [];
|
||||
options.subscribe(value => seen.push(value.optionsRevision));
|
||||
assert.equal(options.state.canStart, false);
|
||||
assert.equal(options.state.canGenerate, false);
|
||||
const opening = options.open();
|
||||
answer({ wd14_general_threshold: 0.5, wd14_character_threshold: 0.9, default_prompt: 'Say it', suggestion_limit: 7,
|
||||
models: { 'wd14-swinv2': { downloaded: true }, 'wd14-vit': { downloaded: false }, gguf: { downloaded: true } } });
|
||||
await opening;
|
||||
assert.equal(options.state.canStart, true);
|
||||
assert.deepEqual(options.request(), { model_type: 'wd14-swinv2', options: { general_threshold: 0.5, character_threshold: 0.9, suggestion_limit: 7 } });
|
||||
options.setOption('suggestion_limit', '3');
|
||||
options.setModel('wd14-vit');
|
||||
assert.equal(options.state.canStart, false);
|
||||
assert.match(options.state.unavailable, /^WD14 ViT is not downloaded\./);
|
||||
options.setModel('gguf');
|
||||
assert.deepEqual(options.request(), { model_type: 'gguf', options: { instructions: 'Say it', suggestion_limit: 3 } });
|
||||
options.setOption('instructions', ' ');
|
||||
assert.equal(options.request(), null);
|
||||
assert.deepEqual(seen, [1, 1, 2, 3, 3]);
|
||||
assert.throws(() => options.setModel('nope'), TypeError);
|
||||
assert.throws(() => options.setOption('nope', 1), TypeError);
|
||||
options.dispose();
|
||||
options.setOption('suggestion_limit', '4');
|
||||
assert.equal(options.state.options.suggestion_limit, '3');
|
||||
});
|
||||
|
||||
test('the bulk options keep the documented defaults when the read fails', async () => {
|
||||
const { createTaggingOptionsController } = state;
|
||||
const options = createTaggingOptionsController({ api: { get: async () => { throw new Error('offline'); } } });
|
||||
await options.open();
|
||||
await options.open();
|
||||
assert.equal(options.state.canStart, true);
|
||||
assert.equal(options.state.unavailable, '');
|
||||
assert.deepEqual(options.request(), { model_type: 'wd14-swinv2', options: { general_threshold: 0.35, character_threshold: 0.85, suggestion_limit: 10 } });
|
||||
assert.throws(() => createTaggingOptionsController({}), TypeError);
|
||||
});
|
||||
|
||||
test('a preset starts the Tag image dialog on the chosen model and options', async () => {
|
||||
const { createImageTaggingController } = state;
|
||||
const target = { root_index: 0, root_scope: SCOPE, path: 'a.png' };
|
||||
const api = {
|
||||
get: async () => ({ wd14_general_threshold: 0.35, wd14_character_threshold: 0.85, default_prompt: 'Describe', suggestion_limit: 10 }),
|
||||
post: async () => ({ success: true, library_epoch: '12345678-1234-1234-1234-123456789abc', filename: 'a.png', title: null, tags: [], tag_revision: 0,
|
||||
capture: { token: 'tok', sha256: 'd'.repeat(64), preview_url: '/prompt_manager/image-annotations/capture/tok', width: 4, height: 4,
|
||||
media_type: 'image', generate: { supported: true, reason: null } } }),
|
||||
put: async () => ({}), del: async () => ({}),
|
||||
};
|
||||
const controller = createImageTaggingController({ api, target, source: 'bulk', preset: { model: 'gguf', options: { instructions: 'Name it', suggestion_limit: 3 } } });
|
||||
assert.equal(controller.state.model, 'gguf');
|
||||
await controller.open();
|
||||
assert.deepEqual([controller.state.model, controller.state.options.instructions, controller.state.options.suggestion_limit], ['gguf', 'Name it', 3]);
|
||||
controller.dispose();
|
||||
assert.throws(() => createImageTaggingController({ api, target, preset: { model: 'nope', options: {} } }), TypeError);
|
||||
assert.throws(() => createImageTaggingController({ api, target, preset: { model: 'gguf', options: { instructions: '' } } }), TypeError);
|
||||
assert.throws(() => createImageTaggingController({ api, target, preset: { model: 'gguf', options: { nope: 1 } } }), TypeError);
|
||||
});
|
||||
@@ -0,0 +1,110 @@
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
|
||||
// Guarded: before the module exists every test fails on an assertion, not on the import.
|
||||
const loaded = await import('../../web/js/workspace/images-tagging-review.mjs').catch(() => null);
|
||||
const module = () => { assert.ok(loaded, 'images-tagging-review.mjs is not implemented'); return loaded; };
|
||||
const SCOPE = 'e'.repeat(64);
|
||||
const images = ['a.png', 'b.png', 'c.png'].map(path => ({ root_index: 0, root_scope: SCOPE, path, filename: path }));
|
||||
const OPTIONS = { general_threshold: 0.4, character_threshold: 0.8, suggestion_limit: 3 };
|
||||
|
||||
/** A session over fake dialogs: `dialogs` holds every open call's arguments and whether it was closed. */
|
||||
function session(extra = {}) {
|
||||
const dialogs = [];
|
||||
const toasts = [];
|
||||
let closes = 0;
|
||||
const saved = [];
|
||||
const handle = module().startReviewSession({
|
||||
api: {}, scope: {}, images, model_type: 'wd14-vit', options: OPTIONS, trigger: 'auto-tag',
|
||||
onSaved: event => saved.push(event.filename),
|
||||
open: args => {
|
||||
const dialog = { args, closed: 0 };
|
||||
dialogs.push(dialog);
|
||||
return { close: () => { dialog.closed += 1; args.onStop(); } };
|
||||
},
|
||||
close: () => { closes += 1; },
|
||||
notify: text => toasts.push(text),
|
||||
...extra,
|
||||
});
|
||||
return { handle, dialogs, toasts, saved, closes: () => closes };
|
||||
}
|
||||
const approve = (dialog, tags = ['x']) => dialog.args.onSaved({ filename: dialog.args.image.filename, tags, target: {} });
|
||||
|
||||
test('the first image opens at once with the session model, options and its position', () => {
|
||||
const { dialogs } = session();
|
||||
assert.equal(dialogs.length, 1);
|
||||
const { args } = dialogs[0];
|
||||
assert.equal(args.image, images[0]);
|
||||
assert.deepEqual(args.position, { index: 1, total: 3 });
|
||||
assert.deepEqual(args.preset, { model: 'wd14-vit', options: OPTIONS });
|
||||
assert.equal(args.autoGenerate, true);
|
||||
assert.equal(args.actions, 'review');
|
||||
assert.equal(args.source, 'bulk');
|
||||
assert.equal(args.trigger, 'auto-tag');
|
||||
assert.throws(() => module().startReviewSession({ images: [] }), TypeError);
|
||||
});
|
||||
|
||||
test('advances only on approve or skip and writes nothing on skip or stop', () => {
|
||||
const { dialogs, toasts, saved } = session();
|
||||
// Nothing from the dialog (a Generate in progress, a model-busy answer): the session waits on that image.
|
||||
assert.equal(dialogs.length, 1);
|
||||
dialogs[0].args.onSkip();
|
||||
assert.equal(dialogs.length, 2);
|
||||
assert.deepEqual(dialogs[1].args.position, { index: 2, total: 3 });
|
||||
// A second report from a dialog already left changes nothing.
|
||||
dialogs[0].args.onSkip();
|
||||
dialogs[0].args.onStop();
|
||||
assert.equal(dialogs.length, 2);
|
||||
approve(dialogs[1]);
|
||||
assert.equal(dialogs.length, 3);
|
||||
dialogs[2].args.onStop();
|
||||
assert.deepEqual(saved, ['b.png']);
|
||||
assert.deepEqual(toasts, ['1 of 3 images tagged.']);
|
||||
assert.equal(dialogs.length, 3);
|
||||
});
|
||||
|
||||
test('approve saves exactly the dialog\'s manual-plus-chips set', () => {
|
||||
// The session never writes itself: each approve is the dialog's own Save (its event carries the saved set).
|
||||
const forwarded = [];
|
||||
const { dialogs } = session({ onSaved: event => forwarded.push(event.tags) });
|
||||
approve(dialogs[0], ['manual', 'chip']);
|
||||
assert.deepEqual(forwarded, [['manual', 'chip']]);
|
||||
assert.deepEqual(dialogs[1].args.preset, { model: 'wd14-vit', options: OPTIONS });
|
||||
});
|
||||
|
||||
test('last image offers Approve and finish', () => {
|
||||
const { dialogs, toasts, closes } = session();
|
||||
approve(dialogs[0]);
|
||||
dialogs[1].args.onSkip();
|
||||
assert.deepEqual(dialogs[2].args.position, { index: 3, total: 3 });
|
||||
approve(dialogs[2]);
|
||||
// The last approve ends the session: the dialog closes and the count is toasted once.
|
||||
assert.equal(closes(), 1);
|
||||
assert.deepEqual(toasts, ['2 of 3 images tagged.']);
|
||||
dialogs[2].args.onStop();
|
||||
assert.deepEqual(toasts, ['2 of 3 images tagged.']);
|
||||
});
|
||||
|
||||
test('model-busy on one image waits for retry or skip', () => {
|
||||
const { dialogs, toasts } = session();
|
||||
// The dialog shows the refusal with Retry; a Retry that succeeds then approves, all on the same image.
|
||||
assert.equal(dialogs.length, 1);
|
||||
approve(dialogs[0]);
|
||||
assert.equal(dialogs.length, 2);
|
||||
assert.deepEqual(toasts, []);
|
||||
});
|
||||
|
||||
test('stop releases listeners and in-flight requests', () => {
|
||||
const { handle, dialogs, toasts, saved } = session();
|
||||
handle.stop();
|
||||
assert.equal(dialogs[0].closed, 1);
|
||||
assert.deepEqual(toasts, ['0 of 3 images tagged.']);
|
||||
// A save that settles after the stop still counts as written, but nothing opens or toasts again.
|
||||
approve(dialogs[0]);
|
||||
dialogs[0].args.onSkip();
|
||||
handle.stop();
|
||||
assert.equal(dialogs.length, 1);
|
||||
assert.deepEqual(saved, ['a.png']);
|
||||
assert.deepEqual(toasts, ['0 of 3 images tagged.']);
|
||||
assert.equal(module().reviewOutcome(1, 1), '1 of 1 images tagged.');
|
||||
});
|
||||
@@ -0,0 +1,35 @@
|
||||
"""A lock stand-in that reports when a second caller has to wait for it.
|
||||
|
||||
Concurrency tests patch it over a module's lock so they can wait until the
|
||||
second caller is really blocked on the lock, instead of hoping a thread
|
||||
start or one event-loop tick got it there.
|
||||
"""
|
||||
|
||||
import threading
|
||||
|
||||
|
||||
class ProbeLock:
|
||||
"""A re-entrant lock whose ``contended`` event is set when a caller blocks."""
|
||||
|
||||
def __init__(self):
|
||||
self._lock = threading.RLock()
|
||||
self.contended = threading.Event()
|
||||
|
||||
def acquire(self, blocking=True, timeout=-1):
|
||||
if self._lock.acquire(blocking=False):
|
||||
return True
|
||||
self.contended.set()
|
||||
if not blocking:
|
||||
return False
|
||||
return self._lock.acquire(timeout=timeout)
|
||||
|
||||
def release(self):
|
||||
self._lock.release()
|
||||
|
||||
def __enter__(self):
|
||||
self.acquire()
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc):
|
||||
self.release()
|
||||
return False
|
||||
+169
-9
@@ -5,6 +5,7 @@ database and a stub ``folder_paths`` module, so the tests run without ComfyUI.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
@@ -734,9 +735,9 @@ class TestSettingsReadMonitorThroughModule(AdminAPITestCase):
|
||||
def test_no_sys_modules_scan(self):
|
||||
import inspect
|
||||
|
||||
from py.api.admin import AdminRoutesMixin
|
||||
from py.api.settings_routes import SettingsRoutesMixin
|
||||
|
||||
source = inspect.getsource(AdminRoutesMixin.get_settings)
|
||||
source = inspect.getsource(SettingsRoutesMixin.get_settings)
|
||||
self.assertNotIn("sys.modules", source)
|
||||
|
||||
async def test_reports_directories_of_running_monitor(self):
|
||||
@@ -1059,14 +1060,29 @@ class TestSettingsMisc(AdminAPITestCase):
|
||||
saved["gallery"]["monitoring"]["directories"], [self._root(self.output_dir)]
|
||||
)
|
||||
|
||||
async def test_save_replaces_a_corrupt_config_file(self):
|
||||
with open(self.config_path, "w") as f:
|
||||
f.write("{not json")
|
||||
async def test_save_refuses_a_corrupt_config_file(self):
|
||||
# #220: a config.json that cannot be parsed is the user's file, never
|
||||
# silently replaced; the save is refused and nothing is published.
|
||||
Path(self.config_path).write_bytes(b"{not json")
|
||||
before_timeout = PromptManagerConfig.RESULT_TIMEOUT
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/settings", json={"result_timeout": 4}
|
||||
)
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(self._read_config()["web_ui"]["result_timeout"], 4)
|
||||
body = await resp.json()
|
||||
self.assertEqual((resp.status, body["code"]), (409, "config-unreadable"))
|
||||
self.assertIs(body["success"], False)
|
||||
self.assertIs(body["persisted"], False)
|
||||
self.assertEqual(PromptManagerConfig.RESULT_TIMEOUT, before_timeout)
|
||||
self.assertEqual(Path(self.config_path).read_bytes(), b"{not json")
|
||||
|
||||
async def test_get_settings_reports_an_unreadable_config(self):
|
||||
Path(self.config_path).write_bytes(b"{not json")
|
||||
resp = await self.client.request("GET", "/prompt_manager/settings")
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 200) # the legacy page still loads
|
||||
self.assertIsNone(body["revision"])
|
||||
self.assertEqual(body["config_error"], "config-unreadable")
|
||||
self.assertIn("result_timeout", body["settings"])
|
||||
|
||||
async def test_saved_config_file_is_private_to_the_user(self):
|
||||
import stat
|
||||
@@ -1107,12 +1123,156 @@ class TestSettingsMisc(AdminAPITestCase):
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertFalse(data["restart_required"])
|
||||
|
||||
async def test_unwritable_config_path_still_succeeds(self):
|
||||
os.environ[CONFIG_PATH_ENV] = os.path.join(self.tmpdir, "missing", "c.json")
|
||||
async def test_unwritable_config_path_reports_failure_without_runtime_change(self):
|
||||
# #220: the parent is a file, so config.json cannot be written. The
|
||||
# save reports the failure and publishes nothing.
|
||||
blocker = Path(self.tmpdir) / "not-a-dir"
|
||||
blocker.write_text("x")
|
||||
os.environ[CONFIG_PATH_ENV] = str(blocker / "c.json")
|
||||
GalleryConfig.MONITORING_DIRECTORIES = []
|
||||
before_timeout = PromptManagerConfig.RESULT_TIMEOUT
|
||||
with self.assertLogs("prompt_manager.api", level="ERROR"):
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/settings",
|
||||
json={
|
||||
"result_timeout": before_timeout + 1,
|
||||
"gallery_root_paths": [self._root(self.output_dir)],
|
||||
},
|
||||
)
|
||||
body = await resp.json()
|
||||
self.assertEqual(
|
||||
(resp.status, body["code"], body["persisted"]),
|
||||
(500, "config-write-failed", False),
|
||||
)
|
||||
self.assertIs(body["success"], False)
|
||||
self.assertIs(body["runtime_applied"], False)
|
||||
self.assertEqual(PromptManagerConfig.RESULT_TIMEOUT, before_timeout)
|
||||
self.assertEqual(GalleryConfig.MONITORING_DIRECTORIES, [])
|
||||
self.assertEqual(blocker.read_text(), "x")
|
||||
|
||||
async def test_save_reports_the_commit(self):
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/settings", json={"result_timeout": 3}
|
||||
)
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(body["message"], "Settings saved successfully")
|
||||
self.assertIs(body["persisted"], True)
|
||||
self.assertIs(body["runtime_applied"], True)
|
||||
self.assertIs(body["restart_required"], False)
|
||||
self.assertEqual(body["restart_required_fields"], [])
|
||||
digest = hashlib.sha256(Path(self.config_path).read_bytes()).hexdigest()
|
||||
self.assertEqual(body["revision"], "sha256:" + digest)
|
||||
|
||||
async def test_changed_roots_name_the_restart_field(self):
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/settings",
|
||||
json={"gallery_root_paths": [self._root(self.output_dir)]},
|
||||
)
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIs(body["restart_required"], True)
|
||||
self.assertEqual(body["restart_required_fields"], ["gallery_root_paths"])
|
||||
|
||||
async def test_runtime_publication_failure_after_commit_is_reported(self):
|
||||
# The values are durable; the response must not claim they are live.
|
||||
self.api.reset_gallery_roots_cache = _raise
|
||||
with self.assertLogs("prompt_manager.api", level="ERROR"):
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/settings",
|
||||
json={"gallery_root_paths": [self._root(self.output_dir)]},
|
||||
)
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIs(body["persisted"], True)
|
||||
self.assertIs(body["runtime_applied"], False)
|
||||
self.assertIs(body["restart_required"], True)
|
||||
self.assertEqual(body["restart_required_fields"], ["gallery_root_paths"])
|
||||
self.assertEqual(
|
||||
self._read_config()["gallery"]["monitoring"]["directories"],
|
||||
[self._root(self.output_dir)],
|
||||
)
|
||||
|
||||
async def test_get_settings_reports_revision(self):
|
||||
resp = await self.client.request("GET", "/prompt_manager/settings")
|
||||
body = await resp.json()
|
||||
self.assertEqual(body["revision"], "absent")
|
||||
self.assertNotIn("config_error", body)
|
||||
|
||||
self._write_config({"web_ui": {"result_timeout": 2}})
|
||||
body = await (
|
||||
await self.client.request("GET", "/prompt_manager/settings")
|
||||
).json()
|
||||
digest = hashlib.sha256(Path(self.config_path).read_bytes()).hexdigest()
|
||||
self.assertEqual(body["revision"], "sha256:" + digest)
|
||||
|
||||
async def test_native_save_with_stale_revision_is_409(self):
|
||||
self._write_config({"web_ui": {"result_timeout": 2}})
|
||||
body = await (
|
||||
await self.client.request("GET", "/prompt_manager/settings")
|
||||
).json()
|
||||
seen = body["revision"]
|
||||
self._write_config({"web_ui": {"result_timeout": 6}}) # another writer
|
||||
before = Path(self.config_path).read_bytes()
|
||||
before_timeout = PromptManagerConfig.RESULT_TIMEOUT
|
||||
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/settings",
|
||||
json={"result_timeout": 9, "expected_revision": seen},
|
||||
)
|
||||
body = await resp.json()
|
||||
self.assertEqual((resp.status, body["code"]), (409, "settings-changed"))
|
||||
self.assertIs(body["persisted"], False)
|
||||
digest = hashlib.sha256(before).hexdigest()
|
||||
self.assertEqual(body["revision"], "sha256:" + digest)
|
||||
self.assertEqual(Path(self.config_path).read_bytes(), before)
|
||||
self.assertEqual(PromptManagerConfig.RESULT_TIMEOUT, before_timeout)
|
||||
|
||||
async def test_native_save_with_current_revision_commits(self):
|
||||
body = await (
|
||||
await self.client.request("GET", "/prompt_manager/settings")
|
||||
).json()
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/settings",
|
||||
json={"result_timeout": 9, "expected_revision": body["revision"]},
|
||||
)
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(self._read_config()["web_ui"]["result_timeout"], 9)
|
||||
|
||||
async def test_expected_revision_must_be_a_string(self):
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/settings",
|
||||
json={"result_timeout": 9, "expected_revision": 5},
|
||||
)
|
||||
self.assertEqual(resp.status, 400)
|
||||
self.assertFalse(os.path.exists(self.config_path))
|
||||
|
||||
async def test_legacy_partial_update_without_revision_still_merges(self):
|
||||
self._write_config(
|
||||
{
|
||||
"integrations": {"lora_manager": {"civitai_api_key": "secret"}},
|
||||
"web_ui": {"result_timeout": 1, "webui_display_mode": "popup"},
|
||||
"custom_section": {"keep": True},
|
||||
}
|
||||
)
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/settings", json={"result_timeout": 8}
|
||||
)
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertNotIn("secret", await resp.text())
|
||||
saved = self._read_config()
|
||||
self.assertEqual(
|
||||
saved["integrations"]["lora_manager"]["civitai_api_key"], "secret"
|
||||
)
|
||||
self.assertEqual(saved["custom_section"], {"keep": True})
|
||||
self.assertEqual(saved["web_ui"]["result_timeout"], 8)
|
||||
self.assertEqual(saved["web_ui"]["webui_display_mode"], "popup") # not sent
|
||||
|
||||
async def test_save_unexpected_error_is_500(self):
|
||||
self.api._parse_gallery_roots = _raise
|
||||
|
||||
@@ -5,12 +5,16 @@ stream the upload to disk, reject anything that is not a healthy prompts
|
||||
database and leave the live database untouched when it does.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import glob
|
||||
import hashlib
|
||||
import os
|
||||
import sqlite3
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
_mock_server = MagicMock()
|
||||
@@ -142,6 +146,85 @@ class TestBackupRoute(BackupRestoreTestCase):
|
||||
self.assertIn("backup", data["error"].lower())
|
||||
|
||||
|
||||
class TestRestoreHoldsTheRegistry(BackupRestoreTestCase):
|
||||
"""A legacy restore and a workspace job refuse each other (#269 gap)."""
|
||||
|
||||
def _digest(self):
|
||||
return {
|
||||
suffix: hashlib.sha256(Path(self.db_path + suffix).read_bytes()).hexdigest()
|
||||
for suffix in ("", "-wal")
|
||||
if os.path.exists(self.db_path + suffix)
|
||||
}
|
||||
|
||||
async def test_legacy_restore_refused_while_scan_job_runs(self):
|
||||
self._save("keep me")
|
||||
upload = self._make_db_file(["restored"])
|
||||
gate = asyncio.Event()
|
||||
|
||||
async def runner(job):
|
||||
await gate.wait()
|
||||
|
||||
scan = self.api._jobs.start(
|
||||
"scan", "Scan for metadata", record_changing=True, scope={}, runner=runner
|
||||
)
|
||||
before = self._digest()
|
||||
try:
|
||||
resp = await self._restore(upload)
|
||||
self.assertEqual(resp.status, 409)
|
||||
self.assertEqual(
|
||||
await resp.json(),
|
||||
{"success": False, "error": "Scan for metadata already running"},
|
||||
)
|
||||
self.assertEqual(self._digest(), before)
|
||||
self.assertEqual(
|
||||
[p["text"] for p in self.api.db.search_prompts()], ["keep me"]
|
||||
)
|
||||
self.assertEqual(glob.glob(self.db_path + ".backup_*"), [])
|
||||
finally:
|
||||
gate.set()
|
||||
await asyncio.wait_for(scan.task, 5)
|
||||
|
||||
async def test_scan_refused_while_legacy_restore_holds(self):
|
||||
upload = self._make_db_file(["restored"])
|
||||
entered = threading.Event()
|
||||
release = threading.Event()
|
||||
model = self.api.db.model
|
||||
real_verify = model.verify_database_file
|
||||
|
||||
def gated(path, **kwargs):
|
||||
entered.set()
|
||||
release.wait(10)
|
||||
return real_verify(path, **kwargs)
|
||||
|
||||
loop = asyncio.get_running_loop()
|
||||
with patch.object(model, "verify_database_file", gated):
|
||||
restore = asyncio.ensure_future(self._restore(upload))
|
||||
self.assertTrue(await loop.run_in_executor(None, entered.wait, 10))
|
||||
resp = await self.client.post("/prompt_manager/jobs/scan")
|
||||
body = await resp.json()
|
||||
listed = (await (await self.client.get("/prompt_manager/jobs")).json())[
|
||||
"jobs"
|
||||
]
|
||||
release.set()
|
||||
restored = await restore
|
||||
|
||||
self.assertEqual((resp.status, body["code"]), (409, "busy"))
|
||||
self.assertEqual(body["running"]["title"], "Database restore (admin page)")
|
||||
self.assertEqual(
|
||||
[(j["kind"], j["state"], j["cancellable"]) for j in listed],
|
||||
[("restore", "running", False)],
|
||||
)
|
||||
self.assertEqual(restored.status, 200)
|
||||
self.assertEqual(self.api._jobs.list()[0].state, "finished")
|
||||
self.assertIsNone(self.api._jobs.active("restore"))
|
||||
|
||||
async def test_rejected_upload_ends_its_hold_failed(self):
|
||||
resp = await self._restore(self._write("notes.txt", b"not a database"))
|
||||
self.assertEqual(resp.status, 400)
|
||||
job = self.api._jobs.list()[0]
|
||||
self.assertEqual((job.kind, job.state), ("restore", "failed"))
|
||||
|
||||
|
||||
class TestRestoreRoute(BackupRestoreTestCase):
|
||||
async def test_restore_replaces_database_and_reports_backup(self):
|
||||
self._save("old prompt")
|
||||
|
||||
@@ -462,6 +462,15 @@ class TestUiRoutes(CoreRoutesTestCase):
|
||||
):
|
||||
self.assertIn(entry, table)
|
||||
|
||||
async def test_image_tag_job_route_is_registered_beside_the_scan_job(self):
|
||||
table = {
|
||||
(route.method, route.resource.canonical)
|
||||
for route in self.app.router.routes()
|
||||
if route.resource is not None
|
||||
}
|
||||
self.assertIn(("POST", "/prompt_manager/jobs/scan"), table)
|
||||
self.assertIn(("POST", "/prompt_manager/jobs/image-tags"), table)
|
||||
|
||||
|
||||
class TestStaticRoutes(CoreRoutesTestCase):
|
||||
|
||||
|
||||
@@ -5,11 +5,13 @@ Uses aiohttp's test client against a real PromptManagerAPI wired to a
|
||||
temporary SQLite database and a temporary output directory.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
from contextlib import ExitStack
|
||||
from pathlib import Path
|
||||
@@ -638,6 +640,93 @@ class TestGenerateThumbnailsPost(ThumbnailTestCase):
|
||||
self.assertEqual(data["total_images"], 0)
|
||||
|
||||
|
||||
class TestThumbnailRoutesHoldTheRegistry(ThumbnailTestCase):
|
||||
"""Thumbnail work is ``guarded``: never beside a record-changing job."""
|
||||
|
||||
def _run_scan(self):
|
||||
gate = asyncio.Event()
|
||||
|
||||
async def runner(job):
|
||||
await gate.wait()
|
||||
|
||||
job = self.api._jobs.start(
|
||||
"scan", "Scan for metadata", admission="records", scope={}, runner=runner
|
||||
)
|
||||
|
||||
async def release():
|
||||
gate.set()
|
||||
await asyncio.wait_for(job.task, 5)
|
||||
|
||||
self.addAsyncCleanup(release)
|
||||
return job
|
||||
|
||||
async def test_thumbnail_generate_refused_while_scan_runs(self):
|
||||
make_png(self.output_dir / "a.png")
|
||||
self._run_scan()
|
||||
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/images/generate-thumbnails", json={}
|
||||
)
|
||||
self.assertEqual(resp.status, 409)
|
||||
self.assertEqual(
|
||||
await resp.json(),
|
||||
{"success": False, "error": "Scan for metadata already running"},
|
||||
)
|
||||
|
||||
resp = await self.client.request("POST", self.PROGRESS_URL)
|
||||
text = await resp.text()
|
||||
self.assertEqual(resp.status, 200) # the gallery reads an SSE stream
|
||||
self.assertEqual(
|
||||
text,
|
||||
"event: error\ndata: "
|
||||
+ json.dumps(
|
||||
{
|
||||
"error": "Scan for metadata already running",
|
||||
"message": "Scan for metadata already running",
|
||||
}
|
||||
)
|
||||
+ "\n\n",
|
||||
)
|
||||
|
||||
thumbs = self.output_dir / "thumbnails"
|
||||
make_png(thumbs / "b_thumb.png")
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/images/clear-thumbnails"
|
||||
)
|
||||
self.assertEqual(resp.status, 409)
|
||||
self.assertFalse(self._thumb_path("a").exists())
|
||||
self.assertTrue((thumbs / "b_thumb.png").exists())
|
||||
|
||||
async def test_scan_refused_while_thumbnails_generate(self):
|
||||
make_png(self.output_dir / "a.png")
|
||||
entered = threading.Event()
|
||||
release = threading.Event()
|
||||
real = self.api._generate_gallery_thumbnails_sync
|
||||
|
||||
def gated(*args):
|
||||
entered.set()
|
||||
release.wait(10)
|
||||
return real(*args)
|
||||
|
||||
self.api._generate_gallery_thumbnails_sync = gated
|
||||
loop = asyncio.get_running_loop()
|
||||
generate = asyncio.ensure_future(
|
||||
self.client.request(
|
||||
"POST", "/prompt_manager/images/generate-thumbnails", json={}
|
||||
)
|
||||
)
|
||||
self.assertTrue(await loop.run_in_executor(None, entered.wait, 10))
|
||||
resp = await self.client.request("POST", "/prompt_manager/jobs/scan")
|
||||
body = await resp.json()
|
||||
self.assertEqual((resp.status, body["code"]), (409, "busy"))
|
||||
self.assertEqual(
|
||||
body["running"]["title"], "Thumbnail generation (gallery page)"
|
||||
)
|
||||
release.set()
|
||||
self.assertEqual((await generate).status, 200)
|
||||
self.assertTrue(self._thumb_path("a").exists())
|
||||
|
||||
|
||||
class TestRecentImagesCaps(ImageAPITestCase):
|
||||
"""Every path through /images/recent hands the DB a bounded window."""
|
||||
|
||||
|
||||
+340
-12
@@ -4,11 +4,13 @@ Filesystem detection and network downloads are patched out so the tests run
|
||||
anywhere without a LoraManager install or a CivitAI connection.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
@@ -34,11 +36,20 @@ from aiohttp.test_utils import AioHTTPTestCase
|
||||
|
||||
from database.operations import PromptDatabase
|
||||
from py.api import PromptManagerAPI
|
||||
from py import lora_utils
|
||||
from py import config_store, lora_utils
|
||||
from py.config import IntegrationConfig, PromptManagerConfig
|
||||
from py.api import settings_routes as settings_module
|
||||
from py.api import lora_integration as lora_module
|
||||
from py.lora_utils import get_trigger_cache
|
||||
from utils.hashing import generate_prompt_hash
|
||||
|
||||
try:
|
||||
from tests.lock_probe import ProbeLock
|
||||
from tests.open_handles import assert_closed
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from lock_probe import ProbeLock
|
||||
from open_handles import assert_closed
|
||||
|
||||
SECRET_KEY = "sk-civitai-super-secret-0123456789"
|
||||
|
||||
|
||||
@@ -56,7 +67,13 @@ class LoraAPITestCase(AioHTTPTestCase):
|
||||
return_value=[self.custom_nodes],
|
||||
)
|
||||
self._custom_nodes_patch.start()
|
||||
self._real_save = PromptManagerConfig.save_to_file
|
||||
# Never the repository's config.json (live user data): every save in
|
||||
# these tests goes to a file in this test's temp directory.
|
||||
self.config_file = self.tmp_root / "config.json"
|
||||
self._env_patch = patch.dict(
|
||||
os.environ, {"PROMPT_MANAGER_CONFIG_PATH": str(self.config_file)}
|
||||
)
|
||||
self._env_patch.start()
|
||||
|
||||
self._config_patch = patch.multiple(
|
||||
IntegrationConfig,
|
||||
@@ -70,8 +87,6 @@ class LoraAPITestCase(AioHTTPTestCase):
|
||||
"py.lora_utils.detect_lora_manager", return_value=None
|
||||
)
|
||||
self.detect_mock = self._detect_patch.start()
|
||||
self._save_patch = patch.object(PromptManagerConfig, "save_to_file")
|
||||
self.save_mock = self._save_patch.start()
|
||||
get_trigger_cache().clear()
|
||||
|
||||
app = web.Application()
|
||||
@@ -83,7 +98,7 @@ class LoraAPITestCase(AioHTTPTestCase):
|
||||
|
||||
async def tearDownAsync(self):
|
||||
await super().tearDownAsync() # closes the aiohttp test client
|
||||
self._save_patch.stop()
|
||||
self._env_patch.stop()
|
||||
self._detect_patch.stop()
|
||||
self._config_patch.stop()
|
||||
self._custom_nodes_patch.stop()
|
||||
@@ -96,8 +111,19 @@ class LoraAPITestCase(AioHTTPTestCase):
|
||||
):
|
||||
if os.path.exists(path):
|
||||
os.unlink(path)
|
||||
assert_closed(self, self.tmp_root)
|
||||
shutil.rmtree(self.tmp_root, ignore_errors=True)
|
||||
|
||||
def _saved(self):
|
||||
return json.loads(self.config_file.read_text())
|
||||
|
||||
def _make_unwritable(self):
|
||||
"""Point the config path under a regular file, so no write can succeed."""
|
||||
blocker = self.tmp_root / "not-a-dir"
|
||||
blocker.write_text("x")
|
||||
self.config_file = blocker / "config.json"
|
||||
os.environ["PROMPT_MANAGER_CONFIG_PATH"] = str(self.config_file)
|
||||
|
||||
async def _post_json(self, url, body):
|
||||
return await self.client.request(
|
||||
"POST",
|
||||
@@ -250,7 +276,7 @@ class TestLoraEnableRoute(LoraAPITestCase):
|
||||
)
|
||||
self.assertEqual(resp.status, 400)
|
||||
self.assertFalse(IntegrationConfig.LORA_MANAGER_ENABLED)
|
||||
self.save_mock.assert_not_called()
|
||||
self.assertFalse(self.config_file.exists())
|
||||
|
||||
async def test_enable_persists_and_loads_trigger_cache(self):
|
||||
lm = _make_lora_manager(self.custom_nodes)
|
||||
@@ -285,7 +311,15 @@ class TestLoraEnableRoute(LoraAPITestCase):
|
||||
os.path.normcase(os.path.realpath(str(lm))),
|
||||
)
|
||||
self.assertEqual(IntegrationConfig.CIVITAI_API_KEY, SECRET_KEY)
|
||||
self.save_mock.assert_called_once_with()
|
||||
saved = self._saved()["integrations"]["lora_manager"]
|
||||
self.assertEqual(saved["civitai_api_key"], SECRET_KEY)
|
||||
self.assertEqual(saved["path"], IntegrationConfig.LORA_MANAGER_PATH)
|
||||
self.assertIs(saved["enabled"], True)
|
||||
self.assertIs(saved["trigger_words_enabled"], True)
|
||||
self.assertEqual(
|
||||
(data["persisted"], data["runtime_applied"], data["cache_loaded"]),
|
||||
(True, True, True),
|
||||
)
|
||||
self.assertTrue(get_trigger_cache().is_loaded)
|
||||
self.assertEqual(get_trigger_cache().get_trigger_words("neon"), ["glow"])
|
||||
|
||||
@@ -319,7 +353,7 @@ class TestLoraEnableRoute(LoraAPITestCase):
|
||||
self.assertFalse(json.loads(body)["success"])
|
||||
self.assertNotIn(str(self.tmp_root), body)
|
||||
self.assertFalse(IntegrationConfig.LORA_MANAGER_ENABLED)
|
||||
self.save_mock.assert_not_called()
|
||||
self.assertFalse(self.config_file.exists())
|
||||
|
||||
async def test_enable_with_custom_node_lacking_lora_in_its_name_is_400(self):
|
||||
lm = _make_lora_manager(self.custom_nodes).parent / "ComfyUI-Other"
|
||||
@@ -344,7 +378,6 @@ class TestLoraEnableRoute(LoraAPITestCase):
|
||||
|
||||
async def test_enable_writes_the_configured_config_file(self):
|
||||
config_file = self.tmp_root / "cfg" / "config.json"
|
||||
self.save_mock.side_effect = self._real_save
|
||||
|
||||
with patch.dict(os.environ, {"PROMPT_MANAGER_CONFIG_PATH": str(config_file)}):
|
||||
resp = await self._post_json(
|
||||
@@ -353,7 +386,6 @@ class TestLoraEnableRoute(LoraAPITestCase):
|
||||
)
|
||||
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.save_mock.assert_called_once_with()
|
||||
self.assertTrue(config_file.is_file())
|
||||
saved = json.loads(config_file.read_text())["integrations"]["lora_manager"]
|
||||
self.assertFalse(saved["enabled"])
|
||||
@@ -414,9 +446,299 @@ class TestLoraEnableRoute(LoraAPITestCase):
|
||||
self.assertEqual(resp.status, 400)
|
||||
|
||||
async def test_persist_failure_is_500(self):
|
||||
self.save_mock.side_effect = OSError("read-only")
|
||||
resp = await self._post_json("/prompt_manager/lora/enable", {"enabled": False})
|
||||
self._make_unwritable()
|
||||
with self.assertLogs("prompt_manager.api", level="ERROR"):
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable", {"enabled": False}
|
||||
)
|
||||
self.assertEqual(resp.status, 500)
|
||||
self.assertEqual((await resp.json())["code"], "config-write-failed")
|
||||
|
||||
async def test_enable_write_failure_reports_and_keeps_runtime(self):
|
||||
# #220: a save that cannot be written publishes nothing and says so.
|
||||
lm = _make_lora_manager(self.custom_nodes)
|
||||
self.detect_mock.return_value = str(lm)
|
||||
IntegrationConfig.CIVITAI_API_KEY = SECRET_KEY
|
||||
self._make_unwritable()
|
||||
|
||||
with self.assertLogs("prompt_manager.api", level="ERROR"):
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{
|
||||
"enabled": True,
|
||||
"path": str(lm),
|
||||
"trigger_words_enabled": True,
|
||||
"clear_civitai_api_key": True,
|
||||
},
|
||||
)
|
||||
|
||||
text = await resp.text()
|
||||
body = json.loads(text)
|
||||
self.assertEqual(resp.status, 500)
|
||||
self.assertEqual(body["code"], "config-write-failed")
|
||||
self.assertEqual((body["persisted"], body["runtime_applied"]), (False, False))
|
||||
self.assertFalse(IntegrationConfig.LORA_MANAGER_ENABLED)
|
||||
self.assertEqual(IntegrationConfig.LORA_MANAGER_PATH, "")
|
||||
self.assertFalse(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED)
|
||||
self.assertEqual(IntegrationConfig.CIVITAI_API_KEY, SECRET_KEY)
|
||||
self.assertFalse(get_trigger_cache().is_loaded)
|
||||
self.assertNotIn(SECRET_KEY, text)
|
||||
self.assertNotIn(str(self.tmp_root), text)
|
||||
|
||||
async def test_enable_preserves_unknown_sections(self):
|
||||
stored = "stored-value" # a plain marker: what is on disk must stay
|
||||
self.config_file.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"custom_section": {"keep": True},
|
||||
"web_ui": {"result_timeout": 11},
|
||||
"integrations": {
|
||||
"lora_manager": {"civitai_api_key": stored, "extra": 1},
|
||||
"other_tool": {"on": True},
|
||||
},
|
||||
}
|
||||
)
|
||||
)
|
||||
IntegrationConfig.CIVITAI_API_KEY = stored
|
||||
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": False, "trigger_words_enabled": True},
|
||||
)
|
||||
|
||||
self.assertEqual(resp.status, 200)
|
||||
saved = self._saved()
|
||||
self.assertEqual(saved["custom_section"], {"keep": True})
|
||||
self.assertEqual(saved["web_ui"], {"result_timeout": 11})
|
||||
self.assertEqual(saved["integrations"]["other_tool"], {"on": True})
|
||||
lora = saved["integrations"]["lora_manager"]
|
||||
self.assertEqual(lora["extra"], 1)
|
||||
self.assertEqual(lora["civitai_api_key"], stored)
|
||||
self.assertIs(lora["trigger_words_enabled"], True)
|
||||
self.assertNotIn("database", saved) # no runtime defaults written
|
||||
|
||||
async def test_enable_refuses_a_corrupt_config_file(self):
|
||||
self.config_file.write_bytes(b"{not json")
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": False, "trigger_words_enabled": True},
|
||||
)
|
||||
body = await resp.json()
|
||||
self.assertEqual((resp.status, body["code"]), (409, "config-unreadable"))
|
||||
self.assertEqual(self.config_file.read_bytes(), b"{not json")
|
||||
self.assertFalse(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED)
|
||||
|
||||
async def test_enable_requires_boolean_flags_and_body_cap(self):
|
||||
for body in (
|
||||
{"enabled": "yes"},
|
||||
{"enabled": False, "trigger_words_enabled": 1},
|
||||
{"enabled": False, "clear_civitai_api_key": "true"},
|
||||
):
|
||||
with self.subTest(body=body):
|
||||
resp = await self._post_json("/prompt_manager/lora/enable", body)
|
||||
self.assertEqual(resp.status, 400)
|
||||
resp = await self._post_json("/prompt_manager/lora/enable", [True])
|
||||
self.assertEqual(resp.status, 400)
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": False, "civitai_api_key": "k" * (2 * 1024 * 1024)},
|
||||
)
|
||||
self.assertEqual(resp.status, 413)
|
||||
self.assertFalse(self.config_file.exists())
|
||||
self.assertFalse(IntegrationConfig.LORA_MANAGER_ENABLED)
|
||||
|
||||
async def test_blank_key_unchanged_and_explicit_clear(self):
|
||||
self.config_file.write_text(
|
||||
json.dumps({"integrations": {"lora_manager": {"civitai_api_key": "old"}}})
|
||||
)
|
||||
IntegrationConfig.CIVITAI_API_KEY = "old"
|
||||
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable", {"enabled": False, "civitai_api_key": " "}
|
||||
)
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(
|
||||
self._saved()["integrations"]["lora_manager"]["civitai_api_key"], "old"
|
||||
)
|
||||
self.assertEqual(IntegrationConfig.CIVITAI_API_KEY, "old")
|
||||
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": False, "clear_civitai_api_key": True},
|
||||
)
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(
|
||||
self._saved()["integrations"]["lora_manager"]["civitai_api_key"], ""
|
||||
)
|
||||
self.assertEqual(IntegrationConfig.CIVITAI_API_KEY, "")
|
||||
|
||||
async def test_cache_load_failure_reports_persisted_but_unloaded(self):
|
||||
lm = _make_lora_manager(self.custom_nodes)
|
||||
self.detect_mock.return_value = str(lm)
|
||||
|
||||
with (
|
||||
patch.object(
|
||||
type(get_trigger_cache()), "load", side_effect=OSError("unreadable")
|
||||
),
|
||||
self.assertLogs("prompt_manager.api", level="WARNING"),
|
||||
):
|
||||
resp = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": True, "path": str(lm), "trigger_words_enabled": True},
|
||||
)
|
||||
|
||||
text = await resp.text()
|
||||
body = json.loads(text)
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIs(body["success"], True)
|
||||
self.assertEqual((body["persisted"], body["cache_loaded"]), (True, False))
|
||||
self.assertTrue(IntegrationConfig.LORA_MANAGER_ENABLED)
|
||||
self.assertIs(self._saved()["integrations"]["lora_manager"]["enabled"], True)
|
||||
self.assertNotIn("unreadable", text)
|
||||
|
||||
|
||||
class TestConfigWritersShareOneStore(LoraAPITestCase):
|
||||
|
||||
async def asyncSetUp(self):
|
||||
await super().asyncSetUp()
|
||||
self._orig_timeout = PromptManagerConfig.RESULT_TIMEOUT
|
||||
|
||||
async def asyncTearDown(self):
|
||||
PromptManagerConfig.RESULT_TIMEOUT = self._orig_timeout
|
||||
await super().asyncTearDown()
|
||||
|
||||
async def test_settings_and_lora_saves_do_not_overwrite_each_other(self):
|
||||
# #220: the LoRA save is sent while the settings save is inside its
|
||||
# commit (lock held, file already read). The LoRA commit re-reads the
|
||||
# file under the same lock, so both edits survive and nothing else
|
||||
# is lost (before, the LoRA save rewrote the whole runtime config).
|
||||
self.config_file.write_text(json.dumps({"custom_section": {"keep": True}}))
|
||||
inside = threading.Event()
|
||||
release = threading.Event()
|
||||
real_mutation = settings_module._settings_mutation
|
||||
|
||||
def gated_mutation(updates, new_roots):
|
||||
mutate = real_mutation(updates, new_roots)
|
||||
|
||||
def gated(config):
|
||||
inside.set()
|
||||
release.wait(10)
|
||||
return mutate(config)
|
||||
|
||||
return gated
|
||||
|
||||
lora_calls = []
|
||||
real_lora_mutation = lora_module._lora_settings_mutation
|
||||
|
||||
def recorded_lora_mutation(values):
|
||||
mutate = real_lora_mutation(values)
|
||||
|
||||
def recorded(config):
|
||||
lora_calls.append(dict(config))
|
||||
return mutate(config)
|
||||
|
||||
return recorded
|
||||
|
||||
probe = ProbeLock()
|
||||
loop = asyncio.get_running_loop()
|
||||
with (
|
||||
patch.object(settings_module, "_settings_mutation", gated_mutation),
|
||||
patch.object(
|
||||
lora_module, "_lora_settings_mutation", recorded_lora_mutation
|
||||
),
|
||||
patch.object(config_store, "_COMMIT_LOCK", probe),
|
||||
):
|
||||
settings = asyncio.ensure_future(
|
||||
self._post_json("/prompt_manager/settings", {"result_timeout": 13})
|
||||
)
|
||||
self.assertTrue(await loop.run_in_executor(None, inside.wait, 10))
|
||||
lora = asyncio.ensure_future(
|
||||
self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": False, "trigger_words_enabled": True},
|
||||
)
|
||||
)
|
||||
# Wait until the LoRA commit is blocked on the lock the settings
|
||||
# commit holds; its mutation must not have run yet.
|
||||
self.assertTrue(await loop.run_in_executor(None, probe.contended.wait, 10))
|
||||
self.assertEqual(lora_calls, [])
|
||||
release.set()
|
||||
settings_resp, lora_resp = await asyncio.gather(settings, lora)
|
||||
self.assertEqual(len(lora_calls), 1)
|
||||
self.assertEqual(lora_calls[0]["web_ui"]["result_timeout"], 13)
|
||||
|
||||
self.assertEqual((settings_resp.status, lora_resp.status), (200, 200))
|
||||
saved = self._saved()
|
||||
self.assertEqual(saved["custom_section"], {"keep": True})
|
||||
self.assertEqual(saved["web_ui"]["result_timeout"], 13)
|
||||
self.assertIs(
|
||||
saved["integrations"]["lora_manager"]["trigger_words_enabled"], True
|
||||
)
|
||||
settings_body = await settings_resp.json()
|
||||
lora_body = await lora_resp.json()
|
||||
self.assertNotEqual(settings_body["revision"], lora_body["revision"])
|
||||
|
||||
def _delay_first_commit_result(self):
|
||||
"""Hold back the first config commit's result from its handler.
|
||||
|
||||
The commit itself has run (the file is written); only the handler's
|
||||
continuation waits until the returned event is set. Before #220's fix
|
||||
round the handler published after this point, so a later save could
|
||||
publish first and the earlier one overwrite it at runtime.
|
||||
"""
|
||||
original = self.api._run_in_executor
|
||||
written = asyncio.Event()
|
||||
resume = asyncio.Event()
|
||||
delayed = []
|
||||
|
||||
async def run(func, *args, **kwargs):
|
||||
result = await original(func, *args, **kwargs)
|
||||
if getattr(func, "__func__", None) is config_store.ConfigStore.commit:
|
||||
if not delayed:
|
||||
delayed.append(func)
|
||||
written.set()
|
||||
await resume.wait()
|
||||
return result
|
||||
|
||||
patcher = patch.object(self.api, "_run_in_executor", run)
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
return written, resume
|
||||
|
||||
async def test_concurrent_settings_saves_end_on_the_last_written(self):
|
||||
written, resume = self._delay_first_commit_result()
|
||||
first = asyncio.ensure_future(
|
||||
self._post_json("/prompt_manager/settings", {"result_timeout": 11})
|
||||
)
|
||||
await asyncio.wait_for(written.wait(), 10)
|
||||
second = await self._post_json(
|
||||
"/prompt_manager/settings", {"result_timeout": 22}
|
||||
)
|
||||
resume.set()
|
||||
first = await first
|
||||
self.assertEqual((first.status, second.status), (200, 200))
|
||||
self.assertEqual(self._saved()["web_ui"]["result_timeout"], 22)
|
||||
self.assertEqual(PromptManagerConfig.RESULT_TIMEOUT, 22)
|
||||
|
||||
async def test_concurrent_lora_saves_end_on_the_last_written(self):
|
||||
written, resume = self._delay_first_commit_result()
|
||||
first = asyncio.ensure_future(
|
||||
self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": False, "trigger_words_enabled": True},
|
||||
)
|
||||
)
|
||||
await asyncio.wait_for(written.wait(), 10)
|
||||
second = await self._post_json(
|
||||
"/prompt_manager/lora/enable",
|
||||
{"enabled": False, "trigger_words_enabled": False},
|
||||
)
|
||||
resume.set()
|
||||
first = await first
|
||||
self.assertEqual((first.status, second.status), (200, 200))
|
||||
lora = self._saved()["integrations"]["lora_manager"]
|
||||
self.assertIs(lora["trigger_words_enabled"], False)
|
||||
self.assertIs(IntegrationConfig.LORA_TRIGGER_WORDS_ENABLED, False)
|
||||
|
||||
|
||||
class TestLoraScanRoute(LoraAPITestCase):
|
||||
@@ -445,6 +767,12 @@ class TestLoraScanRoute(LoraAPITestCase):
|
||||
body = await resp.text()
|
||||
self.assertNotIn(str(self.tmp_root), body)
|
||||
self.assertIn("x.db", body)
|
||||
# The admin page's modal shows the failure as its status and closes.
|
||||
error = self._sse_events(body)[-1]
|
||||
self.assertEqual(
|
||||
(error["type"], error["status"], error["progress"]),
|
||||
("error", error["message"], 0),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _sse_events(text):
|
||||
|
||||
@@ -0,0 +1,419 @@
|
||||
"""Tests for the serialized config store (py/config_store.py, #220).
|
||||
|
||||
Every test works on a config.json inside its own temporary directory; the
|
||||
repository's config.json (live user data) is never read or written.
|
||||
"""
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
_mock_server = MagicMock()
|
||||
_mock_server.PromptServer.instance.routes = MagicMock()
|
||||
sys.modules.setdefault("server", _mock_server)
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from py import config_store # noqa: E402
|
||||
from py.config_store import ( # noqa: E402
|
||||
ABSENT_REVISION,
|
||||
ConfigStore,
|
||||
ConfigUnreadable,
|
||||
RevisionConflict,
|
||||
get_config_store,
|
||||
revision_of,
|
||||
)
|
||||
|
||||
try:
|
||||
from tests.lock_probe import ProbeLock # noqa: E402
|
||||
from tests.open_handles import assert_closed # noqa: E402
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from lock_probe import ProbeLock # noqa: E402
|
||||
from open_handles import assert_closed # noqa: E402
|
||||
|
||||
|
||||
class ConfigStoreTests(unittest.TestCase):
|
||||
def setUp(self):
|
||||
self.tmpdir = os.path.realpath(tempfile.mkdtemp())
|
||||
self.path = os.path.join(self.tmpdir, "config.json")
|
||||
self.store = ConfigStore(lambda: self.path)
|
||||
|
||||
def tearDown(self):
|
||||
try:
|
||||
assert_closed(self, self.tmpdir)
|
||||
finally:
|
||||
shutil.rmtree(self.tmpdir, ignore_errors=True)
|
||||
|
||||
def _write(self, data):
|
||||
Path(self.path).write_text(json.dumps(data), encoding="utf-8")
|
||||
|
||||
def _bytes(self):
|
||||
return Path(self.path).read_bytes()
|
||||
|
||||
def _leftovers(self):
|
||||
return sorted(name for name in os.listdir(self.tmpdir) if name != "config.json")
|
||||
|
||||
# ── read ──────────────────────────────────────────────────────────
|
||||
|
||||
def test_read_of_a_missing_file_is_empty_and_absent(self):
|
||||
self.assertEqual(self.store.read(), ({}, ABSENT_REVISION))
|
||||
|
||||
def test_read_reports_the_sha256_of_the_file_bytes(self):
|
||||
self._write({"web_ui": {"result_timeout": 3}})
|
||||
data, revision = self.store.read()
|
||||
self.assertEqual(data, {"web_ui": {"result_timeout": 3}})
|
||||
expected = "sha256:" + hashlib.sha256(self._bytes()).hexdigest()
|
||||
self.assertEqual(revision, expected)
|
||||
self.assertEqual(revision_of(self._bytes()), expected)
|
||||
|
||||
def test_non_object_config_is_unreadable(self):
|
||||
self._write([1, 2])
|
||||
with self.assertRaises(ConfigUnreadable):
|
||||
self.store.read()
|
||||
|
||||
def test_unreadable_path_is_unreadable(self):
|
||||
os.mkdir(self.path) # a directory where the file should be
|
||||
with self.assertRaises(ConfigUnreadable):
|
||||
self.store.read()
|
||||
|
||||
# ── commit ────────────────────────────────────────────────────────
|
||||
|
||||
def test_commit_preserves_unknown_sections_and_secret(self):
|
||||
self._write(
|
||||
{
|
||||
"custom_section": {"keep": True},
|
||||
"integrations": {"lora_manager": {"civitai_api_key": "secret"}},
|
||||
"web_ui": {"result_timeout": 1, "show_test_button": True},
|
||||
}
|
||||
)
|
||||
|
||||
def mutate(config):
|
||||
config.setdefault("web_ui", {})["result_timeout"] = 7
|
||||
return config
|
||||
|
||||
saved, revision = self.store.commit(mutate, expected_revision=None)
|
||||
|
||||
on_disk = json.loads(self._bytes())
|
||||
self.assertEqual(saved, on_disk)
|
||||
self.assertEqual(on_disk["custom_section"], {"keep": True})
|
||||
self.assertEqual(
|
||||
on_disk["integrations"]["lora_manager"]["civitai_api_key"], "secret"
|
||||
)
|
||||
self.assertEqual(
|
||||
on_disk["web_ui"], {"result_timeout": 7, "show_test_button": True}
|
||||
)
|
||||
self.assertEqual(revision, revision_of(self._bytes()))
|
||||
self.assertEqual(self.store.read(), (on_disk, revision))
|
||||
|
||||
def test_commit_creates_a_missing_file_and_its_parent(self):
|
||||
self.path = os.path.join(self.tmpdir, "nested", "config.json")
|
||||
saved, revision = self.store.commit(
|
||||
lambda config: {**config, "a": 1}, expected_revision=ABSENT_REVISION
|
||||
)
|
||||
self.assertEqual(saved, {"a": 1})
|
||||
self.assertEqual(json.loads(Path(self.path).read_text()), {"a": 1})
|
||||
self.assertNotEqual(revision, ABSENT_REVISION)
|
||||
|
||||
def test_mutate_works_on_a_copy(self):
|
||||
self._write({"web_ui": {"result_timeout": 1}})
|
||||
seen = {}
|
||||
|
||||
def mutate(config):
|
||||
seen["config"] = config
|
||||
config["web_ui"]["result_timeout"] = 2
|
||||
raise RuntimeError("validation failed late")
|
||||
|
||||
before = self._bytes()
|
||||
with self.assertRaises(RuntimeError):
|
||||
self.store.commit(mutate, expected_revision=None)
|
||||
self.assertEqual(self._bytes(), before)
|
||||
self.assertEqual(self.store.read()[0], {"web_ui": {"result_timeout": 1}})
|
||||
|
||||
def test_mutate_must_return_an_object(self):
|
||||
self._write({"a": 1})
|
||||
before = self._bytes()
|
||||
with self.assertRaises(TypeError):
|
||||
self.store.commit(lambda config: None, expected_revision=None)
|
||||
self.assertEqual(self._bytes(), before)
|
||||
|
||||
def test_malformed_existing_config_is_refused_and_untouched(self):
|
||||
Path(self.path).write_bytes(b"{not json")
|
||||
calls = []
|
||||
with self.assertRaises(ConfigUnreadable):
|
||||
self.store.commit(
|
||||
lambda config: calls.append(config) or config, expected_revision=None
|
||||
)
|
||||
self.assertEqual(calls, [])
|
||||
self.assertEqual(self._bytes(), b"{not json")
|
||||
self.assertEqual(self._leftovers(), [])
|
||||
|
||||
def test_write_failure_leaves_bytes_and_raises(self):
|
||||
self._write({"a": 1})
|
||||
before = self._bytes()
|
||||
with patch.object(config_store.os, "fsync", side_effect=OSError("disk full")):
|
||||
with self.assertRaises(OSError):
|
||||
self.store.commit(lambda c: {**c, "b": 2}, expected_revision=None)
|
||||
self.assertEqual(self._bytes(), before)
|
||||
self.assertEqual(self._leftovers(), [])
|
||||
|
||||
def test_unwritable_directory_raises_oserror(self):
|
||||
blocker = os.path.join(self.tmpdir, "afile")
|
||||
Path(blocker).write_text("x")
|
||||
self.path = os.path.join(blocker, "config.json") # parent is a file
|
||||
with self.assertRaises(OSError):
|
||||
self.store.commit(lambda c: {"a": 1}, expected_revision=None)
|
||||
|
||||
def test_stale_revision_conflicts_without_write(self):
|
||||
self._write({"a": 1})
|
||||
_data, revision = self.store.read()
|
||||
self._write({"a": 2}) # another writer got there first
|
||||
before = self._bytes()
|
||||
calls = []
|
||||
with self.assertRaises(RevisionConflict) as caught:
|
||||
self.store.commit(
|
||||
lambda c: calls.append(c) or c, expected_revision=revision
|
||||
)
|
||||
self.assertEqual(caught.exception.current, revision_of(before))
|
||||
self.assertEqual(calls, [])
|
||||
self.assertEqual(self._bytes(), before)
|
||||
|
||||
def test_matching_revision_commits(self):
|
||||
self._write({"a": 1})
|
||||
_data, revision = self.store.read()
|
||||
saved, _ = self.store.commit(
|
||||
lambda c: {**c, "b": 2}, expected_revision=revision
|
||||
)
|
||||
self.assertEqual(saved, {"a": 1, "b": 2})
|
||||
|
||||
def test_temp_file_removed_after_replace_failure(self):
|
||||
self._write({"a": 1})
|
||||
before = self._bytes()
|
||||
with patch.object(
|
||||
config_store.os, "replace", side_effect=PermissionError("locked")
|
||||
):
|
||||
with self.assertRaises(OSError):
|
||||
self.store.commit(lambda c: {**c, "b": 2}, expected_revision=None)
|
||||
self.assertEqual(self._bytes(), before)
|
||||
self.assertEqual(self._leftovers(), [])
|
||||
|
||||
@unittest.skipUnless(os.name == "posix", "POSIX file modes")
|
||||
def test_committed_file_is_private_to_the_user(self):
|
||||
import stat
|
||||
|
||||
self._write({"a": 1})
|
||||
os.chmod(self.path, 0o644)
|
||||
self.store.commit(lambda c: c, expected_revision=None)
|
||||
self.assertEqual(stat.S_IMODE(os.stat(self.path).st_mode), 0o600)
|
||||
|
||||
def _probe_lock(self):
|
||||
probe = ProbeLock()
|
||||
patcher = patch.object(config_store, "_COMMIT_LOCK", probe)
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
return probe
|
||||
|
||||
def _hold_lock_in_a_commit(self, value="first"):
|
||||
"""Start a commit that holds the lock until the returned event is set."""
|
||||
inside = threading.Event()
|
||||
release = threading.Event()
|
||||
errors = []
|
||||
|
||||
def slow(config):
|
||||
inside.set()
|
||||
release.wait(10)
|
||||
config[value] = 1
|
||||
return config
|
||||
|
||||
def run():
|
||||
try:
|
||||
self.store.commit(slow, expected_revision=None)
|
||||
except Exception as exc: # pragma: no cover - reported by the test
|
||||
errors.append(exc)
|
||||
|
||||
thread = threading.Thread(target=run)
|
||||
thread.start()
|
||||
self.assertTrue(inside.wait(10))
|
||||
return thread, release, errors
|
||||
|
||||
def test_concurrent_commits_serialize(self):
|
||||
# The second commit is observed blocked on the lock before the first
|
||||
# one may finish, so this fails whenever commit stops taking the lock.
|
||||
self._write({"keep": True})
|
||||
probe = self._probe_lock()
|
||||
first, release, errors = self._hold_lock_in_a_commit()
|
||||
calls = []
|
||||
|
||||
def fast(config):
|
||||
calls.append(dict(config))
|
||||
config["second"] = 2
|
||||
return config
|
||||
|
||||
second = threading.Thread(
|
||||
target=lambda: self.store.commit(fast, expected_revision=None)
|
||||
)
|
||||
second.start()
|
||||
self.assertTrue(probe.contended.wait(10), "second commit never waited")
|
||||
self.assertEqual(calls, []) # not called while the first holds the lock
|
||||
release.set()
|
||||
first.join(10)
|
||||
second.join(10)
|
||||
|
||||
self.assertEqual(errors, [])
|
||||
self.assertEqual(calls, [{"keep": True, "first": 1}]) # re-read inside
|
||||
self.assertEqual(
|
||||
json.loads(self._bytes()), {"keep": True, "first": 1, "second": 2}
|
||||
)
|
||||
|
||||
def test_read_waits_for_a_running_commit(self):
|
||||
# Windows cannot replace a file another handle has open, so a read
|
||||
# runs under the commit lock and never overlaps a replace.
|
||||
self._write({"keep": True})
|
||||
probe = self._probe_lock()
|
||||
first, release, errors = self._hold_lock_in_a_commit()
|
||||
seen = []
|
||||
reader = threading.Thread(target=lambda: seen.append(self.store.read()))
|
||||
reader.start()
|
||||
self.assertTrue(probe.contended.wait(10), "read did not take the lock")
|
||||
self.assertEqual(seen, [])
|
||||
release.set()
|
||||
first.join(10)
|
||||
reader.join(10)
|
||||
self.assertEqual(errors, [])
|
||||
self.assertEqual(seen[0][0], {"keep": True, "first": 1})
|
||||
self.assertEqual(seen[0][1], revision_of(self._bytes()))
|
||||
|
||||
def test_replace_is_retried_after_a_transient_permission_error(self):
|
||||
# Windows: an antivirus or indexer handle makes os.replace fail with
|
||||
# PermissionError for a moment; the commit retries before failing.
|
||||
self._write({"a": 1})
|
||||
real_replace = os.replace
|
||||
attempts = []
|
||||
|
||||
def flaky(src, dst):
|
||||
attempts.append(dst)
|
||||
if len(attempts) == 1:
|
||||
raise PermissionError("in use by another process")
|
||||
return real_replace(src, dst)
|
||||
|
||||
with (
|
||||
patch.object(config_store, "_REPLACE_ATTEMPTS", 5),
|
||||
patch.object(config_store.os, "replace", side_effect=flaky),
|
||||
patch.object(config_store.time, "sleep") as sleep,
|
||||
):
|
||||
saved, _ = self.store.commit(
|
||||
lambda c: {**c, "b": 2}, expected_revision=None
|
||||
)
|
||||
self.assertEqual(saved, {"a": 1, "b": 2})
|
||||
self.assertEqual(json.loads(self._bytes()), {"a": 1, "b": 2})
|
||||
self.assertEqual(len(attempts), 2)
|
||||
sleep.assert_called_once_with(config_store._REPLACE_BACKOFF_SECONDS)
|
||||
self.assertEqual(self._leftovers(), [])
|
||||
|
||||
def test_replace_gives_up_after_the_last_attempt(self):
|
||||
self._write({"a": 1})
|
||||
before = self._bytes()
|
||||
with (
|
||||
patch.object(config_store, "_REPLACE_ATTEMPTS", 3),
|
||||
patch.object(
|
||||
config_store.os, "replace", side_effect=PermissionError("locked")
|
||||
) as replace,
|
||||
patch.object(config_store.time, "sleep") as sleep,
|
||||
):
|
||||
with self.assertRaises(PermissionError):
|
||||
self.store.commit(lambda c: {**c, "b": 2}, expected_revision=None)
|
||||
self.assertEqual((replace.call_count, sleep.call_count), (3, 2))
|
||||
self.assertEqual(self._bytes(), before)
|
||||
self.assertEqual(self._leftovers(), [])
|
||||
|
||||
def test_replace_is_tried_once_outside_windows(self):
|
||||
expected = 5 if os.name == "nt" else 1
|
||||
self.assertEqual(config_store._REPLACE_ATTEMPTS, expected)
|
||||
|
||||
@unittest.skipUnless(hasattr(os, "symlink"), "needs symlinks")
|
||||
def test_symlinked_config_is_written_through_the_link(self):
|
||||
# A config.json that is a symlink (or a single-file bind mount) is
|
||||
# written beside its real target, so the link stays a link.
|
||||
real_dir = os.path.join(self.tmpdir, "real")
|
||||
os.mkdir(real_dir)
|
||||
target = os.path.join(real_dir, "settings.json")
|
||||
Path(target).write_text(json.dumps({"a": 1}), encoding="utf-8")
|
||||
try:
|
||||
os.symlink(target, self.path)
|
||||
except (OSError, NotImplementedError): # pragma: no cover - Windows
|
||||
self.skipTest("symlinks are not permitted here")
|
||||
|
||||
saved, revision = self.store.commit(
|
||||
lambda c: {**c, "b": 2}, expected_revision=None
|
||||
)
|
||||
|
||||
self.assertTrue(os.path.islink(self.path))
|
||||
self.assertEqual(json.loads(Path(target).read_text()), {"a": 1, "b": 2})
|
||||
self.assertEqual(self.store.read(), (saved, revision))
|
||||
self.assertEqual(os.listdir(real_dir), ["settings.json"])
|
||||
|
||||
def test_publish_runs_under_the_lock_after_the_write(self):
|
||||
# Runtimes publish inside the commit, so two saves publish in the
|
||||
# order they were written and the runtime ends on the last one.
|
||||
self._write({"a": 1})
|
||||
seen = []
|
||||
|
||||
# The lock is re-entrant, so another thread probes whether it is held.
|
||||
def probe_publish(config):
|
||||
result = []
|
||||
thread = threading.Thread(
|
||||
target=lambda: result.append(
|
||||
config_store._COMMIT_LOCK.acquire(blocking=False)
|
||||
)
|
||||
)
|
||||
thread.start()
|
||||
thread.join(10)
|
||||
seen.append((config, json.loads(self._bytes()), result[0]))
|
||||
|
||||
saved, _ = self.store.commit(
|
||||
lambda c: {**c, "b": 2}, expected_revision=None, publish=probe_publish
|
||||
)
|
||||
self.assertEqual(seen, [(saved, {"a": 1, "b": 2}, False)])
|
||||
|
||||
def test_publish_is_skipped_when_the_write_fails(self):
|
||||
self._write({"a": 1})
|
||||
published = []
|
||||
with patch.object(config_store.os, "fsync", side_effect=OSError("disk full")):
|
||||
with self.assertRaises(OSError):
|
||||
self.store.commit(
|
||||
lambda c: {**c, "b": 2},
|
||||
expected_revision=None,
|
||||
publish=published.append,
|
||||
)
|
||||
self.assertEqual(published, [])
|
||||
|
||||
def test_handles_closed_after_commit(self):
|
||||
self._write({"a": 1})
|
||||
self.store.read()
|
||||
self.store.commit(lambda c: {**c, "b": 2}, expected_revision=None)
|
||||
assert_closed(self, self.tmpdir)
|
||||
os.unlink(self.path) # Windows refuses this while a handle is open
|
||||
self.assertEqual(self._leftovers(), [])
|
||||
|
||||
|
||||
class SingletonTests(unittest.TestCase):
|
||||
def test_get_config_store_is_one_process_store_on_the_configured_path(self):
|
||||
store = get_config_store()
|
||||
self.assertIs(store, get_config_store())
|
||||
tmpdir = tempfile.mkdtemp()
|
||||
try:
|
||||
custom = os.path.join(tmpdir, "c.json")
|
||||
with patch.dict(os.environ, {"PROMPT_MANAGER_CONFIG_PATH": custom}):
|
||||
self.assertEqual(store.path, custom)
|
||||
finally:
|
||||
shutil.rmtree(tmpdir, ignore_errors=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,804 @@
|
||||
"""Bulk image tagging as an Activity job: Add tags and Auto-tag (#249 D1).
|
||||
|
||||
``POST /prompt_manager/jobs/image-tags`` starts one record-changing
|
||||
``image-tags`` job over 1..500 listed images. Add tags unions a fixed set
|
||||
into each file's byte record; Auto-tag runs one legacy engine on a private
|
||||
capture of each still image (under the shared model gate, never
|
||||
downloading) and unions its top N suggestions. No real model is loaded: the
|
||||
engine is B2's fake service.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import sqlite3
|
||||
import threading
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
try:
|
||||
from test_api_annotation_suggest import SCORED, WD14, FakeTagger, SuggestTestCase
|
||||
from test_api_images import make_png
|
||||
except ImportError:
|
||||
from tests.test_api_annotation_suggest import (
|
||||
SCORED,
|
||||
WD14,
|
||||
FakeTagger,
|
||||
SuggestTestCase,
|
||||
)
|
||||
from tests.test_api_images import make_png
|
||||
|
||||
from database.execution_history_schema import rotate_history_epoch
|
||||
from py import model_gate
|
||||
from py.api import jobs as jobs_module
|
||||
from py.config import AutotagConfig
|
||||
from utils.media_capture import CaptureCapacity
|
||||
|
||||
START = "/prompt_manager/jobs/image-tags"
|
||||
WAIT = 30.0
|
||||
KIND = getattr(jobs_module, "IMAGE_TAG_JOB_KIND", "image-tags")
|
||||
GROUPS = ("tagged", "unchanged", "skipped", "errors")
|
||||
|
||||
|
||||
class HookedTagger(FakeTagger):
|
||||
"""The B2 fake plus a hook that runs inside each inference."""
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.on_generate = None
|
||||
|
||||
def generate_tag_suggestions(self, image_path, **kwargs):
|
||||
result = super().generate_tag_suggestions(image_path, **kwargs)
|
||||
if self.on_generate is not None:
|
||||
self.on_generate(image_path)
|
||||
return result
|
||||
|
||||
|
||||
class ImageTagJobTestCase(SuggestTestCase):
|
||||
async def get_application(self):
|
||||
app = await super().get_application()
|
||||
self.tagger = HookedTagger()
|
||||
patcher = patch("py.autotag.get_autotag_service", return_value=self.tagger)
|
||||
patcher.start()
|
||||
self.addCleanup(patcher.stop)
|
||||
return app
|
||||
|
||||
async def tearDownAsync(self):
|
||||
# Every job of the test has ended before the app goes away.
|
||||
for job in self.api._jobs.list():
|
||||
if job.task is not None and not job.task.done():
|
||||
job.task.cancel()
|
||||
await asyncio.gather(job.task, return_exceptions=True)
|
||||
await super().tearDownAsync()
|
||||
|
||||
# -- helpers ---------------------------------------------------------
|
||||
|
||||
def png(self, path, size=(8, 6), color=(200, 120, 40)):
|
||||
make_png(self.output_dir / path, size=size, color=color)
|
||||
return self.output_dir / path
|
||||
|
||||
def digest(self, path):
|
||||
return hashlib.sha256((self.output_dir / path).read_bytes()).hexdigest()
|
||||
|
||||
def stored(self, path):
|
||||
return self.api.db.read_image_tags(self.locator(path), self.digest(path))
|
||||
|
||||
def add_body(self, paths, tags, **changes):
|
||||
return {
|
||||
"mode": "add",
|
||||
"targets": [self.target(p) for p in paths],
|
||||
"tags": list(tags),
|
||||
**changes,
|
||||
}
|
||||
|
||||
def auto_body(self, paths, model_type="wd14-vit", options=None, **changes):
|
||||
return {
|
||||
"mode": "auto",
|
||||
"targets": [self.target(p) for p in paths],
|
||||
"model_type": model_type,
|
||||
"options": dict(WD14) if options is None else options,
|
||||
**changes,
|
||||
}
|
||||
|
||||
async def start(self, body):
|
||||
response = await self.client.post(START, json=body)
|
||||
return response.status, await self.json_of(response)
|
||||
|
||||
async def start_ok(self, body):
|
||||
status, data = await self.start(body)
|
||||
self.assertEqual(status, 202, data)
|
||||
return self.api._jobs.get(data["job"]["id"])
|
||||
|
||||
async def run_job(self, body):
|
||||
job = await self.start_ok(body)
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
return job
|
||||
|
||||
def rows(self, job, group):
|
||||
return [(row["path"], row["detail"]) for row in job.outcomes[group]]
|
||||
|
||||
async def wait_for(self, event):
|
||||
ok = await asyncio.get_running_loop().run_in_executor(None, event.wait, WAIT)
|
||||
self.assertTrue(ok, "the fake engine was never entered")
|
||||
|
||||
|
||||
class TestAddTags(ImageTagJobTestCase):
|
||||
async def test_add_job_unions_tags_on_each_target_and_records_outcomes(self):
|
||||
for name in ("a.png", "b.png", "c.png"):
|
||||
self.png(name, color=(len(name), ord(name[0]), 9))
|
||||
self.saved_set("a.png", (self.output_dir / "a.png").read_bytes(), ["keep_me"])
|
||||
self.saved_set(
|
||||
"c.png", (self.output_dir / "c.png").read_bytes(), ["Sandstone", "desert"]
|
||||
)
|
||||
|
||||
job = await self.run_job(
|
||||
self.add_body(
|
||||
["a.png", "b.png", "c.png", "gone.png"], ["sandstone", "desert"]
|
||||
)
|
||||
)
|
||||
|
||||
self.assertEqual(job.kind, KIND)
|
||||
self.assertEqual(job.title, "Add tags to images")
|
||||
self.assertEqual(job.state, "finished")
|
||||
self.assertTrue(job.record_changing)
|
||||
self.assertEqual(job.scope, {"mode": "add", "images": 4})
|
||||
self.assertEqual(
|
||||
job.counters, {"tagged": 2, "unchanged": 1, "skipped": 1, "errors": 0}
|
||||
)
|
||||
self.assertEqual(
|
||||
self.stored("a.png")["tags"], ["keep_me", "sandstone", "desert"]
|
||||
)
|
||||
self.assertEqual(self.stored("b.png")["tags"], ["sandstone", "desert"])
|
||||
# The stored spelling wins; nothing new means no write.
|
||||
self.assertEqual(self.stored("c.png")["tags"], ["Sandstone", "desert"])
|
||||
self.assertEqual(self.stored("c.png")["tag_revision"], 1)
|
||||
self.assertEqual(
|
||||
self.rows(job, "tagged"),
|
||||
[("a.png", "Added 2 tags"), ("b.png", "Added 2 tags")],
|
||||
)
|
||||
self.assertEqual(self.rows(job, "unchanged"), [("c.png", "No new tags")])
|
||||
self.assertEqual(self.rows(job, "skipped"), [("gone.png", "File not found")])
|
||||
self.assertEqual(job.outcomes["tagged"][0]["root_index"], 0)
|
||||
self.assertEqual((job.processed, job.total, job.progress), (4, 4, 100))
|
||||
self.assertTrue(any("+ " in line and "a.png" in line for line in job.log))
|
||||
self.assertTrue(job.log[-1].endswith("Add tags finished · 4 files processed"))
|
||||
# Add tags never needs the model.
|
||||
self.assertEqual(self.tagger.load_calls, [])
|
||||
self.assertIsNone(model_gate.holder())
|
||||
|
||||
async def test_add_tags_vocabulary_names_become_global_tags(self):
|
||||
self.png("a.png")
|
||||
|
||||
await self.run_job(self.add_body(["a.png"], ["brand_new_tag"]))
|
||||
|
||||
names = set(self.api.db.get_all_tags())
|
||||
self.assertIn("brand_new_tag", names)
|
||||
|
||||
async def test_add_over_the_tag_cap_is_an_error_row_with_zero_writes(self):
|
||||
self.png("a.png")
|
||||
full = [f"t{i:03d}" for i in range(100)]
|
||||
self.saved_set("a.png", (self.output_dir / "a.png").read_bytes(), full)
|
||||
|
||||
job = await self.run_job(self.add_body(["a.png"], ["one_more"]))
|
||||
|
||||
self.assertEqual(job.counters["errors"], 1)
|
||||
self.assertEqual(
|
||||
self.rows(job, "errors"),
|
||||
[("a.png", "An image can carry at most 100 tags.")],
|
||||
)
|
||||
self.assertEqual(self.stored("a.png")["tags"], full)
|
||||
|
||||
async def test_video_skipped_in_auto_and_tagged_in_add(self):
|
||||
(self.output_dir / "clip.mp4").write_bytes(b"\x00\x00\x00\x18ftypmp42" * 8)
|
||||
|
||||
added = await self.run_job(self.add_body(["clip.mp4"], ["motion"]))
|
||||
self.assertEqual(self.rows(added, "tagged"), [("clip.mp4", "Added 1 tag")])
|
||||
self.assertEqual(self.stored("clip.mp4")["tags"], ["motion"])
|
||||
|
||||
auto = await self.run_job(self.auto_body(["clip.mp4"]))
|
||||
self.assertEqual(
|
||||
self.rows(auto, "skipped"), [("clip.mp4", "Video: manual tags only")]
|
||||
)
|
||||
self.assertEqual(self.tagger.calls, [])
|
||||
self.assertEqual(self.stored("clip.mp4")["tags"], ["motion"])
|
||||
|
||||
async def test_not_a_still_image_and_too_large_are_skipped(self):
|
||||
(self.output_dir / "broken.png").write_bytes(b"not an image")
|
||||
self.png("big.png")
|
||||
self.api._capture_store().max_file = 16
|
||||
|
||||
auto = await self.run_job(self.auto_body(["big.png"]))
|
||||
self.assertEqual(
|
||||
self.rows(auto, "skipped"), [("big.png", "Too large to verify")]
|
||||
)
|
||||
|
||||
self.api._capture_store().max_file = 2**20
|
||||
auto = await self.run_job(self.auto_body(["broken.png"]))
|
||||
self.assertEqual(
|
||||
self.rows(auto, "skipped"), [("broken.png", "Not a supported still image")]
|
||||
)
|
||||
|
||||
self.api._capture_store().max_file = 16
|
||||
added = await self.run_job(self.add_body(["big.png"], ["x"]))
|
||||
self.assertEqual(
|
||||
self.rows(added, "skipped"), [("big.png", "Too large to verify")]
|
||||
)
|
||||
self.assertEqual(self.tagger.calls, [])
|
||||
|
||||
|
||||
class TestAutoTag(ImageTagJobTestCase):
|
||||
async def test_auto_job_applies_the_requested_count_per_file_without_removing_existing( # noqa: E501
|
||||
self,
|
||||
):
|
||||
self.png("a.png")
|
||||
self.png("b.png", color=(1, 2, 3))
|
||||
self.saved_set("a.png", (self.output_dir / "a.png").read_bytes(), ["keep_me"])
|
||||
options = {**WD14, "suggestion_limit": 2}
|
||||
|
||||
job = await self.run_job(self.auto_body(["a.png", "b.png"], options=options))
|
||||
|
||||
self.assertEqual(job.title, "Auto-tag images")
|
||||
self.assertEqual(
|
||||
job.scope, {"mode": "auto", "images": 2, "model_type": "wd14-vit"}
|
||||
)
|
||||
self.assertEqual(job.state, "finished")
|
||||
self.assertEqual(
|
||||
self.api.db.read_image_tags(self.locator("a.png"), self.digest("a.png"))[
|
||||
"tags"
|
||||
],
|
||||
["keep_me", "sandstone", "desert"],
|
||||
)
|
||||
self.assertEqual(self.stored("b.png")["tags"], ["sandstone", "desert"])
|
||||
self.assertEqual(
|
||||
job.counters, {"tagged": 2, "unchanged": 0, "skipped": 0, "errors": 0}
|
||||
)
|
||||
# Inference read the private captures only, never the current files.
|
||||
for path in self.tagger.read_paths:
|
||||
self.assertNotIn(str(self.output_dir), path)
|
||||
self.assertEqual([call["limit"] for call in self.tagger.calls], [2, 2])
|
||||
|
||||
# limit 3: at most three names are added to a file.
|
||||
self.png("c.png", color=(9, 9, 9))
|
||||
options = {**WD14, "suggestion_limit": 3}
|
||||
job = await self.run_job(self.auto_body(["c.png"], options=options))
|
||||
self.assertEqual(self.stored("c.png")["tags"], [name for name, _ in SCORED[:3]])
|
||||
self.assertEqual(self.rows(job, "tagged"), [("c.png", "Added 3 tags")])
|
||||
|
||||
# Omitted: the configured default.
|
||||
AutotagConfig.SUGGESTION_LIMIT = 4
|
||||
self.png("d.png", color=(7, 7, 7))
|
||||
await self.run_job(self.auto_body(["d.png"]))
|
||||
self.assertEqual(self.tagger.calls[-1]["limit"], 4)
|
||||
self.assertEqual(len(self.stored("d.png")["tags"]), 4)
|
||||
|
||||
# Suggestions the file already carries add nothing.
|
||||
job = await self.run_job(self.auto_body(["d.png"]))
|
||||
self.assertEqual(self.rows(job, "unchanged"), [("d.png", "No new tags")])
|
||||
|
||||
async def test_auto_job_holds_gate_and_never_downloads(self):
|
||||
self.png("a.png")
|
||||
self.png("b.png", color=(1, 2, 3))
|
||||
entered, proceed = threading.Event(), threading.Event()
|
||||
self.tagger.block = (entered, proceed)
|
||||
|
||||
job = await self.start_ok(self.auto_body(["a.png", "b.png"]))
|
||||
await self.wait_for(entered)
|
||||
holder = model_gate.holder()
|
||||
self.assertIsNotNone(holder)
|
||||
self.assertTrue(holder.startswith("image-tag-job:"), holder)
|
||||
# A legacy route cannot swap the model under the running job.
|
||||
response = await self.client.post("/prompt_manager/autotag/unload")
|
||||
self.assertEqual(response.status, 409)
|
||||
self.assertEqual((await response.json())["code"], "model-busy")
|
||||
proceed.set()
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
|
||||
self.assertEqual(job.counters["tagged"], 2)
|
||||
self.assertEqual(self.tagger.load_calls, [("wd14-vit", True)])
|
||||
self.tagger.download_model.assert_not_called()
|
||||
self.assertIsNone(model_gate.holder())
|
||||
|
||||
async def test_model_unavailable_fails_the_job_before_the_first_file(self):
|
||||
self.png("a.png")
|
||||
self.tagger.downloaded["wd14-vit"] = False
|
||||
|
||||
job = await self.run_job(self.auto_body(["a.png"]))
|
||||
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertEqual(job.processed, 0)
|
||||
self.assertEqual(job.counters, dict.fromkeys(GROUPS, 0))
|
||||
self.assertTrue(
|
||||
any(
|
||||
line.endswith(
|
||||
"is not downloaded. Download it from the legacy Auto-tag page."
|
||||
)
|
||||
for line in job.log
|
||||
),
|
||||
job.log,
|
||||
)
|
||||
self.assertEqual(self.tagger.load_calls, [])
|
||||
self.tagger.download_model.assert_not_called()
|
||||
self.assertIsNone(model_gate.holder())
|
||||
self.assertEqual(self.api._capture_store().stats()["captures"], 0)
|
||||
|
||||
async def test_model_load_failure_fails_the_job_and_frees_the_gate(self):
|
||||
self.png("a.png")
|
||||
self.tagger.load_error = RuntimeError("/secret/models/wd14 exploded")
|
||||
|
||||
job = await self.run_job(self.auto_body(["a.png"]))
|
||||
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertEqual(job.processed, 0)
|
||||
self.assertNotIn("/secret", json.dumps(job.detail()))
|
||||
self.assertIsNone(model_gate.holder())
|
||||
|
||||
async def test_inference_failure_is_an_error_row_and_the_job_goes_on(self):
|
||||
self.png("a.png")
|
||||
self.png("b.png", color=(1, 2, 3))
|
||||
failing = {"first": True}
|
||||
|
||||
def explode(_path):
|
||||
if failing.pop("first", False):
|
||||
raise RuntimeError("/secret/path boom")
|
||||
|
||||
self.tagger.on_generate = explode
|
||||
|
||||
job = await self.run_job(self.auto_body(["a.png", "b.png"]))
|
||||
|
||||
self.assertEqual(job.counters["errors"], 1)
|
||||
self.assertEqual(job.counters["tagged"], 1)
|
||||
self.assertEqual(
|
||||
self.rows(job, "errors"),
|
||||
[("a.png", "Auto-tagging failed. Check server logs for details.")],
|
||||
)
|
||||
self.assertNotIn("/secret", json.dumps(job.report()))
|
||||
self.assertIsNone(model_gate.holder())
|
||||
|
||||
async def test_changed_file_is_skipped_not_tagged(self):
|
||||
self.png("a.png")
|
||||
before = self.digest("a.png")
|
||||
|
||||
def rewrite(_capture_path):
|
||||
make_png(self.output_dir / "a.png", size=(8, 6), color=(1, 1, 1))
|
||||
|
||||
self.tagger.on_generate = rewrite
|
||||
|
||||
job = await self.run_job(self.auto_body(["a.png"]))
|
||||
|
||||
self.assertEqual(self.rows(job, "skipped"), [("a.png", "File changed")])
|
||||
self.assertEqual(job.counters["tagged"], 0)
|
||||
locator = self.locator("a.png")
|
||||
self.assertEqual(self.api.db.read_image_tags(locator, before)["tags"], [])
|
||||
self.assertEqual(self.stored("a.png")["tags"], [])
|
||||
|
||||
async def test_captures_released_and_handles_closed(self):
|
||||
try:
|
||||
from open_handles import assert_closed
|
||||
except ImportError:
|
||||
from tests.open_handles import assert_closed
|
||||
for index in range(3):
|
||||
self.png(f"f{index}.png", color=(index, 0, 0))
|
||||
|
||||
await self.run_job(self.auto_body([f"f{index}.png" for index in range(3)]))
|
||||
|
||||
store = self.api._capture_store()
|
||||
self.assertEqual(store.stats()["captures"], 0)
|
||||
assert_closed(self, store.directory)
|
||||
assert_closed(self, str(self.output_dir))
|
||||
|
||||
|
||||
class TestStartAndExclusion(ImageTagJobTestCase):
|
||||
async def test_start_returns_model_busy_or_job_busy(self):
|
||||
self.png("a.png")
|
||||
other = model_gate.new_owner("legacy-batch")
|
||||
self.assertTrue(model_gate.try_acquire(other))
|
||||
try:
|
||||
status, body = await self.start(self.auto_body(["a.png"]))
|
||||
self.assertEqual((status, body["code"]), (409, "model-busy"))
|
||||
self.assertEqual(self.api._jobs.list(), [])
|
||||
# Add tags needs no model: it starts while the gate is held.
|
||||
job = await self.run_job(self.add_body(["a.png"], ["x"]))
|
||||
self.assertEqual(job.state, "finished")
|
||||
finally:
|
||||
model_gate.release(other)
|
||||
|
||||
entered, proceed = threading.Event(), threading.Event()
|
||||
self.tagger.block = (entered, proceed)
|
||||
job = await self.start_ok(self.auto_body(["a.png"]))
|
||||
await self.wait_for(entered)
|
||||
try:
|
||||
for body in (self.add_body(["a.png"], ["y"]), self.auto_body(["a.png"])):
|
||||
resp = await self.client.post(START, json=body)
|
||||
body = await resp.json()
|
||||
self.assertEqual((resp.status, body["code"]), (409, "busy"))
|
||||
self.assertEqual(
|
||||
body["running"], {"id": job.id, "title": "Auto-tag images"}
|
||||
)
|
||||
finally:
|
||||
proceed.set()
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
# A refused start never leaves the gate taken.
|
||||
self.assertIsNone(model_gate.holder())
|
||||
|
||||
async def test_image_tag_job_refused_while_legacy_restore_holds(self):
|
||||
self.png("a.png")
|
||||
async with self.api._jobs.hold(
|
||||
"restore", "Database restore (admin page)", admission="records"
|
||||
):
|
||||
status, body = await self.start(self.add_body(["a.png"], ["x"]))
|
||||
self.assertEqual((status, body["code"]), (409, "busy"))
|
||||
self.assertEqual(body["running"]["title"], "Database restore (admin page)")
|
||||
status, body = await self.start(self.auto_body(["a.png"]))
|
||||
self.assertEqual((status, body["code"]), (409, "busy"))
|
||||
self.assertIsNone(model_gate.holder())
|
||||
async with self.api._jobs.hold(
|
||||
"thumbnails", "Thumbnail generation (gallery page)", admission="guarded"
|
||||
):
|
||||
status, body = await self.start(self.add_body(["a.png"], ["x"]))
|
||||
self.assertEqual((status, body["code"]), (409, "busy"))
|
||||
|
||||
async def test_record_changing_excludes_running_scan(self):
|
||||
self.png("a.png")
|
||||
gate = asyncio.Event()
|
||||
|
||||
async def scan_runner(job):
|
||||
await gate.wait()
|
||||
|
||||
scan = self.api._jobs.start(
|
||||
"scan",
|
||||
"Scan for metadata",
|
||||
record_changing=True,
|
||||
scope={"files": "new"},
|
||||
runner=scan_runner,
|
||||
)
|
||||
try:
|
||||
status, body = await self.start(self.add_body(["a.png"], ["x"]))
|
||||
self.assertEqual((status, body["code"]), (409, "busy"))
|
||||
self.assertEqual(body["running"]["title"], "Scan for metadata")
|
||||
status, body = await self.start(self.auto_body(["a.png"]))
|
||||
self.assertEqual((status, body["code"]), (409, "busy"))
|
||||
self.assertIsNone(model_gate.holder())
|
||||
finally:
|
||||
gate.set()
|
||||
await scan.task
|
||||
|
||||
# The other way round: a running tagging job keeps the scan out.
|
||||
entered, proceed = threading.Event(), threading.Event()
|
||||
self.tagger.block = (entered, proceed)
|
||||
job = await self.start_ok(self.auto_body(["a.png"]))
|
||||
await self.wait_for(entered)
|
||||
try:
|
||||
resp = await self.client.post("/prompt_manager/jobs/scan", json={})
|
||||
self.assertEqual(resp.status, 409)
|
||||
self.assertEqual((await resp.json())["running"]["id"], job.id)
|
||||
finally:
|
||||
proceed.set()
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
|
||||
async def test_a_start_that_loses_the_slot_race_frees_the_gate(self):
|
||||
self.png("a.png")
|
||||
gate = asyncio.Event()
|
||||
|
||||
async def scan_runner(job):
|
||||
await gate.wait()
|
||||
|
||||
scan = self.api._jobs.start(
|
||||
"scan",
|
||||
"Scan for metadata",
|
||||
record_changing=True,
|
||||
scope={},
|
||||
runner=scan_runner,
|
||||
)
|
||||
try:
|
||||
# The pre-check saw a free slot; registry.start still refuses.
|
||||
with patch.object(self.api._jobs, "blocker", side_effect=[None, scan]):
|
||||
status, body = await self.start(self.auto_body(["a.png"]))
|
||||
self.assertEqual((status, body["code"]), (409, "busy"))
|
||||
self.assertIsNone(model_gate.holder())
|
||||
finally:
|
||||
gate.set()
|
||||
await scan.task
|
||||
|
||||
async def test_cancel_stops_at_file_boundary_and_keeps_written_files(self):
|
||||
for index in range(3):
|
||||
self.png(f"f{index}.png", color=(index, 5, 5))
|
||||
entered, proceed = threading.Event(), threading.Event()
|
||||
self.tagger.block = (entered, proceed)
|
||||
|
||||
job = await self.start_ok(self.auto_body([f"f{i}.png" for i in range(3)]))
|
||||
await self.wait_for(entered)
|
||||
response = await self.client.post(f"/prompt_manager/jobs/{job.id}/cancel")
|
||||
self.assertTrue((await response.json())["cancellation_requested"])
|
||||
proceed.set()
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
|
||||
self.assertEqual(job.state, "cancelled")
|
||||
self.assertEqual(job.counters["tagged"], 1)
|
||||
self.assertEqual(job.processed, 1)
|
||||
self.assertEqual(len(self.stored("f0.png")["tags"]), 10)
|
||||
self.assertEqual(self.stored("f1.png")["tags"], [])
|
||||
self.assertEqual(len(self.tagger.calls), 1)
|
||||
self.assertTrue(job.log[-1].endswith("Auto-tag cancelled · 1 file processed"))
|
||||
self.assertIsNone(model_gate.holder())
|
||||
self.assertEqual(self.api._capture_store().stats()["captures"], 0)
|
||||
|
||||
async def test_stale_root_scope_refuses_the_start(self):
|
||||
self.png("a.png")
|
||||
body = self.add_body(["a.png"], ["x"])
|
||||
body["targets"][0]["root_scope"] = "0" * 64
|
||||
|
||||
status, data = await self.start(body)
|
||||
|
||||
self.assertEqual((status, data["code"]), (409, "source-changed"))
|
||||
self.assertEqual(self.api._jobs.list(), [])
|
||||
|
||||
async def test_restore_during_the_job_stops_it_without_writes(self):
|
||||
self.png("a.png")
|
||||
self.png("b.png", color=(1, 2, 3))
|
||||
|
||||
def rotate(_path):
|
||||
conn = sqlite3.connect(self._temp_db.name)
|
||||
try:
|
||||
rotate_history_epoch(conn)
|
||||
finally:
|
||||
conn.close()
|
||||
|
||||
self.tagger.on_generate = rotate
|
||||
|
||||
job = await self.run_job(self.auto_body(["a.png", "b.png"]))
|
||||
|
||||
self.assertEqual(job.state, "failed")
|
||||
self.assertEqual(job.counters["tagged"], 0)
|
||||
self.assertEqual(
|
||||
self.rows(job, "errors"),
|
||||
[("a.png", "Prompt storage was restored; start the job again.")],
|
||||
)
|
||||
self.assertEqual(self.stored("a.png")["tags"], [])
|
||||
self.assertEqual(len(self.tagger.calls), 1)
|
||||
self.assertIsNone(model_gate.holder())
|
||||
|
||||
async def test_invalid_bodies_rejected(self):
|
||||
self.png("a.png")
|
||||
target = self.target("a.png")
|
||||
bad_bodies = [
|
||||
{"mode": "add", "targets": [], "tags": ["x"]},
|
||||
{"mode": "add", "targets": [target] * 501, "tags": ["x"]},
|
||||
{"mode": "add", "targets": "a.png", "tags": ["x"]},
|
||||
{"mode": "add", "targets": [{**target, "path": "../a.png"}], "tags": ["x"]},
|
||||
{"mode": "add", "targets": [{**target, "root_index": True}], "tags": ["x"]},
|
||||
{"mode": "add", "targets": [{**target, "extra": 1}], "tags": ["x"]},
|
||||
{"mode": "add", "targets": [{**target, "path": "a.txt"}], "tags": ["x"]},
|
||||
{"mode": "add", "targets": [target], "tags": []},
|
||||
{"mode": "add", "targets": [target], "tags": "x"},
|
||||
{"mode": "add", "targets": [target], "tags": ["bad\nname"]},
|
||||
{"mode": "add", "targets": [target]},
|
||||
{"mode": "add", "targets": [target], "tags": ["x"], "extra": 1},
|
||||
{"mode": "add", "targets": [target], "tags": ["x"], "model_type": "hf"},
|
||||
{"mode": "remove", "targets": [target], "tags": ["x"]},
|
||||
{"targets": [target], "tags": ["x"]},
|
||||
{"mode": "auto", "targets": [target], "model_type": "wd14-vit"},
|
||||
{"mode": "auto", "targets": [target], "model_type": "x", "options": {}},
|
||||
{
|
||||
"mode": "auto",
|
||||
"targets": [target],
|
||||
"model_type": "wd14-vit",
|
||||
"options": {},
|
||||
},
|
||||
{
|
||||
"mode": "auto",
|
||||
"targets": [target],
|
||||
"model_type": "wd14-vit",
|
||||
"options": {**WD14, "general_threshold": 2},
|
||||
},
|
||||
{
|
||||
"mode": "auto",
|
||||
"targets": [target],
|
||||
"model_type": "wd14-vit",
|
||||
"options": {**WD14, "suggestion_limit": 0},
|
||||
},
|
||||
{
|
||||
"mode": "auto",
|
||||
"targets": [target],
|
||||
"model_type": "gguf",
|
||||
"options": dict(WD14),
|
||||
},
|
||||
{
|
||||
"mode": "auto",
|
||||
"targets": [target],
|
||||
"model_type": "wd14-vit",
|
||||
"options": dict(WD14),
|
||||
"tags": ["x"],
|
||||
},
|
||||
]
|
||||
for body in bad_bodies:
|
||||
with self.subTest(body=json.dumps(body)[:120]):
|
||||
status, data = await self.start(body)
|
||||
self.assertIn(status, (400, 403), data)
|
||||
self.assertFalse(data["success"])
|
||||
too_many = {
|
||||
"mode": "add",
|
||||
"targets": [target],
|
||||
"tags": [f"t{i}" for i in range(101)],
|
||||
}
|
||||
status, data = await self.start(too_many)
|
||||
self.assertEqual((status, data["code"]), (400, "tag-limit"))
|
||||
response = await self.client.post(START, data="not json")
|
||||
self.assertEqual(response.status, 400)
|
||||
self.assertEqual(self.api._jobs.list(), [])
|
||||
self.assertIsNone(model_gate.holder())
|
||||
|
||||
async def test_duplicate_targets_are_tagged_once(self):
|
||||
self.png("a.png")
|
||||
|
||||
job = await self.run_job(self.add_body(["a.png", "a.png"], ["x"]))
|
||||
|
||||
self.assertEqual(job.scope["images"], 1)
|
||||
self.assertEqual(job.counters["tagged"], 1)
|
||||
|
||||
async def test_rows_and_report_carry_no_absolute_paths(self):
|
||||
self.png("sub/a.png")
|
||||
self.png("sub/b.png", color=(3, 3, 3))
|
||||
job = await self.run_job(
|
||||
self.auto_body(["sub/a.png", "sub/b.png", "sub/missing.png"])
|
||||
)
|
||||
|
||||
for url in (
|
||||
START.rsplit("/", 1)[0],
|
||||
f"/prompt_manager/jobs/{job.id}",
|
||||
f"/prompt_manager/jobs/{job.id}/report",
|
||||
):
|
||||
response = await self.client.get(url)
|
||||
self.assertEqual(response.status, 200)
|
||||
body = json.loads(await response.text())
|
||||
self.assert_no_paths(body)
|
||||
detail = job.detail()
|
||||
self.assertEqual(
|
||||
[row["path"] for row in detail["outcomes"]["tagged"]],
|
||||
["sub/a.png", "sub/b.png"],
|
||||
)
|
||||
self.assertTrue(
|
||||
all(row["root"] == "output" for row in detail["outcomes"]["tagged"])
|
||||
)
|
||||
self.assertTrue(any("output/sub/a.png" in line for line in detail["log"]))
|
||||
|
||||
|
||||
class TestFixRound(ImageTagJobTestCase):
|
||||
"""The final review's fixes: estimate, body bound, busy captures, roots."""
|
||||
|
||||
async def test_status_line_estimates_minutes_left_after_the_minimum(self):
|
||||
for index in range(3):
|
||||
self.png(f"f{index}.png", color=(index, 7, 7))
|
||||
clock = iter([0.0, 0.0, 120.0, 240.0])
|
||||
lines = []
|
||||
self.tagger.on_generate = lambda _path: lines.append(
|
||||
self.api._jobs.list()[0].status_line
|
||||
)
|
||||
|
||||
with patch(
|
||||
"py.api.image_tag_jobs._monotonic", side_effect=lambda: next(clock, 240.0)
|
||||
):
|
||||
await self.run_job(self.auto_body([f"f{i}.png" for i in range(3)]))
|
||||
|
||||
# Two files make the minimum (the scan's MIN_BATCHES_FOR_ESTIMATE); then
|
||||
# 120 s per file leaves about 2 min for the third.
|
||||
self.assertEqual(
|
||||
lines,
|
||||
[
|
||||
"Processing file 1 of 3",
|
||||
"Processing file 2 of 3",
|
||||
"Processing file 3 of 3 · about 2 min left",
|
||||
],
|
||||
)
|
||||
|
||||
def maximal_target(self, number):
|
||||
"""A distinct target whose path is 1024 non-ASCII code points long."""
|
||||
head = f"{number:03d}/" + ("界" * 200 + "/") * 5
|
||||
path = head + "界" * (1024 - len(head) - 4) + ".png"
|
||||
self.assertEqual(len(path), 1024)
|
||||
return self.target(path)
|
||||
|
||||
async def test_start_body_bounds(self):
|
||||
from py.api import image_tag_jobs
|
||||
|
||||
body = {
|
||||
"mode": "add",
|
||||
"targets": [self.maximal_target(n) for n in range(500)],
|
||||
"tags": ["x"],
|
||||
}
|
||||
# Escaped as \uXXXX (6 bytes a code point), far over the old 1 MiB.
|
||||
payload = json.dumps(body).encode("utf-8")
|
||||
self.assertGreater(len(payload), 3 * 1000 * 1000)
|
||||
response = await self.client.post(
|
||||
START, data=payload, headers={"Content-Type": "application/json"}
|
||||
)
|
||||
data = await self.json_of(response)
|
||||
self.assertEqual(response.status, 202, data)
|
||||
self.assertEqual(data["job"]["scope"]["images"], 500)
|
||||
job = self.api._jobs.get(data["job"]["id"])
|
||||
await self.client.post(f"/prompt_manager/jobs/{job.id}/cancel")
|
||||
await asyncio.wait_for(asyncio.shield(job.task), WAIT)
|
||||
|
||||
cap = image_tag_jobs.MAX_JOB_BODY_BYTES
|
||||
self.assertGreaterEqual(cap, len(payload))
|
||||
over = b'{"mode":"add","targets":[],"tags":["x"]'
|
||||
over += b" " * (cap + 1 - len(over) - 1) + b"}"
|
||||
self.assertEqual(len(over), cap + 1)
|
||||
response = await self.client.post(
|
||||
START, data=over, headers={"Content-Type": "application/json"}
|
||||
)
|
||||
self.assertEqual(response.status, 413)
|
||||
|
||||
def saturate(self, store):
|
||||
with store._lock:
|
||||
store._workers += store.max_workers
|
||||
self.addCleanup(self.unsaturate, store)
|
||||
|
||||
def unsaturate(self, store):
|
||||
with store._lock:
|
||||
if store._workers >= store.max_workers:
|
||||
store._workers -= store.max_workers
|
||||
|
||||
async def test_busy_capture_workers_are_retried_then_left_for_a_rerun(self):
|
||||
self.png("a.png")
|
||||
self.png("b.png", color=(1, 2, 3))
|
||||
store = self.api._capture_store()
|
||||
self.saturate(store)
|
||||
|
||||
with patch("py.api.image_tag_jobs.CAPACITY_BACKOFF", (0.001, 0.001)):
|
||||
added = await self.run_job(self.add_body(["a.png"], ["x"]))
|
||||
auto = await self.run_job(self.auto_body(["b.png"]))
|
||||
|
||||
retry = "The file could not be read now. Run the job again for it."
|
||||
for job, path in ((added, "a.png"), (auto, "b.png")):
|
||||
self.assertEqual(job.state, "finished")
|
||||
self.assertEqual(self.rows(job, "skipped"), [(path, retry)])
|
||||
self.assertEqual(job.counters["errors"], 0)
|
||||
self.assertNotIn("Tag dialog", json.dumps(job.detail()))
|
||||
self.assertEqual(self.tagger.calls, [])
|
||||
self.assertEqual(self.stored("a.png")["tags"], [])
|
||||
|
||||
async def test_a_busy_capture_worker_that_frees_up_is_tagged(self):
|
||||
self.png("a.png")
|
||||
store = self.api._capture_store()
|
||||
real = store.verify
|
||||
tries = []
|
||||
|
||||
def verify(root, relative):
|
||||
tries.append(relative)
|
||||
if len(tries) < 3:
|
||||
raise CaptureCapacity("Too many captures are in progress")
|
||||
return real(root, relative)
|
||||
|
||||
with (
|
||||
patch.object(store, "verify", side_effect=verify),
|
||||
patch("py.api.image_tag_jobs.CAPACITY_BACKOFF", (0.001, 0.001, 0.001)),
|
||||
):
|
||||
job = await self.run_job(self.add_body(["a.png"], ["x"]))
|
||||
|
||||
self.assertEqual(len(tries), 3)
|
||||
self.assertEqual(self.rows(job, "tagged"), [("a.png", "Added 1 tag")])
|
||||
self.assertEqual(self.stored("a.png")["tags"], ["x"])
|
||||
|
||||
async def test_output_roots_are_read_once_per_job_off_the_event_loop(self):
|
||||
for index in range(3):
|
||||
self.png(f"f{index}.png", color=(index, 1, 1))
|
||||
body = self.add_body([f"f{i}.png" for i in range(3)], ["x"])
|
||||
original = self.api._get_all_output_dirs
|
||||
threads = []
|
||||
|
||||
def lookup():
|
||||
threads.append(threading.current_thread())
|
||||
return original()
|
||||
|
||||
with patch.object(self.api, "_get_all_output_dirs", side_effect=lookup):
|
||||
job = await self.run_job(body)
|
||||
|
||||
self.assertEqual(job.counters["tagged"], 3)
|
||||
# One read when the start checks the scopes, one for the whole run.
|
||||
self.assertEqual(len(threads), 2)
|
||||
self.assertNotIn(threading.main_thread(), threads)
|
||||
self.assertEqual(job.outcomes["tagged"][0]["root"], "output")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,552 @@
|
||||
"""Registry admission classes and the legacy routes that hold the registry.
|
||||
|
||||
Record-changing work never runs together (Activity design): a ``records``
|
||||
job is refused while any ``records`` or ``guarded`` job runs, a ``guarded``
|
||||
job (thumbnails) while a ``records`` job runs, and ``read`` jobs only share
|
||||
the one-per-kind rule. The legacy admin and gallery routes that change
|
||||
records or files take a registry hold for their duration, so a workspace job
|
||||
and a legacy-page operation refuse each other, naming the one that runs.
|
||||
|
||||
Fake runners and gated executor calls only; ordering is driven by events.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import threading
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
_mock_server = MagicMock()
|
||||
_mock_server.PromptServer.instance.routes = MagicMock()
|
||||
sys.modules.setdefault("server", _mock_server)
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from py.api.jobs import ( # noqa: E402
|
||||
ADMISSIONS,
|
||||
Job,
|
||||
JobBusy,
|
||||
JobRegistry,
|
||||
)
|
||||
|
||||
try:
|
||||
from tests.test_api_admin import AdminAPITestCase # noqa: E402
|
||||
except ImportError: # discovered with tests/ as the top-level directory
|
||||
from test_api_admin import AdminAPITestCase # noqa: E402
|
||||
|
||||
|
||||
def _gated(gate):
|
||||
async def runner(job):
|
||||
await gate.wait()
|
||||
|
||||
return runner
|
||||
|
||||
|
||||
async def _noop(job):
|
||||
return None
|
||||
|
||||
|
||||
class TestAdmissionClasses(unittest.IsolatedAsyncioTestCase):
|
||||
|
||||
def _start(self, registry, kind, admission, gate=None, **kwargs):
|
||||
return registry.start(
|
||||
kind,
|
||||
kind.title(),
|
||||
admission=admission,
|
||||
scope={},
|
||||
runner=_gated(gate) if gate is not None else _noop,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
async def test_admission_names_and_explicit_read(self):
|
||||
self.assertEqual(ADMISSIONS, ("read", "guarded", "records"))
|
||||
registry = JobRegistry()
|
||||
job = registry.start(
|
||||
"report", "Report", admission="read", scope={}, runner=_noop
|
||||
)
|
||||
await asyncio.wait_for(job.task, 2)
|
||||
self.assertEqual(job.admission, "read")
|
||||
self.assertFalse(job.record_changing)
|
||||
with self.assertRaises(ValueError):
|
||||
registry.start("x", "X", admission="exclusive", scope={}, runner=_noop)
|
||||
with self.assertRaises(ValueError):
|
||||
registry.start(
|
||||
"x",
|
||||
"X",
|
||||
admission="read",
|
||||
record_changing=True,
|
||||
scope={},
|
||||
runner=_noop,
|
||||
)
|
||||
|
||||
async def test_start_requires_an_admission_class(self):
|
||||
# A job that forgets its class must not silently run as "read" beside
|
||||
# a scan or restore: every start call decides.
|
||||
registry = JobRegistry()
|
||||
with self.assertRaises(TypeError) as raised:
|
||||
registry.start("review", "Review", scope={}, runner=_noop)
|
||||
self.assertIn("admission", str(raised.exception))
|
||||
self.assertEqual(registry.list(), [])
|
||||
job = registry.start(
|
||||
"review", "Review", record_changing=False, scope={}, runner=_noop
|
||||
)
|
||||
await asyncio.wait_for(job.task, 2)
|
||||
self.assertEqual(job.admission, "read")
|
||||
|
||||
async def test_record_changing_is_an_alias_for_records(self):
|
||||
registry = JobRegistry()
|
||||
gate = asyncio.Event()
|
||||
job = registry.start(
|
||||
"scan", "Scan", record_changing=True, scope={}, runner=_gated(gate)
|
||||
)
|
||||
self.assertEqual(job.admission, "records")
|
||||
self.assertTrue(job.record_changing)
|
||||
self.assertEqual(job.summary()["admission"], "records")
|
||||
self.assertIs(
|
||||
registry.blocker("image-tags", record_changing=True), job
|
||||
) # the existing call shape keeps working
|
||||
gate.set()
|
||||
await asyncio.wait_for(job.task, 2)
|
||||
|
||||
async def test_records_job_refused_while_guarded_runs(self):
|
||||
registry = JobRegistry()
|
||||
gate = asyncio.Event()
|
||||
thumbs = self._start(registry, "thumbnails", "guarded", gate)
|
||||
with self.assertRaises(JobBusy) as raised:
|
||||
self._start(registry, "scan", "records")
|
||||
self.assertIs(raised.exception.running, thumbs)
|
||||
self.assertIs(registry.blocker("restore", admission="records"), thumbs)
|
||||
gate.set()
|
||||
await asyncio.wait_for(thumbs.task, 2)
|
||||
scan = self._start(registry, "scan", "records")
|
||||
await asyncio.wait_for(scan.task, 2)
|
||||
self.assertEqual(scan.state, "finished")
|
||||
|
||||
async def test_guarded_refused_while_records_runs(self):
|
||||
registry = JobRegistry()
|
||||
gate = asyncio.Event()
|
||||
scan = self._start(registry, "scan", "records", gate)
|
||||
with self.assertRaises(JobBusy) as raised:
|
||||
self._start(registry, "thumbnails", "guarded")
|
||||
self.assertIs(raised.exception.running, scan)
|
||||
# A second guarded kind may run beside a guarded one, not beside records.
|
||||
gate.set()
|
||||
await asyncio.wait_for(scan.task, 2)
|
||||
hold = asyncio.Event()
|
||||
thumbs = self._start(registry, "thumbnails", "guarded", hold)
|
||||
backup = self._start(registry, "backup", "guarded")
|
||||
await asyncio.wait_for(backup.task, 2)
|
||||
self.assertEqual(backup.state, "finished")
|
||||
hold.set()
|
||||
await asyncio.wait_for(thumbs.task, 2)
|
||||
|
||||
async def test_read_jobs_run_in_parallel(self):
|
||||
registry = JobRegistry()
|
||||
gate = asyncio.Event()
|
||||
scan = self._start(registry, "scan", "records", gate)
|
||||
health = self._start(registry, "health", "read", gate)
|
||||
duplicates = self._start(registry, "duplicates", "read", gate)
|
||||
self.assertTrue(all(j.running for j in (scan, health, duplicates)))
|
||||
with self.assertRaises(JobBusy) as raised: # one job of a kind
|
||||
self._start(registry, "health", "read")
|
||||
self.assertIs(raised.exception.running, health)
|
||||
gate.set()
|
||||
await asyncio.wait_for(
|
||||
asyncio.gather(scan.task, health.task, duplicates.task), 2
|
||||
)
|
||||
|
||||
async def test_non_cancellable_job_ignores_cancel(self):
|
||||
registry = JobRegistry()
|
||||
gate = asyncio.Event()
|
||||
job = self._start(registry, "restore", "records", gate, cancellable=False)
|
||||
self.assertIs(job.summary()["cancellable"], False)
|
||||
self.assertFalse(registry.cancel(job.id))
|
||||
self.assertFalse(job.cancelled)
|
||||
gate.set()
|
||||
await asyncio.wait_for(job.task, 2)
|
||||
self.assertEqual(job.state, "finished")
|
||||
|
||||
async def test_hold_registers_a_running_non_cancellable_job(self):
|
||||
registry = JobRegistry()
|
||||
held = registry.hold(
|
||||
"restore", "Database restore (admin page)", admission="records"
|
||||
)
|
||||
async with held as job:
|
||||
self.assertIs(registry.active("restore"), job)
|
||||
self.assertEqual(job.state, "running")
|
||||
self.assertEqual(
|
||||
(job.summary()["cancellable"], job.summary()["admission"]),
|
||||
(False, "records"),
|
||||
)
|
||||
self.assertFalse(registry.cancel(job.id))
|
||||
with self.assertRaises(JobBusy) as raised:
|
||||
self._start(registry, "scan", "records")
|
||||
self.assertIn("Database restore (admin page)", str(raised.exception))
|
||||
self.assertEqual(job.state, "finished")
|
||||
self.assertIsNone(registry.active("restore"))
|
||||
self.assertIs(registry.list()[0], job)
|
||||
|
||||
async def test_hold_refused_without_waiting(self):
|
||||
registry = JobRegistry()
|
||||
gate = asyncio.Event()
|
||||
scan = self._start(registry, "scan", "records", gate)
|
||||
with self.assertRaises(JobBusy) as raised:
|
||||
async with registry.hold("maintenance", "Maintenance", admission="records"):
|
||||
self.fail("a refused hold must not run its body")
|
||||
self.assertIs(raised.exception.running, scan)
|
||||
self.assertEqual([j.kind for j in registry.list()], ["scan"])
|
||||
gate.set()
|
||||
await asyncio.wait_for(scan.task, 2)
|
||||
|
||||
async def test_hold_released_on_exception(self):
|
||||
registry = JobRegistry()
|
||||
with self.assertRaises(RuntimeError):
|
||||
async with registry.hold(
|
||||
"maintenance", "Maintenance", admission="records"
|
||||
) as job:
|
||||
raise RuntimeError("boom")
|
||||
self.assertEqual(job.state, "failed")
|
||||
again = registry.hold("maintenance", "Maintenance", admission="records")
|
||||
async with again as second:
|
||||
second.finish("failed") # the body may end it itself
|
||||
self.assertEqual(second.state, "failed")
|
||||
|
||||
async def test_hold_admits_on_entry_not_at_the_call(self):
|
||||
# A hold created but never entered (an await or an exception between
|
||||
# the call and ``async with``) must not leave a running records job
|
||||
# behind that blocks all record work until a restart.
|
||||
registry = JobRegistry()
|
||||
held = registry.hold("restore", "Restore", admission="records")
|
||||
self.assertEqual(registry.list(), [])
|
||||
self.assertIsNone(registry.blocker("scan", admission="records"))
|
||||
del held
|
||||
async with registry.hold("restore", "Restore", admission="records") as job:
|
||||
self.assertIs(registry.active("restore"), job)
|
||||
self.assertEqual(job.state, "finished")
|
||||
|
||||
def test_job_defaults_stay_read_and_cancellable(self):
|
||||
job = Job("duplicates", "Duplicate scan")
|
||||
self.assertEqual((job.admission, job.cancellable), ("read", True))
|
||||
|
||||
|
||||
class _LegacyHoldCase(AdminAPITestCase):
|
||||
"""The admin test app plus a running workspace job on demand."""
|
||||
|
||||
def _run_job(self, kind="scan", title="Scan for metadata", admission="records"):
|
||||
gate = asyncio.Event()
|
||||
job = self.api._jobs.start(
|
||||
kind, title, admission=admission, scope={}, runner=_gated(gate)
|
||||
)
|
||||
self.addAsyncCleanup(self._release, gate, job)
|
||||
return gate, job
|
||||
|
||||
@staticmethod
|
||||
async def _release(gate, job):
|
||||
gate.set()
|
||||
await asyncio.wait_for(job.task, 5)
|
||||
|
||||
def _db_digest(self):
|
||||
path = Path(self.api.db.model.db_path)
|
||||
return {
|
||||
suffix: hashlib.sha256(Path(f"{path}{suffix}").read_bytes()).hexdigest()
|
||||
for suffix in ("", "-wal")
|
||||
if Path(f"{path}{suffix}").exists()
|
||||
}
|
||||
|
||||
async def _busy(self, resp, title="Scan for metadata"):
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 409)
|
||||
self.assertEqual(body, {"success": False, "error": f"{title} already running"})
|
||||
|
||||
|
||||
class TestLegacyRoutesHoldTheRegistry(_LegacyHoldCase):
|
||||
|
||||
async def test_hold_is_listed_as_running_job(self):
|
||||
held = self.api._jobs.hold(
|
||||
"restore", "Database restore (admin page)", admission="records"
|
||||
)
|
||||
async with held:
|
||||
resp = await self.client.request("GET", "/prompt_manager/jobs")
|
||||
jobs = (await resp.json())["jobs"]
|
||||
self.assertEqual(len(jobs), 1)
|
||||
job = jobs[0]
|
||||
self.assertEqual(
|
||||
(job["kind"], job["title"], job["state"]),
|
||||
("restore", "Database restore (admin page)", "running"),
|
||||
)
|
||||
self.assertIs(job["cancellable"], False)
|
||||
self.assertEqual(job["admission"], "records")
|
||||
|
||||
async def test_legacy_maintenance_and_cleanup_refused_while_scan_runs(self):
|
||||
self.api.db.save_prompt(text="keep me", prompt_hash="h1")
|
||||
before = self._db_digest()
|
||||
self._run_job()
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/maintenance",
|
||||
json={"operations": ["cleanup_duplicates", "vacuum"]},
|
||||
)
|
||||
await self._busy(resp)
|
||||
resp = await self.client.request("POST", "/prompt_manager/cleanup")
|
||||
await self._busy(resp)
|
||||
self.assertEqual(self._db_digest(), before)
|
||||
|
||||
async def test_read_only_maintenance_runs_during_scan(self):
|
||||
# The legacy statistics and consistency checks only read: they hold
|
||||
# the registry as "read", so a long scan does not block them.
|
||||
self.api.db.save_prompt(text="keep me", prompt_hash="h1")
|
||||
_gate, scan = self._run_job()
|
||||
listed = []
|
||||
real = self.api._run_maintenance_operations
|
||||
|
||||
def observed(operations):
|
||||
listed.extend(
|
||||
(job.kind, job.admission, job.state) for job in self.api._jobs.list()
|
||||
)
|
||||
return real(operations)
|
||||
|
||||
self.api._run_maintenance_operations = observed
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/maintenance",
|
||||
json={
|
||||
"operations": [
|
||||
"statistics",
|
||||
"check_consistency",
|
||||
"check_hash_duplicates",
|
||||
]
|
||||
},
|
||||
)
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIs(body["success"], True)
|
||||
self.assertEqual(
|
||||
set(body["results"]),
|
||||
{"statistics", "check_consistency", "check_hash_duplicates"},
|
||||
)
|
||||
self.assertIn(("maintenance", "read", "running"), listed)
|
||||
self.assertEqual(scan.state, "running")
|
||||
maintenance = self.api._jobs.list()[-1]
|
||||
self.assertEqual(
|
||||
(maintenance.kind, maintenance.admission, maintenance.state),
|
||||
("maintenance", "read", "finished"),
|
||||
)
|
||||
|
||||
async def test_mixed_maintenance_selection_is_refused_during_scan(self):
|
||||
# One write operation among read-only ones makes the run "records".
|
||||
self.api.db.save_prompt(text="keep me", prompt_hash="h1")
|
||||
before = self._db_digest()
|
||||
self._run_job()
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/maintenance",
|
||||
json={"operations": ["statistics", "vacuum"]},
|
||||
)
|
||||
await self._busy(resp)
|
||||
self.assertEqual(self._db_digest(), before)
|
||||
|
||||
async def test_default_maintenance_selection_is_refused_during_scan(self):
|
||||
# No "operations" key means the default list, which vacuums.
|
||||
self.api.db.save_prompt(text="keep me", prompt_hash="h1")
|
||||
before = self._db_digest()
|
||||
self._run_job()
|
||||
resp = await self.client.request("POST", "/prompt_manager/maintenance", json={})
|
||||
await self._busy(resp)
|
||||
self.assertEqual(self._db_digest(), before)
|
||||
self.assertEqual([j.kind for j in self.api._jobs.list()], ["scan"])
|
||||
|
||||
async def test_legacy_maintenance_refused_while_image_tags_runs(self):
|
||||
self._run_job("image-tags", "Add tags to 3 images")
|
||||
resp = await self.client.request("POST", "/prompt_manager/maintenance")
|
||||
await self._busy(resp, "Add tags to 3 images")
|
||||
|
||||
async def test_scan_refused_while_legacy_maintenance_holds(self):
|
||||
entered = threading.Event()
|
||||
release = threading.Event()
|
||||
real = self.api._run_maintenance_operations
|
||||
|
||||
def gated(operations):
|
||||
entered.set()
|
||||
release.wait(10)
|
||||
return real(operations)
|
||||
|
||||
self.api._run_maintenance_operations = gated
|
||||
loop = asyncio.get_running_loop()
|
||||
maintenance = asyncio.ensure_future(
|
||||
self.client.request(
|
||||
"POST", "/prompt_manager/maintenance", json={"operations": ["vacuum"]}
|
||||
)
|
||||
)
|
||||
self.assertTrue(await loop.run_in_executor(None, entered.wait, 10))
|
||||
resp = await self.client.request("POST", "/prompt_manager/jobs/scan")
|
||||
body = await resp.json()
|
||||
self.assertEqual((resp.status, body["code"]), (409, "busy"))
|
||||
self.assertEqual(body["running"]["title"], "Maintenance (admin page)")
|
||||
release.set()
|
||||
self.assertEqual((await maintenance).status, 200)
|
||||
listed = (
|
||||
await (await self.client.request("GET", "/prompt_manager/jobs")).json()
|
||||
)["jobs"]
|
||||
self.assertEqual(
|
||||
[(j["kind"], j["state"]) for j in listed], [("maintenance", "finished")]
|
||||
)
|
||||
|
||||
async def test_legacy_duplicate_delete_refused_while_scan_runs(self):
|
||||
victim = self.output_dir / "dup.png"
|
||||
victim.write_bytes(b"x")
|
||||
self._run_job()
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/delete_duplicate_images",
|
||||
json={"image_paths": [str(victim)]},
|
||||
)
|
||||
await self._busy(resp)
|
||||
self.assertTrue(victim.exists())
|
||||
# The duplicate family lock was not taken by the refused request.
|
||||
self.assertIsNotNone(self.api._acquire_long_job(self.api.LONG_JOB_DUPLICATES))
|
||||
|
||||
async def test_lora_scan_refused_while_scan_runs(self):
|
||||
from py.config import IntegrationConfig
|
||||
|
||||
lm = Path(self.tmpdir) / "custom_nodes" / "ComfyUI-Lora-Manager"
|
||||
(lm / "py").mkdir(parents=True)
|
||||
self._run_job()
|
||||
imported = []
|
||||
with (
|
||||
patch.multiple(
|
||||
IntegrationConfig, LORA_MANAGER_ENABLED=True, LORA_MANAGER_PATH=str(lm)
|
||||
),
|
||||
patch("py.lora_utils.resolve_lora_manager_path", return_value=str(lm)),
|
||||
patch.object(
|
||||
self.api, "_lora_scan_stream", side_effect=lambda *a: imported.append(a)
|
||||
),
|
||||
):
|
||||
resp = await self.client.request("POST", "/prompt_manager/lora/scan")
|
||||
text = await resp.text()
|
||||
self.assertEqual(resp.status, 200) # the legacy page reads a stream
|
||||
events = [json.loads(line[6:]) for line in text.split("\n\n") if line]
|
||||
# The admin page's modal shows ``status`` and closes on ``type: error``.
|
||||
message = "Scan for metadata already running"
|
||||
self.assertEqual(
|
||||
events,
|
||||
[{"type": "error", "status": message, "progress": 0, "message": message}],
|
||||
)
|
||||
self.assertEqual(imported, [])
|
||||
|
||||
async def test_legacy_autotag_refused_while_scan_runs(self):
|
||||
# Both pages read this route as a stream (a 409 reads as a lost
|
||||
# connection there), so a refusal by another job is one error frame:
|
||||
# the admin page keys on ``type``, the gallery on ``status``.
|
||||
self._run_job()
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/autotag/start", json={}
|
||||
)
|
||||
text = await resp.text()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(resp.headers["Content-Type"], "text/event-stream")
|
||||
events = [json.loads(line[6:]) for line in text.split("\n\n") if line]
|
||||
message = "Scan for metadata already running"
|
||||
self.assertEqual(
|
||||
events, [{"type": "error", "status": "error", "message": message}]
|
||||
)
|
||||
self.assertEqual(
|
||||
[j.kind for j in self.api._jobs.list() if j.kind == "autotag"], []
|
||||
)
|
||||
|
||||
async def test_second_legacy_autotag_batch_keeps_its_json_409(self):
|
||||
from py.api.autotag_routes import _BUSY_ERROR
|
||||
|
||||
self._run_job("autotag", "Auto-tag images (legacy page)")
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/autotag/start", json={}
|
||||
)
|
||||
self.assertEqual(resp.status, 409)
|
||||
self.assertEqual(await resp.json(), {"success": False, "error": _BUSY_ERROR})
|
||||
|
||||
async def test_legacy_payloads_unchanged_when_idle(self):
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/maintenance", json={"operations": ["vacuum"]}
|
||||
)
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(
|
||||
set(body),
|
||||
{"success", "operations_completed", "all_successful", "results", "message"},
|
||||
)
|
||||
resp = await self.client.request("POST", "/prompt_manager/cleanup")
|
||||
body = await resp.json()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertEqual(set(body), {"success", "message", "duplicates_removed"})
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/delete_duplicate_images", json={"image_paths": []}
|
||||
)
|
||||
self.assertEqual(
|
||||
(resp.status, await resp.json()),
|
||||
(400, {"success": False, "error": "No image paths provided"}),
|
||||
)
|
||||
# Each hold ended with the request; refused input never took one.
|
||||
listed = await self._listed()
|
||||
self.assertEqual(
|
||||
[(j["kind"], j["state"]) for j in listed],
|
||||
[("maintenance", "finished"), ("maintenance", "finished")],
|
||||
)
|
||||
self.assertEqual(
|
||||
[j["title"] for j in listed],
|
||||
["Prompt cleanup (admin page)", "Maintenance (admin page)"],
|
||||
)
|
||||
|
||||
async def _listed(self):
|
||||
resp = await self.client.request("GET", "/prompt_manager/jobs")
|
||||
return (await resp.json())["jobs"]
|
||||
|
||||
|
||||
class TestHeldJobStates(_LegacyHoldCase):
|
||||
"""Activity lists what a held legacy operation really did (Minor 7)."""
|
||||
|
||||
async def _listed(self):
|
||||
resp = await self.client.request("GET", "/prompt_manager/jobs")
|
||||
return [(j["kind"], j["state"]) for j in (await resp.json())["jobs"]]
|
||||
|
||||
async def test_refused_input_takes_no_hold(self):
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/delete_duplicate_images", json={}
|
||||
)
|
||||
self.assertEqual(resp.status, 400)
|
||||
resp = await self.client.request(
|
||||
"POST",
|
||||
"/prompt_manager/images/generate-thumbnails",
|
||||
data=b"{not json",
|
||||
headers={"Content-Type": "application/json"},
|
||||
)
|
||||
self.assertEqual(resp.status, 400)
|
||||
self.assertEqual(await self._listed(), [])
|
||||
|
||||
async def test_thumbnail_stream_that_ends_in_error_is_failed(self):
|
||||
with patch.object(self.api, "_get_all_output_dirs", return_value=[]):
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/images/generate-thumbnails/progress"
|
||||
)
|
||||
text = await resp.text()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIn("event: error", text)
|
||||
self.assertEqual(await self._listed(), [("thumbnails", "failed")])
|
||||
|
||||
async def test_legacy_autotag_that_ends_in_error_is_failed(self):
|
||||
service = MagicMock()
|
||||
service.get_models_status.return_value = {} # model not downloaded
|
||||
with patch("py.autotag.get_autotag_service", return_value=service):
|
||||
resp = await self.client.request(
|
||||
"POST", "/prompt_manager/autotag/start", json={}
|
||||
)
|
||||
text = await resp.text()
|
||||
self.assertEqual(resp.status, 200)
|
||||
self.assertIn('"type": "error"', text)
|
||||
self.assertEqual(await self._listed(), [("autotag", "failed")])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -12,6 +12,7 @@ import unittest
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
|
||||
from py.api import jobs as jobs_module # noqa: E402
|
||||
from py.api.jobs import ( # noqa: E402
|
||||
DEFAULT_SCAN_SCOPE,
|
||||
LOG_CAP_LINE,
|
||||
@@ -181,8 +182,13 @@ class TestJobRegistry(unittest.IsolatedAsyncioTestCase):
|
||||
"scope",
|
||||
"listed",
|
||||
"log_length",
|
||||
"cancellable",
|
||||
"admission",
|
||||
},
|
||||
)
|
||||
self.assertEqual(
|
||||
(summary["cancellable"], summary["admission"]), (True, "records")
|
||||
)
|
||||
self.assertEqual(summary["scope"], {"files": "new", "output_folders": 2})
|
||||
# What the detail would list, so a view reads it only when that changed.
|
||||
self.assertEqual((summary["listed"], summary["log_length"]), ({}, 0))
|
||||
@@ -415,6 +421,14 @@ class TestJobOutcomesAndLog(unittest.TestCase):
|
||||
SCAN_GROUP_NAMES, ("added", "known", "review", "no_metadata", "errors")
|
||||
)
|
||||
|
||||
def test_image_tag_job_kind_and_groups(self):
|
||||
# #249 D1: bulk Add tags / Auto-tag, counter-tile order.
|
||||
self.assertEqual(getattr(jobs_module, "IMAGE_TAG_JOB_KIND", None), "image-tags")
|
||||
self.assertEqual(
|
||||
getattr(jobs_module, "IMAGE_TAG_GROUP_NAMES", None),
|
||||
("tagged", "unchanged", "skipped", "errors"),
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
+28
-16
@@ -1,9 +1,9 @@
|
||||
{
|
||||
"admin.js": {
|
||||
"lines": 28,
|
||||
"lines": 31,
|
||||
"branches": 58,
|
||||
"functions": 23,
|
||||
"statements": 28
|
||||
"functions": 26,
|
||||
"statements": 31
|
||||
},
|
||||
"api-client.js": {
|
||||
"lines": 98,
|
||||
@@ -42,10 +42,10 @@
|
||||
"statements": 98
|
||||
},
|
||||
"gallery.js": {
|
||||
"lines": 39,
|
||||
"lines": 40,
|
||||
"branches": 63,
|
||||
"functions": 43,
|
||||
"statements": 39
|
||||
"functions": 44,
|
||||
"statements": 40
|
||||
},
|
||||
"html-escape.js": {
|
||||
"lines": 98,
|
||||
@@ -157,27 +157,27 @@
|
||||
},
|
||||
"workspace/activity-job-view.mjs": {
|
||||
"lines": 97,
|
||||
"branches": 83,
|
||||
"branches": 86,
|
||||
"functions": 98,
|
||||
"statements": 97
|
||||
},
|
||||
"workspace/activity-scan.mjs": {
|
||||
"lines": 95,
|
||||
"branches": 64,
|
||||
"lines": 96,
|
||||
"branches": 73,
|
||||
"functions": 98,
|
||||
"statements": 95
|
||||
"statements": 96
|
||||
},
|
||||
"workspace/activity-store.mjs": {
|
||||
"lines": 98,
|
||||
"branches": 95,
|
||||
"branches": 96,
|
||||
"functions": 98,
|
||||
"statements": 98
|
||||
},
|
||||
"workspace/activity.mjs": {
|
||||
"lines": 95,
|
||||
"lines": 96,
|
||||
"branches": 87,
|
||||
"functions": 98,
|
||||
"statements": 95
|
||||
"statements": 96
|
||||
},
|
||||
"workspace/file-inspector.mjs": {
|
||||
"lines": 96,
|
||||
@@ -229,8 +229,8 @@
|
||||
},
|
||||
"workspace/image-tagging.mjs": {
|
||||
"lines": 97,
|
||||
"branches": 88,
|
||||
"functions": 87,
|
||||
"branches": 90,
|
||||
"functions": 88,
|
||||
"statements": 97
|
||||
},
|
||||
"workspace/images-category.mjs": {
|
||||
@@ -253,7 +253,7 @@
|
||||
},
|
||||
"workspace/images-selection.mjs": {
|
||||
"lines": 98,
|
||||
"branches": 92,
|
||||
"branches": 93,
|
||||
"functions": 98,
|
||||
"statements": 98
|
||||
},
|
||||
@@ -263,6 +263,18 @@
|
||||
"functions": 98,
|
||||
"statements": 98
|
||||
},
|
||||
"workspace/images-tagging-bulk.mjs": {
|
||||
"lines": 98,
|
||||
"branches": 93,
|
||||
"functions": 86,
|
||||
"statements": 98
|
||||
},
|
||||
"workspace/images-tagging-review.mjs": {
|
||||
"lines": 98,
|
||||
"branches": 93,
|
||||
"functions": 98,
|
||||
"statements": 98
|
||||
},
|
||||
"workspace/images-tagging.mjs": {
|
||||
"lines": 98,
|
||||
"branches": 98,
|
||||
|
||||
+46
-7
@@ -25,8 +25,10 @@
|
||||
this.tagsPage = null;
|
||||
|
||||
this.api = ApiClient.createApiClient();
|
||||
// Search and page loads share one guard: whichever list request is newest wins
|
||||
this.listRequest = ApiClient.latestOnly();
|
||||
// The page's lifetime ends when it starts to leave. WebKit cancels the page's reads
|
||||
// between beforeunload and pagehide; a save's follow-up read started then fails and is
|
||||
// logged as a page error ("Fetch API cannot load … due to access control checks.").
|
||||
this.startPageLifetime();
|
||||
this.infiniteScroll = false; // session value; seeded from settings, toggled in the header
|
||||
this.infiniteScrollChosen = false;
|
||||
this.loadingMore = false;
|
||||
@@ -150,7 +152,18 @@
|
||||
// Output scan: a server-side job that survives modal dismissal and reloads
|
||||
this.scanState = ScanProgress.initialState();
|
||||
this.scanSource = null;
|
||||
window.addEventListener("beforeunload", (e) => this.warnIfScanRunning(e));
|
||||
window.addEventListener("beforeunload", (e) => {
|
||||
this.warnIfScanRunning(e);
|
||||
// A page that asks to stay (a running scan) keeps its reads until it really goes.
|
||||
if (!e.defaultPrevented) this.pageLifetime.abort();
|
||||
});
|
||||
window.addEventListener("pagehide", () => this.pageLifetime.abort());
|
||||
// A page restored from the back/forward cache lives again, and so does one whose
|
||||
// navigation never left (stopped, or answered with a download): its next input revives it.
|
||||
window.addEventListener("pageshow", (e) => { if (e.persisted) this.revivePageLifetime(); });
|
||||
for (const type of ["pointerdown", "keydown"]) {
|
||||
window.addEventListener(type, () => this.revivePageLifetime(), true);
|
||||
}
|
||||
|
||||
this.bindModalEvents();
|
||||
|
||||
@@ -319,9 +332,24 @@
|
||||
}
|
||||
}
|
||||
|
||||
/** A new lifetime once the old one has ended; a living page keeps its list guard. */
|
||||
revivePageLifetime() {
|
||||
if (this.pageLifetime.signal.aborted) this.startPageLifetime();
|
||||
}
|
||||
|
||||
/** A fresh page lifetime, and the list guard bound to it. */
|
||||
startPageLifetime() {
|
||||
this.pageLifetime = new AbortController();
|
||||
// Search and page loads share one guard: whichever list request is newest wins
|
||||
this.listRequest = ApiClient.latestOnly({ lifetime: this.pageLifetime.signal });
|
||||
}
|
||||
|
||||
async loadStatistics() {
|
||||
// A leaving page starts no read; one in flight is aborted with the page and not reported.
|
||||
const { signal } = this.pageLifetime;
|
||||
if (signal.aborted) return;
|
||||
try {
|
||||
const response = await ApiClient.fetchUrl("/prompt_manager/stats");
|
||||
const response = await ApiClient.fetchUrl("/prompt_manager/stats", { signal });
|
||||
if (response.ok) {
|
||||
const data = await response.json();
|
||||
|
||||
@@ -340,6 +368,7 @@
|
||||
console.error("Stats API HTTP error:", response.status, response.statusText);
|
||||
}
|
||||
} catch (error) {
|
||||
if (signal.aborted) return;
|
||||
console.error("Stats error:", error);
|
||||
// Set some default values if API fails
|
||||
document.getElementById("totalPrompts").textContent = this.prompts.length || 0;
|
||||
@@ -1151,6 +1180,11 @@
|
||||
`Done — ${data.imported} imported, ${data.skipped} skipped`;
|
||||
this.loadStatistics();
|
||||
setTimeout(() => this.hideModal("loraImportModal"), 2000);
|
||||
} else if (data.type === "error") {
|
||||
// A refusal (another job runs) or a failed import: name it and close the modal.
|
||||
document.getElementById("loraImportStatus").textContent = `Error: ${data.message}`;
|
||||
this.showNotification(`LoRA import failed: ${data.message}`, "error");
|
||||
setTimeout(() => this.hideModal("loraImportModal"), 3000);
|
||||
}
|
||||
} catch (e) { /* skip malformed SSE */ }
|
||||
}
|
||||
@@ -1202,10 +1236,13 @@
|
||||
}
|
||||
this.hideModal("settingsModal");
|
||||
} else {
|
||||
throw new Error("Failed to save settings");
|
||||
// A refusal names its reason (a corrupt config.json, a changed file): show it.
|
||||
const data = await response.json().catch(() => ({}));
|
||||
throw new Error(data.error ? `Failed to save settings: ${data.error}` : "Failed to save settings");
|
||||
}
|
||||
} catch (error) {
|
||||
this.showNotification("Failed to save settings", "error");
|
||||
const reason = error.message.startsWith("Failed to save settings:") ? error.message : "Failed to save settings";
|
||||
this.showNotification(reason, "error");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1819,7 +1856,9 @@
|
||||
throw new Error(data.error || 'Maintenance failed');
|
||||
}
|
||||
} else {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`);
|
||||
// A refusal names the job that runs ("Scan for metadata already running").
|
||||
const data = await response.json().catch(() => ({}));
|
||||
throw new Error(data.error || `HTTP ${response.status}: ${response.statusText}`);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Maintenance error:', error);
|
||||
|
||||
+11
-3
@@ -107,20 +107,28 @@
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{lifetime?: AbortSignal}} [options] - `lifetime` ends with the page (its pagehide):
|
||||
* then no call starts a request, the one in flight is aborted, and both resolve to STALE.
|
||||
* WebKit fails a fetch that a leaving page starts and logs it as a page error.
|
||||
* @returns {(run: (signal: AbortSignal) => Promise<*>) => Promise<*>} wrapper that
|
||||
* aborts the previous call's signal and resolves superseded calls to STALE
|
||||
*/
|
||||
function latestOnly() {
|
||||
function latestOnly({ lifetime } = {}) {
|
||||
let current = null;
|
||||
lifetime?.addEventListener("abort", () => current?.abort(), { once: true });
|
||||
return async function (run) {
|
||||
if (current) current.abort();
|
||||
if (lifetime?.aborted) {
|
||||
current = null;
|
||||
return STALE;
|
||||
}
|
||||
const controller = new AbortController();
|
||||
current = controller;
|
||||
try {
|
||||
const result = await run(controller.signal);
|
||||
return controller === current ? result : STALE;
|
||||
return controller === current && !lifetime?.aborted ? result : STALE;
|
||||
} catch (error) {
|
||||
if (controller !== current) return STALE;
|
||||
if (controller !== current || lifetime?.aborted) return STALE;
|
||||
throw error;
|
||||
}
|
||||
};
|
||||
|
||||
+17
-3
@@ -17,6 +17,14 @@
|
||||
this.dupResults = null;
|
||||
|
||||
this.api = ApiClient.createApiClient();
|
||||
// The page's lifetime ends when it starts to leave: WebKit fails a fetch a leaving page starts
|
||||
// ("Fetch API cannot load … due to access control checks.") and logs it as a page error.
|
||||
this.pageLifetime = new AbortController();
|
||||
window.addEventListener("beforeunload", () => this.pageLifetime.abort());
|
||||
window.addEventListener("pagehide", () => this.pageLifetime.abort());
|
||||
window.addEventListener("pageshow", (e) => {
|
||||
if (e.persisted && this.pageLifetime.signal.aborted) this.pageLifetime = new AbortController();
|
||||
});
|
||||
// Only the newest page request may update the grid
|
||||
this.imagesRequest = ApiClient.latestOnly();
|
||||
|
||||
@@ -2129,8 +2137,11 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
|
||||
headers: { 'Content-Type': 'application/json' }
|
||||
});
|
||||
|
||||
if (response.ok) {
|
||||
const data = await response.json();
|
||||
if (!response.ok) {
|
||||
// Refused (a running job holds the thumbnails): say why and claim nothing.
|
||||
const data = await response.json().catch(() => ({}));
|
||||
this.showNotification(`Thumbnails were not cleared: ${data.error || `HTTP ${response.status}`}`, 'error');
|
||||
return;
|
||||
}
|
||||
} catch (e) {
|
||||
console.warn('Could not clear server thumbnails:', e);
|
||||
@@ -2531,8 +2542,11 @@ Seed: ${this.currentMetadata.seed || 'Unknown'}`;
|
||||
return;
|
||||
}
|
||||
|
||||
// A leaving page starts no read, and one it aborts is not reported.
|
||||
const signal = this.pageLifetime?.signal;
|
||||
if (signal?.aborted) return;
|
||||
try {
|
||||
const data = await this.api.get('/prompt_manager/images/thumbnails/status');
|
||||
const data = await this.api.get('/prompt_manager/images/thumbnails/status', { signal });
|
||||
const total = data.total_images;
|
||||
const pending = data.pending;
|
||||
if (!data.success || !Number.isSafeInteger(total) ||
|
||||
|
||||
@@ -11,6 +11,9 @@
|
||||
* filter, and the selected outcome's listed rows (at most 200 per outcome, so "Showing N of M"). Rows under Added and
|
||||
* Already known open the file in the existing viewer through its listing identity (D57); the others are plain text.
|
||||
* - Log: `details.term` "Scan log", collapsed, holding the raw timestamped lines in `pre[tabindex=0]`.
|
||||
* - D89: the tiles, the outcome list, the openable rows and the log and cancel labels are the job kind's (`jobView`):
|
||||
* an image tagging job shows Tagged, Unchanged, Skipped and Errors, opens Tagged and Unchanged rows, and reads
|
||||
* "Tagging log" and "Cancel tagging"; another kind keeps the scan's tiles as "Job log" and "Cancel job".
|
||||
*
|
||||
* The summary follows the Activity store (one SSE stream per running job); the listed rows and the log are read from
|
||||
* `GET /prompt_manager/jobs/<id>` at most once a second while the job runs, and once more when it ends.
|
||||
@@ -19,7 +22,7 @@ import { createButton } from '../shared/toolkit/core.js';
|
||||
import { pageHeading } from './heading.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
import { noun } from './plural.mjs';
|
||||
import { parseJob, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
|
||||
import { parseJob, jobView, JOB_VIEWS, JOBS_URL, LOST_DETAIL } from './activity-store.mjs';
|
||||
import { jobDetail, CANCEL_FAILED_TOAST } from './activity.mjs';
|
||||
import { downloadJobReport } from './activity-scan.mjs';
|
||||
import { createMediaViewer } from './media-viewer.mjs';
|
||||
@@ -36,18 +39,6 @@ export const FILE_GONE = 'This file is no longer in Images. Rescan from Import a
|
||||
export const OPEN_FAILED = 'The file could not be opened. Try again.';
|
||||
const LISTING_LIMIT = 5;
|
||||
|
||||
/** The counter tiles, in the spec's order and the mockup's tones. */
|
||||
export const TILES = Object.freeze([
|
||||
['added', 'Added to Library', 'added'], ['known', 'Already known', ''], ['review', 'Needs review', 'warn'],
|
||||
['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad'],
|
||||
]);
|
||||
/** The outcome list, in the mockup's order: what got added and what went wrong first. */
|
||||
export const OUTCOMES = Object.freeze([
|
||||
['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
|
||||
['no_metadata', 'No usable metadata'], ['known', 'Already known'],
|
||||
]);
|
||||
const OPENABLE = new Set(['added', 'known']);
|
||||
|
||||
function element(tag, text, className) {
|
||||
const node = document.createElement(tag);
|
||||
if (text !== undefined) node.textContent = text;
|
||||
@@ -103,9 +94,9 @@ export function filterRows(rows, filter) {
|
||||
return rows.filter(row => `${row.root}/${row.path}`.toLocaleLowerCase().includes(wanted));
|
||||
}
|
||||
|
||||
/** The first outcome with files, so a view opens on what happened; Added when nothing has yet. */
|
||||
export function defaultOutcome(counters) {
|
||||
return OUTCOMES.find(([group]) => (counters[group] ?? 0) > 0)?.[0] ?? 'added';
|
||||
/** The first outcome with files, so a view opens on what happened; the first outcome when nothing has yet. */
|
||||
export function defaultOutcome(counters, outcomes = JOB_VIEWS.scan.outcomes) {
|
||||
return (outcomes.find(([group]) => (counters[group] ?? 0) > 0) ?? outcomes[0])[0];
|
||||
}
|
||||
|
||||
function section(title, id) {
|
||||
@@ -132,13 +123,6 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
bar.max = 100;
|
||||
const stats = element('dl', undefined, 'stat-row');
|
||||
const tileValues = new Map();
|
||||
for (const [group, label, tone] of TILES) {
|
||||
const tile = element('div', undefined, tone ? `stat ${tone}` : 'stat');
|
||||
const value = element('dd', '0');
|
||||
tile.append(element('dt', label), value);
|
||||
tileValues.set(group, value);
|
||||
stats.append(tile);
|
||||
}
|
||||
progressPart.node.append(bar, stats);
|
||||
|
||||
const resultsPart = section('Results', 'job-results-title');
|
||||
@@ -155,15 +139,6 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
list.setAttribute('aria-label', 'Outcomes');
|
||||
const files = element('div', undefined, 'outcome-files');
|
||||
const buttons = new Map();
|
||||
for (const [group, label] of OUTCOMES) {
|
||||
const button = createButton('');
|
||||
const count = element('span', '0');
|
||||
button.append(element('span', label), count);
|
||||
button.setAttribute('aria-pressed', 'false');
|
||||
button.count = count;
|
||||
buttons.set(group, button);
|
||||
list.append(button);
|
||||
}
|
||||
outcomes.append(list, files);
|
||||
resultsPart.node.append(outcomes);
|
||||
|
||||
@@ -181,6 +156,8 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
missing.append(element('p', NOT_FOUND, 'empty-state'), back);
|
||||
|
||||
let detail = null;
|
||||
// The job's kind decides its tiles, outcomes and labels (D89); it is known once the job is.
|
||||
let view = null;
|
||||
let selected = null;
|
||||
let mounted = false;
|
||||
let lastRead = 0;
|
||||
@@ -200,13 +177,35 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
const viewer = createMediaViewer({ api, container: main, parentScope: scope, inspectionController: inspection,
|
||||
sender: createPromptSender({ scope }), onSelect: image => inspection.select(image) });
|
||||
|
||||
/** The tiles and the outcome list of the job's kind, built once. */
|
||||
function layout(kind) {
|
||||
view = jobView(kind); // D89
|
||||
for (const [group, label, tone] of view.tiles) {
|
||||
const tile = element('div', undefined, tone ? `stat ${tone}` : 'stat');
|
||||
const value = element('dd', '0');
|
||||
tile.append(element('dt', label), value);
|
||||
tileValues.set(group, value);
|
||||
stats.append(tile);
|
||||
}
|
||||
for (const [group, label] of view.outcomes) {
|
||||
const button = createButton('');
|
||||
const count = element('span', '0');
|
||||
button.append(element('span', label), count);
|
||||
button.setAttribute('aria-pressed', 'false');
|
||||
button.count = count;
|
||||
buttons.set(group, button);
|
||||
list.append(button);
|
||||
}
|
||||
}
|
||||
|
||||
function showBody(job) {
|
||||
if (mounted) return;
|
||||
mounted = true;
|
||||
if (!view) layout(job.kind);
|
||||
missing.remove();
|
||||
parts.actions.append(download, cancel);
|
||||
main.append(progressPart.node, resultsPart.node, log);
|
||||
selected ??= defaultOutcome(job.counters);
|
||||
selected ??= defaultOutcome(job.counters, view.outcomes);
|
||||
}
|
||||
|
||||
/** The job is not (or no longer) in this session: say so instead of an empty view. */
|
||||
@@ -242,7 +241,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
|
||||
function renderRows(job) {
|
||||
const rows = detail?.outcomes[selected] ?? [];
|
||||
const openable = OPENABLE.has(selected);
|
||||
const openable = view.openable.has(selected);
|
||||
const wanted = filterRows(rows.map((row, index) => ({ ...row, index })), input.value);
|
||||
const kept = new Map(wanted.map(row => rowNode(row, row.index, openable)));
|
||||
rowNodes = kept;
|
||||
@@ -270,12 +269,13 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
parts.h1.textContent = job.title;
|
||||
document.title = `${job.title} · Prompt Manager`;
|
||||
parts.description.textContent = jobDetail(job);
|
||||
cancel.textContent = scan ? 'Cancel scan' : 'Cancel job';
|
||||
cancel.textContent = view.cancel;
|
||||
// A job lost with the server has no report there (review M9).
|
||||
const lost = job.state === 'lost';
|
||||
if (lost && document.activeElement === download) parts.h1.focus();
|
||||
download.disabled = lost;
|
||||
if (job.state === 'running') {
|
||||
// Truth rule: a job that cannot be cancelled (a held legacy-page job) shows no Cancel.
|
||||
if (job.state === 'running' && job.cancellable) {
|
||||
if (!cancel.isConnected) parts.actions.append(cancel);
|
||||
} else if (cancel.isConnected) {
|
||||
// A focused Cancel that goes away hands its focus to the job's heading, never to the page body.
|
||||
@@ -291,7 +291,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) {
|
||||
button.count.textContent = number(job.counters[group] ?? 0);
|
||||
button.setAttribute('aria-pressed', String(group === selected));
|
||||
}
|
||||
logLabel.textContent = `● ${scan ? 'Scan' : 'Job'} log`;
|
||||
logLabel.textContent = `● ${view.log}`;
|
||||
const text = detail?.log.length ? detail.log.join('\n') : 'No log lines yet.';
|
||||
if (pre.textContent !== text) {
|
||||
// Replacing the text clamps the scroll position, so keep it: pinned readers follow the end, others stay.
|
||||
|
||||
@@ -23,10 +23,52 @@ export const LOST_DETAIL = 'This job ended when ComfyUI restarted.';
|
||||
const SERVER_STATES = new Set(['running', 'finished', 'cancelled', 'failed']);
|
||||
const FINAL_STATES = new Set(['finished', 'cancelled', 'failed']);
|
||||
const JOB_ID = /^[0-9a-f]{32}$/;
|
||||
// The pill's verb for each job kind; a kind without one shows its title.
|
||||
const PILL_VERBS = Object.freeze({ scan: 'Scanning' });
|
||||
// The pill's verb for each job kind; a kind without one shows its title (D89: image tagging jobs read "Tagging").
|
||||
const PILL_VERBS = Object.freeze({ scan: 'Scanning', 'image-tags': 'Tagging' });
|
||||
const START_TOASTS = Object.freeze({ scan: 'Scan started.' });
|
||||
|
||||
/** The scan's job view (D56): its counter tiles `[group, label, tone]` and outcome list in the mockup's orders. */
|
||||
const SCAN_VIEW = Object.freeze({
|
||||
tiles: Object.freeze([
|
||||
['added', 'Added to Library', 'added'], ['known', 'Already known', ''], ['review', 'Needs review', 'warn'],
|
||||
['no_metadata', 'No usable metadata', ''], ['errors', 'Errors', 'bad'],
|
||||
].map(Object.freeze)),
|
||||
// What got added and what went wrong first.
|
||||
outcomes: Object.freeze([
|
||||
['added', 'Added to Library'], ['review', 'Needs review'], ['errors', 'Errors'],
|
||||
['no_metadata', 'No usable metadata'], ['known', 'Already known'],
|
||||
].map(Object.freeze)),
|
||||
// Rows that open the file in the viewer (D57); the others are plain text.
|
||||
openable: Object.freeze(new Set(['added', 'known'])),
|
||||
log: 'Scan log',
|
||||
cancel: 'Cancel scan',
|
||||
});
|
||||
|
||||
/**
|
||||
* Each job kind's view: its tiles, outcome list, openable rows, log label and cancel label. D89: an image tagging job
|
||||
* (`image-tags`) counts Tagged (green), Unchanged, Skipped (warning) and Errors (red), lists what got tagged and what
|
||||
* went wrong first, and opens Tagged and Unchanged rows in the viewer like the scan's (D57).
|
||||
*/
|
||||
export const JOB_VIEWS = Object.freeze({
|
||||
scan: SCAN_VIEW,
|
||||
'image-tags': Object.freeze({
|
||||
tiles: Object.freeze([['tagged', 'Tagged', 'added'], ['unchanged', 'Unchanged', ''], ['skipped', 'Skipped', 'warn'],
|
||||
['errors', 'Errors', 'bad']].map(Object.freeze)),
|
||||
outcomes: Object.freeze([['tagged', 'Tagged'], ['errors', 'Errors'], ['skipped', 'Skipped'],
|
||||
['unchanged', 'Unchanged']].map(Object.freeze)),
|
||||
openable: Object.freeze(new Set(['tagged', 'unchanged'])),
|
||||
log: 'Tagging log',
|
||||
cancel: 'Cancel tagging',
|
||||
}),
|
||||
});
|
||||
// Another kind keeps the scan's tiles and outcomes under the generic labels.
|
||||
const GENERIC_VIEW = Object.freeze({ ...SCAN_VIEW, log: 'Job log', cancel: 'Cancel job' });
|
||||
|
||||
/** The view of a job `kind`; an unknown kind gets the generic one. */
|
||||
export function jobView(kind) {
|
||||
return Object.hasOwn(JOB_VIEWS, kind) ? JOB_VIEWS[kind] : GENERIC_VIEW;
|
||||
}
|
||||
|
||||
/** The wait before the `attempt`-th re-read (0-based): 1 s, 2 s, 4 s … never more than 30 s. */
|
||||
export function backoffDelay(attempt) {
|
||||
const step = Number.isSafeInteger(attempt) && attempt > 0 ? Math.min(attempt, 16) : 0;
|
||||
@@ -61,6 +103,8 @@ export function parseJob(raw) {
|
||||
scope: plainObject(raw.scope) ? { ...raw.scope } : {},
|
||||
listed,
|
||||
log_length: count(raw.log_length),
|
||||
// A held legacy-page job cannot be cancelled; an older server sends no key (cancellable then).
|
||||
cancellable: raw.cancellable !== false,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -70,20 +114,31 @@ const errorsOf = job => `${number(job.counters.errors ?? 0)} ${noun(job.counters
|
||||
/** "1,420 added". */
|
||||
export const addedOf = job => `${number(job.counters.added ?? 0)} added`;
|
||||
|
||||
/** The toast when a job ends: "Scan finished · N added · M errors", "Scan cancelled · N added". */
|
||||
/**
|
||||
* The toast when a job ends: "Scan finished · N added · M errors", "Scan cancelled · N added"; an image tagging job
|
||||
* the same way with its own title and count (D89): "Auto-tag images finished · N tagged · M errors".
|
||||
*/
|
||||
export function finishToast(job) {
|
||||
if (job.kind === 'scan') {
|
||||
if (job.state === 'cancelled') return `Scan cancelled · ${addedOf(job)}`;
|
||||
if (job.state === 'failed') return `Scan failed · ${addedOf(job)} · ${errorsOf(job)}`;
|
||||
return `Scan finished · ${addedOf(job)} · ${errorsOf(job)}`;
|
||||
}
|
||||
if (job.kind === 'image-tags') {
|
||||
if (job.state === 'cancelled') return `${job.title} cancelled · ${taggedOf(job)}`;
|
||||
return `${job.title} ${job.state === 'failed' ? 'failed' : 'finished'} · ${outcomeLine(job)}`;
|
||||
}
|
||||
if (job.state === 'cancelled') return `${job.title} was cancelled.`;
|
||||
if (job.state === 'failed') return `${job.title} failed.`;
|
||||
return `${job.title} finished.`;
|
||||
}
|
||||
|
||||
// D85: an image tagging job says which of its two modes started.
|
||||
const IMAGE_TAG_START_TOASTS = Object.freeze({ add: 'Tagging started.', auto: 'Auto-tagging started.' });
|
||||
|
||||
/** The toast after this tab starts a job. */
|
||||
export function startToast(job) {
|
||||
if (job.kind === 'image-tags' && IMAGE_TAG_START_TOASTS[job.scope.mode]) return IMAGE_TAG_START_TOASTS[job.scope.mode];
|
||||
return START_TOASTS[job.kind] ?? `${job.title} started.`;
|
||||
}
|
||||
|
||||
@@ -94,9 +149,14 @@ export function pillParts(job) {
|
||||
return { label, files, text: label + files };
|
||||
}
|
||||
|
||||
/** The scan's counters as one line: "N added · M errors"; other kinds have none yet. */
|
||||
/** "12 tagged" (D89). */
|
||||
export const taggedOf = job => `${number(job.counters.tagged ?? 0)} tagged`;
|
||||
|
||||
/** The counters as one line: "N added · M errors" for a scan, "N tagged · M errors" for image tagging (D89). */
|
||||
export function outcomeLine(job) {
|
||||
return job.kind === 'scan' ? `${addedOf(job)} · ${errorsOf(job)}` : '';
|
||||
if (job.kind === 'scan') return `${addedOf(job)} · ${errorsOf(job)}`;
|
||||
if (job.kind === 'image-tags') return `${taggedOf(job)} · ${errorsOf(job)}`;
|
||||
return '';
|
||||
}
|
||||
|
||||
function defaultConnect(url) {
|
||||
|
||||
@@ -11,13 +11,14 @@
|
||||
* - D54: each `.activity-item` is a real job. Its chip names the state (green "Completed", or blue "Running",
|
||||
* "Cancelled", "Failed", "Ended"), its line says when and what, and a running job shows its `<progress>` and
|
||||
* "Cancel job" (no "Simulate completion").
|
||||
* - D89: an image tagging job's line names its images and, once ended, "N tagged · M errors".
|
||||
*/
|
||||
import { createButton } from '../shared/toolkit/core.js';
|
||||
import { navigateTo } from './state.mjs';
|
||||
import { pageHeading } from './heading.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
import { countOf } from './plural.mjs';
|
||||
import { pillParts, outcomeLine, addedOf, LOST_DETAIL } from './activity-store.mjs';
|
||||
import { countOf, noun } from './plural.mjs';
|
||||
import { pillParts, outcomeLine, addedOf, taggedOf, LOST_DETAIL } from './activity-store.mjs';
|
||||
|
||||
export const ACTIVITY_DESCRIPTION = 'Scans, tagging, downloads and maintenance share one history.';
|
||||
export const EMPTY_ACTIVITY = 'No jobs have run since ComfyUI started. Scans and other long operations appear here.';
|
||||
@@ -44,12 +45,21 @@ export function clockTime(seconds) {
|
||||
|
||||
const join = parts => parts.filter(Boolean).join(' · ');
|
||||
|
||||
/** What a job covers: "2 output folders" for a scan. */
|
||||
const whole = value => Number.isSafeInteger(value) && value >= 0;
|
||||
|
||||
/** What a job covers: "2 output folders" for a scan, "4 images" for image tagging (D89). */
|
||||
function scopeLine(job) {
|
||||
// D89: grouped as the server writes its lines ("19,622 images").
|
||||
const images = job.scope?.images;
|
||||
if (job.kind === 'image-tags') return whole(images) ? `${images.toLocaleString('en-US')} ${noun(images, 'image')}` : '';
|
||||
const folders = job.scope?.output_folders;
|
||||
return Number.isSafeInteger(folders) && folders >= 0 ? countOf(folders, 'output folder') : '';
|
||||
return whole(folders) ? countOf(folders, 'output folder') : '';
|
||||
}
|
||||
|
||||
/** What a cancelled job kept: "N added" for a scan, "N tagged" for image tagging (D89). */
|
||||
const KEPT = Object.freeze({ __proto__: null, scan: addedOf, 'image-tags': taggedOf }); // D89
|
||||
const keptOf = job => KEPT[job.kind]?.(job) ?? '';
|
||||
|
||||
/** The item's line under its title, by state. */
|
||||
export function jobDetail(job) {
|
||||
const at = clockTime(job.finished_at);
|
||||
@@ -58,8 +68,7 @@ export function jobDetail(job) {
|
||||
}
|
||||
if (job.state === 'lost') return LOST_DETAIL;
|
||||
if (job.state === 'cancelled') {
|
||||
return join([`Cancelled ${at} at a file boundary`, job.kind === 'scan' ? addedOf(job) : '',
|
||||
'completed work is kept']);
|
||||
return join([`Cancelled ${at} at a file boundary`, keptOf(job), 'completed work is kept']);
|
||||
}
|
||||
if (job.state === 'failed') return join([`Stopped on an error at ${at}`, outcomeLine(job)]);
|
||||
return join([`Finished ${at}`, outcomeLine(job)]);
|
||||
@@ -144,11 +153,15 @@ function itemFor(job, scope) {
|
||||
progress.textContent = `${next.progress}%`;
|
||||
progress.setAttribute('aria-label', `${next.title} progress`);
|
||||
if (!progress.isConnected) item.insertBefore(progress, actions.isConnected ? actions : null);
|
||||
if (!actions.isConnected) item.append(actions);
|
||||
} else {
|
||||
progress.remove();
|
||||
}
|
||||
// Truth rule: a job that cannot be cancelled (a held legacy-page job) shows no Cancel job.
|
||||
if (running && next.cancellable) {
|
||||
if (!actions.isConnected) item.append(actions);
|
||||
} else if (actions.isConnected) {
|
||||
// A focused Cancel job that goes away hands its focus to the item's title link, never to the page body.
|
||||
if (actions.contains(document.activeElement)) link.focus();
|
||||
progress.remove();
|
||||
actions.remove();
|
||||
}
|
||||
};
|
||||
|
||||
@@ -128,7 +128,8 @@ export function optionError(model, options) {
|
||||
return null;
|
||||
}
|
||||
|
||||
function requestOptions(model, options) {
|
||||
/** The options a request sends for `model`: numbers parsed, only the model's own keys. */
|
||||
export function requestOptions(model, options) {
|
||||
const body = {suggestion_limit: numberOf(options.suggestion_limit)};
|
||||
if (modelOf(model).classifier) {
|
||||
return {general_threshold: numberOf(options.general_threshold), character_threshold: numberOf(options.character_threshold), ...body};
|
||||
@@ -177,28 +178,48 @@ function validateReceipt(value) {
|
||||
return value;
|
||||
}
|
||||
|
||||
/** Thresholds, instructions and suggestion count from the server, else the documented defaults. */
|
||||
/**
|
||||
* Thresholds, instructions and suggestion count from the server, else the documented defaults, and the models'
|
||||
* download status (`models`, null when unknown).
|
||||
*/
|
||||
async function readDefaults(api, signal) {
|
||||
try {
|
||||
const value = await api.get(MODELS_URL, {signal, cache: 'no-store'});
|
||||
if (!object(value)) return DEFAULTS;
|
||||
if (!object(value)) return {defaults: DEFAULTS, models: null};
|
||||
return {
|
||||
general_threshold: unit(value.wd14_general_threshold) ? value.wd14_general_threshold : DEFAULTS.general_threshold,
|
||||
character_threshold: unit(value.wd14_character_threshold) ? value.wd14_character_threshold : DEFAULTS.character_threshold,
|
||||
instructions: typeof value.default_prompt === 'string' ? value.default_prompt : DEFAULTS.instructions,
|
||||
suggestion_limit: limit(value.suggestion_limit) ? value.suggestion_limit : DEFAULTS.suggestion_limit,
|
||||
defaults: {
|
||||
general_threshold: unit(value.wd14_general_threshold) ? value.wd14_general_threshold : DEFAULTS.general_threshold,
|
||||
character_threshold: unit(value.wd14_character_threshold) ? value.wd14_character_threshold : DEFAULTS.character_threshold,
|
||||
instructions: typeof value.default_prompt === 'string' ? value.default_prompt : DEFAULTS.instructions,
|
||||
suggestion_limit: limit(value.suggestion_limit) ? value.suggestion_limit : DEFAULTS.suggestion_limit,
|
||||
},
|
||||
models: object(value.models) ? value.models : null,
|
||||
};
|
||||
} catch {
|
||||
// The defaults keep the dialog usable; Generate reports a real failure itself.
|
||||
return DEFAULTS;
|
||||
return {defaults: DEFAULTS, models: null};
|
||||
}
|
||||
}
|
||||
|
||||
function initialState(source) {
|
||||
const UNAVAILABLE_HINT = 'Download it from the legacy Auto-tag page.';
|
||||
/**
|
||||
* Why `model` cannot run, from the `/autotag/models` status (the server's own `model-unavailable` sentence), else ''
|
||||
* (downloaded, or the status is unknown: the job then reports a missing model itself).
|
||||
*/
|
||||
export function modelUnavailable(models, model) {
|
||||
const option = modelOf(model);
|
||||
requireValue(option, 'Unknown tagging model');
|
||||
const status = object(models) && object(models[model]) ? models[model] : null;
|
||||
if (!status || status.downloaded !== false) return '';
|
||||
const name = typeof status.name === 'string' && status.name.trim() ? status.name : option.label.split(' · ')[0];
|
||||
return `${name} is not downloaded. ${UNAVAILABLE_HINT}`;
|
||||
}
|
||||
|
||||
function initialState(source, preset) {
|
||||
return {
|
||||
phase: 'observing', closed: false, source, showScopeNote: source === 'review',
|
||||
image: null, savedTags: [], tagRevision: 0, currentTags: null, manual: '',
|
||||
model: MODEL_OPTIONS[0].id, options: {...DEFAULTS}, generated: false, suggestions: [],
|
||||
model: preset?.model ?? MODEL_OPTIONS[0].id, options: {...DEFAULTS, ...preset?.options}, generated: false, suggestions: [],
|
||||
suggestionsText: COPY.modelChanged, statusText: '', generateSupported: false, generateReason: null, toast: null,
|
||||
// Bumped whenever the controller itself rewrites the option fields or the manual draft, never while typing.
|
||||
optionsRevision: 0, draftRevision: 0,
|
||||
@@ -219,13 +240,31 @@ function derive(state) {
|
||||
};
|
||||
}
|
||||
|
||||
export function createImageTaggingController({api, target, source = 'images', onLateSave = () => {}} = {}) {
|
||||
/** A `{model, options}` preset (the bulk review's choice) when valid, else a TypeError. */
|
||||
function validatePreset(preset) {
|
||||
if (preset === null || preset === undefined) return null;
|
||||
requireValue(object(preset) && modelOf(preset.model) && object(preset.options), 'Invalid tagging preset');
|
||||
const options = {};
|
||||
for (const [key, value] of Object.entries(preset.options)) {
|
||||
requireValue(OPTION_KEYS.has(key), 'Unknown tagging option');
|
||||
options[key] = value;
|
||||
}
|
||||
requireValue(!optionError(preset.model, {...DEFAULTS, ...options}), 'Invalid tagging preset');
|
||||
return Object.freeze({model: preset.model, options: Object.freeze(options)});
|
||||
}
|
||||
|
||||
/**
|
||||
* `preset` ({model, options}, D90's review session) starts the dialog on that model, and every observation applies
|
||||
* those options over the server's defaults.
|
||||
*/
|
||||
export function createImageTaggingController({api, target, source = 'images', onLateSave = () => {}, preset = null} = {}) {
|
||||
requireValue(object(api) && ['get', 'post', 'put', 'del'].every(name => typeof api[name] === 'function'), 'An api client is required');
|
||||
const imageTarget = validateTarget(target);
|
||||
requireValue(SOURCES.has(source), 'Unknown tagging source');
|
||||
requireValue(typeof onLateSave === 'function', 'onLateSave must be a function');
|
||||
const chosen = validatePreset(preset);
|
||||
|
||||
let state = freeze(derive(initialState(source)));
|
||||
let state = freeze(derive(initialState(source, chosen)));
|
||||
const listeners = new Set();
|
||||
let closed = false;
|
||||
let capture = null;
|
||||
@@ -297,7 +336,7 @@ export function createImageTaggingController({api, target, source = 'images', on
|
||||
}
|
||||
capture = {token: value.capture.token, sha256: value.capture.sha256};
|
||||
epoch = value.library_epoch;
|
||||
defaults = await defaultsRead;
|
||||
({defaults} = await defaultsRead);
|
||||
if (closed || serial !== openSerial) return;
|
||||
observedOnce = true;
|
||||
const {width, height, media_type: mediaType, preview_url: previewUrl, generate} = value.capture;
|
||||
@@ -311,7 +350,7 @@ export function createImageTaggingController({api, target, source = 'images', on
|
||||
savedTags: value.tags.slice(), tagRevision: value.tag_revision,
|
||||
manual: draft === null ? value.tags.join(', ') : changed ? mergeCurrent(value.tags, draft) : draft,
|
||||
draftRevision: state.draftRevision + 1,
|
||||
options: {...defaults}, optionsRevision: state.optionsRevision + 1,
|
||||
options: {...defaults, ...chosen?.options}, optionsRevision: state.optionsRevision + 1,
|
||||
generateSupported: generate.supported, generateReason: generate.reason,
|
||||
statusText: changed ? `${COPY.changedElsewhere} ${savedNow(value.tags)}${reason ? ` ${reason}` : ''}` : reason,
|
||||
});
|
||||
@@ -482,3 +521,66 @@ export function createImageTaggingController({api, target, source = 'images', on
|
||||
subscribe, open, setManual, setModel, setOption, generate, remove, save, retry, cancel, dispose,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The model and options of the bulk "Auto-tag selected images" dialog (D85): the same fields as the Tag image dialog's
|
||||
* model panel, without an image. open() reads the server's defaults and the models' download status once; until it
|
||||
* lands Start waits (`canStart` false). `unavailable` is why the chosen model cannot run ('' when it can), so the
|
||||
* dialog refuses to start a job that would fail at once.
|
||||
*/
|
||||
export function createTaggingOptionsController({api} = {}) {
|
||||
requireValue(object(api) && typeof api.get === 'function', 'An api client is required');
|
||||
const listeners = new Set();
|
||||
let closed = false;
|
||||
let defaults = DEFAULTS;
|
||||
let models = null;
|
||||
let abort = null;
|
||||
const deriveOptions = value => ({...value, canGenerate: false, unavailable: modelUnavailable(models, value.model),
|
||||
canStart: value.phase === 'ready' && !modelUnavailable(models, value.model)});
|
||||
let state = freeze(deriveOptions({phase: 'loading', model: MODEL_OPTIONS[0].id, options: {...DEFAULTS}, optionsRevision: 0}));
|
||||
|
||||
function set(patch) {
|
||||
if (closed) return;
|
||||
state = freeze(deriveOptions({...state, ...patch}));
|
||||
for (const listener of [...listeners]) listener(state);
|
||||
}
|
||||
async function open() {
|
||||
if (closed || abort) return;
|
||||
abort = new AbortController();
|
||||
const read = await readDefaults(api, abort.signal);
|
||||
if (closed) return;
|
||||
({defaults, models} = read);
|
||||
set({phase: 'ready', options: {...defaults}, optionsRevision: state.optionsRevision + 1});
|
||||
}
|
||||
function setModel(id) {
|
||||
requireValue(modelOf(id), 'Unknown tagging model');
|
||||
if (closed || id === state.model) return;
|
||||
// As in the Tag image dialog: the model's fields start from the defaults again; the count is kept (D91).
|
||||
set({model: id, optionsRevision: state.optionsRevision + 1,
|
||||
options: {...state.options, general_threshold: defaults.general_threshold,
|
||||
character_threshold: defaults.character_threshold, instructions: defaults.instructions}});
|
||||
}
|
||||
function setOption(key, value) {
|
||||
requireValue(OPTION_KEYS.has(key), 'Unknown tagging option');
|
||||
set({options: {...state.options, [key]: value}});
|
||||
}
|
||||
/** `{model_type, options}` for a job or a review session, or null while the options are invalid. */
|
||||
function request() {
|
||||
if (optionError(state.model, state.options)) return null;
|
||||
return {model_type: state.model, options: requestOptions(state.model, state.options)};
|
||||
}
|
||||
function dispose() {
|
||||
closed = true;
|
||||
listeners.clear();
|
||||
abort?.abort();
|
||||
}
|
||||
function subscribe(listener) {
|
||||
requireValue(typeof listener === 'function', 'A listener function is required');
|
||||
listeners.add(listener);
|
||||
return () => listeners.delete(listener);
|
||||
}
|
||||
return {
|
||||
get state() {return state;},
|
||||
subscribe, open, setModel, setOption, request, dispose,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -101,10 +101,12 @@ function numberInput(value, {min, max, step}) {
|
||||
|
||||
/**
|
||||
* The concept's `.tag-model-panel` (a1049): `heading`, the Tagging model select, the model's options and Generate.
|
||||
* It follows `controller` until `scope` is disposed (reused by the bulk Auto-tag dialog). Returns the `element` and
|
||||
* `requestGenerate()`, Generate's own path (field validity first), for a Retry after a failed Generate.
|
||||
* It follows `controller` until `scope` is disposed (reused by the bulk Auto-tag dialog, D85, without Generate:
|
||||
* `generate: false`). Returns the `element`, `validate()` (the fields' native validity, then what the controller would
|
||||
* refuse; true when valid, else the first invalid field reports) and `requestGenerate()`, Generate's own path (field
|
||||
* validity first), for a Retry after a failed Generate.
|
||||
*/
|
||||
export function renderModelPanel({scope, controller, heading = 'Auto-tag this image'}) {
|
||||
export function renderModelPanel({scope, controller, heading = 'Auto-tag this image', generate: withGenerate = true}) {
|
||||
const panel = el('div', undefined, 'tag-model-panel');
|
||||
const select = el('select');
|
||||
for (const option of MODEL_OPTIONS) {
|
||||
@@ -116,7 +118,8 @@ export function renderModelPanel({scope, controller, heading = 'Auto-tag this im
|
||||
options.id = 'image-tag-model-options';
|
||||
const generate = createButton('', 'ca-primary');
|
||||
generate.append(createWorkspaceIcon('tagImage'), ' Generate suggestions');
|
||||
panel.append(el('h3', heading), field('Tagging model', select, 'image-tag-model'), options, generate);
|
||||
panel.append(el('h3', heading), field('Tagging model', select, 'image-tag-model'), options);
|
||||
if (withGenerate) panel.append(generate);
|
||||
const inputs = new Map();
|
||||
let shownModel = null, shownOptions = null;
|
||||
|
||||
@@ -161,26 +164,28 @@ export function renderModelPanel({scope, controller, heading = 'Auto-tag this im
|
||||
setDisabled(generate, !state.canGenerate);
|
||||
}
|
||||
select.addEventListener('change', () => controller.setModel(select.value));
|
||||
function requestGenerate() {
|
||||
function validate() {
|
||||
const {state} = controller;
|
||||
if (!state.canGenerate) return;
|
||||
// The concept's native validity first (a1104–1114), then what the controller would refuse (a blank field).
|
||||
for (const input of inputs.values()) input.setCustomValidity('');
|
||||
const invalid = [...inputs.values()].find(input => !input.checkValidity());
|
||||
if (invalid) { invalid.reportValidity(); return; }
|
||||
if (invalid) { invalid.reportValidity(); return false; }
|
||||
const error = optionError(state.model, state.options);
|
||||
if (error) {
|
||||
const input = inputs.get(error.field);
|
||||
input.setCustomValidity(error.message);
|
||||
input.reportValidity();
|
||||
return;
|
||||
return false;
|
||||
}
|
||||
controller.generate();
|
||||
return true;
|
||||
}
|
||||
function requestGenerate() {
|
||||
if (controller.state.canGenerate && validate()) controller.generate();
|
||||
}
|
||||
generate.addEventListener('click', () => { if (!disabled(generate)) requestGenerate(); });
|
||||
scope.cleanup(controller.subscribe(update));
|
||||
update(controller.state);
|
||||
return {element: panel, requestGenerate};
|
||||
return {element: panel, requestGenerate, validate};
|
||||
}
|
||||
|
||||
function previewOf(image, state) {
|
||||
@@ -192,20 +197,33 @@ function previewOf(image, state) {
|
||||
|
||||
/**
|
||||
* Open "Tag image" for one listed `image` (its tagging target). `source` is the page (`images`, `library`, `review`,
|
||||
* `tags`); `onSaved(event)` runs after a confirmed save, as every `onImageTagsSaved` listener does.
|
||||
* `tags`, `bulk`); `onSaved(event)` runs after a confirmed save, as every `onImageTagsSaved` listener does.
|
||||
*
|
||||
* D90 (the bulk review session, `images-tagging-review.mjs`): `actions: 'review'` with `position: {index, total}`
|
||||
* titles the dialog "Tag image · Image <index> of <total>" and makes its actions Stop reviewing, Skip and Approve and
|
||||
* next (Approve and finish on the last image). `preset` ({model, options}) applies the session's model and options,
|
||||
* and `autoGenerate` runs Generate as soon as the file is observed. Approve saves as Save does, without the per-image
|
||||
* toast (the session toasts its count), and calls `onSaved`; Skip writes nothing and calls `onSkip`; Stop reviewing,
|
||||
* Escape and × write nothing and call `onStop` once. Swapping the next image's dialog in is not a stop.
|
||||
*/
|
||||
export function openImageTagging({api, scope, image, source = 'images', trigger = null, onSaved = null}) {
|
||||
export function openImageTagging({api, scope, image, source = 'images', trigger = null, onSaved = null, preset = null,
|
||||
autoGenerate = false, actions: mode = 'tag', position = null, onSkip = null, onStop = null}) {
|
||||
const target = imageTarget(image);
|
||||
if (!target) throw new TypeError('The image has no tagging target');
|
||||
const review = mode === 'review';
|
||||
if (review && !(Number.isSafeInteger(position?.index) && Number.isSafeInteger(position?.total) &&
|
||||
position.index >= 1 && position.index <= position.total)) throw new TypeError('A review needs its position');
|
||||
// Set once this dialog hands over to the session (Approve, Skip): its close is then not a Stop.
|
||||
let advancing = false;
|
||||
const saved = ({filename: name, tags, toast}) => {
|
||||
const event = {target, filename: name, tags, source};
|
||||
announceSaved(event);
|
||||
if (toast) showWorkspaceToast(toast);
|
||||
onSaved?.(event);
|
||||
showWorkspaceToast(toast);
|
||||
};
|
||||
// A save that settles after the dialog closed is still reported: its toast, and the pages' refresh (I5).
|
||||
const onLateSave = outcome => (outcome.saved ? saved(outcome) : showWorkspaceToast(outcome.message));
|
||||
const controller = createImageTaggingController({api, target, source, onLateSave});
|
||||
const controller = createImageTaggingController({api, target, source, onLateSave, preset});
|
||||
const local = createScope();
|
||||
let open = true;
|
||||
let completed = false;
|
||||
@@ -253,9 +271,15 @@ export function openImageTagging({api, scope, image, source = 'images', trigger
|
||||
// 7. The actions; Retry leads them only while the controller allows it.
|
||||
const actions = el('div', undefined, 'dialog-actions');
|
||||
const retry = createButton('Retry');
|
||||
const cancel = createButton('Cancel');
|
||||
const cancel = createButton(review ? 'Stop reviewing' : 'Cancel'); // D90
|
||||
const skip = createButton('Skip');
|
||||
const primary = createButton('Save manual tags', 'ca-primary');
|
||||
actions.append(cancel, primary);
|
||||
const last = review && position.index === position.total;
|
||||
// D90: Approve with suggestions on screen; after a model change cleared them (M2) Save keeps only the manual field.
|
||||
const reviewLabel = state => (state.generated ? (last ? 'Approve and finish' : 'Approve and next')
|
||||
: (last ? 'Save and finish' : 'Save and next'));
|
||||
if (review) actions.append(cancel, skip, primary);
|
||||
else actions.append(cancel, primary);
|
||||
|
||||
const nodes = [heading];
|
||||
if (controller.state.showScopeNote) nodes.push(el('p', SCOPE_NOTE, 'tag-scope-note small'));
|
||||
@@ -301,6 +325,12 @@ export function openImageTagging({api, scope, image, source = 'images', trigger
|
||||
}
|
||||
function complete(state) {
|
||||
completed = true;
|
||||
if (review) {
|
||||
// D90: the session swaps the next image in (or ends), so this dialog is not closed here.
|
||||
advancing = true;
|
||||
saved({filename: state.image.filename, tags: state.savedTags.slice(), toast: null});
|
||||
return;
|
||||
}
|
||||
closeWorkspaceDialog();
|
||||
saved({filename: state.image.filename, tags: state.savedTags.slice(), toast: state.toast});
|
||||
}
|
||||
@@ -315,8 +345,10 @@ export function openImageTagging({api, scope, image, source = 'images', trigger
|
||||
if (state.suggestions !== shownRows || state.suggestionsText !== shownText) renderSuggestions(state);
|
||||
setText(count, state.countText);
|
||||
setText(status, state.statusText);
|
||||
setText(primary, state.primaryLabel);
|
||||
setText(primary, review ? reviewLabel(state) : state.primaryLabel);
|
||||
setDisabled(primary, !state.canSave);
|
||||
// D90: while Approve's save is in flight the session cannot be left, so its count includes that save.
|
||||
if (review) for (const button of [skip, cancel]) setDisabled(button, state.phase === 'saving');
|
||||
if (state.canRetry !== retry.isConnected) {
|
||||
if (state.canRetry) actions.prepend(retry); // D116
|
||||
else {
|
||||
@@ -327,7 +359,12 @@ export function openImageTagging({api, scope, image, source = 'images', trigger
|
||||
if (state.phase === 'saved') complete(state);
|
||||
}
|
||||
|
||||
cancel.addEventListener('click', () => closeWorkspaceDialog());
|
||||
cancel.addEventListener('click', () => { if (!disabled(cancel)) closeWorkspaceDialog(); });
|
||||
skip.addEventListener('click', () => {
|
||||
if (!open || advancing || disabled(skip)) return;
|
||||
advancing = true;
|
||||
onSkip?.();
|
||||
});
|
||||
// Retry after a failed Generate takes Generate's own path, so an invalid field reports its validity (M4).
|
||||
retry.addEventListener('click', () => (controller.state.phase === 'generate-error' ? requestGenerate() : controller.retry()));
|
||||
primary.addEventListener('click', () => {
|
||||
@@ -345,12 +382,17 @@ export function openImageTagging({api, scope, image, source = 'images', trigger
|
||||
document.getElementById('modal')?.classList.remove('tagging-modal');
|
||||
local.dispose();
|
||||
controller.dispose();
|
||||
if (review && !advancing) onStop?.();
|
||||
}
|
||||
openContentDialog({scope, title: 'Tag image', nodes, trigger, onClose: release});
|
||||
const dialogTitle = review ? `Tag image · Image ${position.index} of ${position.total}` : 'Tag image'; // D90
|
||||
openContentDialog({scope, title: dialogTitle, nodes, trigger, onClose: release,
|
||||
canClose: review ? () => controller.state.phase !== 'saving' : null});
|
||||
document.getElementById('modal')?.classList.add('tagging-modal');
|
||||
local.cleanup(controller.subscribe(render));
|
||||
render(controller.state);
|
||||
controller.open();
|
||||
controller.open().then(() => {
|
||||
if (open && autoGenerate && controller.state.canGenerate) controller.generate();
|
||||
});
|
||||
return {
|
||||
close() { if (open) closeWorkspaceDialog(); },
|
||||
get controller() { return controller; },
|
||||
|
||||
@@ -4,6 +4,7 @@ import { inspectPath, toInspectionRecord } from './inspection-state.mjs';
|
||||
import { BUNDLE_FILENAME, metadataBundle, collectImagesMetadata, exportOutcome } from './images-export.mjs';
|
||||
import { CATEGORY_DIALOG, CATEGORY_FAILED, CATEGORY_MAX_LENGTH, CATEGORY_NONE_LINKED, categoryOutcome, categoryTargets, updatedCount } from './images-category.mjs';
|
||||
import { openInputDialog } from './workspace-dialogs.mjs';
|
||||
import { openAddTags, openAutoTag } from './images-tagging-bulk.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
|
||||
const CHECKBOX = '.image-overlays input[type="checkbox"]';
|
||||
@@ -26,8 +27,11 @@ function el(tag, text, className) {
|
||||
* - `onCategorySet()` reloads the categories and the listing after a successful Set category (the reload clears the
|
||||
* selection through `clear`); focus then moves to `focusFallback` if it was on the hidden action.
|
||||
* - `retain(kept)` drops the checked files a refreshed listing no longer holds (a tag save under a filter, I7).
|
||||
* - With the Activity store `activity`, Add tags and Auto-tag selected (D85) follow the count, as the concept's Add tags
|
||||
* leads its actions; `onTagged(keys)` re-reads the files a finished tagging job tagged. The selection stays.
|
||||
*/
|
||||
export function mountSelectionBar({ scope, api, grid, getItems, focusFallback, onCategorySet, checked: initial = Object.freeze(new Set()) }) {
|
||||
export function mountSelectionBar({ scope, api, grid, getItems, focusFallback, onCategorySet, activity = null, onTagged,
|
||||
checked: initial = Object.freeze(new Set()) }) {
|
||||
let checked = initial;
|
||||
let exporting = false, categorizing = false;
|
||||
const element = el('div');
|
||||
@@ -40,7 +44,15 @@ export function mountSelectionBar({ scope, api, grid, getItems, focusFallback, o
|
||||
const clearButton = createButton('Clear');
|
||||
// Clear hides the bar it sits in, so focus moves to the page heading instead of falling to the page.
|
||||
scope.listen(clearButton, 'click', () => { clear(); focusFallback.focus({ preventScroll: true }); });
|
||||
bulk.append(count, setCategory, exportMetadata, clearButton);
|
||||
bulk.append(count);
|
||||
if (activity) {
|
||||
const addTags = createButton('Add tags');
|
||||
scope.listen(addTags, 'click', () => openAddTags({ scope, store: activity, images: checkedImages(), trigger: addTags, onTagged }));
|
||||
const autoTag = createButton('Auto-tag selected'); // D85
|
||||
scope.listen(autoTag, 'click', () => openAutoTag({ api, scope, store: activity, images: checkedImages(), trigger: autoTag, onTagged }));
|
||||
bulk.append(addTags, autoTag);
|
||||
}
|
||||
bulk.append(setCategory, exportMetadata, clearButton);
|
||||
|
||||
/** The count updates in place: the same bar and `strong`, its text written only when it changes. */
|
||||
function render() {
|
||||
|
||||
@@ -0,0 +1,192 @@
|
||||
/**
|
||||
* The Images selection bar's tagging actions (#249 PR D, Mock inventory M3):
|
||||
*
|
||||
* - **Add tags** (the concept's `bulk-tags` simpleInput, app.js a1378–1383): "Tag selected images", "Tags, separated
|
||||
* by commas", Apply. The names are parsed as the Tag image dialog's manual field parses them, then the `image-tags`
|
||||
* Activity job adds them to every selected image.
|
||||
* - **Auto-tag selected** (D85, ruling L1, beyond the concept): "Auto-tag selected images" with the Tag image dialog's
|
||||
* model panel (its fields show the server's defaults, D91's Suggestions included) and no Generate, then **How to
|
||||
* apply** (D90, ruling T3-bulk): **Apply automatically** starts the job, **Review each image** starts the browser
|
||||
* review session (images-tagging-review.mjs). A model that is not downloaded keeps Start disabled and says why.
|
||||
*
|
||||
* The toast says the job started (the Activity store's "Tagging started." / "Auto-tagging started."), or why it did
|
||||
* not: a running record-changing job (named, with its link), a held model, a refused body. Activity shows progress.
|
||||
* The selection stays either way. When a started job ends having tagged files, `onTagged(keys)` re-reads them.
|
||||
*/
|
||||
import { createButton, createScope } from '../shared/toolkit/core.js';
|
||||
import { createTaggingOptionsController, parseManualTags } from './image-tagging-state.mjs';
|
||||
import { imageTarget, renderModelPanel } from './image-tagging.mjs';
|
||||
import { startReviewSession } from './images-tagging-review.mjs';
|
||||
import { closeWorkspaceDialog, openContentDialog, openInputDialog } from './workspace-dialogs.mjs';
|
||||
import { busyToast } from './activity-scan.mjs';
|
||||
import { imageKey } from './images-state.mjs';
|
||||
import { countOf } from './plural.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
|
||||
export const IMAGE_TAG_JOB = 'image-tags';
|
||||
export const MAX_BULK_IMAGES = 500;
|
||||
export const TOO_MANY = 'Select at most 500 images.';
|
||||
export const NO_TAGS = 'Enter at least one tag.'; // D118
|
||||
export const ADD_DIALOG = Object.freeze({title: 'Tag selected images', label: 'Tags, separated by commas', action: 'Apply'});
|
||||
export const AUTO_TITLE = 'Auto-tag selected images';
|
||||
export const APPLY_CHOICES = Object.freeze([
|
||||
Object.freeze({value: 'auto', label: 'Apply automatically', action: 'Start auto-tagging'}),
|
||||
Object.freeze({value: 'review', label: 'Review each image', action: 'Start review'}),
|
||||
]);
|
||||
// D118: the concept has no copy for these; they follow D55's "The scan could not start. Try again.".
|
||||
const START_FAILED = Object.freeze({add: 'Tagging could not start. Try again.', auto: 'Auto-tagging could not start. Try again.'});
|
||||
const FINAL = new Set(['finished', 'cancelled', 'failed']);
|
||||
|
||||
function el(tag, text, className) {
|
||||
const node = document.createElement(tag);
|
||||
if (text !== undefined) node.textContent = text;
|
||||
if (className) node.className = className;
|
||||
return node;
|
||||
}
|
||||
|
||||
/**
|
||||
* The job targets of the checked `images` in listing order (their reveal identity), or null when more than 500 are
|
||||
* checked. A file without a tagging identity is left out.
|
||||
*/
|
||||
export function bulkTargets(images) {
|
||||
if (images.length > MAX_BULK_IMAGES) return null;
|
||||
return images.map(imageTarget).filter(Boolean);
|
||||
}
|
||||
|
||||
/** The Add tags field's names, deduplicated as the Tag image dialog's manual field is. */
|
||||
export const parseBulkTags = parseManualTags;
|
||||
|
||||
// Duck-typed: this ES module cannot import the UMD api-client.js.
|
||||
const apiMessage = error => (error?.name === 'ApiError' && error.code && error.message ? error.message : null);
|
||||
|
||||
/**
|
||||
* Start one `image-tags` job with `body` through `store.start`. Returns the job, or null after toasting why not: the
|
||||
* busy answer naming the running job, a coded refusal's own sentence (`model-busy`, a bad body), else a fixed one.
|
||||
*/
|
||||
export async function startTagJob({store, scope, body, notify = showWorkspaceToast}) {
|
||||
try {
|
||||
const result = await store.start(IMAGE_TAG_JOB, body);
|
||||
if (scope.signal.aborted) return null;
|
||||
if (result.busy !== undefined) {
|
||||
notify(busyToast(result.busy));
|
||||
return null;
|
||||
}
|
||||
return result.job;
|
||||
} catch (error) {
|
||||
if (!scope.signal.aborted) notify(apiMessage(error) ?? START_FAILED[body.mode]);
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
/** Call `onEnd(job)` once the job `id` has ended in `store`; returns the unsubscribe. */
|
||||
export function followJobEnd({store, scope, id, onEnd}) {
|
||||
let done = false;
|
||||
const check = () => {
|
||||
const job = store.find(id);
|
||||
if (done || !job || !FINAL.has(job.state)) return;
|
||||
done = true;
|
||||
stop();
|
||||
onEnd(job);
|
||||
};
|
||||
const stop = store.subscribe(check);
|
||||
scope.cleanup(stop);
|
||||
check();
|
||||
return stop;
|
||||
}
|
||||
|
||||
/**
|
||||
* Wire one started job: when it ends having tagged files, `onTagged(keys)` gets the image keys it was started for.
|
||||
*/
|
||||
function followTagged({store, scope, job, images, onTagged}) {
|
||||
const keys = new Set(images.map(imageKey));
|
||||
followJobEnd({store, scope, id: job.id, onEnd: ended => { if ((ended.counters.tagged ?? 0) > 0) onTagged(keys); }});
|
||||
}
|
||||
|
||||
/**
|
||||
* Add tags: the concept's simpleInput, then the job over `images` (the checked files, taken now). A field that parses
|
||||
* to no names keeps the dialog and its text, and reports NO_TAGS on the field (D118). `openInput` is the dialog seam.
|
||||
*/
|
||||
export function openAddTags({scope, store, images, trigger, onTagged = () => {}, notify = showWorkspaceToast,
|
||||
openInput = openInputDialog}) {
|
||||
const targets = bulkTargets(images);
|
||||
if (!targets) { notify(TOO_MANY); return; }
|
||||
openInput({scope, trigger, ...ADD_DIALOG, validate: text => (parseBulkTags(text).length ? null : NO_TAGS),
|
||||
onApply: async text => {
|
||||
const tags = parseBulkTags(text);
|
||||
const job = await startTagJob({store, scope, body: {mode: 'add', targets, tags}, notify});
|
||||
if (job) followTagged({store, scope, job, images, onTagged});
|
||||
}});
|
||||
}
|
||||
|
||||
/** The D90 How to apply radio group: Apply automatically (checked) or Review each image. */
|
||||
function applyChoice(onChange) {
|
||||
const group = el('div', undefined, 'field');
|
||||
group.setAttribute('role', 'radiogroup');
|
||||
const caption = el('span', 'How to apply');
|
||||
caption.id = 'auto-tag-apply-label';
|
||||
group.setAttribute('aria-labelledby', caption.id);
|
||||
group.append(caption);
|
||||
for (const choice of APPLY_CHOICES) {
|
||||
const label = el('label');
|
||||
const input = el('input');
|
||||
Object.assign(input, {type: 'radio', name: 'auto-tag-apply', value: choice.value, checked: choice.value === 'auto'});
|
||||
input.addEventListener('change', () => { if (input.checked) onChange(choice); });
|
||||
label.append(input, ` ${choice.label}`);
|
||||
group.append(label);
|
||||
}
|
||||
return group;
|
||||
}
|
||||
|
||||
/**
|
||||
* Auto-tag selected (D85): the model panel, How to apply (D90), Cancel and Start. Start validates the fields as
|
||||
* Generate does, then starts the job or the review session over `images` (the checked files, taken now).
|
||||
*/
|
||||
export function openAutoTag({api, scope, store, images, trigger, onTagged = () => {}, notify = showWorkspaceToast}) {
|
||||
const targets = bulkTargets(images);
|
||||
if (!targets) { notify(TOO_MANY); return; }
|
||||
const listed = images.filter(imageTarget);
|
||||
const local = createScope();
|
||||
const options = createTaggingOptionsController({api});
|
||||
// D85: the Tag image dialog's model panel without Generate; its Suggestions field is the job's count (D91).
|
||||
const {element: panel, validate} = renderModelPanel({scope: local, controller: options,
|
||||
heading: `Auto-tag ${countOf(listed.length, 'image')}`, generate: false}); // D118
|
||||
let choice = APPLY_CHOICES[0];
|
||||
const actions = el('div', undefined, 'dialog-actions');
|
||||
const cancel = createButton('Cancel');
|
||||
const start = createButton(choice.action, 'ca-primary');
|
||||
actions.append(cancel, start);
|
||||
// A model that is not downloaded: the reason, as the Tag image dialog's status line words it.
|
||||
const status = el('div', undefined, 'small');
|
||||
status.id = 'tag-generation-status';
|
||||
status.setAttribute('role', 'status');
|
||||
const choices = applyChoice(next => { choice = next; start.textContent = next.action; }); // D90
|
||||
const render = state => {
|
||||
if (status.textContent !== state.unavailable) status.textContent = state.unavailable;
|
||||
if (state.canStart) start.removeAttribute('aria-disabled');
|
||||
else start.setAttribute('aria-disabled', 'true');
|
||||
};
|
||||
local.cleanup(options.subscribe(render));
|
||||
render(options.state);
|
||||
cancel.addEventListener('click', () => closeWorkspaceDialog());
|
||||
start.addEventListener('click', () => {
|
||||
if (start.getAttribute('aria-disabled') === 'true' || !options.state.canStart || !validate()) return;
|
||||
const request = options.request();
|
||||
if (!request) return;
|
||||
if (choice.value === 'review') {
|
||||
// The first image's dialog replaces this one in the shell (D90).
|
||||
startReviewSession({api, scope, images: listed, ...request, trigger});
|
||||
return;
|
||||
}
|
||||
closeWorkspaceDialog();
|
||||
startTagJob({store, scope, body: {mode: 'auto', targets, ...request}, notify}).then(job => {
|
||||
if (job) followTagged({store, scope, job, images: listed, onTagged});
|
||||
});
|
||||
});
|
||||
openContentDialog({scope, title: AUTO_TITLE, nodes: [panel, choices, status, actions], trigger, onClose: () => {
|
||||
document.getElementById('modal')?.classList.remove('tagging-modal');
|
||||
local.dispose();
|
||||
options.dispose();
|
||||
}});
|
||||
document.getElementById('modal')?.classList.add('tagging-modal');
|
||||
options.open();
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
/**
|
||||
* D90 (ruling T3-bulk): "Review each image", the bulk Auto-tag session in the browser. The selected images are walked
|
||||
* one Tag image dialog at a time (image-tagging.mjs `actions: 'review'`), each opened with the session's model and
|
||||
* options and Generate already running. Approve saves that image as the dialog's Save does and advances; Skip
|
||||
* advances without a write; Stop reviewing, Escape and × end the session. Each image is one gated `suggest` call, so
|
||||
* the session holds no model between images; a refused one (`model-busy`) stays on its image with the dialog's own
|
||||
* status and Retry until the user retries or skips. Ending toasts "<n> of <M> images tagged."; the selection stays.
|
||||
*/
|
||||
import { closeWorkspaceDialog } from './workspace-dialogs.mjs';
|
||||
import { openImageTagging } from './image-tagging.mjs';
|
||||
import { showWorkspaceToast } from './toast.mjs';
|
||||
|
||||
/** The closing toast: approved saves of the session's images (the plan's copy verbatim). */
|
||||
export function reviewOutcome(tagged, total) {
|
||||
return `${tagged} of ${total} images tagged.`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk `images` (listing items with a tagging target, in selection order) with `model_type` and `options`.
|
||||
* `onSaved(event)` follows every approved save. Returns `{stop()}`, which ends the session (its open dialog closes and
|
||||
* writes nothing more). `open`, `close` and `notify` are the dialog seams (the defaults in the page).
|
||||
*/
|
||||
export function startReviewSession({api, scope, images, model_type: model, options, trigger = null, onSaved = null,
|
||||
open = openImageTagging, close = closeWorkspaceDialog, notify = showWorkspaceToast}) {
|
||||
if (!Array.isArray(images) || !images.length) throw new TypeError('A review needs its images');
|
||||
const total = images.length;
|
||||
let index = 0, tagged = 0, ended = false, handle = null;
|
||||
|
||||
function end() {
|
||||
if (ended) return;
|
||||
ended = true;
|
||||
handle = null;
|
||||
notify(reviewOutcome(tagged, total));
|
||||
}
|
||||
function advance() {
|
||||
if (ended) return;
|
||||
index += 1;
|
||||
if (index >= total) {
|
||||
// The last dialog handed over (advancing), so closing it is not a Stop.
|
||||
end();
|
||||
close();
|
||||
return;
|
||||
}
|
||||
show();
|
||||
}
|
||||
function show() {
|
||||
let current = true;
|
||||
const settled = () => {
|
||||
const was = current;
|
||||
current = false;
|
||||
return was;
|
||||
};
|
||||
handle = open({api, scope, image: images[index], source: 'bulk', trigger, preset: {model, options},
|
||||
autoGenerate: true, actions: 'review', position: {index: index + 1, total},
|
||||
onSaved: event => {
|
||||
// A save that settles after its dialog was left (Skip or Stop while it ran) still counts.
|
||||
tagged += 1;
|
||||
onSaved?.(event);
|
||||
if (settled()) advance();
|
||||
},
|
||||
onSkip: () => { if (settled()) advance(); },
|
||||
onStop: () => { if (settled()) end(); },
|
||||
});
|
||||
}
|
||||
|
||||
show();
|
||||
return {
|
||||
stop() {
|
||||
if (ended) return;
|
||||
const open = handle;
|
||||
end();
|
||||
open?.close();
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -23,10 +23,13 @@ export function openCardTagging({api, scope, image, action, embedded}) {
|
||||
/**
|
||||
* Follow confirmed tag saves until `scope` ends. `filtered()` tells whether a Tags filter or a search is active,
|
||||
* `selection()` is the selected image key, and `onRefreshing(busy)` marks Load more busy while the re-read runs.
|
||||
* Returns `refresh(keys)`, the same re-read for the files a bulk tagging job tagged (once per job, D85).
|
||||
*/
|
||||
export function followImageTagSaves({scope, stream, inspection, filtered, selection, onRefreshing}) {
|
||||
scope.cleanup(onImageTagsSaved(async ({target}) => {
|
||||
const key = `${target.root_index}:${target.path}`;
|
||||
scope.cleanup(onImageTagsSaved(({target}) => refresh(new Set([`${target.root_index}:${target.path}`]))));
|
||||
return refresh;
|
||||
|
||||
async function refresh(keys) {
|
||||
if (filtered()) {
|
||||
onRefreshing(true);
|
||||
try {
|
||||
@@ -37,6 +40,6 @@ export function followImageTagSaves({scope, stream, inspection, filtered, select
|
||||
if (!scope.signal.aborted) onRefreshing(false);
|
||||
}
|
||||
}
|
||||
if (!scope.signal.aborted && selection() === key) inspection.refresh();
|
||||
}));
|
||||
if (!scope.signal.aborted && keys.has(selection())) inspection.refresh();
|
||||
}
|
||||
}
|
||||
|
||||
@@ -254,7 +254,7 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
|
||||
// The checked cards and the concept's selection bar (#249 D8), between the filter row and the gallery layout. The
|
||||
// selection is kept across appended batches and cleared with the collection; the Library-embedded gallery has none.
|
||||
const bar = embedded ? null : mountSelectionBar({scope, api, grid, getItems:() => state.items, focusFallback,
|
||||
checked:visit?.checked, onCategorySet:() => { loadCategories(); reset(); }});
|
||||
checked:visit?.checked, onCategorySet:() => { loadCategories(); reset(); }, activity, onTagged:keys => refreshTagged(keys)});
|
||||
if (bar) main.append(bar.element);
|
||||
main.append(body);
|
||||
const mediaViewer = createMediaViewer({api,container:main,parentScope:scope,inspectionController:inspection,sender,
|
||||
@@ -529,7 +529,7 @@ export function mountImages(main, api, parentScope, {embedded = false, fetchPage
|
||||
scheduleLoad();
|
||||
}
|
||||
const stream = createImageStream({ api, fetchPage, onChange: renderState, restore: visit ? {filters, state:visit.state} : undefined });
|
||||
followImageTagSaves({scope, stream, inspection, selection:() => selection, filtered:() => !embedded && !recovering && Boolean(filters.tags.length || filters.q),
|
||||
const refreshTagged = followImageTagSaves({scope, stream, inspection, selection:() => selection, filtered:() => !embedded && !recovering && Boolean(filters.tags.length || filters.q),
|
||||
onRefreshing:busy => more.setAttribute('aria-disabled', String(busy || Boolean(state?.loading) || recovering))});
|
||||
scope.cleanup(() => {
|
||||
stream.dispose();
|
||||
|
||||
@@ -59,10 +59,10 @@ function createShell() {
|
||||
body.id = 'modal-body';
|
||||
header.append(title, close);
|
||||
dialog.append(header, body);
|
||||
close.addEventListener('click', () => finish());
|
||||
close.addEventListener('click', () => dismiss());
|
||||
dialog.addEventListener('cancel', event => {
|
||||
event.preventDefault();
|
||||
finish();
|
||||
dismiss();
|
||||
});
|
||||
// A browser may still close a modal on its own; that is a dismissal too. `close` arrives as a later task, so it
|
||||
// is ignored once this shell was reopened or replaced.
|
||||
@@ -82,6 +82,12 @@ function finish() {
|
||||
if (trigger?.isConnected) trigger.focus();
|
||||
}
|
||||
|
||||
/** × or Escape: close, unless the open dialog's `canClose()` holds it (a save in flight). */
|
||||
function dismiss() {
|
||||
if (session?.canClose && !session.canClose()) return;
|
||||
finish();
|
||||
}
|
||||
|
||||
const boundScopes = new WeakSet();
|
||||
|
||||
function requireScope(scope) {
|
||||
@@ -104,13 +110,13 @@ function bindScope(scope) {
|
||||
});
|
||||
}
|
||||
|
||||
function show(heading, nodes, { scope, trigger, focusTarget, onClose = null }) {
|
||||
function show(heading, nodes, { scope, trigger, focusTarget, onClose = null, canClose = null }) {
|
||||
shell ??= createShell();
|
||||
if (!shell.dialog.isConnected) document.body.append(shell.dialog);
|
||||
bindScope(scope);
|
||||
// A swapped-out body is gone from the shell, so its owner releases it.
|
||||
const replaced = session?.onClose;
|
||||
session = { scope, trigger: trigger ?? session?.trigger ?? null, onClose };
|
||||
session = { scope, trigger: trigger ?? session?.trigger ?? null, onClose, canClose };
|
||||
replaced?.();
|
||||
shell.title.textContent = heading;
|
||||
shell.body.replaceChildren(...nodes);
|
||||
@@ -298,10 +304,14 @@ export function openCopyFallback(text, { scope, trigger = null } = {}) {
|
||||
* The concept's `simpleInput`: one labelled `input#action-input` (starting at `value`), then Cancel and the primary
|
||||
* `action`. The action reports the trimmed value and closes; an empty value keeps the dialog open on its field, or
|
||||
* reports `fallback` when one is given (Save view's "Saved view", a1339). A positive `maxLength` limits the field.
|
||||
* `validate(value)` may refuse a value with a message: the dialog then stays open with the typed text and the field
|
||||
* reports the message (its native validity) until it is edited.
|
||||
*/
|
||||
export function openInputDialog({ scope, title, label, action, value = '', maxLength, fallback, onApply, trigger = null }) {
|
||||
export function openInputDialog({ scope, title, label, action, value = '', maxLength, fallback, onApply, trigger = null,
|
||||
validate = null }) {
|
||||
requireScope(scope);
|
||||
requireCallbacks({ onApply });
|
||||
if (validate !== null) requireCallbacks({ validate });
|
||||
for (const [name, text] of Object.entries({ title, label, action })) {
|
||||
if (typeof text !== 'string' || !text) throw new TypeError(`The input dialog needs a ${name}`);
|
||||
}
|
||||
@@ -314,9 +324,17 @@ export function openInputDialog({ scope, title, label, action, value = '', maxLe
|
||||
const cancel = createButton('Cancel');
|
||||
const apply = createButton(action, 'ca-primary');
|
||||
cancel.addEventListener('click', () => finish());
|
||||
input.addEventListener('input', () => input.setCustomValidity(''));
|
||||
apply.addEventListener('click', () => {
|
||||
const entered = input.value.trim() || fallback;
|
||||
if (!entered) { input.focus(); return; }
|
||||
const problem = validate?.(entered);
|
||||
if (problem) {
|
||||
input.setCustomValidity(problem);
|
||||
input.reportValidity();
|
||||
input.focus();
|
||||
return;
|
||||
}
|
||||
finish();
|
||||
onApply(entered);
|
||||
});
|
||||
@@ -327,14 +345,16 @@ export function openInputDialog({ scope, title, label, action, value = '', maxLe
|
||||
|
||||
/**
|
||||
* The concept's plain `dialog(title, body)` with page-built `nodes`. `onClose` runs once when the dialog closes or
|
||||
* another dialog replaces its body, so the page can dispose what it mounted there.
|
||||
* another dialog replaces its body, so the page can dispose what it mounted there. While `canClose()` answers false,
|
||||
* × and Escape leave the dialog open (the page, a route change or a replacing dialog still close it).
|
||||
*/
|
||||
export function openContentDialog({ scope, title, nodes, onClose = null, trigger = null }) {
|
||||
export function openContentDialog({ scope, title, nodes, onClose = null, trigger = null, canClose = null }) {
|
||||
requireScope(scope);
|
||||
if (typeof title !== 'string' || !title) throw new TypeError('The dialog needs a title');
|
||||
if (!Array.isArray(nodes) || nodes.some(node => !node || typeof node !== 'object')) throw new TypeError('The dialog needs its nodes');
|
||||
if (onClose !== null && typeof onClose !== 'function') throw new TypeError('The dialog onClose must be a function');
|
||||
show(title, nodes, { scope, trigger, onClose });
|
||||
if (canClose !== null && typeof canClose !== 'function') throw new TypeError('The dialog canClose must be a function');
|
||||
show(title, nodes, { scope, trigger, onClose, canClose });
|
||||
}
|
||||
|
||||
export const SCAN_REVIEW_SENTENCE = 'Enrich positive / negative associations, reuse canonical text and report unchanged files. ' +
|
||||
|
||||
@@ -376,3 +376,9 @@ pre.prompt-text { font: inherit; font-size: 12px; line-height: 1.75; margin-bloc
|
||||
/* D83: the chip's × keeps the concept's 40px coarse target. */
|
||||
.ca-tools #tag-suggestions .chip.tag-suggestion > button { min-block-size: 40px; min-inline-size: 40px; }
|
||||
}
|
||||
|
||||
/* Auto-tag selected images (#249 PR D, D85/D90): the Tag image dialog's model panel, then How to apply, a radio group
|
||||
in the concept's `.field` row whose radios keep their own size (the concept's `.field input` is a full-width text
|
||||
field) beside their label, as the concept's own radio rows (s1160 `.compare-item label`). */
|
||||
.field[role="radiogroup"] > label { display: flex; align-items: center; gap: 8px; /* D90 */ }
|
||||
.field[role="radiogroup"] input[type="radio"] { width: auto; margin: 0; /* D90 */ }
|
||||
|
||||
Reference in New Issue
Block a user