* feat(collections): store collections in the library database * feat(collections): add, remove and rename collections and keep members through duplicate cleanup * feat(collections): add collection routes and the Images collection filter * feat(images): store images' own categories in the library database (#332) One new table, image_categories, and its category index, created empty by a forward-only migration that reads and writes only its own objects. Startup checks the structure; restore also checks the values. The store sets, clears and reads own categories by file key (the deepest output source holding the file and the path below it), and copies a removed duplicate's category to a kept copy that has none. * feat(images): duplicate cleanup keeps images' own categories (#332) The kept copy's own category wins; a kept copy without one receives the removed copy's, keyed by the deepest output source holding each file. The removed copy's row is never deleted, so a give-back or a failed delete loses nothing. The job log reports how many were kept. * feat(images): set, remove and list images' own categories (#332) PUT /prompt_manager/images/category sets the own category of up to 500 listed files, DELETE removes it (listed or not), and GET /prompt_manager/images/categories lists the values whose file is listed. Targets are the Images listing identity of an output source, checked by media_target (shared with Collections). The routes never touch prompts, recipes or links; POST /prompt_manager/bulk/category is unchanged. * fix(images): key symlinked output folders and keep key work off the event loop (#332) Review fixes: a source matches by its configured spelling too, so a detected output folder that is a symlink keys its files (routes and duplicate cleanup). File keys are computed in the executor. A path no source holds is a plain 403, a lone surrogate a 400. The install repairs a table that lacks its category index instead of refusing to open. * fix(collections): address review findings * feat(images): set or remove a category on all N matching images (#332) The category routes also accept the Images listing filter of "Select all N matching" (#392) instead of explicit items, resolved now and capped at 500 images like an explicit selection. * test(vendor): count the #332 host case in the execution matrix image-attribution.spec.mjs gains the image category route case, so the execution config lists 92 tests (3 attribution cases per project). * test(browser): install the slideshow clock before wrapping setTimeout The slideshow fixture wrapped window.setTimeout before page.clock.install(), so the viewer's timer ran on real time and clock.runFor() did not drive it; a late WebKit timer failed 'slideshow exposes exact state labels' on CI. * test(legacy): record the image category routes in the legacy contract snapshot (#332) #332 adds PUT/DELETE /prompt_manager/images/category and GET /prompt_manager/images/categories; the legacy pages' routes are unchanged. * test(duplicates): a removed copy with a collection and a category moves both (#331, #332)
158 lines
5.9 KiB
Python
158 lines
5.9 KiB
Python
"""Every path a duplicate copy is recorded under (#249 System S6, Task 17).
|
|
|
|
The search walks each source's real path, but a record names a file the way
|
|
it was seen when it was written: under the folder the image monitor watches,
|
|
the configured spelling, ComfyUI's output folder, a symlink to it, or a
|
|
parent of a source listed later. ``Spellings`` finds them by containment:
|
|
for each known folder spelling ``S`` (resolving to the folder ``R``) and a
|
|
file ``P`` inside ``R``, the file is also recorded as ``S + relpath(P, R)``.
|
|
|
|
Folders are compared by ``os.stat`` identity (``st_dev``, ``st_ino``), not
|
|
by text, so a case variant on a case-insensitive filesystem (macOS, Windows)
|
|
or a symlinked spelling matches. A volume that reports no file id falls
|
|
back to ``normcase(realpath)``. Favorites are keyed by a listed root and the
|
|
path below it, so ``favorites`` lists every listed root that holds a file.
|
|
Blocking: each lookup stats the file's folders once (cached).
|
|
"""
|
|
|
|
import os
|
|
from pathlib import Path
|
|
|
|
from .images import _favorite_relative_key, _favorite_root_key, image_category_key
|
|
|
|
|
|
def _identity(path):
|
|
try:
|
|
info = os.stat(path)
|
|
except OSError:
|
|
return None
|
|
if info.st_ino:
|
|
return ("inode", info.st_dev, info.st_ino)
|
|
return ("path", os.path.normcase(os.path.realpath(str(path))))
|
|
|
|
|
|
class Spellings:
|
|
"""Folder spellings and listed roots, matched to files by containment."""
|
|
|
|
def __init__(self, folders, listed_roots):
|
|
self._cache = {}
|
|
self._folders = []
|
|
for spelled in dict.fromkeys(str(f) for f in folders if f):
|
|
identity = self._identity(spelled)
|
|
if identity is not None:
|
|
self._folders.append((spelled, identity))
|
|
self._listed = []
|
|
for root in dict.fromkeys(str(r) for r in listed_roots if r):
|
|
identity = self._identity(root)
|
|
if identity is not None:
|
|
self._listed.append((root, identity))
|
|
|
|
def _identity(self, path):
|
|
if path not in self._cache:
|
|
self._cache[path] = _identity(path)
|
|
return self._cache[path]
|
|
|
|
def _ancestors(self, path):
|
|
"""``{identity: folder}`` of every folder above ``path``, nearest first."""
|
|
found = {}
|
|
folder = os.path.dirname(os.path.abspath(str(path)))
|
|
while True:
|
|
identity = self._identity(folder)
|
|
if identity is not None:
|
|
found.setdefault(identity, folder)
|
|
parent = os.path.dirname(folder)
|
|
if parent == folder:
|
|
return found
|
|
folder = parent
|
|
|
|
def of(self, path):
|
|
"""``{folder spelling: path spelled under it}``; ``""`` is ``path`` itself."""
|
|
path = os.path.abspath(str(path))
|
|
spelled = {"": path}
|
|
ancestors = self._ancestors(path)
|
|
for folder, identity in self._folders:
|
|
inside = ancestors.get(identity)
|
|
if inside is not None:
|
|
spelled.setdefault(
|
|
folder, os.path.join(folder, os.path.relpath(path, inside))
|
|
)
|
|
return spelled
|
|
|
|
@property
|
|
def listed_roots(self):
|
|
"""The listed roots (the output sources), in order."""
|
|
return [root for root, _identity in self._listed]
|
|
|
|
def category_key(self, path):
|
|
"""*path*'s image category file key (#332), or None outside every source."""
|
|
return image_category_key(self.listed_roots, path)
|
|
|
|
def favorites(self, path):
|
|
"""``{listed root: path below it (POSIX)}`` for each listed root holding it."""
|
|
path = os.path.abspath(str(path))
|
|
ancestors = self._ancestors(path)
|
|
held = {}
|
|
for root, identity in self._listed:
|
|
inside = ancestors.get(identity)
|
|
if inside is not None:
|
|
held[root] = Path(os.path.relpath(path, inside)).as_posix()
|
|
return held
|
|
|
|
|
|
def copy_move(target, kept_stat, spellings):
|
|
"""What ``move_copy_records`` moves for one removal.
|
|
|
|
A record found under a folder spelling moves to the kept copy under
|
|
the same spelling when that folder holds it too, else to its real
|
|
path. Its favorite and its collection memberships move under every
|
|
listed root holding the file.
|
|
"""
|
|
removed = spellings.of(target["abs"])
|
|
kept = spellings.of(target["keeper_abs"])
|
|
targets = {
|
|
spelled: kept.get(folder, target["keeper_abs"])
|
|
for folder, spelled in removed.items()
|
|
}
|
|
kept_favorites = spellings.favorites(target["keeper_abs"])
|
|
favorites = []
|
|
for root, relative in spellings.favorites(target["abs"]).items():
|
|
kept_root, kept_rel = (
|
|
(root, kept_favorites[root])
|
|
if root in kept_favorites
|
|
else (target["keeper_root"], target["keeper_rel"])
|
|
)
|
|
favorites.append(
|
|
(
|
|
_favorite_root_key(root),
|
|
_favorite_relative_key(relative),
|
|
_favorite_root_key(kept_root),
|
|
_favorite_relative_key(kept_rel),
|
|
)
|
|
)
|
|
return {
|
|
"removed": list(removed.values()),
|
|
"kept": list(kept.values()),
|
|
"targets": targets,
|
|
"kept_path": target["keeper_abs"],
|
|
"favorites": favorites,
|
|
"categories": _category_pairs(target, spellings),
|
|
"removed_locator": target["locator"],
|
|
"kept_locator": target["keeper_locator"],
|
|
"sha256": target["hash"],
|
|
"kept_size": kept_stat.st_size,
|
|
"kept_mtime_ns": kept_stat.st_mtime_ns,
|
|
}
|
|
|
|
|
|
def _category_pairs(target, spellings):
|
|
"""``[(removed file key, kept file key)]`` for the image category (#332).
|
|
|
|
From the two absolute paths, not the favorites tuples: those follow the
|
|
listed root, which differs from the file key when sources nest.
|
|
"""
|
|
removed = spellings.category_key(target["abs"])
|
|
kept = spellings.category_key(target["keeper_abs"])
|
|
if removed is None or kept is None:
|
|
return []
|
|
return [(removed, kept)]
|