fix(maintenance): never remove the kept copy; move records before unlink

Final review of #313:

- C1: the duplicate find tells files apart by (st_dev, st_ino), so a
  source listed twice, a symlinked source inside another and a hardlink
  are one file. The preview and the job refuse a removal that
  os.path.samestat matches with its kept copy.
- I1: each file's records move to its verified kept copy in one short
  transaction before that file is unlinked; a failed move deletes
  nothing and a failed unlink leaves the records on the identical kept
  copy. The status line counts files removed and not removed.
- I2: links are looked up under every spelling of the source (the real
  path, the configured and monitored folders, ComfyUI's output).
- I5: image tags join the kept copy as their union and the removed
  copy's pending, undecided Needs review items are settled; the plan
  says so.
- I3: the VACUUM disk check measures SQLite's real temporary folder.
- M1: the token is looked at again with no await before the job starts.
- M2: the backup decision is made once and handed to the job.
- M3: backups are linked into place; startup sweeps interrupted temps.
- M4: Optimize runs alone (new "exclusive" admission) and its plan says
  the database is locked while VACUUM runs.
- M5: the reviewed duplicate merge is strict: collisions and Library
  titles are protected, and errors roll back. The legacy cleanup is
  unchanged.
- M7: 2,000 keep entries within a 1 MiB body. M8: event waits in tests.
- M9: the preview binds files by size, mtime and inode without hashing;
  the job hashes right before each unlink.
This commit is contained in:
Vito Sansevero
2026-10-23 19:11:01 -07:00
parent 237a97d7cb
commit 58ededde6d
15 changed files with 1422 additions and 340 deletions
+23 -3
View File
@@ -24,6 +24,7 @@ from .execution_contexts import ExecutionContextOperations
from .execution_outputs import ExecutionOutputOperations
from .history_reads import HistoryReadOperations
from .health import HealthOperations, iter_consistency_issues
from .duplicate_copies import DuplicateCopyOperations
from .maintenance import ( # noqa: F401 (ORPHANED_PROMPTS_SQL is re-exported)
ORPHANED_PROMPTS_SQL,
_LINK_ROWS_SQL,
@@ -337,6 +338,7 @@ class PromptDatabase(
SavedViewOperations,
HealthOperations,
MaintenanceOperations,
DuplicateCopyOperations,
):
"""Database operations class for managing prompts."""
@@ -1860,7 +1862,11 @@ class PromptDatabase(
return obj
def _merge_duplicate_metadata(
self, conn: sqlite3.Connection, primary_id: int, duplicate_ids: List[int]
self,
conn: sqlite3.Connection,
primary_id: int,
duplicate_ids: List[int],
strict: bool = False,
) -> Dict[str, Any]:
"""
Merge metadata from duplicate prompts, prioritizing non-empty values.
@@ -1920,11 +1926,17 @@ class PromptDatabase(
return merged
except Exception as e:
if strict: # a reviewed apply rolls back instead
raise
self.logger.error(f"Error merging metadata: {e}", exc_info=True)
return {}
def _transfer_images_to_primary(
self, conn: sqlite3.Connection, primary_id: int, duplicate_ids: List[int]
self,
conn: sqlite3.Connection,
primary_id: int,
duplicate_ids: List[int],
strict: bool = False,
) -> int:
"""
Transfer all images from duplicate prompts to the primary prompt.
@@ -1956,11 +1968,17 @@ class PromptDatabase(
return transferred_count
except Exception as e:
if strict: # a reviewed apply rolls back instead
raise
self.logger.error(f"Error transferring images: {e}", exc_info=True)
return 0
def _update_primary_with_merged_metadata(
self, conn: sqlite3.Connection, primary_id: int, merged_metadata: Dict[str, Any]
self,
conn: sqlite3.Connection,
primary_id: int,
merged_metadata: Dict[str, Any],
strict: bool = False,
) -> None:
"""
Update the primary prompt with merged metadata.
@@ -1991,6 +2009,8 @@ class PromptDatabase(
)
except Exception as e:
if strict: # a reviewed apply rolls back instead
raise
self.logger.error(
f"Error updating primary prompt metadata: {e}", exc_info=True
)