perf(images): refresh the listing when the monitor sees a new output (#348, PR 2) (#353)

* perf(images): refresh the listing when the monitor sees a new output (#348, PR 2)

The image monitor now reports every settled output (linked or not, also
when linking raised) with its resolved path through set_output_listener.
The API's notify_output_settled hops from the monitor's worker thread onto
the server's loop (call_soon_threadsafe) and does nothing before the loop is
known or after cleanup. mark_root_dirty matches the path against each
walk's resolved_root (normcase(realpath), computed in the executor by
walk_root, now its fourth value) and marks the root dirty.

A dirty root gets one background walk, no sooner than
rewalk_interval = min(15, max(5, 5 x last walk's duration)) after the
previous walk started (last_started_at / last_duration are recorded on
every walk). _schedule_walk is shared with PR 1's delayed stale refresh:
one deferred walk per root, never armed while a walk runs (the walk's end
re-checks dirty: the trailing walk), at least 0.5 s away, the earlier
deadline wins, and every walk start cancels it. A failed walk clears dirty.
Rescan semantics are unchanged.

* fix(images): keep removed roots and cancelled walks from walking again (#348, PR 2)

Review findings on the monitor-triggered walks:

- reset_gallery_roots_cache now retires every listing record: its pending
  timer is cancelled, dirty is cleared, and no mark, timer or trailing walk
  starts a walk of it until a reader or the warm-up asks for that root
  again. Before, a timer or trailing walk pending when the roots changed
  stored a listing for the removed root, and the monitor (which watches the
  old roots until restart) kept it walking on every new output.
- A walk task cancelled outside cleanup_gallery_listings no longer starts a
  trailing walk while the loop shuts down.
- root_contains ignores a Windows extended-length prefix (\\?\ and
  \\?\UNC\) on either side, which realpath can keep on a file past MAX_PATH.
- The REWALK comment states the ~20% GIL share only for walks up to 3 s.

* test(images): prove a cancelled walk starts no trailing walk (#348, PR 2)

The previous assertions held even with the trailing walk, because that
walk finished before they ran; assert the walk number and the scan count.

* perf(images): add a settled output to the listing without a walk (#348, PR 2)

Live test: each landing in a 20,000-file folder cost a full walk (~0.4 s of
GIL) plus a full workspace rebuild, so Images reads during a run took up to
2.1 s. The monitor's report now patches the stored listing instead.

- output_settled (the monitor hop's target) slices the file's part below
  the root in its own case (relative_part; Windows compares folded), and
  _run_adds places one root's pending files in one executor batch:
  _settled_entry applies the walk's filters (suffix, thumbnails, depth,
  symlink by lstat; macOS on-disk spelling), insert_entry puts the
  (Path, mtime) entry where the newest-first sort does, add_folder copies
  the folder sets. Stored only when no walk started and no drop happened
  meanwhile; walked_at is untouched, so the 30 s refresh still walks.
- Anything inexact marks the root dirty and walks as before: a running
  walk, an unsliceable path, a failed lstat, a tie in mtime, the cap, an
  unexpected error. Rescan, the refresh, the warm-up and invalidation stay
  full walks; a walk start forgets the pending adds.
- The workspace caches follow a patched listing by entry identity
  (listing_delta): only added values are built, the newest order places
  them by bisection, the favorites lookup is copied and extended.

Synthetic 20,000 files: per landing 252 ms (walk + full build) -> 18 ms
(3 ms add + 15 ms warm build).

* fix(images): review fixes for adds without a walk (#348, PR 2)

- _newest_step steps the cached newest order only when the per-file cache
  still holds exactly the values this call built. A stale page read racing
  a newer warm build (both run in executor threads) stored an order
  labelled with older values than it held, and the next read inserted the
  new file a second time. It also sorts in full when a roots reset dropped
  the per-file values meanwhile.
- The stored spelling is checked on Windows too (nt._getfinalpathname:
  realpath keeps the given case when it cannot open the file), and before
  the name filters, which compare the walk's stored spelling. macOS opens
  non-blocking, so a FIFO named like an image never holds the executor.
- An output in a folder the walk cannot list (no read permission) is not
  added, as the walk skips it.
- Windows-portable unit tests: POSIX paths in the insert tests, and a
  str.lower fold for the length guard (Windows' normcase keeps lengths).

* test(images): reproduce the newest-order race and fix a Windows clock flake (#348, PR 2)

- The race test now has the newer build step from the values the cached
  order was built from, which is the interleaving that inserted a file
  twice; it fails without the parts check in _newest_step.
- test_a_failed_walk_records_its_timing_and_clears_dirty compared two
  monotonic readings that Windows' ~15.6 ms clock can make equal; the
  previous start is moved back instead.
- TESTING.md: os.access ignores ACLs on Windows.

utils/image_monitor.py

@@ -106,6 +106,10 @@ class ImageGenerationHandler(FileSystemEventHandler):
         # ``queue_capture_thumbnail``, wired at start-up; #249). It only
         # queues: the preview is written on its own worker. None: no queue.
         self.thumbnail_queue: Optional[Callable[[str], str]] = None
+        # Told about every settled output, with its resolved path (the API's
+        # ``notify_output_settled``, wired at start-up; #348): the Images
+        # listing walks that root again. None: nobody is told.
+        self.output_listener: Optional[Callable[[str], None]] = None
 
         # Read from GalleryConfig if available, otherwise use defaults
         try:
@@ -259,8 +263,12 @@ class ImageGenerationHandler(FileSystemEventHandler):
                         f"Image never settled or vanished, skipping: {image_path}"
                     )
                 return
-            self.process_new_image(image_path, prompt_snapshot=prompt_snapshot)
-            self._queue_thumbnail(image_path)
+            try:
+                self.process_new_image(image_path, prompt_snapshot=prompt_snapshot)
+                self._queue_thumbnail(image_path)
+            finally:
+                # Linked or not, even if linking raised: the file exists.
+                self._report_output(image_path)
         finally:
             with self._pending_lock:
                 self._pending.pop(image_path, None)
@@ -285,6 +293,25 @@ class ImageGenerationHandler(FileSystemEventHandler):
             return
         self.logger.debug(f"Preview {outcome}: {os.path.basename(image_path)}")
 
+    def _report_output(self, image_path: str) -> None:
+        """Tell the output listener a settled output exists, best effort.
+
+        The path is resolved here, on the worker thread, so the listener's
+        event loop does no file I/O. A failure is logged, never raised: the
+        output is then listed by the listing's 30 s refresh.
+        """
+        listener = getattr(self, "output_listener", None)
+        if listener is None:
+            return
+        try:
+            listener(os.path.realpath(image_path))
+        except Exception:
+            self.logger.warning(
+                f"{os.path.basename(image_path)} could not be reported to the "
+                "Images listing",
+                exc_info=True,
+            )
+
     def wait_until_settled(self, image_path: str, stop=None) -> bool:
         """True once the file's size is non-zero and unchanged across two reads.
 
@@ -848,6 +875,14 @@ class ImageMonitor:
         self.logger = get_logger("prompt_manager.image_monitor")
         self.review_recorder: Optional[Callable[[str], str]] = None
         self.thumbnail_queue: Optional[Callable[[str], str]] = None
+        self.output_listener: Optional[Callable[[str], None]] = None
+
+    def set_output_listener(self, listener: Optional[Callable[[str], None]]) -> None:
+        """Report every settled output's resolved path to ``listener`` (None
+        stops it), on the running handler and every later one (#348)."""
+        self.output_listener = listener
+        if self.handler is not None:
+            self.handler.output_listener = listener
 
     def set_thumbnail_queue(
         self, queue_preview: Optional[Callable[[str], str]]
@@ -925,6 +960,7 @@ class ImageMonitor:
         handler = ImageGenerationHandler(self.db_manager, self.prompt_tracker)
         handler.review_recorder = self.review_recorder
         handler.thumbnail_queue = self.thumbnail_queue
+        handler.output_listener = self.output_listener
         observer = Observer()
         scheduled = []
         for output_dir in valid_directories: