fix(thumbnails): keep the video fallback tested and stop the preview worker before cleanup
A video listed without a preview still renders its own element and is released on refresh. Test teardowns and the browser reset stop the automatic preview worker before the temp tree goes, so no preview lands after cleanup (Windows). TESTING.md documents automatic previews.
This commit is contained in:
+40
-3
@@ -1932,8 +1932,8 @@ pending count (`GET /prompt_manager/review/items?limit=1`, its `total`).
|
||||
unavailable file's empty media button keeps the concept's 1.33 media area, so its
|
||||
card lines up with the others. A long file name in the panel's `p.small` wraps inside
|
||||
the panel (`overflow-wrap: anywhere`), where the concept's short sample name never
|
||||
overflows. A queue card shows the Images listing's thumbnail when the server knows one
|
||||
(`thumbnail_url`, the same URL the Images page uses), else the original; a re-read
|
||||
overflows. A queue card shows the Images listing's thumbnail (`thumbnail_url`, the
|
||||
same URL the Images page uses; a preview not made yet is made on first view); a re-read
|
||||
rebuilds a card whose media, target or availability changed (a root came or went), so
|
||||
Open and Tag follow it. When an unanswered decision's item leaves the queue in a
|
||||
re-read, the decision was recorded: the page finishes as a recorded decision does (its
|
||||
@@ -2230,6 +2230,42 @@ through the Activity store, which toasts "Thumbnail generation started."; the jo
|
||||
in Activity. A start refused because another job holds the slot toasts the D57 busy
|
||||
link toast and stays. The legacy gallery's thumbnail tools are unchanged.
|
||||
|
||||
Automatic previews (#249, `py/api/thumbnail_auto.py`). The approved mock has no
|
||||
repeated global thumbnail prompt (`docs/concepts/workspace/README.md`), so the
|
||||
workspace never asks; previews are a cache the server keeps:
|
||||
|
||||
- Every listing gives an image or video a thumbnail URL (`media_thumbnail_url`: Images,
|
||||
saved views and favorites, Library cards and galleries, Needs review, duplicate
|
||||
groups). An existing preview's URL keeps its `v=<mtime>`; a missing one is listed
|
||||
without `v=`.
|
||||
- `GET /images/serve/thumbnails/<rel>_thumb<ext>` for a missing preview maps it back to
|
||||
its media file under the same root (traversal and name checks on the text first; the
|
||||
source must be a regular file inside the root, never through a link), makes it with
|
||||
the job's defaults (512 px, medium) and serves it. At most 3 views write at once and
|
||||
one flight runs per file. When it cannot be made the original is served with its own
|
||||
content type and `Cache-Control: no-store`; an undecodable file is remembered for the
|
||||
session by path, size and mtime (not retried per view, retried once it changes) and
|
||||
counted as `failed` in `/images/thumbnails/status`, which Overview's row adds
|
||||
("N missing thumbnails · M failed"). A preview another program holds open (Windows)
|
||||
is not remembered. A `thumbnails` folder linked out of the root is never written.
|
||||
The recorded-history thumbnail route makes a missing preview the same way.
|
||||
- The image monitor hands every settled output to `queue_capture_thumbnail`, which
|
||||
queues it for one background worker: each wake takes at most Worker threads files and
|
||||
writes them on that many threads; a queued file is not queued twice; the queue holds
|
||||
10,000 and a file left out is made on view.
|
||||
- A finished scan that added files counts the missing previews of the sources it
|
||||
walked: 200 or more start the thumbnail job (missing only, visible and cancellable in
|
||||
Activity, never silent bulk work); fewer go through the worker.
|
||||
- Holds: automatic previews write only `thumbnails/` files and no records, so they run
|
||||
beside a scan, image tagging, the thumbnail job and the Needs review recording (S5
|
||||
M7). They wait (a view gets the original, a queued file is left for its view) while a
|
||||
restore, a duplicate removal, a sources change, the gallery's thumbnail clear or an
|
||||
`exclusive` job runs (`_auto_thumbnail_blocker`).
|
||||
- Cards and strips load previews only: Images in every layout and size, Library cards,
|
||||
Needs review cards and the inspector's cover (a video's cover is its preview frame).
|
||||
The media viewer's main media and downloads load the original
|
||||
(`workspace-thumbnails.spec.mjs` records every media request; `tests/test_thumbnail_auto.py`).
|
||||
|
||||
Overview (`system-overview.mjs`) reads `GET /prompt_manager/system/overview` once per
|
||||
visit through `latestOnly()` on the section scope; a late answer after the section left
|
||||
paints nothing. Until it answers, the section shows "Loading the overview…"; a failure
|
||||
@@ -2277,7 +2313,8 @@ Recorded deviations (plan D60–D67, D79–D81 renumbered D92–D99 and D111–D
|
||||
"Backup status" reads "No retained backup record." until verified backups exist (S8).
|
||||
The thumbnail row is "Checking thumbnails…" until its own read answers, then "No
|
||||
missing thumbnails", "N missing thumbnails" (the pending count of
|
||||
`/images/thumbnails/status`) or "Thumbnail status unavailable"; Review thumbnails
|
||||
`/images/thumbnails/status`, followed by " · M failed" when automatic previews
|
||||
failed this session) or "Thumbnail status unavailable"; Review thumbnails
|
||||
opens the thumbnail dialog (D99). "One display configuration" says "Grid
|
||||
and density settings are shared across views." (no paging). The sources row is
|
||||
"N output sources" with "Monitoring and enrichment share one source list.", prefixed
|
||||
|
||||
@@ -423,8 +423,13 @@ class AutoThumbnailMixin:
|
||||
return self._auto_thumbnailer
|
||||
|
||||
def close_auto_thumbnails(self):
|
||||
"""Stop the capture worker (shutdown; safe when it never started)."""
|
||||
thumbnailer = self._auto_thumbnailer
|
||||
"""Stop the capture worker and forget it (shutdown, a test's reset).
|
||||
|
||||
Waits for the batch in hand, so nothing writes into the output tree
|
||||
after it returns; the next use starts a fresh one.
|
||||
"""
|
||||
with AutoThumbnailMixin._auto_thumbnailer_lock:
|
||||
thumbnailer, self._auto_thumbnailer = self._auto_thumbnailer, None
|
||||
if thumbnailer is not None:
|
||||
thumbnailer.close()
|
||||
|
||||
|
||||
@@ -386,6 +386,8 @@ def create_app(root: Path) -> web.Application:
|
||||
async def reset(request: web.Request) -> web.Response:
|
||||
nonlocal history_fixture
|
||||
request_holds.release()
|
||||
# No preview queued by the last test may land in the new fixture.
|
||||
await api._run_in_executor(api.close_auto_thumbnails)
|
||||
library_editor_commands.reset()
|
||||
data = await request.json() if request.can_read_body else {}
|
||||
if not isinstance(data, dict):
|
||||
|
||||
@@ -730,6 +730,32 @@ test('refresh releases the previous inspector cover source', async ({ page, requ
|
||||
expect(await page.evaluate(() => window.__oldInspectorCover.getAttribute('src'))).toBeNull();
|
||||
});
|
||||
|
||||
/** A listing that names no preview for its videos (the fallback a card and the cover keep for one). */
|
||||
async function videosWithoutPreview(page) {
|
||||
await page.route(url => url.pathname === '/prompt_manager/images/output', async route => {
|
||||
const response = await route.fetch();
|
||||
const body = await response.json();
|
||||
if (Array.isArray(body.images)) body.images = body.images.map(image => image.is_video ? { ...image, thumbnail_url: null } : image);
|
||||
await route.fulfill({ response, json: body });
|
||||
});
|
||||
}
|
||||
|
||||
test('a video listed without a preview keeps its own element, released on refresh', async ({ page, request }) => {
|
||||
await expect(await request.post('/__test__/reset', { data: { include_video: true } })).toBeOK();
|
||||
await videosWithoutPreview(page);
|
||||
await page.goto(workspace);
|
||||
const card = cards(page).filter({ hasText: 'clip.webm' });
|
||||
await expect(card.locator('.media-art > .image-open > video')).toHaveCount(1);
|
||||
await expect(card.locator('img')).toHaveCount(0);
|
||||
await card.getByRole('button', { name: 'Inspect clip.webm', exact: true }).click();
|
||||
await expect(details(page).locator('video')).toBeVisible();
|
||||
await details(page).locator('video').evaluate(element => { window.__oldInspectorVideo = element; });
|
||||
await card.locator('video').evaluate(element => { window.__oldCardVideo = element; });
|
||||
await rescan(page);
|
||||
await expect.poll(() => page.evaluate(() => !window.__oldInspectorVideo.isConnected && !window.__oldCardVideo.isConnected)).toBe(true);
|
||||
expect(await page.evaluate(() => [window.__oldInspectorVideo.getAttribute('src'), window.__oldCardVideo.getAttribute('src')])).toEqual([null, null]);
|
||||
});
|
||||
|
||||
test('Import Rescan rescans cached files and preserves the collection when rescan fails', async ({ page, request }) => {
|
||||
await page.goto(workspace);
|
||||
await expect(cards(page)).toHaveCount(2);
|
||||
|
||||
@@ -79,6 +79,8 @@ class AdminAPITestCase(AioHTTPTestCase):
|
||||
|
||||
async def tearDownAsync(self):
|
||||
await super().tearDownAsync() # closes the aiohttp test client
|
||||
# The automatic preview worker writes into the temp tree: stop it first.
|
||||
self.api.close_auto_thumbnails()
|
||||
GalleryConfig.MONITORING_DIRECTORIES = self._orig_gallery_dirs
|
||||
PromptManagerConfig.update_config(self._orig_pm_config)
|
||||
for key, value in self._orig_env.items():
|
||||
|
||||
@@ -131,6 +131,8 @@ class AutotagAPITestCase(AioHTTPTestCase):
|
||||
|
||||
async def tearDownAsync(self):
|
||||
await super().tearDownAsync() # closes the aiohttp test client
|
||||
# The automatic preview worker writes into the temp tree: stop it first.
|
||||
self.api.close_auto_thumbnails()
|
||||
self._service_patch.stop()
|
||||
if model_gate is not None:
|
||||
leaked = model_gate.holder()
|
||||
|
||||
@@ -62,6 +62,8 @@ class BackupRestoreTestCase(AioHTTPTestCase):
|
||||
|
||||
async def tearDownAsync(self):
|
||||
await super().tearDownAsync() # closes the aiohttp test client
|
||||
# The automatic preview worker writes into the temp tree: stop it first.
|
||||
self.api.close_auto_thumbnails()
|
||||
PromptModel.close_all_instances() # restore may have swapped api.db
|
||||
assert_closed(self, self.tmpdir)
|
||||
for name in os.listdir(self.tmpdir):
|
||||
|
||||
@@ -384,6 +384,8 @@ class CoreRoutesTestCase(AioHTTPTestCase):
|
||||
|
||||
async def tearDownAsync(self):
|
||||
await super().tearDownAsync() # closes the aiohttp test client
|
||||
# The automatic preview worker writes into the temp tree: stop it first.
|
||||
self.api.close_auto_thumbnails()
|
||||
self.api.db.close_all()
|
||||
assert_closed(self, self.tmpdir)
|
||||
shutil.rmtree(self.tmpdir, ignore_errors=True)
|
||||
|
||||
@@ -1116,6 +1116,28 @@ class HistoryOutputAPITests(ImageAPITestCase):
|
||||
response = await self.current(descriptor["thumbnail_url"])
|
||||
self.assert_error(response, await response.json(), 403, "not-authorized")
|
||||
|
||||
async def test_missing_preview_is_never_made_through_a_linked_out_folder(self):
|
||||
original = make_png(self.output_dir / "image.png", size=(64, 64))
|
||||
elsewhere = self.temp_dir / "elsewhere"
|
||||
elsewhere.mkdir()
|
||||
(self.output_dir / "thumbnails").symlink_to(elsewhere, target_is_directory=True)
|
||||
start = self.capture([self.ref()])
|
||||
descriptor = await self.media(start)
|
||||
response = await self.current(descriptor["thumbnail_url"])
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(await response.read(), original.read_bytes())
|
||||
self.assertEqual(os.listdir(elsewhere), [])
|
||||
|
||||
async def test_a_preview_that_cannot_be_made_serves_the_original(self):
|
||||
broken = self.output_dir / "image.png"
|
||||
broken.write_bytes(b"\x89PNG\r\n\x1a\n not an image")
|
||||
start = self.capture([self.ref()])
|
||||
descriptor = await self.media(start)
|
||||
response = await self.current(descriptor["thumbnail_url"])
|
||||
self.assertEqual(response.status, 200)
|
||||
self.assertEqual(await response.read(), broken.read_bytes())
|
||||
self.assertFalse((self.output_dir / "thumbnails" / "image_thumb.png").exists())
|
||||
|
||||
async def test_disappearance_before_open_is_missing_and_closes_handle(
|
||||
self,
|
||||
):
|
||||
|
||||
@@ -80,6 +80,8 @@ class ImageAPITestCase(AioHTTPTestCase):
|
||||
|
||||
async def tearDownAsync(self):
|
||||
await super().tearDownAsync() # closes the aiohttp test client
|
||||
# The automatic preview worker writes into the temp tree: stop it first.
|
||||
self.api.close_auto_thumbnails()
|
||||
self._anchor_patch.stop()
|
||||
# Windows refuses to unlink a database that still has open handles.
|
||||
self.api.db.close_all()
|
||||
|
||||
Reference in New Issue
Block a user