From 34df48c031cbdf1f9040d81c9ab89dc85a94f4d9 Mon Sep 17 00:00:00 2001 From: Vito Sansevero Date: Sat, 31 Oct 2026 20:48:22 -0700 Subject: [PATCH] feat(system): choose where thumbnails are kept and move them (#320) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit System › Sources & scans gains the Thumbnail location row (D196): Inside each source or One folder, the folder path, the status line and Review move…, whose review (D197) lists what is copied, removed and left before Move thumbnails starts; a refused folder is explained under the field, a folder of your own asks for a confirmation first. Activity shows a move and a prune with their own tiles and step line (D198). An unavailable folder shows a notice on Overview and Sources & scans with Recreate folder or Use this folder (D199). Thumbnail maintenance offers Remove thumbnails of removed sources in folder mode (D200), and Move library's review says where the thumbnails stay (D201). Closes #320 --- TESTING.md | 67 +++ py/api/library_location.py | 10 + py/api/preview_location.py | 5 + py/api/thumbnail_move.py | 12 + tests/browser/system-concept.spec.mjs | 5 +- tests/browser/thumbnail_location_fixture.py | 55 ++- .../workspace-thumbnail-location.spec.mjs | 148 +++++++ ...orkspace-system-thumbnail-location.test.js | 137 ++++++ tests/test_api_library_location.py | 2 + tests/test_api_thumbnail_location.py | 2 +- tools/vendor/frontend-coverage-baseline.json | 6 + web/js/api-client.js | 7 + web/js/workspace/activity-job-view.mjs | 7 +- web/js/workspace/activity-kinds.mjs | 50 ++- web/js/workspace/library-location.mjs | 3 + web/js/workspace/system-overview.mjs | 15 +- web/js/workspace/system-sources.mjs | 19 +- .../workspace/system-thumbnail-location.mjs | 396 ++++++++++++++++++ web/js/workspace/system.mjs | 4 +- web/js/workspace/thumbnail-review.mjs | 35 +- web/js/workspace/workspace-dialogs.mjs | 13 +- web/js/workspace/workspace.css | 29 ++ 22 files changed, 1006 insertions(+), 21 deletions(-) create mode 100644 tests/browser/workspace-thumbnail-location.spec.mjs create mode 100644 tests/js/workspace-system-thumbnail-location.test.js create mode 100644 web/js/workspace/system-thumbnail-location.mjs diff --git a/TESTING.md b/TESTING.md index 00393f34..7e4947dc 100644 --- a/TESTING.md +++ b/TESTING.md @@ -5723,6 +5723,73 @@ are closed before `os.replace`; renames stay inside one folder; folder identity folder are refused. The browser test server closes its own log handlers before its temp root (which now holds the logs) is removed. +### Thumbnail location contract (#320) + +Previews live in each source's `thumbnails/` (the default) or in one folder, `//…`. Every reader, +writer and remover asks `py/api/preview_store.py`; `tests/test_preview_store.py` fails on a `thumbnails` path join +anywhere else. The setting is `thumbnails.location` in config.json, and only the Move thumbnails job writes it: POST +`/settings` answers 409 `location-needs-move`. + +- **Keys and marker.** A key is a slug plus a hash of the source's identity. The folder's + `.promptmanager-thumbnails.json` stores each key with its identity and install. Only a single `-` name + is ever joined onto the folder, and anything else in a marker is dropped. Every write and delete passes + `preview_store.may_write`: below a key folder, with no link, junction or reparse point on the way + (`tests/test_preview_store_folder.py`, including tampered markers). +- **Unavailable.** A folder is unavailable when it is missing, is a link, holds no marker or holds another + folder's marker. A probe checks it every 5 s, and a store call that fails or stalls marks it at once. While it is + unavailable: + - originals are served and nothing is written anywhere; + - generate, convert, clear and prune answer 409 `location-unavailable`; + - the folder is never made again except by Recreate folder. + + The configured location is in force before any request. A bad value is `invalid`, never a fall-back to the + sources. +- **Move thumbnails.** The job is a `thumbnails` job with `scope.mode` `move` (`tests/test_thumbnail_move.py`). It + runs in this order: + 1. The review answers `POST /images/thumbnails/plan`; not enough space answers 507. + 2. A token starts the job. + 3. Automatic previews are held. + 4. Copies run on the job pool in chunks of 2 or 4 files, each with its size checked and its mtime rounded up to + 2 s. + 5. One config commit makes the switch and records the old stores under `cleanup`. The job cannot be cancelled + from here. + 6. The old copies are removed. + + If it fails or is cancelled before the switch, nothing changes. A crash leaves + `/data/thumbnail-move.json`, which the next start undoes. Old copies still held open are removed by the + next start's "Move thumbnails (finishing)" job. Leaving a folder also removes the previews of this install's + sources that are no longer listed (Q320-8). +- **Remove thumbnails of removed sources** (`scope.mode` `prune`). It is offered in Thumbnail maintenance only in + folder mode, and it touches only this install's keys. +- **Page tests.** `tests/browser/workspace-thumbnail-location.spec.mjs` covers the row, both moves, the in-place + refusal, the confirmation, the notice with Recreate and with Use this folder. The legacy gallery runs once in + folder mode (`workflows.spec.mjs`). The hooks are `tests/browser/thumbnail_location_fixture.py`. + +Beyond the concept, against the approved static mock `docs/egm/post-3.4.1/320-mock`: + +- D196: the Thumbnail location row in System › Sources & scans, after Thumbnail generation. + - The status line is "Now: inside each source." or "Now: one folder, .". It does not show the mock's + "(3 sources, 8,512 thumbnails)" counts, because counting means walking every preview on each view. It says + "Using the default folder next to your library." and names old copies that wait to be removed. +- D197: the Move thumbnails review uses the shared review. + - A grid value may be a lead sentence with a list (`[lead, items]`). + - Sources no longer listed appear under "Removed after the move", where the mock has a "Not moved" row + (maintainer answer to Q320-8: remove). + - The Free space line is " free", without naming the disk. +- D198: Activity's view of `mode: move` and `mode: prune`. + - Tiles for a move: Moved, Already there, Left in place, Errors. Tiles for a prune: Removed, Left in place, + Errors. + - The step line under the bar is `#job-phase`. + - Cancel shows only until the switch. +- D199: the unavailable notice on Overview and Sources & scans. + - Recreate folder appears for a missing folder, after a confirmation. + - Use this folder appears for another folder's marker. + - Change location focuses the row; on Overview it opens Sources & scans. +- D200: Thumbnail maintenance's "Remove thumbnails of removed sources" scope, with the plan line and a grid of + key folders. +- D201: Move library's review adds "Thumbnails: Stay in . Move them in System › Sources & scans." in folder + mode. + ### Backup and restore contract (#249 System S8) A restore is staged, then published (`database/restore_stage.py`). The upload is diff --git a/py/api/library_location.py b/py/api/library_location.py index 70b3b94d..30590c53 100644 --- a/py/api/library_location.py +++ b/py/api/library_location.py @@ -75,6 +75,14 @@ def _failure(message, status, code): ) +def _thumbnail_folder(): + """The thumbnail folder in use, or None (previews inside each source).""" + from . import preview_store + + setting = preview_store.setting() + return setting.path if setting.mode == preview_store.FOLDER_MODE else None + + def _inside_thumbnail_folder(path): """Whether *path* lies in the thumbnail folder in use (#320).""" from . import preview_store @@ -236,6 +244,8 @@ class LibraryLocationRoutesMixin: ), "problem": location.problem or self._node_folder_risk(location), "size_bytes": size, + # #320 D201: thumbnails kept in one folder stay there (D8A). + "thumbnails_folder": _thumbnail_folder(), } # The node folder (tests point it at a temp tree). diff --git a/py/api/preview_location.py b/py/api/preview_location.py index 7cbd7c71..dab96151 100644 --- a/py/api/preview_location.py +++ b/py/api/preview_location.py @@ -119,6 +119,11 @@ MESSAGES = { 400, "This path is too long for Windows. Choose a shorter folder path.", ), + "insufficient-space": ( + 507, + "There is not enough space for the thumbnails. Free some space or choose " + "another folder.", + ), "location-needs-move": ( 409, "Use Move thumbnails in System › Sources & scans.", diff --git a/py/api/thumbnail_move.py b/py/api/thumbnail_move.py index 3d4d6472..afc156c2 100644 --- a/py/api/thumbnail_move.py +++ b/py/api/thumbnail_move.py @@ -128,6 +128,18 @@ class ThumbnailMoveMixin: ) self.logger.warning(f"Move thumbnails could not start: {error}") return + left = [ + { + "path": "/".join(item), + "root": s.label, + "detail": "Not a thumbnail; left in place", + } + for s in plan.sources + if s.scan is not None + for item in s.scan.left + ] + if left: + job.record_counted("left", len(left), left) items = [ (source, item) for source in plan.sources diff --git a/tests/browser/system-concept.spec.mjs b/tests/browser/system-concept.spec.mjs index 6018d5ce..33b453bd 100644 --- a/tests/browser/system-concept.spec.mjs +++ b/tests/browser/system-concept.spec.mjs @@ -656,9 +656,10 @@ function mapSourcesConcept(reference, { settings, sources }) { // D113: the range ends at the server's core count (the concept's fixed 16) and shows the real worker count. return [{ ...row, control: { ...row.control, max: String(settings.cpu_count), value: String(settings.worker_threads) } }]; }), - // D158: the inserted row after the first setting row; D98: "Refresh files" closes the surface. + // D158: the inserted row after the first setting row; D196 (#320): the Thumbnail location block after + // "Thumbnail generation"; D98: "Refresh files" closes the surface. surface: { ...reference.surface, tree: [...reference.surface.tree.slice(0, rowAt + 1), 'div.setting-row', - ...reference.surface.tree.slice(rowAt + 1), 'div.operation'] }, + ...reference.surface.tree.slice(rowAt + 1), 'div.setting-block', 'div.operation'] }, operations: [...reference.operations, { ...reference.operations[0], title: REFRESH_FILES.title, description: REFRESH_FILES.description, actions: [{ ...reference.operations[0].actions[0], text: REFRESH_FILES.label }] }], }; diff --git a/tests/browser/thumbnail_location_fixture.py b/tests/browser/thumbnail_location_fixture.py index 2bae5228..2028b7e6 100644 --- a/tests/browser/thumbnail_location_fixture.py +++ b/tests/browser/thumbnail_location_fixture.py @@ -31,12 +31,40 @@ def register_thumbnail_location_commands( ) -> Callable[[], None]: """Register the commands; returns the reset every reset runs.""" from py.api import preview_location, preview_store + from py.api import thumbnail_location, thumbnail_move_plan + from py.config import PromptManagerConfig + from py.config_store import get_config_store folder = root / FOLDER_NAME + default = root / "default-thumbs" + custom = root / "custom-thumbs" # a folder a page types in + journal = root / "data" / "thumbnail-move.json" + # The move's journal and default folder live in this server's root, never + # in the checkout's data folder. + thumbnail_move_plan.journal_path = lambda: str(journal) + thumbnail_move_plan.default_path = lambda: str(default) + thumbnail_location.default_path = lambda: str(default) + api.recover_thumbnail_move() + + def forget_config(config: dict) -> dict: + section = config.get("thumbnails") + if isinstance(section, dict): + section.pop("location", None) + return config def reset() -> None: preview_store.configure(None) - shutil.rmtree(folder, ignore_errors=True) + PromptManagerConfig.THUMBNAIL_LOCATION = None + store = get_config_store() + config, _revision = store.read() + if ( + isinstance(config.get("thumbnails"), dict) + and "location" in config["thumbnails"] + ): + store.commit(forget_config, expected_revision=None) + for path in (folder, default, custom): + shutil.rmtree(path, ignore_errors=True) + journal.unlink(missing_ok=True) def use(mode: str) -> dict: reset() @@ -63,14 +91,18 @@ def register_thumbnail_location_commands( return web.json_response(await api._run_in_executor(use, mode)) def files() -> dict: + setting = preview_store.setting() + where = Path(setting.path) if setting.path else folder previews = sorted( - path.relative_to(folder).as_posix() - for path in folder.rglob("*_thumb.*") + path.relative_to(where).as_posix() + for path in where.rglob("*_thumb.*") if path.is_file() ) sources = [str(path) for path in api._get_all_output_dirs()] return { "folder": previews, + "folder_exists": folder.exists() or default.exists() or custom.exists(), + "custom_path": str(custom), "source_thumbnails": any( (Path(source) / "thumbnails").exists() for source in sources ), @@ -81,7 +113,8 @@ def register_thumbnail_location_commands( return web.json_response(await api._run_in_executor(files)) def remove() -> bool: - shutil.rmtree(folder, ignore_errors=True) + setting = preview_store.setting() + shutil.rmtree(setting.path or folder, ignore_errors=True) return preview_store.refresh() @routes.post("/__test__/thumbnail-folder/remove") @@ -89,4 +122,18 @@ def register_thumbnail_location_commands( available = await api._run_in_executor(remove) return web.json_response({"available": available}) + def foreign() -> bool: + """Another install recreated the folder: its marker has another id.""" + setting = preview_store.setting() + + def other(marker: dict) -> None: + marker["id"] = "another-install" + + preview_location.write_marker(setting.path, other) + return preview_store.refresh() + + @routes.post("/__test__/thumbnail-folder/foreign") + async def foreign_folder(_request: web.Request) -> web.Response: + return web.json_response({"available": await api._run_in_executor(foreign)}) + return reset diff --git a/tests/browser/workspace-thumbnail-location.spec.mjs b/tests/browser/workspace-thumbnail-location.spec.mjs new file mode 100644 index 00000000..a58311cf --- /dev/null +++ b/tests/browser/workspace-thumbnail-location.spec.mjs @@ -0,0 +1,148 @@ +import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs'; +import { test, expect } from './fixtures.mjs'; +import { seedConcept } from './concept-inventory.mjs'; + +// #320 Thumbnail location (D196–D201, the approved 320 mock) on the concept-shaped real fixture: the row in System › +// Sources & scans reviews and runs Move thumbnails both ways, the review refuses a folder in place, and an unplugged +// thumbnail folder shows the notice whose Recreate folder brings it back. The hooks are thumbnail_location_fixture.py's. +const workspace = '/prompt_manager/workspace'; +const NOTICE_KEY = 'pm_update_notice_dismissed'; + +test.beforeEach(async ({ page, request, browserName }) => { + await startCoverage(page, browserName); + await seedConcept(request); + await expect(await request.post('/__test__/system/reset')).toBeOK(); + await expect(await request.post('/__test__/thumbnail-location', { data: { mode: 'source' } })).toBeOK(); + page.locationErrors = []; + page.on('pageerror', error => page.locationErrors.push(error.message)); + await page.addInitScript(key => { if (location.protocol.startsWith('http')) localStorage.setItem(key, '3.2.0'); }, NOTICE_KEY); +}); + +test.afterEach(async ({ page, request, browserName }) => { + await saveCoverage(page, browserName); + await request.post('/__test__/thumbnail-location', { data: { mode: 'source' } }); + expect(page.locationErrors).toEqual([]); +}); + +const block = page => page.locator('#location-block'); +const dialog = page => page.locator('dialog#modal[open]'); + +async function files(request) { + const response = await request.get('/__test__/thumbnail-files'); + await expect(response).toBeOK(); + return response.json(); +} + +async function openSources(page) { + await page.goto(`${workspace}#/system/sources`); + await expect(block(page).locator('#location-status')).not.toHaveText(''); +} + +/** Review the row's choice and start the move; resolves once its job view says it finished. */ +async function moveFromRow(page) { + const planned = page.waitForResponse(response => response.url().endsWith('/images/thumbnails/plan') + && response.request().method() === 'POST'); + await block(page).getByRole('button', { name: 'Review move…' }).click(); + expect((await planned).status()).toBe(200); + await expect(dialog(page).locator('h2')).toHaveText('Move thumbnails'); + await dialog(page).getByRole('button', { name: 'Move thumbnails' }).click(); + await expect(page.locator('#workspace-main h1')).toHaveText('Move thumbnails'); + await expect(page.locator('#workspace-main')).toContainText('Thumbnails moved to'); +} + +test('the row moves thumbnails into one folder and back', async ({ page, request }) => { + await openSources(page); + const row = block(page); + await expect(row.getByRole('radio', { name: /Inside each source/ })).toBeChecked(); + await expect(row.locator('#location-status')).toHaveText('Now: inside each source.'); + await expect(row.getByRole('button', { name: 'Review move…' })).toHaveAttribute('aria-disabled', 'true'); + await expect(row.locator('#location-path')).toBeDisabled(); + + await row.getByRole('radio', { name: /One folder/ }).check(); + await expect(row.locator('#location-path')).toBeEnabled(); + await expect(row.locator('#location-path')).toHaveValue(/default-thumbs$/); + await expect(row.locator('#location-status')).toHaveText(/^Now: inside each source\. The suggested folder is next to your library/); + await expect(row.getByRole('button', { name: 'Review move…' })).toHaveAttribute('aria-disabled', 'false'); + await moveFromRow(page); + await expect(page.locator('#job-phase')).toHaveText(/^Thumbnails are now read from the new location/); + const moved = await files(request); + expect(moved.folder.length).toBeGreaterThan(0); + expect(moved.folder.every(name => /^[a-z0-9_-]+-[0-9a-f]{8}\//.test(name))).toBe(true); + expect(moved.source_thumbnails).toBe(false); + + // Images shows thumbnails served from the folder. + const served = page.waitForResponse(response => response.url().includes('/images/serve/thumbnails/') + && response.status() === 200); + await page.goto(`${workspace}#/images`); + await served; + + await openSources(page); + await expect(block(page).locator('#location-status')).toHaveText(/^Now: one folder, .*default-thumbs\. Using the default folder/); + await block(page).getByRole('radio', { name: /Inside each source/ }).check(); + await moveFromRow(page); + const back = await files(request); + expect(back.source_thumbnails).toBe(true); + expect(back.folder_exists).toBe(false); +}); + +test('a folder the server refuses is explained under the field', async ({ page }) => { + await openSources(page); + const row = block(page); + await row.getByRole('radio', { name: /One folder/ }).check(); + await row.locator('#location-path').fill('thumbnails'); + await row.getByRole('button', { name: 'Review move…' }).click(); + await expect(row.locator('#location-problem')).toHaveText( + 'Enter a full folder path, such as D:\\Thumbnails or /mnt/fast/thumbnails.'); + await expect(row.locator('.location-path')).toHaveAttribute('aria-invalid', 'true'); + await expect(dialog(page)).toHaveCount(0); + await row.locator('#location-path').fill('/somewhere/else'); + await expect(row.locator('#location-problem')).toHaveText(''); +}); + +test('an unplugged folder shows the notice and Recreate folder brings it back', async ({ page, request }) => { + await expect(await request.post('/__test__/thumbnail-location', { data: { mode: 'folder' } })).toBeOK(); + const removed = await request.post('/__test__/thumbnail-folder/remove'); + expect((await removed.json()).available).toBe(false); + await openSources(page); + const notice = page.locator('#unavailable-notice'); + await expect(notice.locator('h2')).toHaveText('The thumbnail folder is unavailable'); + await expect(notice).toContainText('is missing. Originals are shown and no thumbnails are written until it is back.'); + await expect(block(page).locator('#location-status')).toHaveText(/· unavailable\. Originals are shown\.$/); + await notice.getByRole('button', { name: 'Recreate folder' }).click(); + await expect(dialog(page).locator('h2')).toHaveText('Recreate the thumbnail folder?'); + const recreated = page.waitForResponse(response => response.url().endsWith('/location/recreate')); + await dialog(page).getByRole('button', { name: 'Recreate folder' }).click(); + expect((await recreated).status()).toBe(200); + await expect(notice).toHaveCount(0); + await expect(block(page).locator('#location-status')).toHaveText(/^Now: one folder, .*thumbs\.$/); +}); + +test('a folder of your own needs the confirmation before it moves', async ({ page, request }) => { + const custom = (await files(request)).custom_path; + await openSources(page); + const row = block(page); + await row.getByRole('radio', { name: /One folder/ }).check(); + await row.locator('#location-path').fill(custom); + await row.getByRole('button', { name: 'Review move…' }).click(); + const review = dialog(page); + await expect(review.locator('dt')).toContainText(['From', 'To', 'Thumbnails', 'Free space', 'Removed after the move']); + await expect(review.locator('dd').nth(1)).toHaveText(`${custom} (new folder)`); + const move = review.getByRole('button', { name: 'Move thumbnails' }); + await expect(move).toHaveAttribute('aria-disabled', 'true'); + await review.getByRole('checkbox', { name: /may create its own folders/ }).check(); + await expect(move).toHaveAttribute('aria-disabled', 'false'); + await move.click(); + await expect(page.locator('#workspace-main')).toContainText('Thumbnails moved to'); + expect((await files(request)).folder.length).toBeGreaterThan(0); +}); + +test('another folder’s marker offers Use this folder', async ({ page, request }) => { + await expect(await request.post('/__test__/thumbnail-location', { data: { mode: 'folder' } })).toBeOK(); + await expect(await request.post('/__test__/thumbnail-folder/foreign')).toBeOK(); + await openSources(page); + const notice = page.locator('#unavailable-notice'); + await expect(notice).toContainText('holds another thumbnail folder'); + await notice.getByRole('button', { name: 'Use this folder' }).click(); + await expect(notice).toHaveCount(0); + await expect(block(page).locator('#location-status')).toHaveText(/^Now: one folder, .*thumbs\.$/); +}); diff --git a/tests/js/workspace-system-thumbnail-location.test.js b/tests/js/workspace-system-thumbnail-location.test.js new file mode 100644 index 00000000..f89ff10e --- /dev/null +++ b/tests/js/workspace-system-thumbnail-location.test.js @@ -0,0 +1,137 @@ +// Thumbnail location (#320, D196–D201): the row's state and status line, the move review's rows, the unavailable +// notice's text, the Activity views of a move and a prune, the prune scope's plan line and the Move library row. +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const locationModule = import('../../web/js/workspace/system-thumbnail-location.mjs'); +const kindsModule = import('../../web/js/workspace/activity-kinds.mjs'); +const reviewModule = import('../../web/js/workspace/thumbnail-review.mjs'); +const libraryModule = import('../../web/js/workspace/library-location.mjs'); + +const SOURCE = Object.freeze({ mode: 'source', path: null, default_path: '/data/thumbnails', available: true, + problem: null, cleanup_pending: 0 }); +const FOLDER = Object.freeze({ mode: 'folder', path: 'E:\\Thumbnails', default_path: '/data/thumbnails', + available: true, problem: null, cleanup_pending: 0 }); + +const PLAN = Object.freeze({ + from: { mode: 'source', path: null, available: true }, + to: { mode: 'folder', path: 'E:\\Thumbnails', exists: true, will_create: false, default: false }, + totals: { sources: 3, files: 8512, bytes: 1.2 * 1024 ** 3, orphans: 41, temps: 2, links: 1, left_in_place: 1, + too_long: 32, unlisted_sources: 0, unlisted_files: 0 }, + too_long_listed: ['Archive 2025 · a_thumb.webp', 'Archive 2025 · b_thumb.webp'], + left_listed: ['Archive 2025 · notes.txt'], + unlisted: [], + free_bytes: 1.6 * 1024 ** 4, + confirm_required: true, + token: 't', +}); + +test('the status line says where previews are now', async () => { + const { statusText } = await locationModule; + assert.equal(statusText(SOURCE), 'Now: inside each source.'); + assert.equal(statusText(FOLDER), 'Now: one folder, E:\\Thumbnails.'); + assert.equal(statusText({ ...FOLDER, path: '/data/thumbnails' }), + 'Now: one folder, /data/thumbnails. Using the default folder next to your library.'); + assert.equal(statusText({ ...FOLDER, available: false, problem: 'missing' }), + 'Now: one folder, E:\\Thumbnails · unavailable. Originals are shown.'); + assert.match(statusText({ ...SOURCE, cleanup_pending: 1 }), /Old thumbnails in 1 place wait to be removed/); + assert.equal(statusText(undefined), ''); + assert.match(statusText(SOURCE, { mode: 'folder', path: '/data/thumbnails' }), + /^Now: inside each source\. The suggested folder is next to your library/); + assert.equal(statusText(SOURCE, { mode: 'folder', path: '/other' }), 'Now: inside each source.'); +}); + +test('Review move… is offered only for another location', async () => { + const { choiceChanged } = await locationModule; + assert.equal(choiceChanged(SOURCE, { mode: 'source' }), false); + assert.equal(choiceChanged(SOURCE, { mode: 'folder', path: ' ' }), false); + assert.equal(choiceChanged(SOURCE, { mode: 'folder', path: '/data/thumbnails' }), true); + assert.equal(choiceChanged(FOLDER, { mode: 'folder', path: 'E:\\Thumbnails' }), false); + assert.equal(choiceChanged(FOLDER, { mode: 'folder', path: 'F:\\Thumbs' }), true); + assert.equal(choiceChanged(FOLDER, { mode: 'source' }), true); + assert.equal(choiceChanged({ ...SOURCE, problem: 'invalid' }, { mode: 'source' }), true); + assert.equal(choiceChanged(undefined, { mode: 'source' }), false); +}); + +test('the review lists what moves, what is removed and what stays', async () => { + const { reviewRows } = await locationModule; + const rows = Object.fromEntries(reviewRows(PLAN)); + assert.equal(rows.From, 'Inside each source (3 sources)'); + assert.equal(rows.To, 'E:\\Thumbnails (empty folder)'); + assert.equal(rows.Thumbnails, '8,512 in 3 sources · 1.2 GB'); + assert.equal(rows['Free space'], '1638.4 GB free'); + assert.match(rows['Too long for Windows'][0], /^32 thumbnails cannot be stored there/); + assert.deepEqual(rows['Too long for Windows'][1], [...PLAN.too_long_listed, 'and 30 more']); + assert.deepEqual(rows['Removed after the move'][1], + ['41 thumbnails whose original is gone', '2 unfinished temporary files']); + assert.equal(rows['Left in place'][0], '1 file that is not a thumbnail:'); + assert.equal(rows['Links skipped'], '1 link inside a thumbnails folder is not followed'); +}); + +test('the review of a move back names the folder and the removed sources', async () => { + const { reviewRows } = await locationModule; + const back = reviewRows({ ...PLAN, from: { mode: 'folder', path: '/data/thumbnails' }, + to: { mode: 'source', path: null }, unlisted: [{ key: 'old-1c22e0a4', files: 240 }], + totals: { sources: 1, files: 1, bytes: 10 }, free_bytes: undefined }); + const rows = Object.fromEntries(back); + assert.equal(rows.From, '/data/thumbnails'); + assert.equal(rows.To, 'Inside each source (1 source)'); + assert.equal(rows['Free space'], undefined); + assert.match(rows['Removed after the move'][0], /The folder itself is removed once empty/); + assert.deepEqual(rows['Removed after the move'][1], + ['1 source no longer listed: 240 thumbnails are removed; they are made again if it returns']); + const fresh = Object.fromEntries(reviewRows({ ...PLAN, to: { mode: 'folder', path: '/d', default: true, will_create: true } })); + assert.equal(fresh.To, '/d (new, next to your library)'); +}); + +test('the notice says why the folder is unavailable', async () => { + const { noticeText } = await locationModule; + assert.match(noticeText({ ...FOLDER, problem: 'missing' }), /^E:\\Thumbnails is missing\. Originals are shown/); + assert.match(noticeText({ ...FOLDER, problem: 'unreachable' }), /can’t be reached/); + assert.match(noticeText({ ...FOLDER, problem: 'marker-mismatch' }), /holds another thumbnail folder/); + assert.match(noticeText({ ...FOLDER, problem: 'invalid' }), /config\.json can’t be used/); + assert.match(noticeText({ ...FOLDER, problem: null }), /is being checked/); +}); + +test('sizes are written as the server writes them', async () => { + const { sizeText } = await locationModule; + assert.equal(sizeText(512), '512 B'); + assert.equal(sizeText(85.2 * 1024 * 1024), '85.2 MB'); +}); + +test('a move and a prune show their own tiles in Activity', async () => { + const { viewOf, movePhase, finishToast } = await kindsModule; + const move = { kind: 'thumbnails', state: 'running', cancellable: true, scope: { mode: 'move', from: 'source' }, + counters: { moved: 3, skipped: 0, left: 1, errors: 2 }, status_line: '' }; + assert.deepEqual(viewOf(move).tiles.map(([group, label]) => label), ['Moved', 'Already there', 'Left in place', 'Errors']); + assert.equal(viewOf(move).cancelLabel, 'Cancel move'); + assert.match(movePhase(move), /^Step 1 of 3: copying\. Thumbnails are still read from inside each source/); + assert.match(movePhase({ ...move, cancellable: false }), /can no longer be cancelled/); + assert.match(movePhase({ ...move, scope: { mode: 'move', phase: 'remove' } }), /Removing the old copies/); + assert.match(movePhase({ ...move, state: 'finished' }), /1 file that is not a thumbnail was left in place/); + assert.match(movePhase({ ...move, state: 'cancelled' }), /^Nothing changed/); + const done = { ...move, state: 'finished', status_line: 'Thumbnails moved to E:\\T: 3 moved · old copies removed.' }; + assert.equal(finishToast(done), done.status_line); + const prune = { kind: 'thumbnails', state: 'running', cancellable: true, scope: { mode: 'prune' }, + counters: { removed: 1, left: 0, errors: 0 } }; + assert.deepEqual(viewOf(prune).tiles.map(([, label]) => label), ['Removed', 'Left in place', 'Errors']); + const generate = { kind: 'thumbnails', state: 'running', cancellable: true, scope: { mode: 'missing' }, + counters: { generated: 0, skipped: 0, errors: 0 } }; + assert.deepEqual(viewOf(generate).tiles.map(([, label]) => label), ['Generated', 'Skipped', 'Errors']); +}); + +test('the prune scope reads its plan line from the server', async () => { + const { planText, PLAN_FAILED, PRUNE_OPTION } = await reviewModule; + assert.equal(PRUNE_OPTION[1], 'Remove thumbnails of removed sources'); + assert.equal(planText({ mode: 'prune', line: '2 sources no longer listed: 1,240 thumbnails (85.2 MB) are removed.' }), + '2 sources no longer listed: 1,240 thumbnails (85.2 MB) are removed.'); + assert.equal(planText({ mode: 'prune' }), PLAN_FAILED); +}); + +test('Move library says where thumbnails kept in one folder stay', async () => { + const { reviewGrid } = await libraryModule; + const base = { path: 'p', target: 't', size_bytes: 0, config_moves: true }; + assert.equal(reviewGrid(base).some(([term]) => term === 'Thumbnails'), false); + const rows = Object.fromEntries(reviewGrid({ ...base, thumbnails_folder: '/data/thumbnails' })); + assert.equal(rows.Thumbnails, 'Stay in /data/thumbnails. Move them in System › Sources & scans.'); +}); diff --git a/tests/test_api_library_location.py b/tests/test_api_library_location.py index 7b898aa0..d3edcc6d 100644 --- a/tests/test_api_library_location.py +++ b/tests/test_api_library_location.py @@ -91,6 +91,8 @@ class TestLocationRoute(LibraryAPITestCase): "size_bytes": os.path.getsize(self.api.db.model.db_path), "target_source": "default", "config_moves": True, + # #320 D201: previews kept inside each source (no folder). + "thumbnails_folder": None, }, ) diff --git a/tests/test_api_thumbnail_location.py b/tests/test_api_thumbnail_location.py index 1650f9d4..d1eb2fd8 100644 --- a/tests/test_api_thumbnail_location.py +++ b/tests/test_api_thumbnail_location.py @@ -302,7 +302,7 @@ class TestValidation(unittest.TestCase): def test_every_code_has_a_message_without_a_path(self): for code, (status, text) in pl.MESSAGES.items(): with self.subTest(code=code): - self.assertIn(status, (400, 409)) + self.assertIn(status, (400, 409, 507)) self.assertTrue(text.endswith(".")) diff --git a/tools/vendor/frontend-coverage-baseline.json b/tools/vendor/frontend-coverage-baseline.json index eb695acc..fd99db77 100644 --- a/tools/vendor/frontend-coverage-baseline.json +++ b/tools/vendor/frontend-coverage-baseline.json @@ -641,6 +641,12 @@ "functions": 95, "statements": 98 }, + "workspace/system-thumbnail-location.mjs": { + "lines": 80, + "branches": 65, + "functions": 75, + "statements": 80 + }, "workspace/system-thumbnails.mjs": { "lines": 98, "branches": 95, diff --git a/web/js/api-client.js b/web/js/api-client.js index 009d7cb1..bc3e9523 100644 --- a/web/js/api-client.js +++ b/web/js/api-client.js @@ -62,6 +62,13 @@ (typeof job.title === "string" && job.title.length > 0 && job.title.length <= 200))) { this.job = { id: job.id, title: job.title }; } + // A move refused for space (#320) carries its review, so the page + // can show what would have moved and why it cannot start. + this.plan = null; + if (this.code === "insufficient-space" && current && current.plan && + typeof current.plan === "object" && !Array.isArray(current.plan)) { + this.plan = current.plan; + } this.source = null; if (this.code === "invalid-source" && current && typeof current.source === "string" && current.source.length > 0 && current.source.length <= 4096) { diff --git a/web/js/workspace/activity-job-view.mjs b/web/js/workspace/activity-job-view.mjs index 5d125636..3b607c67 100644 --- a/web/js/workspace/activity-job-view.mjs +++ b/web/js/workspace/activity-job-view.mjs @@ -170,7 +170,10 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) { bar.max = 100; const stats = element('dl', undefined, 'stat-row'); const tileValues = new Map(); - progressPart.node.append(bar, stats); + // #320 D198: a kind with phases (Move thumbnails) says which one runs and what it means, under the bar. + const phase = element('p', '', 'small'); + phase.id = 'job-phase'; + progressPart.node.append(bar, phase, stats); const resultsPart = section('Results', 'job-results-title'); const search = element('label', undefined, 'search-field'); @@ -228,6 +231,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) { function layout(job) { view = viewOf(job); if (!view.tiles.length) stats.remove(); + if (typeof view.phase !== 'function') phase.remove(); for (const [group, label, tone] of view.tiles) { const tile = element('div', undefined, tone ? `stat ${tone}` : 'stat'); const value = element('dd', '0'); @@ -341,6 +345,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) { cancel.remove(); } status.textContent = progressLine(job); + if (typeof view?.phase === 'function') phase.textContent = view.phase(job); // D111: no value while the total is unknown; a job that ended with nothing counted shows how it ended. showProgress(bar, job.total > 0 || job.state !== 'finished' ? job : { ...job, progress: 100 }); bar.setAttribute('aria-label', scan ? 'Scan progress' : `${job.title} progress`); diff --git a/web/js/workspace/activity-kinds.mjs b/web/js/workspace/activity-kinds.mjs index 18690c73..8cf81377 100644 --- a/web/js/workspace/activity-kinds.mjs +++ b/web/js/workspace/activity-kinds.mjs @@ -51,6 +51,26 @@ export const removedOf = job => counted(job, 'removed', 'removed'); /** "12 imported" (System S10: the LoRA reference import). */ export const importedOf = job => counted(job, 'imported', 'imported'); +/** #320 D198: the step a Move thumbnails job is in, and what it means meanwhile (under the progress bar). */ +export function movePhase(job) { + const from = job.scope?.from === 'folder' ? 'the old folder' : 'inside each source'; + if (job.state === 'running') { + if (job.scope?.phase === 'remove') return 'Removing the old copies a move left. Thumbnails are read from the new location.'; + if (job.cancellable !== false) { + return `Step 1 of 3: copying. Thumbnails are still read from ${from} until the switch. New thumbnails are not ` + + 'written until the move ends; originals are shown instead.'; + } + return 'Step 2 and 3 of 3: switching, then removing the old copies. This can no longer be cancelled.'; + } + if (job.state === 'finished') { + const left = job.counters?.left ?? 0; + return left ? `Thumbnails are now read from the new location. ${number(left)} ${noun(left, 'file')} that ` + + `${left === 1 ? 'is' : 'are'} not ${left === 1 ? 'a thumbnail was' : 'thumbnails were'} left in place.` + : 'Thumbnails are now read from the new location.'; + } + return 'Nothing changed: thumbnails are read from where they were.'; +} + /** "2 output folders": what a scan or a thumbnail job walks. */ function foldersOf(job) { const folders = job.scope?.output_folders; @@ -159,6 +179,32 @@ export const KINDS = Object.freeze({ outcomes: tiles([['generated', 'Generated'], ['errors', 'Errors'], ['skipped', 'Skipped']]), openable: groups('generated', 'skipped'), }), + // #320 D198: Move thumbnails, a `thumbnails` job with `scope.mode` "move": Moved (green), Already there, Left in + // place and Errors (red); Cancel only until the switch (the server says so through `cancellable`). + 'thumbnails:move': Object.freeze({ + ...GENERIC, + pillVerb: 'Moving thumbnails', + finishToast: job => (job.status_line && job.state !== 'running' ? job.status_line : titleFinished(job)), + line: job => (job.state === 'running' ? '' : job.status_line || ''), + kept: job => counted(job, 'moved', 'moved'), + tiles: tiles([['moved', 'Moved', 'added'], ['skipped', 'Already there', ''], ['left', 'Left in place', ''], + ['errors', 'Errors', 'bad']]), + outcomes: tiles([['moved', 'Moved'], ['errors', 'Errors'], ['skipped', 'Already there'], ['left', 'Left in place']]), + log: 'Move log', + phase: movePhase, + cancelLabel: 'Cancel move', + cancel: 'Cancel move', + }), + // #320 D200: Remove thumbnails of removed sources (`scope.mode` "prune"): Removed, Left in place and Errors. + 'thumbnails:prune': Object.freeze({ + ...GENERIC, + pillVerb: 'Removing thumbnails', + finishToast: job => (job.status_line && job.state !== 'running' ? job.status_line : titleFinished(job)), + line: job => (job.state === 'running' ? '' : job.status_line || ''), + kept: removedOf, + tiles: tiles([['removed', 'Removed', 'added'], ['left', 'Left in place', ''], ['errors', 'Errors', 'bad']]), + outcomes: tiles([['removed', 'Removed'], ['errors', 'Errors'], ['left', 'Left in place']]), + }), // System S7: a reviewed maintenance plan's apply, titled after its operation; it runs without a known total. maintenance: Object.freeze({ ...GENERIC, @@ -243,7 +289,9 @@ export const held = job => job.cancellable === false && Object.keys(countsOf(job */ export function viewOf(job) { if (held(job)) return GENERIC; - const view = kindOf(job.kind); + // #320 D198: a thumbnails job chooses its tiles by its scope's mode (move, prune). + const mode = job.kind === 'thumbnails' ? job.scope?.mode : null; + const view = (mode === 'move' || mode === 'prune') ? KINDS[`thumbnails:${mode}`] : kindOf(job.kind); const counts = countsOf(job); const has = ([group]) => Object.hasOwn(counts, group); if (view.tiles.every(has) && view.outcomes.every(has)) return view; diff --git a/web/js/workspace/library-location.mjs b/web/js/workspace/library-location.mjs index 447d5aeb..f3ddde7c 100644 --- a/web/js/workspace/library-location.mjs +++ b/web/js/workspace/library-location.mjs @@ -97,6 +97,9 @@ export function reviewGrid(location) { ? 'The database backups (config.json stays where PROMPT_MANAGER_CONFIG_PATH names it)' : 'config.json and the database backups'], ['Old files', 'Renamed *.moved-to-…, never deleted'], + // #320 D201: previews kept in one folder do not move with the library (D8A). + ...(typeof location.thumbnails_folder === 'string' && location.thumbnails_folder + ? [['Thumbnails', `Stay in ${location.thumbnails_folder}. Move them in System › Sources & scans.`]] : []), ]; } diff --git a/web/js/workspace/system-overview.mjs b/web/js/workspace/system-overview.mjs index 7356213d..d05a2c0d 100644 --- a/web/js/workspace/system-overview.mjs +++ b/web/js/workspace/system-overview.mjs @@ -28,6 +28,7 @@ import { showWorkspaceToast } from './toast.mjs'; import { noun } from './plural.mjs'; import { operationRow } from './system-controls.mjs'; import { openThumbnailReview } from './thumbnail-review.mjs'; +import { noticeSlot } from './system-thumbnail-location.mjs'; import { isLocation, locationRow, onMoveEnded } from './library-location.mjs'; import { STATUS_LOADING, STATUS_URL, backupStatusText } from './system-backups.mjs'; @@ -406,7 +407,7 @@ function overviewNodes(overview, { api, activity, notify, scope, navigate, openS * `counts` is the shell's sidebar counts (`{get(id), subscribe(listener), freshen(ids)}`, index.mjs): the Images metric * shows its `images` count and follows it, and a visit re-reads it as the workspace Overview does (#555). */ -export function mountOverview(container, { api, activity, scope, counts = null, navigate = navigateTo, +export function mountOverview(container, { api, activity, scope, counts = null, drafts = null, navigate = navigateTo, notify = showWorkspaceToast, latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) { const overviewRead = latestOnly({ lifetime: scope.signal }); const statisticsRead = latestOnly({ lifetime: scope.signal }); @@ -466,7 +467,8 @@ export function mountOverview(container, { api, activity, scope, counts = null, if (overview === STALE || scope.signal.aborted) return; const { nodes, thumbnails, backup, metrics } = overviewNodes(overview, { api, activity, notify, scope, navigate, openStatistics, images: imagesCount(), - openThumbnails: trigger => openThumbnailReview({ api, store: activity, scope, trigger, navigate }) }); + openThumbnails: trigger => openThumbnailReview({ api, store: activity, scope, trigger, navigate, + location: drafts?.get('host', 'thumbnail_location') }) }); imagesValue = metrics.children[2].querySelector('strong'); thumbnailRow = thumbnails; backupRow = backup; @@ -522,5 +524,12 @@ export function mountOverview(container, { api, activity, scope, counts = null, load(); loadThumbnails(); loadBackup(); - return first; + // #320 D199: the unavailable thumbnail folder's notice, first on the page while it lasts. + if (!drafts) return first; + const unavailable = noticeSlot({ api, scope, notify, location: () => drafts.get('host', 'thumbnail_location'), + onChange: () => navigate('#/system/sources'), onDone: () => drafts.reloadHost?.() }); + scope.cleanup(drafts.subscribe(() => unavailable.paint())); + // The anchor goes in first, so the notice (inserted after it) leads the section once the anchor is mounted. + unavailable.paint(); + return [unavailable.anchor, ...first]; } diff --git a/web/js/workspace/system-sources.mjs b/web/js/workspace/system-sources.mjs index cd71d330..13d966f4 100644 --- a/web/js/workspace/system-sources.mjs +++ b/web/js/workspace/system-sources.mjs @@ -35,6 +35,7 @@ import { createButton } from '../shared/toolkit/core.js'; import { closeWorkspaceDialog, field, openContentDialog } from './workspace-dialogs.mjs'; import { openMetadataScan } from './activity-scan.mjs'; import { openThumbnailReview } from './thumbnail-review.mjs'; +import { locationBlock, noticeSlot } from './system-thumbnail-location.mjs'; import { settingRow, operationRow } from './system-controls.mjs'; import { HOST_UNAVAILABLE } from './system-appearance.mjs'; import { showWorkspaceToast } from './toast.mjs'; @@ -234,6 +235,19 @@ export function mountSources(container, { api, activity, scope, drafts, notify = nodes.push(notice); } + // #320 D199: the unavailable thumbnail folder's notice, first on the page while it lasts. + const thumbnailLocation = () => drafts.get('host', 'thumbnail_location'); + const location = locationBlock({ api, store: activity, scope, location: thumbnailLocation, notify }); + const unavailable = noticeSlot({ api, scope, location: thumbnailLocation, notify, onChange: () => location.focus(), + onDone: () => drafts.reloadHost?.() }); + const paintLocation = () => { + location.paint(); + unavailable.paint(); + }; + scope.cleanup(drafts.subscribe(paintLocation)); + nodes.push(unavailable.anchor); + paintLocation(); + const surface = el('section', undefined, 'surface'); const heading = el('div', undefined, 'panel-heading'); const addButton = createButton('+ Add source'); @@ -266,7 +280,10 @@ export function mountSources(container, { api, activity, scope, drafts, notify = operationRow({ scope, title: 'Enrich existing metadata', description: 'Resolve both prompts, deduplicate recipes, preserve uncertain links.', label: 'Review scan', onClick: scanAll }), operationRow({ scope, title: 'Thumbnail generation', description: 'Generate missing previews; rebuilding all is a separate explicit option.', - label: 'Review options', onClick: trigger => openThumbnailReview({ api, store: activity, scope, trigger }) }), + label: 'Review options', onClick: trigger => openThumbnailReview({ api, store: activity, scope, trigger, + location: thumbnailLocation() }) }), + // #320 D196: where previews are kept; never a draft (Move thumbnails is its only writer). + location.node, // D98: the legacy gallery's Refresh, which rereads every source's file list. operationRow({ scope, title: 'Refresh files', description: 'Read every source again to find files added or removed outside ComfyUI.', label: 'Refresh', onClick: refresh })); diff --git a/web/js/workspace/system-thumbnail-location.mjs b/web/js/workspace/system-thumbnail-location.mjs new file mode 100644 index 00000000..92175b67 --- /dev/null +++ b/web/js/workspace/system-thumbnail-location.mjs @@ -0,0 +1,396 @@ +/** + * Thumbnail location (#320, beyond the concept: D196–D199, approved static mock `320-mock`): where previews are kept. + * + * - D196 `locationBlock`: a `.setting-block` in System › Sources & scans after "Thumbnail generation": the radios + * "Inside each source" / "One folder", a "Folder path" field (enabled with One folder, prefilled with the folder in + * use or the server's `default_path`), the status line and "Review move…" (enabled when the choice differs from the + * location in use). Not a host draft: the Save row never saves it; the Move thumbnails job is the only writer (D4B). + * A refusal of the review shows in place under the field (`aria-invalid`), in the server's words. + * - D197 `openMoveReview`: the shared review (Move library's shape) over the server's plan + * (`POST /images/thumbnails/plan`): From, To, Thumbnails, Free space, Too long for Windows, Removed after the move, + * Left in place, Links skipped; a folder that is not the default asks for the confirmation checkbox first. "Move + * thumbnails" starts the job (`POST /jobs/thumbnails {mode: "move", token, confirmed}`) and opens it in Activity. + * - D199 `locationNotice`: while the thumbnail folder is unavailable, a `.notice` on Overview and Sources & scans + * says why, with "Recreate folder" (a missing folder, after a confirmation) or "Use this folder" (another folder's + * marker), and "Change location". + * + * The location comes from GET /settings' `thumbnail_location` (`{mode, path, default_path, available, problem, + * cleanup_pending}`), read with the host settings. + */ +import { createButton } from '../shared/toolkit/core.js'; +import { closeWorkspaceDialog, openContentDialog, openReviewDialog } from './workspace-dialogs.mjs'; +import { busyToast } from './activity-scan.mjs'; +import { showWorkspaceToast } from './toast.mjs'; +import { navigateTo } from './state.mjs'; +import { noun } from './plural.mjs'; + +export const PLAN_URL = '/prompt_manager/images/thumbnails/plan'; +export const RECREATE_URL = '/prompt_manager/images/thumbnails/location/recreate'; +export const ADOPT_URL = '/prompt_manager/images/thumbnails/location/adopt'; +export const LOCATION_HINT = 'Where previews are kept. Originals are never moved. Avoid cloud-synced folders.'; +export const SENTENCE = 'Every thumbnail is copied and checked before Prompt Manager switches. Scans, deletes and ' + + 'tagging wait until it finishes; closing this page does not stop it. Originals are never touched.'; +export const REVIEW_FAILED = 'The move could not be reviewed. Try again.'; +export const START_FAILED = 'Move thumbnails could not start. Try again.'; +export const NOTICE_TITLE = 'The thumbnail folder is unavailable'; +const BACK = 'Originals are shown and no thumbnails are written until it is back.'; +const CHECK = 'If its drive is unplugged, connect it; Prompt Manager checks again every few seconds.'; + +const number = value => Number(value || 0).toLocaleString('en-US'); +const isFolder = location => location?.mode === 'folder'; + +/** "1.2 GB" (binary units, one decimal, as the server writes them). */ +export function sizeText(bytes) { + let size = Number(bytes) || 0; + for (const unit of ['B', 'KB', 'MB', 'GB']) { + if (size < 1024 || unit === 'GB') return unit === 'B' ? `${size} B` : `${size.toFixed(1)} ${unit}`; + size /= 1024; + } + return `${size.toFixed(1)} GB`; +} + +export const SUGGESTED = 'The suggested folder is next to your library; Review move… shows what happens before anything ' + + 'is copied.'; + +/** The status line under the row; `choice` (`{mode, path}`) adds the suggestion while the default folder is chosen. */ +export function statusText(location, choice = null) { + if (!location) return ''; + if (!isFolder(location) && choice?.mode === 'folder' && location.default_path + && choice.path?.trim() === location.default_path) { + return `Now: inside each source. ${SUGGESTED}`; + } + let text = isFolder(location) ? `Now: one folder, ${location.path}` : 'Now: inside each source'; + if (isFolder(location) && location.available === false) return `${text} · unavailable. Originals are shown.`; + text += '.'; + if (isFolder(location) && location.path && location.path === location.default_path) { + text += ' Using the default folder next to your library.'; + } + if (location.cleanup_pending > 0) { + text += ` Old thumbnails in ${number(location.cleanup_pending)} ${noun(location.cleanup_pending, 'place')} wait to ` + + 'be removed until it is connected.'; + } + return text; +} + +/** Whether the row's choice (`{mode, path}`) differs from the location in use (Review move… enabled). */ +export function choiceChanged(location, choice) { + if (!location) return false; + if (choice.mode !== 'folder') return isFolder(location) || Boolean(location.problem); + const path = (choice.path || '').trim(); + return Boolean(path) && (!isFolder(location) || path !== location.path); +} + +/** The D197 review rows (`[term, value]`, a value may be `[lead, items]`) for the server's plan. */ +export function reviewRows(plan) { + const totals = plan.totals ?? {}; + const sources = totals.sources ?? plan.sources?.length ?? 0; + const inside = `Inside each source (${number(sources)} ${noun(sources, 'source')})`; + const from = plan.from?.mode === 'folder' ? plan.from.path : inside; + let to = inside; + if (plan.to?.mode === 'folder') { + const state = plan.to.default ? (plan.to.will_create ? 'new, next to your library' : 'next to your library') + : (plan.to.will_create ? 'new folder' : 'empty folder'); + to = `${plan.to.path} (${state})`; + } + const rows = [['From', from], ['To', to], + ['Thumbnails', `${number(totals.files)} in ${number(sources)} ${noun(sources, 'source')} · ${sizeText(totals.bytes)}`]]; + if (Number.isFinite(plan.free_bytes)) rows.push(['Free space', `${sizeText(plan.free_bytes)} free`]); + if (totals.too_long > 0) { + const more = totals.too_long - (plan.too_long_listed?.length ?? 0); + rows.push(['Too long for Windows', [`${number(totals.too_long)} ${noun(totals.too_long, 'thumbnail')} cannot be ` + + 'stored there. They show the original instead; choose a shorter folder to keep them.', + [...(plan.too_long_listed ?? []), ...(more > 0 ? [`and ${number(more)} more`] : [])]]]); + } + const removed = []; + if (totals.orphans > 0) removed.push(`${number(totals.orphans)} ${noun(totals.orphans, 'thumbnail')} whose original is gone`); + if (totals.temps > 0) removed.push(`${number(totals.temps)} unfinished temporary ${noun(totals.temps, 'file')}`); + for (const source of plan.unlisted ?? []) { + removed.push(`1 source no longer listed: ${number(source.files)} ${noun(source.files, 'thumbnail')} are removed; ` + + 'they are made again if it returns'); + } + const oldCopies = plan.from?.mode === 'folder' + ? 'The thumbnail folders of these sources inside it. The folder itself is removed once empty if Prompt Manager created it.' + : 'The old copies in each source’s thumbnails folder'; + rows.push(['Removed after the move', [removed.length ? `${oldCopies}, and:` : `${oldCopies}.`, removed]]); + if (totals.left_in_place > 0) { + rows.push(['Left in place', [`${number(totals.left_in_place)} ${noun(totals.left_in_place, 'file')} that ` + + `${totals.left_in_place === 1 ? 'is' : 'are'} not ${totals.left_in_place === 1 ? 'a thumbnail' : 'thumbnails'}:`, + plan.left_listed ?? []]]); + } + if (totals.links > 0) { + rows.push(['Links skipped', `${number(totals.links)} ${noun(totals.links, 'link')} inside a thumbnails folder ` + + `${totals.links === 1 ? 'is' : 'are'} not followed`]); + } + return rows; +} + +/** The notice text for an unavailable `location` (D199). */ +export function noticeText(location) { + const where = location?.path || 'The thumbnail folder'; + switch (location?.problem) { + case 'missing': return `${where} is missing. ${BACK} ${CHECK}`; + case 'unreachable': return `${where} can’t be reached. ${BACK} ${CHECK}`; + case 'marker-mismatch': return `${where} holds another thumbnail folder. ${BACK} Use it if it is yours, or change ` + + 'the location.'; + case 'invalid': return `The thumbnail location in config.json can’t be used. ${BACK} Change the location.`; + default: return `${where} is being checked. ${BACK}`; + } +} + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +function radio(value, label, hint) { + const node = el('label'); + const input = el('input'); + input.type = 'radio'; + input.name = 'thumb-location'; + input.value = value; + input.id = `loc-${value}`; + node.append(input, ` ${label} `, el('span', hint, 'small')); + return { node, input }; +} + +/** Start the move with the plan's `token`; opens it in Activity, or says why not. */ +export async function startMove({ store, scope, token, confirmed, notify = showWorkspaceToast, navigate = navigateTo }) { + try { + const result = await store.start('thumbnails', { mode: 'move', token, confirmed }); + if (scope.signal.aborted) return null; + if (result.busy !== undefined) { + notify(busyToast(result.busy)); + return null; + } + navigate(`#/activity/${result.job.id}`); + return result.job; + } catch (error) { + if (!scope.signal.aborted) notify(typeof error?.code === 'string' && error.message ? error.message : START_FAILED); + return null; + } +} + +/** Open the D197 review over `plan`; a plan the server refused for space keeps Move thumbnails unavailable. */ +export function openMoveReview({ scope, store, plan, refusal = null, trigger = null, notify, navigate }) { + const options = []; + let box = null; + if (plan.confirm_required && !refusal) { + const label = el('label', undefined, 'review-confirm'); + box = el('input'); + box.type = 'checkbox'; + box.id = 'review-confirmed'; + label.append(box, el('span', `Prompt Manager may create its own folders and a .promptmanager-thumbnails.json file in ${plan.to.path}.`)); + options.push(label); + } + if (refusal) options.push(el('p', refusal, 'small review-refusal')); + const review = openReviewDialog({ scope, trigger, title: 'Move thumbnails', grid: reviewRows(plan), options, + sentence: refusal ? null : SENTENCE, cancelLabel: 'Cancel', primary: { label: 'Move thumbnails' }, + onStart: () => { startMove({ store, scope, token: plan.token, confirmed: Boolean(box?.checked), notify, navigate }); } }); + const locked = () => Boolean(refusal) || (box !== null && !box.checked); + review.setPrimary({ label: 'Move thumbnails', disabled: locked() }); + if (box) scope.listen(box, 'change', () => review.setPrimary({ label: 'Move thumbnails', disabled: locked() })); + return review; +} + +/** + * The D196 row; `location()` gives the server's `thumbnail_location` (undefined while read). Returns `{node, paint, + * focus}`; `paint()` follows a new location. + */ +export function locationBlock({ api, store, scope, location, notify = showWorkspaceToast, navigate = navigateTo }) { + const block = el('div', undefined, 'setting-block'); + block.id = 'location-block'; + const text = el('div', undefined, 'setting-block-text'); + const label = el('span', 'Thumbnail location', 'setting-block-label'); + label.id = 'location-label'; + text.append(label, el('p', LOCATION_HINT)); + const choice = el('div', undefined, 'location-choice'); + choice.setAttribute('role', 'radiogroup'); + choice.setAttribute('aria-labelledby', 'location-label'); + const source = radio('source', 'Inside each source', '/thumbnails'); + const folder = radio('folder', 'One folder', 'one subfolder per source'); + choice.append(source.node, folder.node); + const pathField = el('div', undefined, 'location-path field'); + const pathLabel = el('label', 'Folder path'); + pathLabel.htmlFor = 'location-path'; + const input = el('input'); + input.id = 'location-path'; + input.autocomplete = 'off'; + input.spellcheck = false; + const problem = el('p', '', 'small location-problem'); + problem.id = 'location-problem'; + problem.setAttribute('role', 'alert'); + pathField.append(pathLabel, input, problem); + const status = el('p', '', 'small location-status'); + status.id = 'location-status'; + status.setAttribute('role', 'status'); + const actions = el('div', undefined, 'row-actions'); + const review = createButton('Review move…'); + review.id = 'review-move'; + actions.append(review); + block.append(text, choice, pathField, status, actions); + + let shown; + const setProblem = message => { + problem.textContent = message || ''; + pathField.setAttribute('aria-invalid', String(Boolean(message))); + }; + const current = () => ({ mode: folder.input.checked ? 'folder' : 'source', path: input.value }); + const repaintButton = () => { + const now = location(); + input.disabled = !folder.input.checked || !now; + source.input.disabled = folder.input.disabled = !now; + const changed = choiceChanged(now, current()); + status.textContent = now ? statusText(now, current()) : ''; + review.setAttribute('aria-disabled', String(!changed)); + review.classList.toggle('ca-primary', changed); + }; + const paint = () => { + const now = location(); + if (now && now !== shown) { + shown = now; + (isFolder(now) ? folder : source).input.checked = true; + input.value = (isFolder(now) ? now.path : now.default_path) || ''; + setProblem(null); + } + repaintButton(); + }; + for (const control of [source.input, folder.input]) { + scope.listen(control, 'change', () => { + if (folder.input.checked && !input.value) input.value = location()?.default_path || ''; + setProblem(null); + repaintButton(); + }); + } + scope.listen(input, 'input', () => { setProblem(null); repaintButton(); }); + scope.listen(review, 'click', async () => { + if (review.getAttribute('aria-disabled') === 'true') return; + const chosen = current(); + const body = { mode: 'move', location: chosen.mode === 'folder' ? { mode: 'folder', path: chosen.path.trim() } + : { mode: 'source' } }; + review.setAttribute('aria-disabled', 'true'); + try { + const plan = await api.post(PLAN_URL, body, { signal: scope.signal }); + if (!scope.signal.aborted) openMoveReview({ scope, store, plan, trigger: review, notify, navigate }); + } catch (error) { + if (scope.signal.aborted) return; + if (error?.status === 507 && error?.plan) { + openMoveReview({ scope, store, plan: error.plan, refusal: error.plan.insufficient?.text || error.message, + trigger: review }); + } else if (error?.status === 409 && error?.code === 'busy') { + notify(busyToast(error.running ?? null)); + } else if (typeof error?.code === 'string' && error.code.startsWith('location-') && error.message) { + setProblem(error.message); + input.focus(); + } else { + notify(typeof error?.message === 'string' && error?.code ? error.message : REVIEW_FAILED); + } + } finally { + repaintButton(); + } + }); + paint(); + return { + node: block, + paint, + focus() { + folder.input.checked = true; + repaintButton(); + input.focus(); + block.scrollIntoView?.({ block: 'center' }); + }, + }; +} + +/** Open the Recreate confirmation for `location`; resolves once it closed. */ +export function openRecreate({ api, scope, location, trigger = null, onDone, notify = showWorkspaceToast }) { + const grid = el('dl', undefined, 'meta-grid'); + for (const [term, value] of [['Folder', location.path], ['Made', 'An empty folder with a new .promptmanager-thumbnails.json'], + ['Thumbnails', 'Made again as you browse, or with Thumbnail maintenance']]) { + const entry = el('div'); + entry.append(el('dt', term), el('dd', value)); + grid.append(entry); + } + const why = el('p', '', 'small location-problem'); + const cancel = createButton('Cancel'); + const go = createButton('Recreate folder', 'ca-primary'); + const actions = el('div', undefined, 'dialog-actions'); + actions.append(cancel, go); + scope.listen(cancel, 'click', () => closeWorkspaceDialog()); + scope.listen(go, 'click', async () => { + try { + const result = await api.post(RECREATE_URL, {}, { signal: scope.signal }); + if (scope.signal.aborted) return; + closeWorkspaceDialog(); + onDone?.(result.location); + } catch (error) { + if (scope.signal.aborted) return; + why.textContent = error?.message || 'The folder could not be recreated.'; + if (!error?.message) notify(why.textContent); + } + }); + openContentDialog({ scope, title: 'Recreate the thumbnail folder?', trigger, nodes: [grid, + el('p', 'If the folder is on a drive that is not connected, connect it instead: recreating would put thumbnails on ' + + 'another disk.', 'small'), why, actions] }); +} + +/** + * The D199 notice for `location` (null while it is available): Recreate folder for a missing folder, Use this folder + * for another folder's marker, and Change location (`onChange`). + */ +export function locationNotice({ api, scope, location, onChange, onDone, notify = showWorkspaceToast }) { + if (!isFolder(location) || location.available !== false) return null; + const notice = el('div', undefined, 'notice'); + notice.id = 'unavailable-notice'; + const text = el('div'); + text.append(el('h2', NOTICE_TITLE), el('p', noticeText(location))); + const actions = el('div', undefined, 'row-actions'); + if (location.problem === 'missing') { + const recreate = createButton('Recreate folder'); + scope.listen(recreate, 'click', () => openRecreate({ api, scope, location, trigger: recreate, onDone, notify })); + actions.append(recreate); + } else if (location.problem === 'marker-mismatch') { + const adopt = createButton('Use this folder'); + scope.listen(adopt, 'click', async () => { + try { + const result = await api.post(ADOPT_URL, {}, { signal: scope.signal }); + if (!scope.signal.aborted) onDone?.(result.location); + } catch (error) { + if (!scope.signal.aborted) notify(error?.message || 'The folder could not be used.'); + } + }); + actions.append(adopt); + } + const change = createButton('Change location', 'ca-primary'); + scope.listen(change, 'click', () => onChange?.()); + actions.append(change); + notice.append(text, actions); + return notice; +} + +/** + * Where a section shows the D199 notice: a comment `anchor` (no element, so the concept comparisons see nothing while the + * folder is available) after which `paint()` keeps the notice for `location()`. + */ +export function noticeSlot({ api, scope, location, onChange, onDone, notify = showWorkspaceToast }) { + const anchor = document.createComment('thumbnail-location-notice'); + let shownFor; + let notice = null; + return { + anchor, + paint() { + const now = location(); + // Not mounted yet: the next paint (a microtask, or the host read) places it. + if (!anchor.parentNode) { + queueMicrotask(() => { if (anchor.parentNode && !scope.signal.aborted) this.paint(); }); + return; + } + if (now === shownFor) return; + shownFor = now; + notice?.remove(); + notice = locationNotice({ api, scope, location: now, onChange, onDone, notify }); + if (notice) anchor.after(notice); + }, + }; +} diff --git a/web/js/workspace/system.mjs b/web/js/workspace/system.mjs index d769f809..1a5e8129 100644 --- a/web/js/workspace/system.mjs +++ b/web/js/workspace/system.mjs @@ -86,7 +86,9 @@ export function systemDrafts(api) { api.get(LORA_STATUS_URL, { cache: 'no-store' }).catch(() => null)]); // The store reads a missing value as none and a missing revision as unreadable (no host save). #317: which // thumbnail formats the server can write rides along as a read-only value (never drafted or sent). - return { values: { ...result.settings, thumbnail_encoders: result.thumbnail_encoders, ...(lora ? loraValues(lora) : {}) }, + // #320: where previews live rides along the same way (read-only; only Move thumbnails changes it). + return { values: { ...result.settings, thumbnail_encoders: result.thumbnail_encoders, + thumbnail_location: result.thumbnail_location, ...(lora ? loraValues(lora) : {}) }, revision: result.revision }; }, async writeHost(changes, { expected_revision }) { diff --git a/web/js/workspace/thumbnail-review.mjs b/web/js/workspace/thumbnail-review.mjs index 177a261b..5a0d850b 100644 --- a/web/js/workspace/thumbnail-review.mjs +++ b/web/js/workspace/thumbnail-review.mjs @@ -23,6 +23,7 @@ import { busyToast } from './activity-scan.mjs'; import { showWorkspaceToast } from './toast.mjs'; import { navigateTo } from './state.mjs'; import { noun } from './plural.mjs'; +import { sizeText } from './system-thumbnail-location.mjs'; export const THUMBNAIL_PLAN_URL = '/prompt_manager/images/thumbnails/plan'; export const THUMBNAILS_KIND = 'thumbnails'; @@ -33,7 +34,9 @@ export const PLAN_FAILED = 'The previews could not be counted.'; export const START_FAILED = 'Thumbnail generation could not start. Try again.'; export const SETTINGS_UNKNOWN = 'Previews follow the thumbnail settings in System › Appearance.'; const FORMAT_LABELS = Object.freeze({ webp: 'WebP', jxl: 'JPEG XL', jpeg: 'JPEG' }); -const MODES = new Set(SCOPE_OPTIONS.map(([value]) => value)); +// #320 D200: offered only while previews are kept in one folder (31A). +export const PRUNE_OPTION = Object.freeze(['prune', 'Remove thumbnails of removed sources']); +const MODES = new Set([...SCOPE_OPTIONS.map(([value]) => value), PRUNE_OPTION[0]]); const count = value => value.toLocaleString('en-US'); const isCount = value => Number.isSafeInteger(value) && value >= 0; @@ -47,6 +50,7 @@ export function settingsText(plan) { /** D99: the plan line for the server's `{mode, targets, fresh, remaining}`; an answer without counts failed. */ export function planText(plan) { + if (plan?.mode === 'prune') return typeof plan.line === 'string' && plan.line ? plan.line : PLAN_FAILED; if (!plan || !MODES.has(plan.mode) || ![plan.targets, plan.fresh, plan.remaining].every(isCount)) { return PLAN_FAILED; } @@ -101,11 +105,18 @@ function select(options, value) { * Activity store; `trigger` gets focus back when the dialog closes. */ export function openThumbnailReview({ api, store, scope, trigger = null, mode: initial = 'missing', notify = showWorkspaceToast, - navigate = navigateTo }) { - const mode = select(SCOPE_OPTIONS, MODES.has(initial) ? initial : 'missing'); + navigate = navigateTo, location = null }) { + const folderMode = location?.mode === 'folder'; + const options = folderMode ? [...SCOPE_OPTIONS, PRUNE_OPTION] : SCOPE_OPTIONS; + const allowed = new Set(options.map(([value]) => value)); + const mode = select(options, allowed.has(initial) ? initial : 'missing'); const line = el('p', PLAN_LOADING, 'small'); line.setAttribute('role', 'status'); const settings = el('p', SETTINGS_UNKNOWN, 'small'); + // D200: what the prune removes, under the plan line. + const pruneDetail = el('p', '', 'small'); + const pruneGrid = el('dl', undefined, 'meta-grid'); + pruneDetail.hidden = pruneGrid.hidden = true; const cancel = createButton('Cancel'); const start = createButton('Start', 'ca-primary'); const actions = el('div', undefined, 'dialog-actions'); @@ -125,7 +136,21 @@ export function openThumbnailReview({ api, store, scope, trigger = null, mode: i // A later choice painted its own line meanwhile, or the dialog closed. if (!open || scope.signal.aborted || mode.value !== chosen) return; line.textContent = planText(plan); - settings.textContent = settingsText(plan); + const prune = chosen === 'prune'; + settings.hidden = prune; + pruneDetail.hidden = pruneGrid.hidden = !prune || !plan?.sources?.length; + if (prune) { + pruneDetail.textContent = `Only Prompt Manager’s thumbnails in their folders inside ${plan?.folder ?? 'the thumbnail ' + + 'folder'} are removed. If a source comes back, its thumbnails are made again as you browse.`; + pruneGrid.replaceChildren(...(plan?.sources ?? []).map(source => { + const entry = el('div'); + entry.append(el('dt', source.key), el('dd', `${count(source.files)} ${noun(source.files, 'thumbnail')} · ` + + `${sizeText(source.bytes)}`)); + return entry; + })); + } else { + settings.textContent = settingsText(plan); + } } scope.listen(mode, 'change', paint); @@ -136,7 +161,7 @@ export function openThumbnailReview({ api, store, scope, trigger = null, mode: i startThumbnails({ store, scope, options, notify, navigate }); }); openContentDialog({ scope, title: 'Thumbnail maintenance', trigger, - nodes: [field('Scope', mode, 'thumb-mode'), line, settings, actions], + nodes: [field('Scope', mode, 'thumb-mode'), line, settings, ...(folderMode ? [pruneDetail, pruneGrid] : []), actions], onClose: () => { open = false; } }); paint(); } diff --git a/web/js/workspace/workspace-dialogs.mjs b/web/js/workspace/workspace-dialogs.mjs index c26d82a0..fb3496f1 100644 --- a/web/js/workspace/workspace-dialogs.mjs +++ b/web/js/workspace/workspace-dialogs.mjs @@ -460,9 +460,13 @@ export function backupNotice({ copy, onBackup }) { return notice; } +/** A grid value: text, or (#320) `[lead, items]`, a lead sentence and the list under it. */ +const gridValue = value => typeof value === 'string' || (Array.isArray(value) && value.length === 2 + && typeof value[0] === 'string' && Array.isArray(value[1]) && value[1].every(item => typeof item === 'string')); + function requireGrid(grid) { if (!Array.isArray(grid) || grid.length === 0 || grid.some(row => !Array.isArray(row) || row.length !== 2 || - typeof row[0] !== 'string' || !row[0] || typeof row[1] !== 'string')) { + typeof row[0] !== 'string' || !row[0] || !gridValue(row[1]))) { throw new TypeError('The review needs its [term, value] grid'); } } @@ -495,7 +499,12 @@ export function openReviewDialog({ scope, title, heading = null, grid, options = values.clear(); list.replaceChildren(...rows.map(([term, value]) => { const entry = el('div'); - const dd = el('dd', value); + const dd = el('dd', typeof value === 'string' ? value : value[0]); + if (typeof value !== 'string' && value[1].length) { + const items = el('ul', undefined, 'review-list'); + items.append(...value[1].map(item => el('li', item))); + dd.append(items); + } values.set(term, dd); entry.append(el('dt', term), dd); return entry; diff --git a/web/js/workspace/workspace.css b/web/js/workspace/workspace.css index 05b569be..756702bf 100644 --- a/web/js/workspace/workspace.css +++ b/web/js/workspace/workspace.css @@ -576,3 +576,32 @@ pre.prompt-text { font: inherit; font-size: 12px; line-height: 1.75; margin-bloc .review-panel { position: sticky; inset-block-start: 20px; } @media (max-width: 700px) { .review-panel { position: static; } } @media (min-width: 701px) and (max-height: 720px) { .review-panel { max-block-size: calc(100dvh - 40px); overflow: auto; } } + +/* #320 D196: Thumbnail location, a setting wider than a .setting-row column. */ +.setting-block { padding: 17px 0; border-top: 1px solid var(--line); display: grid; gap: 12px; } +.setting-block-text .setting-block-label { font-size: 12px; font-weight: 500; display: block; } +.setting-block-text p { font-size: 11px; color: var(--ca-muted); margin-top: 4px; max-width: 640px; } +.location-choice { display: flex; flex-wrap: wrap; gap: 8px 24px; font-size: 12px; } +.location-choice label { display: inline-flex; gap: 6px; align-items: baseline; } +.location-choice .small { color: var(--ca-muted); } +.location-path { max-width: 640px; margin: 0; } +.location-path input { width: 100%; min-width: 0; font-family: var(--ca-mono, ui-monospace, monospace); font-size: 12px; } +.location-path input:disabled { color: var(--ca-muted); } +.location-path[aria-invalid="true"] input { border-color: var(--red); } +.location-problem { color: var(--red); margin-top: 6px; } +.location-problem:empty { display: none; } +.location-status { margin: 0; overflow-wrap: anywhere; } +.setting-block .row-actions { justify-content: flex-start; } + +/* #320 D197: the move review's lists and confirmation. */ +.review-list { margin: 4px 0 0; padding-inline-start: 18px; font-size: 11px; color: var(--ca-muted); } +.review-list li { overflow-wrap: anywhere; } +.review-confirm { display: flex; gap: 8px; align-items: flex-start; font-size: 12px; margin: 12px 0 0; + padding: 10px 12px; border: 1px solid var(--line); border-radius: 6px; background: var(--ca-field); } +.review-confirm input { margin-top: 2px; flex-shrink: 0; } +.review-refusal { color: var(--red); } +.meta-grid dd { overflow-wrap: anywhere; } + +@media (max-width: 700px) { + .location-choice { flex-direction: column; } +}