diff --git a/TESTING.md b/TESTING.md index 00393f34..7e4947dc 100644 --- a/TESTING.md +++ b/TESTING.md @@ -5723,6 +5723,73 @@ are closed before `os.replace`; renames stay inside one folder; folder identity folder are refused. The browser test server closes its own log handlers before its temp root (which now holds the logs) is removed. +### Thumbnail location contract (#320) + +Previews live in each source's `thumbnails/` (the default) or in one folder, `//…`. Every reader, +writer and remover asks `py/api/preview_store.py`; `tests/test_preview_store.py` fails on a `thumbnails` path join +anywhere else. The setting is `thumbnails.location` in config.json, and only the Move thumbnails job writes it: POST +`/settings` answers 409 `location-needs-move`. + +- **Keys and marker.** A key is a slug plus a hash of the source's identity. The folder's + `.promptmanager-thumbnails.json` stores each key with its identity and install. Only a single `-` name + is ever joined onto the folder, and anything else in a marker is dropped. Every write and delete passes + `preview_store.may_write`: below a key folder, with no link, junction or reparse point on the way + (`tests/test_preview_store_folder.py`, including tampered markers). +- **Unavailable.** A folder is unavailable when it is missing, is a link, holds no marker or holds another + folder's marker. A probe checks it every 5 s, and a store call that fails or stalls marks it at once. While it is + unavailable: + - originals are served and nothing is written anywhere; + - generate, convert, clear and prune answer 409 `location-unavailable`; + - the folder is never made again except by Recreate folder. + + The configured location is in force before any request. A bad value is `invalid`, never a fall-back to the + sources. +- **Move thumbnails.** The job is a `thumbnails` job with `scope.mode` `move` (`tests/test_thumbnail_move.py`). It + runs in this order: + 1. The review answers `POST /images/thumbnails/plan`; not enough space answers 507. + 2. A token starts the job. + 3. Automatic previews are held. + 4. Copies run on the job pool in chunks of 2 or 4 files, each with its size checked and its mtime rounded up to + 2 s. + 5. One config commit makes the switch and records the old stores under `cleanup`. The job cannot be cancelled + from here. + 6. The old copies are removed. + + If it fails or is cancelled before the switch, nothing changes. A crash leaves + `/data/thumbnail-move.json`, which the next start undoes. Old copies still held open are removed by the + next start's "Move thumbnails (finishing)" job. Leaving a folder also removes the previews of this install's + sources that are no longer listed (Q320-8). +- **Remove thumbnails of removed sources** (`scope.mode` `prune`). It is offered in Thumbnail maintenance only in + folder mode, and it touches only this install's keys. +- **Page tests.** `tests/browser/workspace-thumbnail-location.spec.mjs` covers the row, both moves, the in-place + refusal, the confirmation, the notice with Recreate and with Use this folder. The legacy gallery runs once in + folder mode (`workflows.spec.mjs`). The hooks are `tests/browser/thumbnail_location_fixture.py`. + +Beyond the concept, against the approved static mock `docs/egm/post-3.4.1/320-mock`: + +- D196: the Thumbnail location row in System › Sources & scans, after Thumbnail generation. + - The status line is "Now: inside each source." or "Now: one folder, .". It does not show the mock's + "(3 sources, 8,512 thumbnails)" counts, because counting means walking every preview on each view. It says + "Using the default folder next to your library." and names old copies that wait to be removed. +- D197: the Move thumbnails review uses the shared review. + - A grid value may be a lead sentence with a list (`[lead, items]`). + - Sources no longer listed appear under "Removed after the move", where the mock has a "Not moved" row + (maintainer answer to Q320-8: remove). + - The Free space line is " free", without naming the disk. +- D198: Activity's view of `mode: move` and `mode: prune`. + - Tiles for a move: Moved, Already there, Left in place, Errors. Tiles for a prune: Removed, Left in place, + Errors. + - The step line under the bar is `#job-phase`. + - Cancel shows only until the switch. +- D199: the unavailable notice on Overview and Sources & scans. + - Recreate folder appears for a missing folder, after a confirmation. + - Use this folder appears for another folder's marker. + - Change location focuses the row; on Overview it opens Sources & scans. +- D200: Thumbnail maintenance's "Remove thumbnails of removed sources" scope, with the plan line and a grid of + key folders. +- D201: Move library's review adds "Thumbnails: Stay in . Move them in System › Sources & scans." in folder + mode. + ### Backup and restore contract (#249 System S8) A restore is staged, then published (`database/restore_stage.py`). The upload is diff --git a/py/api/library_location.py b/py/api/library_location.py index 70b3b94d..30590c53 100644 --- a/py/api/library_location.py +++ b/py/api/library_location.py @@ -75,6 +75,14 @@ def _failure(message, status, code): ) +def _thumbnail_folder(): + """The thumbnail folder in use, or None (previews inside each source).""" + from . import preview_store + + setting = preview_store.setting() + return setting.path if setting.mode == preview_store.FOLDER_MODE else None + + def _inside_thumbnail_folder(path): """Whether *path* lies in the thumbnail folder in use (#320).""" from . import preview_store @@ -236,6 +244,8 @@ class LibraryLocationRoutesMixin: ), "problem": location.problem or self._node_folder_risk(location), "size_bytes": size, + # #320 D201: thumbnails kept in one folder stay there (D8A). + "thumbnails_folder": _thumbnail_folder(), } # The node folder (tests point it at a temp tree). diff --git a/py/api/preview_location.py b/py/api/preview_location.py index 7cbd7c71..dab96151 100644 --- a/py/api/preview_location.py +++ b/py/api/preview_location.py @@ -119,6 +119,11 @@ MESSAGES = { 400, "This path is too long for Windows. Choose a shorter folder path.", ), + "insufficient-space": ( + 507, + "There is not enough space for the thumbnails. Free some space or choose " + "another folder.", + ), "location-needs-move": ( 409, "Use Move thumbnails in System › Sources & scans.", diff --git a/py/api/thumbnail_move.py b/py/api/thumbnail_move.py index 3d4d6472..afc156c2 100644 --- a/py/api/thumbnail_move.py +++ b/py/api/thumbnail_move.py @@ -128,6 +128,18 @@ class ThumbnailMoveMixin: ) self.logger.warning(f"Move thumbnails could not start: {error}") return + left = [ + { + "path": "/".join(item), + "root": s.label, + "detail": "Not a thumbnail; left in place", + } + for s in plan.sources + if s.scan is not None + for item in s.scan.left + ] + if left: + job.record_counted("left", len(left), left) items = [ (source, item) for source in plan.sources diff --git a/tests/browser/system-concept.spec.mjs b/tests/browser/system-concept.spec.mjs index 6018d5ce..33b453bd 100644 --- a/tests/browser/system-concept.spec.mjs +++ b/tests/browser/system-concept.spec.mjs @@ -656,9 +656,10 @@ function mapSourcesConcept(reference, { settings, sources }) { // D113: the range ends at the server's core count (the concept's fixed 16) and shows the real worker count. return [{ ...row, control: { ...row.control, max: String(settings.cpu_count), value: String(settings.worker_threads) } }]; }), - // D158: the inserted row after the first setting row; D98: "Refresh files" closes the surface. + // D158: the inserted row after the first setting row; D196 (#320): the Thumbnail location block after + // "Thumbnail generation"; D98: "Refresh files" closes the surface. surface: { ...reference.surface, tree: [...reference.surface.tree.slice(0, rowAt + 1), 'div.setting-row', - ...reference.surface.tree.slice(rowAt + 1), 'div.operation'] }, + ...reference.surface.tree.slice(rowAt + 1), 'div.setting-block', 'div.operation'] }, operations: [...reference.operations, { ...reference.operations[0], title: REFRESH_FILES.title, description: REFRESH_FILES.description, actions: [{ ...reference.operations[0].actions[0], text: REFRESH_FILES.label }] }], }; diff --git a/tests/browser/thumbnail_location_fixture.py b/tests/browser/thumbnail_location_fixture.py index 2bae5228..2028b7e6 100644 --- a/tests/browser/thumbnail_location_fixture.py +++ b/tests/browser/thumbnail_location_fixture.py @@ -31,12 +31,40 @@ def register_thumbnail_location_commands( ) -> Callable[[], None]: """Register the commands; returns the reset every reset runs.""" from py.api import preview_location, preview_store + from py.api import thumbnail_location, thumbnail_move_plan + from py.config import PromptManagerConfig + from py.config_store import get_config_store folder = root / FOLDER_NAME + default = root / "default-thumbs" + custom = root / "custom-thumbs" # a folder a page types in + journal = root / "data" / "thumbnail-move.json" + # The move's journal and default folder live in this server's root, never + # in the checkout's data folder. + thumbnail_move_plan.journal_path = lambda: str(journal) + thumbnail_move_plan.default_path = lambda: str(default) + thumbnail_location.default_path = lambda: str(default) + api.recover_thumbnail_move() + + def forget_config(config: dict) -> dict: + section = config.get("thumbnails") + if isinstance(section, dict): + section.pop("location", None) + return config def reset() -> None: preview_store.configure(None) - shutil.rmtree(folder, ignore_errors=True) + PromptManagerConfig.THUMBNAIL_LOCATION = None + store = get_config_store() + config, _revision = store.read() + if ( + isinstance(config.get("thumbnails"), dict) + and "location" in config["thumbnails"] + ): + store.commit(forget_config, expected_revision=None) + for path in (folder, default, custom): + shutil.rmtree(path, ignore_errors=True) + journal.unlink(missing_ok=True) def use(mode: str) -> dict: reset() @@ -63,14 +91,18 @@ def register_thumbnail_location_commands( return web.json_response(await api._run_in_executor(use, mode)) def files() -> dict: + setting = preview_store.setting() + where = Path(setting.path) if setting.path else folder previews = sorted( - path.relative_to(folder).as_posix() - for path in folder.rglob("*_thumb.*") + path.relative_to(where).as_posix() + for path in where.rglob("*_thumb.*") if path.is_file() ) sources = [str(path) for path in api._get_all_output_dirs()] return { "folder": previews, + "folder_exists": folder.exists() or default.exists() or custom.exists(), + "custom_path": str(custom), "source_thumbnails": any( (Path(source) / "thumbnails").exists() for source in sources ), @@ -81,7 +113,8 @@ def register_thumbnail_location_commands( return web.json_response(await api._run_in_executor(files)) def remove() -> bool: - shutil.rmtree(folder, ignore_errors=True) + setting = preview_store.setting() + shutil.rmtree(setting.path or folder, ignore_errors=True) return preview_store.refresh() @routes.post("/__test__/thumbnail-folder/remove") @@ -89,4 +122,18 @@ def register_thumbnail_location_commands( available = await api._run_in_executor(remove) return web.json_response({"available": available}) + def foreign() -> bool: + """Another install recreated the folder: its marker has another id.""" + setting = preview_store.setting() + + def other(marker: dict) -> None: + marker["id"] = "another-install" + + preview_location.write_marker(setting.path, other) + return preview_store.refresh() + + @routes.post("/__test__/thumbnail-folder/foreign") + async def foreign_folder(_request: web.Request) -> web.Response: + return web.json_response({"available": await api._run_in_executor(foreign)}) + return reset diff --git a/tests/browser/workspace-thumbnail-location.spec.mjs b/tests/browser/workspace-thumbnail-location.spec.mjs new file mode 100644 index 00000000..a58311cf --- /dev/null +++ b/tests/browser/workspace-thumbnail-location.spec.mjs @@ -0,0 +1,148 @@ +import { startCoverage, saveCoverage } from '../../tools/vendor/browser-coverage.mjs'; +import { test, expect } from './fixtures.mjs'; +import { seedConcept } from './concept-inventory.mjs'; + +// #320 Thumbnail location (D196–D201, the approved 320 mock) on the concept-shaped real fixture: the row in System › +// Sources & scans reviews and runs Move thumbnails both ways, the review refuses a folder in place, and an unplugged +// thumbnail folder shows the notice whose Recreate folder brings it back. The hooks are thumbnail_location_fixture.py's. +const workspace = '/prompt_manager/workspace'; +const NOTICE_KEY = 'pm_update_notice_dismissed'; + +test.beforeEach(async ({ page, request, browserName }) => { + await startCoverage(page, browserName); + await seedConcept(request); + await expect(await request.post('/__test__/system/reset')).toBeOK(); + await expect(await request.post('/__test__/thumbnail-location', { data: { mode: 'source' } })).toBeOK(); + page.locationErrors = []; + page.on('pageerror', error => page.locationErrors.push(error.message)); + await page.addInitScript(key => { if (location.protocol.startsWith('http')) localStorage.setItem(key, '3.2.0'); }, NOTICE_KEY); +}); + +test.afterEach(async ({ page, request, browserName }) => { + await saveCoverage(page, browserName); + await request.post('/__test__/thumbnail-location', { data: { mode: 'source' } }); + expect(page.locationErrors).toEqual([]); +}); + +const block = page => page.locator('#location-block'); +const dialog = page => page.locator('dialog#modal[open]'); + +async function files(request) { + const response = await request.get('/__test__/thumbnail-files'); + await expect(response).toBeOK(); + return response.json(); +} + +async function openSources(page) { + await page.goto(`${workspace}#/system/sources`); + await expect(block(page).locator('#location-status')).not.toHaveText(''); +} + +/** Review the row's choice and start the move; resolves once its job view says it finished. */ +async function moveFromRow(page) { + const planned = page.waitForResponse(response => response.url().endsWith('/images/thumbnails/plan') + && response.request().method() === 'POST'); + await block(page).getByRole('button', { name: 'Review move…' }).click(); + expect((await planned).status()).toBe(200); + await expect(dialog(page).locator('h2')).toHaveText('Move thumbnails'); + await dialog(page).getByRole('button', { name: 'Move thumbnails' }).click(); + await expect(page.locator('#workspace-main h1')).toHaveText('Move thumbnails'); + await expect(page.locator('#workspace-main')).toContainText('Thumbnails moved to'); +} + +test('the row moves thumbnails into one folder and back', async ({ page, request }) => { + await openSources(page); + const row = block(page); + await expect(row.getByRole('radio', { name: /Inside each source/ })).toBeChecked(); + await expect(row.locator('#location-status')).toHaveText('Now: inside each source.'); + await expect(row.getByRole('button', { name: 'Review move…' })).toHaveAttribute('aria-disabled', 'true'); + await expect(row.locator('#location-path')).toBeDisabled(); + + await row.getByRole('radio', { name: /One folder/ }).check(); + await expect(row.locator('#location-path')).toBeEnabled(); + await expect(row.locator('#location-path')).toHaveValue(/default-thumbs$/); + await expect(row.locator('#location-status')).toHaveText(/^Now: inside each source\. The suggested folder is next to your library/); + await expect(row.getByRole('button', { name: 'Review move…' })).toHaveAttribute('aria-disabled', 'false'); + await moveFromRow(page); + await expect(page.locator('#job-phase')).toHaveText(/^Thumbnails are now read from the new location/); + const moved = await files(request); + expect(moved.folder.length).toBeGreaterThan(0); + expect(moved.folder.every(name => /^[a-z0-9_-]+-[0-9a-f]{8}\//.test(name))).toBe(true); + expect(moved.source_thumbnails).toBe(false); + + // Images shows thumbnails served from the folder. + const served = page.waitForResponse(response => response.url().includes('/images/serve/thumbnails/') + && response.status() === 200); + await page.goto(`${workspace}#/images`); + await served; + + await openSources(page); + await expect(block(page).locator('#location-status')).toHaveText(/^Now: one folder, .*default-thumbs\. Using the default folder/); + await block(page).getByRole('radio', { name: /Inside each source/ }).check(); + await moveFromRow(page); + const back = await files(request); + expect(back.source_thumbnails).toBe(true); + expect(back.folder_exists).toBe(false); +}); + +test('a folder the server refuses is explained under the field', async ({ page }) => { + await openSources(page); + const row = block(page); + await row.getByRole('radio', { name: /One folder/ }).check(); + await row.locator('#location-path').fill('thumbnails'); + await row.getByRole('button', { name: 'Review move…' }).click(); + await expect(row.locator('#location-problem')).toHaveText( + 'Enter a full folder path, such as D:\\Thumbnails or /mnt/fast/thumbnails.'); + await expect(row.locator('.location-path')).toHaveAttribute('aria-invalid', 'true'); + await expect(dialog(page)).toHaveCount(0); + await row.locator('#location-path').fill('/somewhere/else'); + await expect(row.locator('#location-problem')).toHaveText(''); +}); + +test('an unplugged folder shows the notice and Recreate folder brings it back', async ({ page, request }) => { + await expect(await request.post('/__test__/thumbnail-location', { data: { mode: 'folder' } })).toBeOK(); + const removed = await request.post('/__test__/thumbnail-folder/remove'); + expect((await removed.json()).available).toBe(false); + await openSources(page); + const notice = page.locator('#unavailable-notice'); + await expect(notice.locator('h2')).toHaveText('The thumbnail folder is unavailable'); + await expect(notice).toContainText('is missing. Originals are shown and no thumbnails are written until it is back.'); + await expect(block(page).locator('#location-status')).toHaveText(/· unavailable\. Originals are shown\.$/); + await notice.getByRole('button', { name: 'Recreate folder' }).click(); + await expect(dialog(page).locator('h2')).toHaveText('Recreate the thumbnail folder?'); + const recreated = page.waitForResponse(response => response.url().endsWith('/location/recreate')); + await dialog(page).getByRole('button', { name: 'Recreate folder' }).click(); + expect((await recreated).status()).toBe(200); + await expect(notice).toHaveCount(0); + await expect(block(page).locator('#location-status')).toHaveText(/^Now: one folder, .*thumbs\.$/); +}); + +test('a folder of your own needs the confirmation before it moves', async ({ page, request }) => { + const custom = (await files(request)).custom_path; + await openSources(page); + const row = block(page); + await row.getByRole('radio', { name: /One folder/ }).check(); + await row.locator('#location-path').fill(custom); + await row.getByRole('button', { name: 'Review move…' }).click(); + const review = dialog(page); + await expect(review.locator('dt')).toContainText(['From', 'To', 'Thumbnails', 'Free space', 'Removed after the move']); + await expect(review.locator('dd').nth(1)).toHaveText(`${custom} (new folder)`); + const move = review.getByRole('button', { name: 'Move thumbnails' }); + await expect(move).toHaveAttribute('aria-disabled', 'true'); + await review.getByRole('checkbox', { name: /may create its own folders/ }).check(); + await expect(move).toHaveAttribute('aria-disabled', 'false'); + await move.click(); + await expect(page.locator('#workspace-main')).toContainText('Thumbnails moved to'); + expect((await files(request)).folder.length).toBeGreaterThan(0); +}); + +test('another folder’s marker offers Use this folder', async ({ page, request }) => { + await expect(await request.post('/__test__/thumbnail-location', { data: { mode: 'folder' } })).toBeOK(); + await expect(await request.post('/__test__/thumbnail-folder/foreign')).toBeOK(); + await openSources(page); + const notice = page.locator('#unavailable-notice'); + await expect(notice).toContainText('holds another thumbnail folder'); + await notice.getByRole('button', { name: 'Use this folder' }).click(); + await expect(notice).toHaveCount(0); + await expect(block(page).locator('#location-status')).toHaveText(/^Now: one folder, .*thumbs\.$/); +}); diff --git a/tests/js/workspace-system-thumbnail-location.test.js b/tests/js/workspace-system-thumbnail-location.test.js new file mode 100644 index 00000000..f89ff10e --- /dev/null +++ b/tests/js/workspace-system-thumbnail-location.test.js @@ -0,0 +1,137 @@ +// Thumbnail location (#320, D196–D201): the row's state and status line, the move review's rows, the unavailable +// notice's text, the Activity views of a move and a prune, the prune scope's plan line and the Move library row. +const test = require('node:test'); +const assert = require('node:assert/strict'); + +const locationModule = import('../../web/js/workspace/system-thumbnail-location.mjs'); +const kindsModule = import('../../web/js/workspace/activity-kinds.mjs'); +const reviewModule = import('../../web/js/workspace/thumbnail-review.mjs'); +const libraryModule = import('../../web/js/workspace/library-location.mjs'); + +const SOURCE = Object.freeze({ mode: 'source', path: null, default_path: '/data/thumbnails', available: true, + problem: null, cleanup_pending: 0 }); +const FOLDER = Object.freeze({ mode: 'folder', path: 'E:\\Thumbnails', default_path: '/data/thumbnails', + available: true, problem: null, cleanup_pending: 0 }); + +const PLAN = Object.freeze({ + from: { mode: 'source', path: null, available: true }, + to: { mode: 'folder', path: 'E:\\Thumbnails', exists: true, will_create: false, default: false }, + totals: { sources: 3, files: 8512, bytes: 1.2 * 1024 ** 3, orphans: 41, temps: 2, links: 1, left_in_place: 1, + too_long: 32, unlisted_sources: 0, unlisted_files: 0 }, + too_long_listed: ['Archive 2025 · a_thumb.webp', 'Archive 2025 · b_thumb.webp'], + left_listed: ['Archive 2025 · notes.txt'], + unlisted: [], + free_bytes: 1.6 * 1024 ** 4, + confirm_required: true, + token: 't', +}); + +test('the status line says where previews are now', async () => { + const { statusText } = await locationModule; + assert.equal(statusText(SOURCE), 'Now: inside each source.'); + assert.equal(statusText(FOLDER), 'Now: one folder, E:\\Thumbnails.'); + assert.equal(statusText({ ...FOLDER, path: '/data/thumbnails' }), + 'Now: one folder, /data/thumbnails. Using the default folder next to your library.'); + assert.equal(statusText({ ...FOLDER, available: false, problem: 'missing' }), + 'Now: one folder, E:\\Thumbnails · unavailable. Originals are shown.'); + assert.match(statusText({ ...SOURCE, cleanup_pending: 1 }), /Old thumbnails in 1 place wait to be removed/); + assert.equal(statusText(undefined), ''); + assert.match(statusText(SOURCE, { mode: 'folder', path: '/data/thumbnails' }), + /^Now: inside each source\. The suggested folder is next to your library/); + assert.equal(statusText(SOURCE, { mode: 'folder', path: '/other' }), 'Now: inside each source.'); +}); + +test('Review move… is offered only for another location', async () => { + const { choiceChanged } = await locationModule; + assert.equal(choiceChanged(SOURCE, { mode: 'source' }), false); + assert.equal(choiceChanged(SOURCE, { mode: 'folder', path: ' ' }), false); + assert.equal(choiceChanged(SOURCE, { mode: 'folder', path: '/data/thumbnails' }), true); + assert.equal(choiceChanged(FOLDER, { mode: 'folder', path: 'E:\\Thumbnails' }), false); + assert.equal(choiceChanged(FOLDER, { mode: 'folder', path: 'F:\\Thumbs' }), true); + assert.equal(choiceChanged(FOLDER, { mode: 'source' }), true); + assert.equal(choiceChanged({ ...SOURCE, problem: 'invalid' }, { mode: 'source' }), true); + assert.equal(choiceChanged(undefined, { mode: 'source' }), false); +}); + +test('the review lists what moves, what is removed and what stays', async () => { + const { reviewRows } = await locationModule; + const rows = Object.fromEntries(reviewRows(PLAN)); + assert.equal(rows.From, 'Inside each source (3 sources)'); + assert.equal(rows.To, 'E:\\Thumbnails (empty folder)'); + assert.equal(rows.Thumbnails, '8,512 in 3 sources · 1.2 GB'); + assert.equal(rows['Free space'], '1638.4 GB free'); + assert.match(rows['Too long for Windows'][0], /^32 thumbnails cannot be stored there/); + assert.deepEqual(rows['Too long for Windows'][1], [...PLAN.too_long_listed, 'and 30 more']); + assert.deepEqual(rows['Removed after the move'][1], + ['41 thumbnails whose original is gone', '2 unfinished temporary files']); + assert.equal(rows['Left in place'][0], '1 file that is not a thumbnail:'); + assert.equal(rows['Links skipped'], '1 link inside a thumbnails folder is not followed'); +}); + +test('the review of a move back names the folder and the removed sources', async () => { + const { reviewRows } = await locationModule; + const back = reviewRows({ ...PLAN, from: { mode: 'folder', path: '/data/thumbnails' }, + to: { mode: 'source', path: null }, unlisted: [{ key: 'old-1c22e0a4', files: 240 }], + totals: { sources: 1, files: 1, bytes: 10 }, free_bytes: undefined }); + const rows = Object.fromEntries(back); + assert.equal(rows.From, '/data/thumbnails'); + assert.equal(rows.To, 'Inside each source (1 source)'); + assert.equal(rows['Free space'], undefined); + assert.match(rows['Removed after the move'][0], /The folder itself is removed once empty/); + assert.deepEqual(rows['Removed after the move'][1], + ['1 source no longer listed: 240 thumbnails are removed; they are made again if it returns']); + const fresh = Object.fromEntries(reviewRows({ ...PLAN, to: { mode: 'folder', path: '/d', default: true, will_create: true } })); + assert.equal(fresh.To, '/d (new, next to your library)'); +}); + +test('the notice says why the folder is unavailable', async () => { + const { noticeText } = await locationModule; + assert.match(noticeText({ ...FOLDER, problem: 'missing' }), /^E:\\Thumbnails is missing\. Originals are shown/); + assert.match(noticeText({ ...FOLDER, problem: 'unreachable' }), /can’t be reached/); + assert.match(noticeText({ ...FOLDER, problem: 'marker-mismatch' }), /holds another thumbnail folder/); + assert.match(noticeText({ ...FOLDER, problem: 'invalid' }), /config\.json can’t be used/); + assert.match(noticeText({ ...FOLDER, problem: null }), /is being checked/); +}); + +test('sizes are written as the server writes them', async () => { + const { sizeText } = await locationModule; + assert.equal(sizeText(512), '512 B'); + assert.equal(sizeText(85.2 * 1024 * 1024), '85.2 MB'); +}); + +test('a move and a prune show their own tiles in Activity', async () => { + const { viewOf, movePhase, finishToast } = await kindsModule; + const move = { kind: 'thumbnails', state: 'running', cancellable: true, scope: { mode: 'move', from: 'source' }, + counters: { moved: 3, skipped: 0, left: 1, errors: 2 }, status_line: '' }; + assert.deepEqual(viewOf(move).tiles.map(([group, label]) => label), ['Moved', 'Already there', 'Left in place', 'Errors']); + assert.equal(viewOf(move).cancelLabel, 'Cancel move'); + assert.match(movePhase(move), /^Step 1 of 3: copying\. Thumbnails are still read from inside each source/); + assert.match(movePhase({ ...move, cancellable: false }), /can no longer be cancelled/); + assert.match(movePhase({ ...move, scope: { mode: 'move', phase: 'remove' } }), /Removing the old copies/); + assert.match(movePhase({ ...move, state: 'finished' }), /1 file that is not a thumbnail was left in place/); + assert.match(movePhase({ ...move, state: 'cancelled' }), /^Nothing changed/); + const done = { ...move, state: 'finished', status_line: 'Thumbnails moved to E:\\T: 3 moved · old copies removed.' }; + assert.equal(finishToast(done), done.status_line); + const prune = { kind: 'thumbnails', state: 'running', cancellable: true, scope: { mode: 'prune' }, + counters: { removed: 1, left: 0, errors: 0 } }; + assert.deepEqual(viewOf(prune).tiles.map(([, label]) => label), ['Removed', 'Left in place', 'Errors']); + const generate = { kind: 'thumbnails', state: 'running', cancellable: true, scope: { mode: 'missing' }, + counters: { generated: 0, skipped: 0, errors: 0 } }; + assert.deepEqual(viewOf(generate).tiles.map(([, label]) => label), ['Generated', 'Skipped', 'Errors']); +}); + +test('the prune scope reads its plan line from the server', async () => { + const { planText, PLAN_FAILED, PRUNE_OPTION } = await reviewModule; + assert.equal(PRUNE_OPTION[1], 'Remove thumbnails of removed sources'); + assert.equal(planText({ mode: 'prune', line: '2 sources no longer listed: 1,240 thumbnails (85.2 MB) are removed.' }), + '2 sources no longer listed: 1,240 thumbnails (85.2 MB) are removed.'); + assert.equal(planText({ mode: 'prune' }), PLAN_FAILED); +}); + +test('Move library says where thumbnails kept in one folder stay', async () => { + const { reviewGrid } = await libraryModule; + const base = { path: 'p', target: 't', size_bytes: 0, config_moves: true }; + assert.equal(reviewGrid(base).some(([term]) => term === 'Thumbnails'), false); + const rows = Object.fromEntries(reviewGrid({ ...base, thumbnails_folder: '/data/thumbnails' })); + assert.equal(rows.Thumbnails, 'Stay in /data/thumbnails. Move them in System › Sources & scans.'); +}); diff --git a/tests/test_api_library_location.py b/tests/test_api_library_location.py index 7b898aa0..d3edcc6d 100644 --- a/tests/test_api_library_location.py +++ b/tests/test_api_library_location.py @@ -91,6 +91,8 @@ class TestLocationRoute(LibraryAPITestCase): "size_bytes": os.path.getsize(self.api.db.model.db_path), "target_source": "default", "config_moves": True, + # #320 D201: previews kept inside each source (no folder). + "thumbnails_folder": None, }, ) diff --git a/tests/test_api_thumbnail_location.py b/tests/test_api_thumbnail_location.py index 1650f9d4..d1eb2fd8 100644 --- a/tests/test_api_thumbnail_location.py +++ b/tests/test_api_thumbnail_location.py @@ -302,7 +302,7 @@ class TestValidation(unittest.TestCase): def test_every_code_has_a_message_without_a_path(self): for code, (status, text) in pl.MESSAGES.items(): with self.subTest(code=code): - self.assertIn(status, (400, 409)) + self.assertIn(status, (400, 409, 507)) self.assertTrue(text.endswith(".")) diff --git a/tools/vendor/frontend-coverage-baseline.json b/tools/vendor/frontend-coverage-baseline.json index eb695acc..fd99db77 100644 --- a/tools/vendor/frontend-coverage-baseline.json +++ b/tools/vendor/frontend-coverage-baseline.json @@ -641,6 +641,12 @@ "functions": 95, "statements": 98 }, + "workspace/system-thumbnail-location.mjs": { + "lines": 80, + "branches": 65, + "functions": 75, + "statements": 80 + }, "workspace/system-thumbnails.mjs": { "lines": 98, "branches": 95, diff --git a/web/js/api-client.js b/web/js/api-client.js index 009d7cb1..bc3e9523 100644 --- a/web/js/api-client.js +++ b/web/js/api-client.js @@ -62,6 +62,13 @@ (typeof job.title === "string" && job.title.length > 0 && job.title.length <= 200))) { this.job = { id: job.id, title: job.title }; } + // A move refused for space (#320) carries its review, so the page + // can show what would have moved and why it cannot start. + this.plan = null; + if (this.code === "insufficient-space" && current && current.plan && + typeof current.plan === "object" && !Array.isArray(current.plan)) { + this.plan = current.plan; + } this.source = null; if (this.code === "invalid-source" && current && typeof current.source === "string" && current.source.length > 0 && current.source.length <= 4096) { diff --git a/web/js/workspace/activity-job-view.mjs b/web/js/workspace/activity-job-view.mjs index 5d125636..3b607c67 100644 --- a/web/js/workspace/activity-job-view.mjs +++ b/web/js/workspace/activity-job-view.mjs @@ -170,7 +170,10 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) { bar.max = 100; const stats = element('dl', undefined, 'stat-row'); const tileValues = new Map(); - progressPart.node.append(bar, stats); + // #320 D198: a kind with phases (Move thumbnails) says which one runs and what it means, under the bar. + const phase = element('p', '', 'small'); + phase.id = 'job-phase'; + progressPart.node.append(bar, phase, stats); const resultsPart = section('Results', 'job-results-title'); const search = element('label', undefined, 'search-field'); @@ -228,6 +231,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) { function layout(job) { view = viewOf(job); if (!view.tiles.length) stats.remove(); + if (typeof view.phase !== 'function') phase.remove(); for (const [group, label, tone] of view.tiles) { const tile = element('div', undefined, tone ? `stat ${tone}` : 'stat'); const value = element('dd', '0'); @@ -341,6 +345,7 @@ export function mountActivityJobView(main, { store, scope, api, jobId }) { cancel.remove(); } status.textContent = progressLine(job); + if (typeof view?.phase === 'function') phase.textContent = view.phase(job); // D111: no value while the total is unknown; a job that ended with nothing counted shows how it ended. showProgress(bar, job.total > 0 || job.state !== 'finished' ? job : { ...job, progress: 100 }); bar.setAttribute('aria-label', scan ? 'Scan progress' : `${job.title} progress`); diff --git a/web/js/workspace/activity-kinds.mjs b/web/js/workspace/activity-kinds.mjs index 18690c73..8cf81377 100644 --- a/web/js/workspace/activity-kinds.mjs +++ b/web/js/workspace/activity-kinds.mjs @@ -51,6 +51,26 @@ export const removedOf = job => counted(job, 'removed', 'removed'); /** "12 imported" (System S10: the LoRA reference import). */ export const importedOf = job => counted(job, 'imported', 'imported'); +/** #320 D198: the step a Move thumbnails job is in, and what it means meanwhile (under the progress bar). */ +export function movePhase(job) { + const from = job.scope?.from === 'folder' ? 'the old folder' : 'inside each source'; + if (job.state === 'running') { + if (job.scope?.phase === 'remove') return 'Removing the old copies a move left. Thumbnails are read from the new location.'; + if (job.cancellable !== false) { + return `Step 1 of 3: copying. Thumbnails are still read from ${from} until the switch. New thumbnails are not ` + + 'written until the move ends; originals are shown instead.'; + } + return 'Step 2 and 3 of 3: switching, then removing the old copies. This can no longer be cancelled.'; + } + if (job.state === 'finished') { + const left = job.counters?.left ?? 0; + return left ? `Thumbnails are now read from the new location. ${number(left)} ${noun(left, 'file')} that ` + + `${left === 1 ? 'is' : 'are'} not ${left === 1 ? 'a thumbnail was' : 'thumbnails were'} left in place.` + : 'Thumbnails are now read from the new location.'; + } + return 'Nothing changed: thumbnails are read from where they were.'; +} + /** "2 output folders": what a scan or a thumbnail job walks. */ function foldersOf(job) { const folders = job.scope?.output_folders; @@ -159,6 +179,32 @@ export const KINDS = Object.freeze({ outcomes: tiles([['generated', 'Generated'], ['errors', 'Errors'], ['skipped', 'Skipped']]), openable: groups('generated', 'skipped'), }), + // #320 D198: Move thumbnails, a `thumbnails` job with `scope.mode` "move": Moved (green), Already there, Left in + // place and Errors (red); Cancel only until the switch (the server says so through `cancellable`). + 'thumbnails:move': Object.freeze({ + ...GENERIC, + pillVerb: 'Moving thumbnails', + finishToast: job => (job.status_line && job.state !== 'running' ? job.status_line : titleFinished(job)), + line: job => (job.state === 'running' ? '' : job.status_line || ''), + kept: job => counted(job, 'moved', 'moved'), + tiles: tiles([['moved', 'Moved', 'added'], ['skipped', 'Already there', ''], ['left', 'Left in place', ''], + ['errors', 'Errors', 'bad']]), + outcomes: tiles([['moved', 'Moved'], ['errors', 'Errors'], ['skipped', 'Already there'], ['left', 'Left in place']]), + log: 'Move log', + phase: movePhase, + cancelLabel: 'Cancel move', + cancel: 'Cancel move', + }), + // #320 D200: Remove thumbnails of removed sources (`scope.mode` "prune"): Removed, Left in place and Errors. + 'thumbnails:prune': Object.freeze({ + ...GENERIC, + pillVerb: 'Removing thumbnails', + finishToast: job => (job.status_line && job.state !== 'running' ? job.status_line : titleFinished(job)), + line: job => (job.state === 'running' ? '' : job.status_line || ''), + kept: removedOf, + tiles: tiles([['removed', 'Removed', 'added'], ['left', 'Left in place', ''], ['errors', 'Errors', 'bad']]), + outcomes: tiles([['removed', 'Removed'], ['errors', 'Errors'], ['left', 'Left in place']]), + }), // System S7: a reviewed maintenance plan's apply, titled after its operation; it runs without a known total. maintenance: Object.freeze({ ...GENERIC, @@ -243,7 +289,9 @@ export const held = job => job.cancellable === false && Object.keys(countsOf(job */ export function viewOf(job) { if (held(job)) return GENERIC; - const view = kindOf(job.kind); + // #320 D198: a thumbnails job chooses its tiles by its scope's mode (move, prune). + const mode = job.kind === 'thumbnails' ? job.scope?.mode : null; + const view = (mode === 'move' || mode === 'prune') ? KINDS[`thumbnails:${mode}`] : kindOf(job.kind); const counts = countsOf(job); const has = ([group]) => Object.hasOwn(counts, group); if (view.tiles.every(has) && view.outcomes.every(has)) return view; diff --git a/web/js/workspace/library-location.mjs b/web/js/workspace/library-location.mjs index 447d5aeb..f3ddde7c 100644 --- a/web/js/workspace/library-location.mjs +++ b/web/js/workspace/library-location.mjs @@ -97,6 +97,9 @@ export function reviewGrid(location) { ? 'The database backups (config.json stays where PROMPT_MANAGER_CONFIG_PATH names it)' : 'config.json and the database backups'], ['Old files', 'Renamed *.moved-to-…, never deleted'], + // #320 D201: previews kept in one folder do not move with the library (D8A). + ...(typeof location.thumbnails_folder === 'string' && location.thumbnails_folder + ? [['Thumbnails', `Stay in ${location.thumbnails_folder}. Move them in System › Sources & scans.`]] : []), ]; } diff --git a/web/js/workspace/system-overview.mjs b/web/js/workspace/system-overview.mjs index 7356213d..d05a2c0d 100644 --- a/web/js/workspace/system-overview.mjs +++ b/web/js/workspace/system-overview.mjs @@ -28,6 +28,7 @@ import { showWorkspaceToast } from './toast.mjs'; import { noun } from './plural.mjs'; import { operationRow } from './system-controls.mjs'; import { openThumbnailReview } from './thumbnail-review.mjs'; +import { noticeSlot } from './system-thumbnail-location.mjs'; import { isLocation, locationRow, onMoveEnded } from './library-location.mjs'; import { STATUS_LOADING, STATUS_URL, backupStatusText } from './system-backups.mjs'; @@ -406,7 +407,7 @@ function overviewNodes(overview, { api, activity, notify, scope, navigate, openS * `counts` is the shell's sidebar counts (`{get(id), subscribe(listener), freshen(ids)}`, index.mjs): the Images metric * shows its `images` count and follows it, and a visit re-reads it as the workspace Overview does (#555). */ -export function mountOverview(container, { api, activity, scope, counts = null, navigate = navigateTo, +export function mountOverview(container, { api, activity, scope, counts = null, drafts = null, navigate = navigateTo, notify = showWorkspaceToast, latestOnly = window.ApiClient.latestOnly, STALE = window.ApiClient.STALE }) { const overviewRead = latestOnly({ lifetime: scope.signal }); const statisticsRead = latestOnly({ lifetime: scope.signal }); @@ -466,7 +467,8 @@ export function mountOverview(container, { api, activity, scope, counts = null, if (overview === STALE || scope.signal.aborted) return; const { nodes, thumbnails, backup, metrics } = overviewNodes(overview, { api, activity, notify, scope, navigate, openStatistics, images: imagesCount(), - openThumbnails: trigger => openThumbnailReview({ api, store: activity, scope, trigger, navigate }) }); + openThumbnails: trigger => openThumbnailReview({ api, store: activity, scope, trigger, navigate, + location: drafts?.get('host', 'thumbnail_location') }) }); imagesValue = metrics.children[2].querySelector('strong'); thumbnailRow = thumbnails; backupRow = backup; @@ -522,5 +524,12 @@ export function mountOverview(container, { api, activity, scope, counts = null, load(); loadThumbnails(); loadBackup(); - return first; + // #320 D199: the unavailable thumbnail folder's notice, first on the page while it lasts. + if (!drafts) return first; + const unavailable = noticeSlot({ api, scope, notify, location: () => drafts.get('host', 'thumbnail_location'), + onChange: () => navigate('#/system/sources'), onDone: () => drafts.reloadHost?.() }); + scope.cleanup(drafts.subscribe(() => unavailable.paint())); + // The anchor goes in first, so the notice (inserted after it) leads the section once the anchor is mounted. + unavailable.paint(); + return [unavailable.anchor, ...first]; } diff --git a/web/js/workspace/system-sources.mjs b/web/js/workspace/system-sources.mjs index cd71d330..13d966f4 100644 --- a/web/js/workspace/system-sources.mjs +++ b/web/js/workspace/system-sources.mjs @@ -35,6 +35,7 @@ import { createButton } from '../shared/toolkit/core.js'; import { closeWorkspaceDialog, field, openContentDialog } from './workspace-dialogs.mjs'; import { openMetadataScan } from './activity-scan.mjs'; import { openThumbnailReview } from './thumbnail-review.mjs'; +import { locationBlock, noticeSlot } from './system-thumbnail-location.mjs'; import { settingRow, operationRow } from './system-controls.mjs'; import { HOST_UNAVAILABLE } from './system-appearance.mjs'; import { showWorkspaceToast } from './toast.mjs'; @@ -234,6 +235,19 @@ export function mountSources(container, { api, activity, scope, drafts, notify = nodes.push(notice); } + // #320 D199: the unavailable thumbnail folder's notice, first on the page while it lasts. + const thumbnailLocation = () => drafts.get('host', 'thumbnail_location'); + const location = locationBlock({ api, store: activity, scope, location: thumbnailLocation, notify }); + const unavailable = noticeSlot({ api, scope, location: thumbnailLocation, notify, onChange: () => location.focus(), + onDone: () => drafts.reloadHost?.() }); + const paintLocation = () => { + location.paint(); + unavailable.paint(); + }; + scope.cleanup(drafts.subscribe(paintLocation)); + nodes.push(unavailable.anchor); + paintLocation(); + const surface = el('section', undefined, 'surface'); const heading = el('div', undefined, 'panel-heading'); const addButton = createButton('+ Add source'); @@ -266,7 +280,10 @@ export function mountSources(container, { api, activity, scope, drafts, notify = operationRow({ scope, title: 'Enrich existing metadata', description: 'Resolve both prompts, deduplicate recipes, preserve uncertain links.', label: 'Review scan', onClick: scanAll }), operationRow({ scope, title: 'Thumbnail generation', description: 'Generate missing previews; rebuilding all is a separate explicit option.', - label: 'Review options', onClick: trigger => openThumbnailReview({ api, store: activity, scope, trigger }) }), + label: 'Review options', onClick: trigger => openThumbnailReview({ api, store: activity, scope, trigger, + location: thumbnailLocation() }) }), + // #320 D196: where previews are kept; never a draft (Move thumbnails is its only writer). + location.node, // D98: the legacy gallery's Refresh, which rereads every source's file list. operationRow({ scope, title: 'Refresh files', description: 'Read every source again to find files added or removed outside ComfyUI.', label: 'Refresh', onClick: refresh })); diff --git a/web/js/workspace/system-thumbnail-location.mjs b/web/js/workspace/system-thumbnail-location.mjs new file mode 100644 index 00000000..92175b67 --- /dev/null +++ b/web/js/workspace/system-thumbnail-location.mjs @@ -0,0 +1,396 @@ +/** + * Thumbnail location (#320, beyond the concept: D196–D199, approved static mock `320-mock`): where previews are kept. + * + * - D196 `locationBlock`: a `.setting-block` in System › Sources & scans after "Thumbnail generation": the radios + * "Inside each source" / "One folder", a "Folder path" field (enabled with One folder, prefilled with the folder in + * use or the server's `default_path`), the status line and "Review move…" (enabled when the choice differs from the + * location in use). Not a host draft: the Save row never saves it; the Move thumbnails job is the only writer (D4B). + * A refusal of the review shows in place under the field (`aria-invalid`), in the server's words. + * - D197 `openMoveReview`: the shared review (Move library's shape) over the server's plan + * (`POST /images/thumbnails/plan`): From, To, Thumbnails, Free space, Too long for Windows, Removed after the move, + * Left in place, Links skipped; a folder that is not the default asks for the confirmation checkbox first. "Move + * thumbnails" starts the job (`POST /jobs/thumbnails {mode: "move", token, confirmed}`) and opens it in Activity. + * - D199 `locationNotice`: while the thumbnail folder is unavailable, a `.notice` on Overview and Sources & scans + * says why, with "Recreate folder" (a missing folder, after a confirmation) or "Use this folder" (another folder's + * marker), and "Change location". + * + * The location comes from GET /settings' `thumbnail_location` (`{mode, path, default_path, available, problem, + * cleanup_pending}`), read with the host settings. + */ +import { createButton } from '../shared/toolkit/core.js'; +import { closeWorkspaceDialog, openContentDialog, openReviewDialog } from './workspace-dialogs.mjs'; +import { busyToast } from './activity-scan.mjs'; +import { showWorkspaceToast } from './toast.mjs'; +import { navigateTo } from './state.mjs'; +import { noun } from './plural.mjs'; + +export const PLAN_URL = '/prompt_manager/images/thumbnails/plan'; +export const RECREATE_URL = '/prompt_manager/images/thumbnails/location/recreate'; +export const ADOPT_URL = '/prompt_manager/images/thumbnails/location/adopt'; +export const LOCATION_HINT = 'Where previews are kept. Originals are never moved. Avoid cloud-synced folders.'; +export const SENTENCE = 'Every thumbnail is copied and checked before Prompt Manager switches. Scans, deletes and ' + + 'tagging wait until it finishes; closing this page does not stop it. Originals are never touched.'; +export const REVIEW_FAILED = 'The move could not be reviewed. Try again.'; +export const START_FAILED = 'Move thumbnails could not start. Try again.'; +export const NOTICE_TITLE = 'The thumbnail folder is unavailable'; +const BACK = 'Originals are shown and no thumbnails are written until it is back.'; +const CHECK = 'If its drive is unplugged, connect it; Prompt Manager checks again every few seconds.'; + +const number = value => Number(value || 0).toLocaleString('en-US'); +const isFolder = location => location?.mode === 'folder'; + +/** "1.2 GB" (binary units, one decimal, as the server writes them). */ +export function sizeText(bytes) { + let size = Number(bytes) || 0; + for (const unit of ['B', 'KB', 'MB', 'GB']) { + if (size < 1024 || unit === 'GB') return unit === 'B' ? `${size} B` : `${size.toFixed(1)} ${unit}`; + size /= 1024; + } + return `${size.toFixed(1)} GB`; +} + +export const SUGGESTED = 'The suggested folder is next to your library; Review move… shows what happens before anything ' + + 'is copied.'; + +/** The status line under the row; `choice` (`{mode, path}`) adds the suggestion while the default folder is chosen. */ +export function statusText(location, choice = null) { + if (!location) return ''; + if (!isFolder(location) && choice?.mode === 'folder' && location.default_path + && choice.path?.trim() === location.default_path) { + return `Now: inside each source. ${SUGGESTED}`; + } + let text = isFolder(location) ? `Now: one folder, ${location.path}` : 'Now: inside each source'; + if (isFolder(location) && location.available === false) return `${text} · unavailable. Originals are shown.`; + text += '.'; + if (isFolder(location) && location.path && location.path === location.default_path) { + text += ' Using the default folder next to your library.'; + } + if (location.cleanup_pending > 0) { + text += ` Old thumbnails in ${number(location.cleanup_pending)} ${noun(location.cleanup_pending, 'place')} wait to ` + + 'be removed until it is connected.'; + } + return text; +} + +/** Whether the row's choice (`{mode, path}`) differs from the location in use (Review move… enabled). */ +export function choiceChanged(location, choice) { + if (!location) return false; + if (choice.mode !== 'folder') return isFolder(location) || Boolean(location.problem); + const path = (choice.path || '').trim(); + return Boolean(path) && (!isFolder(location) || path !== location.path); +} + +/** The D197 review rows (`[term, value]`, a value may be `[lead, items]`) for the server's plan. */ +export function reviewRows(plan) { + const totals = plan.totals ?? {}; + const sources = totals.sources ?? plan.sources?.length ?? 0; + const inside = `Inside each source (${number(sources)} ${noun(sources, 'source')})`; + const from = plan.from?.mode === 'folder' ? plan.from.path : inside; + let to = inside; + if (plan.to?.mode === 'folder') { + const state = plan.to.default ? (plan.to.will_create ? 'new, next to your library' : 'next to your library') + : (plan.to.will_create ? 'new folder' : 'empty folder'); + to = `${plan.to.path} (${state})`; + } + const rows = [['From', from], ['To', to], + ['Thumbnails', `${number(totals.files)} in ${number(sources)} ${noun(sources, 'source')} · ${sizeText(totals.bytes)}`]]; + if (Number.isFinite(plan.free_bytes)) rows.push(['Free space', `${sizeText(plan.free_bytes)} free`]); + if (totals.too_long > 0) { + const more = totals.too_long - (plan.too_long_listed?.length ?? 0); + rows.push(['Too long for Windows', [`${number(totals.too_long)} ${noun(totals.too_long, 'thumbnail')} cannot be ` + + 'stored there. They show the original instead; choose a shorter folder to keep them.', + [...(plan.too_long_listed ?? []), ...(more > 0 ? [`and ${number(more)} more`] : [])]]]); + } + const removed = []; + if (totals.orphans > 0) removed.push(`${number(totals.orphans)} ${noun(totals.orphans, 'thumbnail')} whose original is gone`); + if (totals.temps > 0) removed.push(`${number(totals.temps)} unfinished temporary ${noun(totals.temps, 'file')}`); + for (const source of plan.unlisted ?? []) { + removed.push(`1 source no longer listed: ${number(source.files)} ${noun(source.files, 'thumbnail')} are removed; ` + + 'they are made again if it returns'); + } + const oldCopies = plan.from?.mode === 'folder' + ? 'The thumbnail folders of these sources inside it. The folder itself is removed once empty if Prompt Manager created it.' + : 'The old copies in each source’s thumbnails folder'; + rows.push(['Removed after the move', [removed.length ? `${oldCopies}, and:` : `${oldCopies}.`, removed]]); + if (totals.left_in_place > 0) { + rows.push(['Left in place', [`${number(totals.left_in_place)} ${noun(totals.left_in_place, 'file')} that ` + + `${totals.left_in_place === 1 ? 'is' : 'are'} not ${totals.left_in_place === 1 ? 'a thumbnail' : 'thumbnails'}:`, + plan.left_listed ?? []]]); + } + if (totals.links > 0) { + rows.push(['Links skipped', `${number(totals.links)} ${noun(totals.links, 'link')} inside a thumbnails folder ` + + `${totals.links === 1 ? 'is' : 'are'} not followed`]); + } + return rows; +} + +/** The notice text for an unavailable `location` (D199). */ +export function noticeText(location) { + const where = location?.path || 'The thumbnail folder'; + switch (location?.problem) { + case 'missing': return `${where} is missing. ${BACK} ${CHECK}`; + case 'unreachable': return `${where} can’t be reached. ${BACK} ${CHECK}`; + case 'marker-mismatch': return `${where} holds another thumbnail folder. ${BACK} Use it if it is yours, or change ` + + 'the location.'; + case 'invalid': return `The thumbnail location in config.json can’t be used. ${BACK} Change the location.`; + default: return `${where} is being checked. ${BACK}`; + } +} + +function el(tag, text, className) { + const node = document.createElement(tag); + if (text !== undefined) node.textContent = text; + if (className) node.className = className; + return node; +} + +function radio(value, label, hint) { + const node = el('label'); + const input = el('input'); + input.type = 'radio'; + input.name = 'thumb-location'; + input.value = value; + input.id = `loc-${value}`; + node.append(input, ` ${label} `, el('span', hint, 'small')); + return { node, input }; +} + +/** Start the move with the plan's `token`; opens it in Activity, or says why not. */ +export async function startMove({ store, scope, token, confirmed, notify = showWorkspaceToast, navigate = navigateTo }) { + try { + const result = await store.start('thumbnails', { mode: 'move', token, confirmed }); + if (scope.signal.aborted) return null; + if (result.busy !== undefined) { + notify(busyToast(result.busy)); + return null; + } + navigate(`#/activity/${result.job.id}`); + return result.job; + } catch (error) { + if (!scope.signal.aborted) notify(typeof error?.code === 'string' && error.message ? error.message : START_FAILED); + return null; + } +} + +/** Open the D197 review over `plan`; a plan the server refused for space keeps Move thumbnails unavailable. */ +export function openMoveReview({ scope, store, plan, refusal = null, trigger = null, notify, navigate }) { + const options = []; + let box = null; + if (plan.confirm_required && !refusal) { + const label = el('label', undefined, 'review-confirm'); + box = el('input'); + box.type = 'checkbox'; + box.id = 'review-confirmed'; + label.append(box, el('span', `Prompt Manager may create its own folders and a .promptmanager-thumbnails.json file in ${plan.to.path}.`)); + options.push(label); + } + if (refusal) options.push(el('p', refusal, 'small review-refusal')); + const review = openReviewDialog({ scope, trigger, title: 'Move thumbnails', grid: reviewRows(plan), options, + sentence: refusal ? null : SENTENCE, cancelLabel: 'Cancel', primary: { label: 'Move thumbnails' }, + onStart: () => { startMove({ store, scope, token: plan.token, confirmed: Boolean(box?.checked), notify, navigate }); } }); + const locked = () => Boolean(refusal) || (box !== null && !box.checked); + review.setPrimary({ label: 'Move thumbnails', disabled: locked() }); + if (box) scope.listen(box, 'change', () => review.setPrimary({ label: 'Move thumbnails', disabled: locked() })); + return review; +} + +/** + * The D196 row; `location()` gives the server's `thumbnail_location` (undefined while read). Returns `{node, paint, + * focus}`; `paint()` follows a new location. + */ +export function locationBlock({ api, store, scope, location, notify = showWorkspaceToast, navigate = navigateTo }) { + const block = el('div', undefined, 'setting-block'); + block.id = 'location-block'; + const text = el('div', undefined, 'setting-block-text'); + const label = el('span', 'Thumbnail location', 'setting-block-label'); + label.id = 'location-label'; + text.append(label, el('p', LOCATION_HINT)); + const choice = el('div', undefined, 'location-choice'); + choice.setAttribute('role', 'radiogroup'); + choice.setAttribute('aria-labelledby', 'location-label'); + const source = radio('source', 'Inside each source', '/thumbnails'); + const folder = radio('folder', 'One folder', 'one subfolder per source'); + choice.append(source.node, folder.node); + const pathField = el('div', undefined, 'location-path field'); + const pathLabel = el('label', 'Folder path'); + pathLabel.htmlFor = 'location-path'; + const input = el('input'); + input.id = 'location-path'; + input.autocomplete = 'off'; + input.spellcheck = false; + const problem = el('p', '', 'small location-problem'); + problem.id = 'location-problem'; + problem.setAttribute('role', 'alert'); + pathField.append(pathLabel, input, problem); + const status = el('p', '', 'small location-status'); + status.id = 'location-status'; + status.setAttribute('role', 'status'); + const actions = el('div', undefined, 'row-actions'); + const review = createButton('Review move…'); + review.id = 'review-move'; + actions.append(review); + block.append(text, choice, pathField, status, actions); + + let shown; + const setProblem = message => { + problem.textContent = message || ''; + pathField.setAttribute('aria-invalid', String(Boolean(message))); + }; + const current = () => ({ mode: folder.input.checked ? 'folder' : 'source', path: input.value }); + const repaintButton = () => { + const now = location(); + input.disabled = !folder.input.checked || !now; + source.input.disabled = folder.input.disabled = !now; + const changed = choiceChanged(now, current()); + status.textContent = now ? statusText(now, current()) : ''; + review.setAttribute('aria-disabled', String(!changed)); + review.classList.toggle('ca-primary', changed); + }; + const paint = () => { + const now = location(); + if (now && now !== shown) { + shown = now; + (isFolder(now) ? folder : source).input.checked = true; + input.value = (isFolder(now) ? now.path : now.default_path) || ''; + setProblem(null); + } + repaintButton(); + }; + for (const control of [source.input, folder.input]) { + scope.listen(control, 'change', () => { + if (folder.input.checked && !input.value) input.value = location()?.default_path || ''; + setProblem(null); + repaintButton(); + }); + } + scope.listen(input, 'input', () => { setProblem(null); repaintButton(); }); + scope.listen(review, 'click', async () => { + if (review.getAttribute('aria-disabled') === 'true') return; + const chosen = current(); + const body = { mode: 'move', location: chosen.mode === 'folder' ? { mode: 'folder', path: chosen.path.trim() } + : { mode: 'source' } }; + review.setAttribute('aria-disabled', 'true'); + try { + const plan = await api.post(PLAN_URL, body, { signal: scope.signal }); + if (!scope.signal.aborted) openMoveReview({ scope, store, plan, trigger: review, notify, navigate }); + } catch (error) { + if (scope.signal.aborted) return; + if (error?.status === 507 && error?.plan) { + openMoveReview({ scope, store, plan: error.plan, refusal: error.plan.insufficient?.text || error.message, + trigger: review }); + } else if (error?.status === 409 && error?.code === 'busy') { + notify(busyToast(error.running ?? null)); + } else if (typeof error?.code === 'string' && error.code.startsWith('location-') && error.message) { + setProblem(error.message); + input.focus(); + } else { + notify(typeof error?.message === 'string' && error?.code ? error.message : REVIEW_FAILED); + } + } finally { + repaintButton(); + } + }); + paint(); + return { + node: block, + paint, + focus() { + folder.input.checked = true; + repaintButton(); + input.focus(); + block.scrollIntoView?.({ block: 'center' }); + }, + }; +} + +/** Open the Recreate confirmation for `location`; resolves once it closed. */ +export function openRecreate({ api, scope, location, trigger = null, onDone, notify = showWorkspaceToast }) { + const grid = el('dl', undefined, 'meta-grid'); + for (const [term, value] of [['Folder', location.path], ['Made', 'An empty folder with a new .promptmanager-thumbnails.json'], + ['Thumbnails', 'Made again as you browse, or with Thumbnail maintenance']]) { + const entry = el('div'); + entry.append(el('dt', term), el('dd', value)); + grid.append(entry); + } + const why = el('p', '', 'small location-problem'); + const cancel = createButton('Cancel'); + const go = createButton('Recreate folder', 'ca-primary'); + const actions = el('div', undefined, 'dialog-actions'); + actions.append(cancel, go); + scope.listen(cancel, 'click', () => closeWorkspaceDialog()); + scope.listen(go, 'click', async () => { + try { + const result = await api.post(RECREATE_URL, {}, { signal: scope.signal }); + if (scope.signal.aborted) return; + closeWorkspaceDialog(); + onDone?.(result.location); + } catch (error) { + if (scope.signal.aborted) return; + why.textContent = error?.message || 'The folder could not be recreated.'; + if (!error?.message) notify(why.textContent); + } + }); + openContentDialog({ scope, title: 'Recreate the thumbnail folder?', trigger, nodes: [grid, + el('p', 'If the folder is on a drive that is not connected, connect it instead: recreating would put thumbnails on ' + + 'another disk.', 'small'), why, actions] }); +} + +/** + * The D199 notice for `location` (null while it is available): Recreate folder for a missing folder, Use this folder + * for another folder's marker, and Change location (`onChange`). + */ +export function locationNotice({ api, scope, location, onChange, onDone, notify = showWorkspaceToast }) { + if (!isFolder(location) || location.available !== false) return null; + const notice = el('div', undefined, 'notice'); + notice.id = 'unavailable-notice'; + const text = el('div'); + text.append(el('h2', NOTICE_TITLE), el('p', noticeText(location))); + const actions = el('div', undefined, 'row-actions'); + if (location.problem === 'missing') { + const recreate = createButton('Recreate folder'); + scope.listen(recreate, 'click', () => openRecreate({ api, scope, location, trigger: recreate, onDone, notify })); + actions.append(recreate); + } else if (location.problem === 'marker-mismatch') { + const adopt = createButton('Use this folder'); + scope.listen(adopt, 'click', async () => { + try { + const result = await api.post(ADOPT_URL, {}, { signal: scope.signal }); + if (!scope.signal.aborted) onDone?.(result.location); + } catch (error) { + if (!scope.signal.aborted) notify(error?.message || 'The folder could not be used.'); + } + }); + actions.append(adopt); + } + const change = createButton('Change location', 'ca-primary'); + scope.listen(change, 'click', () => onChange?.()); + actions.append(change); + notice.append(text, actions); + return notice; +} + +/** + * Where a section shows the D199 notice: a comment `anchor` (no element, so the concept comparisons see nothing while the + * folder is available) after which `paint()` keeps the notice for `location()`. + */ +export function noticeSlot({ api, scope, location, onChange, onDone, notify = showWorkspaceToast }) { + const anchor = document.createComment('thumbnail-location-notice'); + let shownFor; + let notice = null; + return { + anchor, + paint() { + const now = location(); + // Not mounted yet: the next paint (a microtask, or the host read) places it. + if (!anchor.parentNode) { + queueMicrotask(() => { if (anchor.parentNode && !scope.signal.aborted) this.paint(); }); + return; + } + if (now === shownFor) return; + shownFor = now; + notice?.remove(); + notice = locationNotice({ api, scope, location: now, onChange, onDone, notify }); + if (notice) anchor.after(notice); + }, + }; +} diff --git a/web/js/workspace/system.mjs b/web/js/workspace/system.mjs index d769f809..1a5e8129 100644 --- a/web/js/workspace/system.mjs +++ b/web/js/workspace/system.mjs @@ -86,7 +86,9 @@ export function systemDrafts(api) { api.get(LORA_STATUS_URL, { cache: 'no-store' }).catch(() => null)]); // The store reads a missing value as none and a missing revision as unreadable (no host save). #317: which // thumbnail formats the server can write rides along as a read-only value (never drafted or sent). - return { values: { ...result.settings, thumbnail_encoders: result.thumbnail_encoders, ...(lora ? loraValues(lora) : {}) }, + // #320: where previews live rides along the same way (read-only; only Move thumbnails changes it). + return { values: { ...result.settings, thumbnail_encoders: result.thumbnail_encoders, + thumbnail_location: result.thumbnail_location, ...(lora ? loraValues(lora) : {}) }, revision: result.revision }; }, async writeHost(changes, { expected_revision }) { diff --git a/web/js/workspace/thumbnail-review.mjs b/web/js/workspace/thumbnail-review.mjs index 177a261b..5a0d850b 100644 --- a/web/js/workspace/thumbnail-review.mjs +++ b/web/js/workspace/thumbnail-review.mjs @@ -23,6 +23,7 @@ import { busyToast } from './activity-scan.mjs'; import { showWorkspaceToast } from './toast.mjs'; import { navigateTo } from './state.mjs'; import { noun } from './plural.mjs'; +import { sizeText } from './system-thumbnail-location.mjs'; export const THUMBNAIL_PLAN_URL = '/prompt_manager/images/thumbnails/plan'; export const THUMBNAILS_KIND = 'thumbnails'; @@ -33,7 +34,9 @@ export const PLAN_FAILED = 'The previews could not be counted.'; export const START_FAILED = 'Thumbnail generation could not start. Try again.'; export const SETTINGS_UNKNOWN = 'Previews follow the thumbnail settings in System › Appearance.'; const FORMAT_LABELS = Object.freeze({ webp: 'WebP', jxl: 'JPEG XL', jpeg: 'JPEG' }); -const MODES = new Set(SCOPE_OPTIONS.map(([value]) => value)); +// #320 D200: offered only while previews are kept in one folder (31A). +export const PRUNE_OPTION = Object.freeze(['prune', 'Remove thumbnails of removed sources']); +const MODES = new Set([...SCOPE_OPTIONS.map(([value]) => value), PRUNE_OPTION[0]]); const count = value => value.toLocaleString('en-US'); const isCount = value => Number.isSafeInteger(value) && value >= 0; @@ -47,6 +50,7 @@ export function settingsText(plan) { /** D99: the plan line for the server's `{mode, targets, fresh, remaining}`; an answer without counts failed. */ export function planText(plan) { + if (plan?.mode === 'prune') return typeof plan.line === 'string' && plan.line ? plan.line : PLAN_FAILED; if (!plan || !MODES.has(plan.mode) || ![plan.targets, plan.fresh, plan.remaining].every(isCount)) { return PLAN_FAILED; } @@ -101,11 +105,18 @@ function select(options, value) { * Activity store; `trigger` gets focus back when the dialog closes. */ export function openThumbnailReview({ api, store, scope, trigger = null, mode: initial = 'missing', notify = showWorkspaceToast, - navigate = navigateTo }) { - const mode = select(SCOPE_OPTIONS, MODES.has(initial) ? initial : 'missing'); + navigate = navigateTo, location = null }) { + const folderMode = location?.mode === 'folder'; + const options = folderMode ? [...SCOPE_OPTIONS, PRUNE_OPTION] : SCOPE_OPTIONS; + const allowed = new Set(options.map(([value]) => value)); + const mode = select(options, allowed.has(initial) ? initial : 'missing'); const line = el('p', PLAN_LOADING, 'small'); line.setAttribute('role', 'status'); const settings = el('p', SETTINGS_UNKNOWN, 'small'); + // D200: what the prune removes, under the plan line. + const pruneDetail = el('p', '', 'small'); + const pruneGrid = el('dl', undefined, 'meta-grid'); + pruneDetail.hidden = pruneGrid.hidden = true; const cancel = createButton('Cancel'); const start = createButton('Start', 'ca-primary'); const actions = el('div', undefined, 'dialog-actions'); @@ -125,7 +136,21 @@ export function openThumbnailReview({ api, store, scope, trigger = null, mode: i // A later choice painted its own line meanwhile, or the dialog closed. if (!open || scope.signal.aborted || mode.value !== chosen) return; line.textContent = planText(plan); - settings.textContent = settingsText(plan); + const prune = chosen === 'prune'; + settings.hidden = prune; + pruneDetail.hidden = pruneGrid.hidden = !prune || !plan?.sources?.length; + if (prune) { + pruneDetail.textContent = `Only Prompt Manager’s thumbnails in their folders inside ${plan?.folder ?? 'the thumbnail ' + + 'folder'} are removed. If a source comes back, its thumbnails are made again as you browse.`; + pruneGrid.replaceChildren(...(plan?.sources ?? []).map(source => { + const entry = el('div'); + entry.append(el('dt', source.key), el('dd', `${count(source.files)} ${noun(source.files, 'thumbnail')} · ` + + `${sizeText(source.bytes)}`)); + return entry; + })); + } else { + settings.textContent = settingsText(plan); + } } scope.listen(mode, 'change', paint); @@ -136,7 +161,7 @@ export function openThumbnailReview({ api, store, scope, trigger = null, mode: i startThumbnails({ store, scope, options, notify, navigate }); }); openContentDialog({ scope, title: 'Thumbnail maintenance', trigger, - nodes: [field('Scope', mode, 'thumb-mode'), line, settings, actions], + nodes: [field('Scope', mode, 'thumb-mode'), line, settings, ...(folderMode ? [pruneDetail, pruneGrid] : []), actions], onClose: () => { open = false; } }); paint(); } diff --git a/web/js/workspace/workspace-dialogs.mjs b/web/js/workspace/workspace-dialogs.mjs index c26d82a0..fb3496f1 100644 --- a/web/js/workspace/workspace-dialogs.mjs +++ b/web/js/workspace/workspace-dialogs.mjs @@ -460,9 +460,13 @@ export function backupNotice({ copy, onBackup }) { return notice; } +/** A grid value: text, or (#320) `[lead, items]`, a lead sentence and the list under it. */ +const gridValue = value => typeof value === 'string' || (Array.isArray(value) && value.length === 2 + && typeof value[0] === 'string' && Array.isArray(value[1]) && value[1].every(item => typeof item === 'string')); + function requireGrid(grid) { if (!Array.isArray(grid) || grid.length === 0 || grid.some(row => !Array.isArray(row) || row.length !== 2 || - typeof row[0] !== 'string' || !row[0] || typeof row[1] !== 'string')) { + typeof row[0] !== 'string' || !row[0] || !gridValue(row[1]))) { throw new TypeError('The review needs its [term, value] grid'); } } @@ -495,7 +499,12 @@ export function openReviewDialog({ scope, title, heading = null, grid, options = values.clear(); list.replaceChildren(...rows.map(([term, value]) => { const entry = el('div'); - const dd = el('dd', value); + const dd = el('dd', typeof value === 'string' ? value : value[0]); + if (typeof value !== 'string' && value[1].length) { + const items = el('ul', undefined, 'review-list'); + items.append(...value[1].map(item => el('li', item))); + dd.append(items); + } values.set(term, dd); entry.append(el('dt', term), dd); return entry; diff --git a/web/js/workspace/workspace.css b/web/js/workspace/workspace.css index 05b569be..756702bf 100644 --- a/web/js/workspace/workspace.css +++ b/web/js/workspace/workspace.css @@ -576,3 +576,32 @@ pre.prompt-text { font: inherit; font-size: 12px; line-height: 1.75; margin-bloc .review-panel { position: sticky; inset-block-start: 20px; } @media (max-width: 700px) { .review-panel { position: static; } } @media (min-width: 701px) and (max-height: 720px) { .review-panel { max-block-size: calc(100dvh - 40px); overflow: auto; } } + +/* #320 D196: Thumbnail location, a setting wider than a .setting-row column. */ +.setting-block { padding: 17px 0; border-top: 1px solid var(--line); display: grid; gap: 12px; } +.setting-block-text .setting-block-label { font-size: 12px; font-weight: 500; display: block; } +.setting-block-text p { font-size: 11px; color: var(--ca-muted); margin-top: 4px; max-width: 640px; } +.location-choice { display: flex; flex-wrap: wrap; gap: 8px 24px; font-size: 12px; } +.location-choice label { display: inline-flex; gap: 6px; align-items: baseline; } +.location-choice .small { color: var(--ca-muted); } +.location-path { max-width: 640px; margin: 0; } +.location-path input { width: 100%; min-width: 0; font-family: var(--ca-mono, ui-monospace, monospace); font-size: 12px; } +.location-path input:disabled { color: var(--ca-muted); } +.location-path[aria-invalid="true"] input { border-color: var(--red); } +.location-problem { color: var(--red); margin-top: 6px; } +.location-problem:empty { display: none; } +.location-status { margin: 0; overflow-wrap: anywhere; } +.setting-block .row-actions { justify-content: flex-start; } + +/* #320 D197: the move review's lists and confirmation. */ +.review-list { margin: 4px 0 0; padding-inline-start: 18px; font-size: 11px; color: var(--ca-muted); } +.review-list li { overflow-wrap: anywhere; } +.review-confirm { display: flex; gap: 8px; align-items: flex-start; font-size: 12px; margin: 12px 0 0; + padding: 10px 12px; border: 1px solid var(--line); border-radius: 6px; background: var(--ca-field); } +.review-confirm input { margin-top: 2px; flex-shrink: 0; } +.review-refusal { color: var(--red); } +.meta-grid dd { overflow-wrap: anywhere; } + +@media (max-width: 700px) { + .location-choice { flex-direction: column; } +}