diff --git a/TESTING.md b/TESTING.md index 728afb1..e8a9f96 100644 --- a/TESTING.md +++ b/TESTING.md @@ -803,8 +803,10 @@ sorted alphabetically). Removed toolbar and filter-row items flow again from the concept's own boxes (`reflow`), checked against the concept page with those items removed at runtime. Masonry tops and heights are compared unrounded, within one layout unit per card above in the column (D3). `one footer height for the collection (D3)` -seeds two seedless files (`seedless_images` on `/__test__/concept/reset`) and asserts -that every row reserves the tallest footer. +seeds two seedless files (`seedless_images` on `/__test__/concept/reset`, with +`short_seedless` so their "96 × 72 · PNG" subtitle stays on one line beside Info and +Tag) and asserts that every row reserves the tallest footer and that their own footer +is shorter than the row it reserves. ### Images page controls and recorded deviations (#249 PR C) @@ -1089,9 +1091,15 @@ Inspect plan keeps D60–D66 (plan D60 → D83, D61 → D84, D65 → D88, D68 checkbox rows: `span.chip.tag-suggestion` in a `.tags` row, in the concept's checked-row tokens (accent border, `--soft-blue`), each with the name, the confidence (`98%` for WD14, "Suggested" for language models) and a × "Remove - ". At most the requested count, all preselected. The help copy reads - "Remove anything you do not want to add." and removing the last chip shows "All - suggestions removed. Generate again or enter tags manually." + ". A chip has the concept `.chip`'s 4px corner (the later `.tag-suggestion` + row rule would give 5px) around the row's own 12px text. At most the requested + count, all preselected. The help copy reads "Remove anything you do not want to + add." and removing the last chip shows "All suggestions removed. Generate again or + enter tags manually." The concept's markup carries that initial sentence ("Uncheck + anything…"), but its `renderTagModelOptions` (a1062–1076) replaces it with "Generate + suggestions with the selected model before approving." as soon as the dialog opens, + so the concept never shows it; production shows ruling T4's sentence on open and + the model-changed sentence after a model change. - **D84** (plan D61, rulings T1/T2): the Tagging model options are legacy's four engines in the mock's order with legacy labels and sizes (`WD14 SwinV2 · ~400 MB`, `WD14 ViT · ~400 MB`, `GGUF JoyCaption (Recommended) · ~4.5 GB` in the @@ -1117,6 +1125,62 @@ Inspect plan keeps D60–D66 (plan D60 → D83, D61 → D84, D65 → D88, D68 its "Auto-tag images" always opens the dialog. Production opens it for the image selected in Images when it was last left; with none it is `aria-disabled` and pressing it toasts "Select an image in Images first." (no new chrome). +- **D29** (no prototype copy) in the dialog: no "Prototype · suggestions are + simulated…" note, the status reads "Suggestions from . Approve tags to save." + (not "Sample suggestions from …"), and the toast has no " in this tab". + +After a confirmed save the selected file's inspection is re-read quietly: the panel +and the viewer keep the current details until the new answer lands, with no "Loading +file details…" in between (`createInspectionController().refresh()`). While a Tags +filter or a search is active the saved file may start or stop matching, so Images +re-reads its loaded range in place (`createImageStream().refresh()`: the same query +from the first page until the loaded count, published once). The cards, the scroll +position and the selection stay while the selected file still matches; when it no +longer does, the panel follows the collection (its first file), as after a reload. +A refresh never runs beside a page load already in flight. Browser tests: "the +post-save re-read keeps the panel content until the new read lands", "a save under a +Tags filter re-reads the listing: …" and "a save under a search keeps the scroll +position and the loaded range". + +Concept comparison (`tests/browser/tagging-concept.spec.mjs` with +`tagging-canonical-comparison.mjs`, plan Task C4): production runs on the concept +fixture with the concept image's own tags (`cinematic, blue_hour, architecture`) +saved as its image tags, and the fake model answers the concept's own sample +(`/__test__/tagging/suggestions`: mountain 0.98, observatory 0.95, snow 0.88, +blue_hour 0.81), so the dialog's inventory (order, text, field attributes, tokens, +preview box, chips, status, sticky actions) must equal the concept's at 1600×1100 and +390×844, dark and light: opened, generated with WD14 and generated with a language +model, plus the empty result (the concept above its own threshold), the review-scoped +dialog with its scope note and the card's Info and Tag group (M1). `mapConceptTagging` +applies only D29, D83, D84 and D91; a mutation check (the model panel's border +removed, the primary relabelled) shows the comparison notices a change. Every +production-versus-concept difference of PR C and its record: + +| Difference | Record | +| --- | --- | +| Removable chips with × instead of checked checkbox rows; "Remove anything…"; the all-removed copy; the initial sentence on open | D83 (ruling T4) | +| Legacy engine labels and values, GGUF in the Ollama slot, server default instructions | D84 (rulings T1/T2) | +| Suggestions count field | D91 (ruling T3-bulk) | +| No prototype note, "Suggestions from …", no " in this tab" | D29 | +| Image tags first in the inspector and viewer chip row | D88 (ruling T7/C2) | +| Auto-tag images `aria-disabled` without a selected image | D115 | +| Retry leads the actions when the controller allows it | ruling T5 (state text only otherwise) | +| States the concept lacks (busy, conflict, changed storage or file, unconfirmed save, unavailable model, video) | Global Constraints: `#tag-generation-status` text only | + +Native image tagging (#249 PR B routes, PR C UI): `POST +/prompt_manager/image-annotations/observe` (capture one listed file, its saved set and +`tag_revision`), `GET`/`DELETE /prompt_manager/image-annotations/capture/` +(the preview and its release), `POST /prompt_manager/image-annotations/suggest` (the +gated top-N suggestions, at most the requested count) and `PUT +/prompt_manager/image-annotations/tags` (re-hash, then save one byte record's set). +Cheap reads follow spec amendment S1 (the verified `(size, mtime_ns)` fingerprint; +residual risk above, under "Image annotations schema"). No automated test downloads +or loads a model: the browser server injects `TestModel`, whose `download_model` +raises, and every tagging spec asserts no `/autotag/download/` request. Capture +limits are those of the capture store above (256 MiB per file, 8 completed captures, +512 MiB reserved, 900 s idle, 1800 s hard). The browser fixtures release every capture +on reset. PR C is HTML, CSS and JS only (no restart); the PR B routes it calls need a +maintainer ComfyUI restart once. Test files: `tests/browser/workspace-tagging.spec.mjs` (the fixture is `tests/browser/workspace_tagging_fixture.py`: two files sharing one prompt and @@ -1127,11 +1191,13 @@ no model files; every test asserts no download request), `tests/js/workspace-images-lifecycle.test.js`, and for D88 `tests/js/workspace-file-inspector.test.js`, `tests/js/workspace-inspection-state.test.js` and `tests/browser/inspector-concept.spec.mjs` -("image tags saved on the concept image keep the inspector equal to the concept"). +("image tags saved on the concept image keep the inspector equal to the concept"), and +for the comparison `tests/browser/tagging-concept.spec.mjs` and +`tests/js/workspace-images-state.test.js` (the in-place refresh). ```sh node --test tests/js/workspace-image-tagging*.test.js tests/js/workspace-images-lifecycle.test.js -npm run --prefix tools/vendor test:workflows -- workspace-tagging.spec.mjs \ +npm run --prefix tools/vendor test:workflows -- workspace-tagging.spec.mjs tagging-concept.spec.mjs \ workspace-vocabulary.spec.mjs images-concept.spec.mjs --project chromium --project touch ``` diff --git a/tests/browser/images-concept.spec.mjs b/tests/browser/images-concept.spec.mjs index 7bf5365..e7e219c 100644 --- a/tests/browser/images-concept.spec.mjs +++ b/tests/browser/images-concept.spec.mjs @@ -533,8 +533,9 @@ test('one image appended by scrolling reads in the singular',async({page,request }); // D3 (one footer height for the collection): bounded rendering cannot measure unrendered cards, so every row reserves -// the tallest measured footer. Two seedless files (D11 "W × H · PNG", one line) form the first row at 390px, where -// every concept subtitle wraps: their row is as tall as the wrapped rows, and every concept row moves down by one. +// the tallest measured footer. Two seedless files (D11 "W × H · PNG", one line: a short "96 × 72" beside Info and Tag) +// form the first row at 390px, where every concept subtitle wraps: their row is as tall as the wrapped rows, and every +// concept row moves down by one. test('one footer height for the collection (D3)',async({page,request})=>{ // A second fixture seed (dozens of PNGs and their thumbnails) runs inside the test. test.setTimeout(120_000); @@ -543,7 +544,7 @@ test('one footer height for the collection (D3)',async({page,request})=>{ const reference=await inventoryOf(page,'#main'); expect(reference.grid.columns).toBe(2); expect(reference.cards.every(card=>card.info.p.height>parseFloat(card.info.p.lineHeight))).toBe(true); - const manifest=await seedConcept(request,{seedless_images:2}); + const manifest=await seedConcept(request,{seedless_images:2,short_seedless:true}); await openProductionLoaded(page,'dark','no-preference',FIXTURE_IMAGES+2); const actual=await inventoryOf(page,'#workspace-main'); const box=card=>({left:card.left,top:card.top,width:card.width,height:card.height}); @@ -552,8 +553,10 @@ test('one footer height for the collection (D3)',async({page,request})=>{ ...reference.cards.slice(0,4).map(card=>({...box(card),top:card.top+pitch}))]); const seedless=actual.cards.slice(0,2); expect(seedless.map(card=>card.open.label)).toEqual(manifest.seedless.map(image=>`Open ${image.filename}`)); - expect(seedless.map(card=>card.info.p.text)).toEqual(['1536 × 1152 · PNG','1536 × 1152 · PNG']); - // Their own footer fits the row it reserves. With Info and Tag (#249 C2, the concept's own pair) their subtitle - // wraps at 390px as the concept's do, so it is no longer one line shorter; the boxes above pin the reservation. - for(const card of seedless)expect(card.info.height).toBeLessThanOrEqual(reference.cards[0].info.height); + expect(seedless.map(card=>card.info.p.text)).toEqual(['96 × 72 · PNG','96 × 72 · PNG']); + // Their own footer is one line shorter than the row it reserves. + for(const card of seedless) { + expect(card.info.p.height).toBeLessThanOrEqual(parseFloat(card.info.p.lineHeight)+0.5); + expect(card.info.height).toBeLessThan(reference.cards[0].info.height); + } }); diff --git a/tests/browser/server.py b/tests/browser/server.py index 5702b22..4ff976d 100644 --- a/tests/browser/server.py +++ b/tests/browser/server.py @@ -60,6 +60,9 @@ class TestModel: loaded = None # Models the Tag image dialog workflows mark as not downloaded. unavailable: set = set() + # Scored names the concept comparisons answer instead of the fixed list + # (``/__test__/tagging/suggestions``); None keeps the fixed list. + suggestions: list | None = None def get_models_status(self) -> dict: return { @@ -91,6 +94,11 @@ class TestModel: ) -> list[tuple[str, float | None]]: """Fixed scored suggestions (WD14) or unscored ones (language models).""" scored = self.loaded is not None and self.loaded.startswith("wd14") + if self.suggestions is not None: + return [ + (name, confidence if scored else None) + for name, confidence in self.suggestions[:limit] + ] names = ["sandstone", "desert", "arch", "sky", "rock", "dune", "canyon"] names += [f"tag_{index:02d}" for index in range(100)] return [ @@ -478,7 +486,8 @@ def create_app(root: Path) -> web.Application: if ( request.query or not isinstance(data, dict) - or set(data) - set(limits) + or set(data) - set(limits) - {"short_seedless"} + or not isinstance(data.get("short_seedless", False), bool) or any( isinstance(data.get(key, 0), bool) or not isinstance(data.get(key, 0), int) @@ -497,6 +506,7 @@ def create_app(root: Path) -> web.Application: root, extra_images=data.get("extra_images", 0), seedless_images=data.get("seedless_images", 0), + short_seedless=data.get("short_seedless", False), ) ) return web.json_response({"success": True, **manifest}) diff --git a/tests/browser/tagging-canonical-comparison.mjs b/tests/browser/tagging-canonical-comparison.mjs new file mode 100644 index 0000000..9edd3b7 --- /dev/null +++ b/tests/browser/tagging-canonical-comparison.mjs @@ -0,0 +1,143 @@ +// Concept-vs-production inventory of the Tag image dialog (plan Mock inventory M2: docs/concepts/workspace/app.js +// tagReview a1037–1061, renderTagModelOptions a1062–1076, generateTags a1077–1135; style.css s1598–1694) and of the +// card's Info and Tag group (M1: app.js mediaCard a398, style.css s1589–1597). The *Of functions run in the page. +// The concept is never rewritten: mapConceptTagging applies only the recorded deviations, each named with its D-number +// or ruling; everything else must be equal. +export {diffInventory as diffTaggingInventory} from './library-editor-canonical-comparison.mjs'; + +/** The open Tag image dialog (`dialog#modal`), its controls in order with their concept tokens. */ +export function taggingInventory(dialog) { + const css=(node,...keys)=>{const style=getComputedStyle(node);return Object.fromEntries(keys.map(key=>[key,style[key]]));}; + const px=value=>Math.round(value*2)/2; + const box=node=>{const rect=node.getBoundingClientRect();return {width:px(rect.width),height:px(rect.height)};}; + const font=node=>css(node,'fontSize','fontWeight','lineHeight','color','marginTop','marginBottom'); + const edges=node=>css(node,'paddingTop','paddingRight','paddingBottom','paddingLeft','borderTopWidth','borderTopStyle','borderTopColor', + 'borderTopLeftRadius','backgroundColor'); + const buttonOf=node=>({text:node.textContent.trim(),label:node.getAttribute('aria-label'),primary:node.classList.contains('ca-primary'), + icon:node.querySelector('svg')!==null,disabled:node.getAttribute('aria-disabled')==='true'||node.disabled, + ...css(node,'minHeight','fontSize','color','backgroundColor','borderTopColor','borderTopLeftRadius')}); + const controlOf=node=>({tag:node.tagName.toLowerCase(),id:node.id,type:node.type??null,value:node.value, + min:node.getAttribute('min'),max:node.getAttribute('max'),step:node.getAttribute('step'), + describedBy:node.getAttribute('aria-describedby'), + options:node.tagName==='SELECT'?[...node.options].map(option=>[option.value,option.textContent]):null, + ...css(node,'fontSize','minHeight','backgroundColor','color','borderTopColor','borderTopLeftRadius','paddingLeft')}); + const fieldOf=field=>{const label=field.querySelector('label'),control=field.querySelector('input,textarea,select'); + return {label:label.textContent,labelled:label.htmlFor===control.id,...css(field,'display','rowGap','marginTop','marginBottom'),control:controlOf(control)};}; + const header=dialog.querySelector('.dialog-header'),title=header.querySelector('h2'),close=header.querySelector('button'); + const body=dialog.querySelector('#modal-body'); + const heading=body.querySelector('.tag-image-heading'); + const preview=heading.querySelector('.tag-image-preview'),copy=heading.querySelector(':scope > div'); + const sizeChip=copy.querySelector('.chip'); + const note=body.querySelector('.tag-scope-note'); + const manual=body.querySelector('#manual-image-tags'),manualField=manual.closest('.field'),help=body.querySelector('#manual-tags-help'); + const panel=body.querySelector('.tag-model-panel'),options=panel.querySelector('#image-tag-model-options'); + const generate=[...panel.querySelectorAll('button')].at(-1); + const suggestions=body.querySelector('section.tag-suggestion-panel'); + const list=suggestions.querySelector('#tag-suggestions'),status=suggestions.querySelector('#tag-generation-status'); + const count=suggestions.querySelector('#suggestion-count'); + const actions=body.querySelector('.dialog-actions'); + // A suggestion is a concept checkbox row (`label.tag-suggestion`) or a production chip (`span.chip.tag-suggestion`). + const rows=[...list.querySelectorAll('.tag-suggestion')].map(row=>{ + const name=row.querySelector('span:not(.tag-confidence)'),confidence=row.querySelector('.tag-confidence'); + const checkbox=row.querySelector('input[type=checkbox]'),remove=row.querySelector('button'); + return {name:name.textContent,confidence:confidence.textContent, + control:checkbox?{kind:'checkbox',checked:checkbox.checked}:remove?{kind:'remove',label:remove.getAttribute('aria-label')}:null, + ...css(row,'display','alignItems','borderTopColor','backgroundColor','borderTopWidth','borderTopStyle','borderTopLeftRadius','fontSize','paddingTop','paddingLeft'), + confidenceToken:css(confidence,'color','fontSize')}; + }); + return { + dialog:{...css(dialog,'paddingTop','paddingRight','paddingBottom','paddingLeft','borderTopLeftRadius','backgroundColor','color','overflowY'), + width:px(dialog.getBoundingClientRect().width),modal:dialog.matches(':modal'),tagging:dialog.classList.contains('tagging-modal'), + labelledBy:document.getElementById(dialog.getAttribute('aria-labelledby'))===title, + noHorizontalOverflow:dialog.scrollWidth<=dialog.clientWidth+1}, + header:{title:{text:title.textContent,...font(title)},close:{label:close.getAttribute('aria-label'),text:close.textContent.trim()}}, + demo:body.querySelector('.demo-note')?.textContent??null, + order:[...body.children].map(node=>node.className), + heading:{...css(heading,'display','columnGap','alignItems','marginBottom'), + preview:{tag:preview.tagName.toLowerCase(),alt:preview.getAttribute('alt'),...box(preview),...css(preview,'objectFit','borderTopLeftRadius','borderTopWidth','borderTopStyle','borderTopColor','flexShrink')}, + h3:{text:copy.querySelector('h3').textContent,...font(copy.querySelector('h3'))}, + filename:{text:copy.querySelector('p.small').textContent,...font(copy.querySelector('p.small'))}, + size:sizeChip?{text:sizeChip.textContent,...css(sizeChip,'marginTop','fontSize','borderTopColor','borderTopLeftRadius','paddingLeft')}:null}, + scopeNote:note?{text:note.textContent,...edges(note),...css(note,'borderLeftWidth','borderLeftStyle','borderLeftColor','fontSize','color')}:null, + manual:{...fieldOf(manualField),help:{text:help.textContent,...font(help)},described:help.id===manual.getAttribute('aria-describedby')}, + modelPanel:{...edges(panel),...css(panel,'marginTop'),h3:{text:panel.querySelector('h3').textContent,...font(panel.querySelector('h3'))}, + fields:[...panel.querySelectorAll('.field')].map(fieldOf), + twoCol:options.querySelector('.two-col')?{columns:getComputedStyle(options.querySelector('.two-col')).gridTemplateColumns.split(' ').length, + ...css(options.querySelector('.two-col'),'display','columnGap')}:null, + generate:buttonOf(generate)}, + suggestions:{labelled:suggestions.getAttribute('aria-labelledby')===suggestions.querySelector('h3').id,...css(suggestions,'marginTop'), + title:{text:suggestions.querySelector('h3').textContent,...font(suggestions.querySelector('h3'))}, + count:{text:count.textContent,...font(count)}, + copy:list.querySelector(':scope > p.small')?.textContent??null, + rows, + status:{text:status.textContent,role:status.getAttribute('role'),...css(status,'marginTop','fontSize','color')}}, + actions:{...css(actions,'position','bottom','marginLeft','marginRight','marginBottom','paddingTop','paddingLeft','backgroundColor', + 'borderTopWidth','borderTopStyle','borderTopColor','zIndex','display','justifyContent','columnGap'), + buttons:[...actions.querySelectorAll('button')].map(buttonOf)}, + noPageScroll:document.documentElement.scrollWidth<=innerWidth+1, + }; +} + +/** The first image card's Info and Tag group (M1), with its card's filename. */ +export function cardActionsInventory(card) { + const css=(node,...keys)=>{const style=getComputedStyle(node);return Object.fromEntries(keys.map(key=>[key,style[key]]));}; + const info=card.querySelector('.media-info'),actions=info.querySelector(':scope > .image-card-actions'); + return { + info:css(info,'display','justifyContent','alignItems','columnGap'), + actions:{...css(actions,'display','alignItems','columnGap','flexShrink','minWidth'),lastInInfo:info.lastElementChild===actions}, + buttons:[...actions.querySelectorAll(':scope > button')].map(node=>{const rect=node.getBoundingClientRect();return { + className:node.className,label:node.getAttribute('aria-label'),icon:node.querySelector('svg')?.innerHTML.replace(/\s+/g,' ').trim()??null, + width:Math.round(rect.width),height:Math.round(rect.height), + ...css(node,'minWidth','minHeight','paddingLeft','paddingTop','borderTopLeftRadius','borderTopColor','backgroundColor','color')};}), + }; +} + +// D84 (plan D61): the legacy engines' values, labels and sizes, GGUF JoyCaption in the concept's Ollama slot. +const MODEL_LABELS=Object.freeze({'wd14-swinv2':['wd14-swinv2','WD14 SwinV2 · ~400 MB'],'wd14-vit':['wd14-vit','WD14 ViT · ~400 MB'], + ollama:['gguf','GGUF JoyCaption (Recommended) · ~4.5 GB'],huggingface:['hf','HuggingFace JoyCaption · ~16 GB']}); +const CONCEPT_LABELS=Object.freeze({'WD14 SwinV2 · fast classifier':'WD14 SwinV2 · ~400 MB','WD14 ViT · visual tags':'WD14 ViT · ~400 MB', + 'Ollama · local language model':'GGUF JoyCaption (Recommended) · ~4.5 GB','HuggingFace · language model':'HuggingFace JoyCaption · ~16 GB'}); + +/** + * The concept inventory with the recorded deviations applied: + * - no prototype note (Global Constraints: `Prototype · suggestions are simulated…` is not shipped); + * - D83 (ruling T4): "Remove anything you do not want to add." and every suggestion a chip with a `Remove ` × + * instead of a checked checkbox row: the concept's own `.chip` corner (`chip`: a concept `.chip` inventory, the + * heading's size chip) around the row's own text (12px) in the checked row's colours (accent border, `--soft-blue`); + * - D84 (plan D61): the legacy engine labels and values, and the server's default instructions (`instructions`); + * - D91 (plan D68): a Suggestions count field after the model's own fields, on the concept's number field tokens + * (`numberField`: the concept's General confidence field) with the configured count (`limit`); + * - D29: the prototype's "Sample suggestions from …" status reads "Suggestions from …"; + * - D83, `opened`: on open the concept's renderTagModelOptions (a1062–1076) replaces its own initial sentence with the + * model-changed one at once; production shows the initial sentence of ruling T4 until a model change. + */ +export function mapConceptTagging(reference,{instructions,limit=10,chip,numberField,opened=false}) { + const [select,...own]=reference.modelPanel.fields; + const suggestionsField={...numberField,label:'Suggestions',control:{...numberField.control,id:'image-suggestion-limit', + value:String(limit),min:'1',max:'100',step:'1'}}; + const mappedOwn=own.map(field=>field.control.id==='image-tag-instructions'?{...field,control:{...field.control,value:instructions}}:field); + const rows=reference.suggestions.rows.map(row=>({...row, + control:row.control?.kind==='checkbox'&&row.control.checked?{kind:'remove',label:`Remove ${row.name}`}:row.control, + borderTopLeftRadius:chip.borderTopLeftRadius})); + const initial='Choose a model, then generate tags. Remove anything you do not want to add.'; + const copy=opened&&reference.suggestions.copy==='Generate suggestions with the selected model before approving.'?initial: + reference.suggestions.copy?.replace('Uncheck anything you do not want to add.','Remove anything you do not want to add.')??null; + const status=reference.suggestions.status.text.replace(/^Sample suggestions from (.+)\. Approve tags to save\.$/, + (_,label)=>`Suggestions from ${CONCEPT_LABELS[label]??label}. Approve tags to save.`); + return {...reference, + demo:null, + order:reference.order.filter(name=>name!=='demo-note'), + modelPanel:{...reference.modelPanel, + fields:[{...select,control:{...select.control,value:MODEL_LABELS[select.control.value]?.[0]??select.control.value, + options:select.control.options.map(([value])=>MODEL_LABELS[value])}},...mappedOwn,suggestionsField]}, + suggestions:{...reference.suggestions,copy,rows,status:{...reference.suggestions.status,text:status}}, + }; +} + +/** + * Production's own chip geometry beyond the concept's `.chip` (D83): the chip's padding hugs its × button. Only the + * leaves D83 replaces are dropped from both sides; names, confidences, colours, radius and font stay compared. + */ +export function withoutChipGeometry(inventory) { + return {...inventory,suggestions:{...inventory.suggestions,rows:inventory.suggestions.rows.map(({paddingTop,paddingLeft,alignItems,...row})=>row)}}; +} diff --git a/tests/browser/tagging-concept.spec.mjs b/tests/browser/tagging-concept.spec.mjs new file mode 100644 index 0000000..cea982b --- /dev/null +++ b/tests/browser/tagging-concept.spec.mjs @@ -0,0 +1,296 @@ +// The Tag image dialog and the card's Info and Tag group against the approved concept (#249 PR C, plan Task C4, Mock +// inventory M1 and M2). Production runs on the concept-shaped fixture with the concept image's own tags saved as its +// image tags and the injected fake model answering the concept's own sample (alpine: mountain 98%, observatory 95%, +// snow 88%, blue_hour 81%), so every remaining difference is a recorded deviation applied by mapConceptTagging +// (D29, D83, D84, D91) or a test failure. No model is ever downloaded: every test checks no download request was sent. +import { test, expect } from './fixtures.mjs'; +import { seedConcept, openConcept, openProduction, saveImageTags, mutate, SIZES, THEMES } from './concept-inventory.mjs'; +import { taggingInventory, cardActionsInventory, mapConceptTagging, diffTaggingInventory, withoutChipGeometry } from './tagging-canonical-comparison.mjs'; + +const MODULE = '/prompt_manager/js/workspace/image-tagging.mjs'; +// The concept's first image (app.js images[0]) and its sample suggestions (generateTags, scene "alpine"). +const CONCEPT_FILE = 'alpine_00001.png'; +const CONCEPT_TAGS = ['cinematic', 'blue_hour', 'architecture']; +const ALPINE = [['mountain', 0.98], ['observatory', 0.95], ['snow', 0.88], ['blue_hour', 0.81]]; +// The fake model's server default instructions (tests/browser/server.py TestModel.default_prompt), D84. +const INSTRUCTIONS = 'Describe the image'; +const CONCEPT_MODEL = { gguf: 'ollama' }; + +const tagDialog = page => page.getByRole('dialog', { name: 'Tag image', exact: true }); +const firstCard = (page, side) => page.locator(side === 'concept' ? '#image-grid > article.media-card' : '#workspace-main article[data-image-key]').first(); + +test.beforeEach(async ({ page, request }) => { + const manifest = await seedConcept(request); + expect(manifest).toBeTruthy(); + await expect(await request.post('/__test__/tagging/suggestions', { data: { scored: ALPINE } })).toBeOK(); + page.downloads = []; + page.on('request', item => { if (/\/autotag\/download\//.test(item.url())) page.downloads.push(item.url()); }); + page.errors = []; + page.on('pageerror', error => page.errors.push(error.message)); +}); +test.afterEach(async ({ page, request }) => { + await leave(page); + expect(page.downloads).toEqual([]); + expect(page.errors).toEqual([]); + await request.post('/__test__/tagging/suggestions', { data: { scored: null } }); +}); + +async function conceptPath(request) { + const listing = await (await request.get('/prompt_manager/images/output?mode=workspace&limit=40')).json(); + const image = listing.images.find(item => item.filename === CONCEPT_FILE); + expect(image).toBeTruthy(); + return image.path; +} + +/** + * Close an open Tag dialog before leaving the page: leaving with it open releases its capture during unload, which + * WebKit reports as a failed fetch. + */ +async function leave(page) { + if (await tagDialog(page).count()) { + await page.keyboard.press('Escape'); + await expect(tagDialog(page)).toHaveCount(0); + } +} +/** Open `side`'s Tag image dialog from its first card, settled (production: the capture preview and saved tags). */ +async function openTagDialog(page, side, theme) { + await leave(page); + if (side === 'concept') await openConcept(page, 'images', theme); + else await openProduction(page, 'images', theme); + await firstCard(page, side).getByRole('button', { name: `Tag or auto-tag ${CONCEPT_FILE}`, exact: true }).click(); + await expect(tagDialog(page)).toBeVisible(); + await settled(page, side); +} +async function settled(page, side) { + if (side === 'production') { + await expect(tagDialog(page).locator('img.tag-image-preview')).toHaveAttribute('src', /\/image-annotations\/capture\//); + await expect(tagDialog(page).locator('#manual-image-tags')).toHaveValue(CONCEPT_TAGS.join(', ')); + } + await page.evaluate(async () => { + await document.fonts.ready; + await Promise.all([...document.querySelectorAll('#modal img')].map(image => image.decode().catch(() => {}))); + }); +} +async function generateWith(page, side, model) { + const select = tagDialog(page).getByRole('combobox', { name: 'Tagging model', exact: true }); + if (model) await select.selectOption(side === 'concept' ? CONCEPT_MODEL[model] ?? model : model); + await tagDialog(page).getByRole('button', { name: 'Generate suggestions', exact: true }).click(); + await expect(tagDialog(page).locator('#suggestion-count')).not.toHaveText('Not generated'); +} +// The pointer rests outside the dialog, so no control is measured in its hover state. +const inventory = async page => { + await page.mouse.move(0, 0); + return page.locator('dialog#modal').evaluate(taggingInventory); +}; + +/** The concept and production inventories of one dialog state, the concept mapped (D29, D83, D84, D91). */ +async function compareState(page, { theme, act = async () => {}, opened = false }) { + await openTagDialog(page, 'concept', theme); + const conceptOpened = await inventory(page); + await act(page, 'concept'); + const concept = await inventory(page); + await openTagDialog(page, 'production', theme); + await act(page, 'production'); + const actual = await inventory(page); + const numberField = conceptOpened.modelPanel.fields.find(field => field.control.id === 'image-general-threshold'); + const reference = mapConceptTagging(concept, { instructions: INSTRUCTIONS, chip: concept.heading.size, numberField, opened }); + return { reference: withoutChipGeometry(reference), actual: withoutChipGeometry(actual) }; +} + +for (const [width, height] of SIZES) for (const theme of THEMES) { + test(`concept and production Tag dialog match at ${width}x${height} ${theme}`, async ({ page, request }) => { + test.setTimeout(90_000); + await saveImageTags(request, await conceptPath(request), CONCEPT_TAGS); + await page.setViewportSize({ width, height }); + // Opened, generated with WD14 (the concept's default), then generated with a language model. + for (const [state, act, opened] of [ + ['opened', async () => {}, true], + ['generated WD14', (page, side) => generateWith(page, side), false], + ['generated GGUF', (page, side) => generateWith(page, side, 'gguf'), false], + ]) { + const { reference, actual } = await compareState(page, { theme, act, opened }); + expect(diffTaggingInventory(reference, actual), `${state} at ${width}x${height} ${theme}`).toEqual([]); + } + // Pinned concept geometry (M2), so a mapping mistake on both sides cannot pass unseen. + const { actual } = await compareState(page, { theme, act: (page, side) => generateWith(page, side) }); + const small = width <= 700; + expect([actual.heading.preview.width, actual.heading.preview.height]).toEqual(small ? [86, 86] : [112, 100]); + expect(actual.heading.columnGap).toBe('16px'); + expect([actual.modelPanel.paddingTop, actual.modelPanel.borderTopLeftRadius]).toEqual(['16px', '6px']); + expect(actual.actions.position).toBe('sticky'); + expect(actual.suggestions.rows.map(row => [row.name, row.confidence, row.control.label])).toEqual( + ALPINE.map(([name, confidence]) => [name, `${Math.round(confidence * 100)}%`, `Remove ${name}`])); + const accent = await page.evaluate(() => { + const probe = document.createElement('i'); + probe.style.cssText = 'border-top:1px solid var(--ca-accent);background:var(--soft-blue)'; + document.body.append(probe); + const style = getComputedStyle(probe), tokens = [style.borderTopColor, style.backgroundColor]; + probe.remove(); + return tokens; + }); + for (const row of actual.suggestions.rows) expect([row.borderTopColor, row.backgroundColor]).toEqual(accent); + expect(actual.dialog.noHorizontalOverflow).toBe(true); + expect(actual.noPageScroll).toBe(true); + }); +} + +test('an empty result matches the concept below its threshold', async ({ page, request }) => { + await saveImageTags(request, await conceptPath(request), CONCEPT_TAGS); + await expect(await request.post('/__test__/tagging/suggestions', { data: { scored: [] } })).toBeOK(); + const { reference, actual } = await compareState(page, { theme: 'dark', act: async (page, side) => { + // The concept filters its sample by the General confidence; above every sample nothing is shown. + if (side === 'concept') await tagDialog(page).getByRole('spinbutton', { name: 'General confidence', exact: true }).fill('1'); + await generateWith(page, side); + if (side === 'concept') await tagDialog(page).getByRole('spinbutton', { name: 'General confidence', exact: true }).fill('0.35'); + } }); + expect(actual.suggestions.copy).toBe('No suggestions met this threshold. Adjust it or enter tags manually.'); + expect(diffTaggingInventory(reference, actual)).toEqual([]); +}); + +test('review-scoped dialog shows the scope note', async ({ page, request }) => { + const path = await conceptPath(request); + await saveImageTags(request, path, CONCEPT_TAGS); + await page.setViewportSize({ width: 1600, height: 1100 }); + await openConcept(page, 'review', 'dark'); + await page.locator('#main section.surface').getByRole('button', { name: 'Tag image', exact: true }).click(); + await expect(tagDialog(page)).toBeVisible(); + await settled(page, 'concept'); + const opened = await inventory(page); + // Needs review ships with PR F; until then the review source opens the same dialog from the module. + await leave(page); + await openProduction(page, 'images', 'dark'); + await page.evaluate(async ({ module, path }) => { + const { openImageTagging } = await import(module); + const { createScope } = await import('/prompt_manager/js/shared/toolkit/core.js'); + const api = window.ApiClient.createApiClient(); + const listing = await api.get('/prompt_manager/images/output?mode=workspace&limit=40'); + openImageTagging({ api, scope: createScope(), image: listing.images.find(item => item.path === path), source: 'review' }); + }, { module: MODULE, path }); + await expect(tagDialog(page)).toBeVisible(); + await settled(page, 'production'); + const actual = await inventory(page); + expect(actual.scopeNote.text).toBe('Tags belong to this image. Resolve its prompt association separately in Needs review.'); + expect(actual.order).toEqual(['tag-image-heading', 'tag-scope-note small', 'field', 'tag-model-panel', 'tag-suggestion-panel', 'dialog-actions']); + const numberField = opened.modelPanel.fields.find(field => field.control.id === 'image-general-threshold'); + const reference = mapConceptTagging(opened, { instructions: INSTRUCTIONS, chip: opened.heading.size, numberField, opened: true }); + expect(diffTaggingInventory(reference, actual)).toEqual([]); +}); + +for (const [width, height] of SIZES) test(`card action group matches M1 at ${width}x${height}`, async ({ page }) => { + await page.setViewportSize({ width, height }); + await openConcept(page, 'images', 'dark'); + const reference = await firstCard(page, 'concept').evaluate(cardActionsInventory); + await openProduction(page, 'images', 'dark'); + const actual = await firstCard(page, 'production').evaluate(cardActionsInventory); + expect(actual.buttons.map(button => button.label)).toEqual([`Inspect ${CONCEPT_FILE}`, `Tag or auto-tag ${CONCEPT_FILE}`]); + expect(diffTaggingInventory(reference, actual)).toEqual([]); +}); + +test('the comparison notices a changed panel border or primary label', async ({ page, request }) => { + await saveImageTags(request, await conceptPath(request), CONCEPT_TAGS); + const { reference, actual } = await compareState(page, { theme: 'dark', opened: true }); + expect(diffTaggingInventory(reference, actual)).toEqual([]); + const undo = await mutate(page, { css: '.tag-model-panel { border: 0 !important; }' }); + const borderless = withoutChipGeometry(await inventory(page)); + expect(diffTaggingInventory(reference, borderless).map(item => item.path)).toEqual( + expect.arrayContaining(['modelPanel.borderTopWidth', 'modelPanel.borderTopStyle'])); + await undo(); + await tagDialog(page).locator('.dialog-actions .ca-primary').evaluate(node => { node.textContent = 'Save'; }); + const relabelled = withoutChipGeometry(await inventory(page)); + expect(diffTaggingInventory(reference, relabelled).map(item => item.path)).toEqual(['actions.buttons.1.text']); +}); + +// Paired captures for the PR (no screenshot baselines): every production state beside the concept's nearest one. The +// states the concept lacks (unavailable model, saving, conflict, file changed) pair with the concept's opened dialog. +test('paired Tag dialog captures', async ({ page, request }, testInfo) => { + test.setTimeout(120_000); + const path = await conceptPath(request); + await saveImageTags(request, path, CONCEPT_TAGS); + await page.setViewportSize({ width: 1600, height: 1100 }); + const shot = async name => { + await page.evaluate(async () => { await document.fonts.ready; await Promise.all([...document.images].map(image => image.decode().catch(() => {}))); }); + const file = testInfo.outputPath(`${name}.png`); + await page.screenshot({ path: file }); + await testInfo.attach(name, { path: file, contentType: 'image/png' }); + }; + const both = async (name, act) => { + for (const side of ['concept', 'production']) { + await openTagDialog(page, side, 'dark'); + await act(page, side); + await shot(`${side}-${name}`); + } + }; + await both('opened', async () => {}); + await both('generated-wd14', (page, side) => generateWith(page, side)); + await both('generated-gguf', (page, side) => generateWith(page, side, 'gguf')); + await expect(await request.post('/__test__/tagging/suggestions', { data: { scored: [] } })).toBeOK(); + await both('empty', async (page, side) => { + if (side === 'concept') await tagDialog(page).getByRole('spinbutton', { name: 'General confidence', exact: true }).fill('1'); + await generateWith(page, side); + }); + await expect(await request.post('/__test__/tagging/suggestions', { data: { scored: ALPINE } })).toBeOK(); + const save = () => tagDialog(page).locator('.dialog-actions .ca-primary').click(); + const status = () => tagDialog(page).locator('#tag-generation-status'); + + // Unavailable model. + await expect(await request.post('/__test__/tagging/models', { data: { unavailable: ['wd14-swinv2'] } })).toBeOK(); + await openTagDialog(page, 'production', 'dark'); + await generateWith(page, 'production').catch(() => {}); + await expect(status()).toHaveText('WD14 SwinV2 is not downloaded. Download it from the legacy Auto-tag page.'); + await shot('production-unavailable'); + await expect(await request.post('/__test__/tagging/models', { data: { unavailable: [] } })).toBeOK(); + + // Saving: the write is held until the capture is taken. + let release; + const held = new Promise(resolve => { release = resolve; }); + const writes = url => url.pathname === '/prompt_manager/image-annotations/tags'; + await page.route(writes, async route => { await held; await route.continue(); }); + await openTagDialog(page, 'production', 'dark'); + const sent = page.waitForRequest(item => writes(new URL(item.url()))); + await save(); + await sent; + await expect(tagDialog(page).locator('.dialog-actions .ca-primary')).toHaveAttribute('aria-disabled', 'true'); + await shot('production-saving'); + release(); + await expect(tagDialog(page)).toHaveCount(0); + await page.unroute(writes); + + // Conflict: another writer saved this file's set after the dialog observed it. + await openTagDialog(page, 'production', 'dark'); + await saveImageTags(request, path, ['changed_elsewhere']); + await save(); + await expect(status()).not.toHaveText(''); + await shot('production-conflict'); + await page.keyboard.press('Escape'); + + // File changed: the server's own envelope for bytes replaced after the preview (B1 `file-changed`). + await page.route(writes, route => route.fulfill({ status: 409, contentType: 'application/json', + body: JSON.stringify({ success: false, code: 'file-changed', error: 'The file changed since it was previewed. Reopen the Tag dialog.' }) })); + await saveImageTags(request, path, CONCEPT_TAGS); + await openTagDialog(page, 'production', 'dark'); + await save(); + await expect(status()).not.toHaveText(''); + await shot('production-file-changed'); + await page.unroute(writes); + await page.keyboard.press('Escape'); + + // Review scope: the concept's Needs review dialog and production's review-sourced dialog. + await leave(page); + await openConcept(page, 'review', 'dark'); + await page.locator('#main section.surface').getByRole('button', { name: 'Tag image', exact: true }).click(); + await expect(tagDialog(page)).toBeVisible(); + await shot('concept-review'); + await leave(page); + await openProduction(page, 'images', 'dark'); + await page.evaluate(async ({ module, path }) => { + const { openImageTagging } = await import(module); + const { createScope } = await import('/prompt_manager/js/shared/toolkit/core.js'); + const api = window.ApiClient.createApiClient(); + const listing = await api.get('/prompt_manager/images/output?mode=workspace&limit=40'); + openImageTagging({ api, scope: createScope(), image: listing.images.find(item => item.path === path), source: 'review' }); + }, { module: MODULE, path }); + await expect(tagDialog(page).locator('.tag-scope-note')).toBeVisible(); + await settled(page, 'production'); + await shot('production-review'); + await leave(page); +}); diff --git a/tests/browser/workspace_concept_fixture.py b/tests/browser/workspace_concept_fixture.py index ea48be8..b22c362 100644 --- a/tests/browser/workspace_concept_fixture.py +++ b/tests/browser/workspace_concept_fixture.py @@ -158,6 +158,9 @@ MAX_EXTRA_IMAGES = 120 # Unlinked, seedless images (D11 "W × H · PNG") newer than the concept's, for D3 checks. MAX_SEEDLESS_IMAGES = 2 SEEDLESS_SIZE = (1536, 1152) +# The same 4:3 shape with a short "96 × 72 · PNG" subtitle that stays on one line at +# 390px beside the card's Info and Tag (the D3 footer check). +SHORT_SEEDLESS_SIZE = (96, 72) SESSION = "5c1f9a7e-3f0b-4a55-9d0e-2b8c4f6a7d10" _TABLES = ( "execution_run_outputs", @@ -326,9 +329,9 @@ def _record_run( return start.run_id -def _write_seedless_png(path: Path) -> None: +def _write_seedless_png(path: Path, size: tuple[int, int] = SEEDLESS_SIZE) -> None: """A plain PNG with no metadata: no prompt, no seed, no link.""" - with Image.new("RGB", SEEDLESS_SIZE, "#2b3442") as image: + with Image.new("RGB", size, "#2b3442") as image: image.save(path) @@ -345,13 +348,17 @@ def seed_workspace_concept( *, extra_images: int = 0, seedless_images: int = 0, + short_seedless: bool = False, ) -> dict: """Replace this server's disposable rows and files with the concept's data. ``extra_images`` continues the concept's rounds (same prompts, dimensions and seed rule) after its 24 images, for bounded-rendering and Load more checks. - ``seedless_images`` adds unlinked PNGs without metadata, newest of all. + ``seedless_images`` adds unlinked PNGs without metadata, newest of all; + ``short_seedless`` makes them ``SHORT_SEEDLESS_SIZE`` instead. """ + if not isinstance(short_seedless, bool): + raise TypeError("short_seedless must be a boolean") _bounded("extra_images", extra_images, MAX_EXTRA_IMAGES) _bounded("seedless_images", seedless_images, MAX_SEEDLESS_IMAGES) _clear(api, root) @@ -459,7 +466,9 @@ def seed_workspace_concept( for number in range(seedless_images): name = f"seedless_{number + 1:05}.png" path = folder / name - _write_seedless_png(path) + _write_seedless_png( + path, SHORT_SEEDLESS_SIZE if short_seedless else SEEDLESS_SIZE + ) stamp = now - 10 - number * 10 os.utime(path, (stamp, stamp)) seedless.append({"filename": name, "path": f"concepts/{name}"}) diff --git a/tests/browser/workspace_tagging_fixture.py b/tests/browser/workspace_tagging_fixture.py index 9ca8aca..777c0db 100644 --- a/tests/browser/workspace_tagging_fixture.py +++ b/tests/browser/workspace_tagging_fixture.py @@ -186,6 +186,7 @@ def register_tagging_commands( if not 0 <= extra <= MAX_EXTRA_IMAGES: raise web.HTTPBadRequest() model.unavailable = set() + model.suggestions = None model.loaded = None # A test that ends with the dialog open leaves its capture behind; drop # every capture so the next test starts below the store's limits. @@ -213,3 +214,45 @@ def register_tagging_commands( raise web.HTTPBadRequest() model.unavailable = set(data["unavailable"]) return web.json_response({"success": True}) + + @routes.post("/__test__/tagging/suggestions") + async def suggestions(request: web.Request) -> web.Response: + """Answer Generate with fixed scored names (the concept's own sample). + + ``{"scored": [[name, confidence], ...]}`` (at most 100, confidence in + [0, 1]) or ``{"scored": null}`` for the default list. Every capture is + released too, so a concept comparison starts below the store's limits + whatever the previous test left open. + """ + data = await request.json() if request.can_read_body else None + scored = data.get("scored") if isinstance(data, dict) else None + if ( + request.query + or not isinstance(data, dict) + or set(data) != {"scored"} + or not (scored is None or _valid_scored(scored)) + ): + raise web.HTTPBadRequest() + model.suggestions = ( + None if scored is None else [(name, float(value)) for name, value in scored] + ) + model.unavailable = set() + await api._run_in_executor(api.close_annotation_captures) + return web.json_response({"success": True}) + + +def _valid_scored(scored: object) -> bool: + return ( + isinstance(scored, list) + and len(scored) <= 100 + and all( + isinstance(row, list) + and len(row) == 2 + and isinstance(row[0], str) + and 0 < len(row[0]) <= 50 + and isinstance(row[1], (int, float)) + and not isinstance(row[1], bool) + and 0 <= row[1] <= 1 + for row in scored + ) + )