fd747a0ffc9526ea5ff87b1745354932b4d0fd39
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vito
|
0095db8cbc |
feat(inspect): local evidence readers for images and videos (#277)
* feat(inspect): bounded browser PNG reader matching the server's
The Inspect page will read a chosen file entirely in the browser, and the
record it shows must equal what Images shows for the same bytes. This adds
to web/js/png-metadata.js (loaded by the legacy pages as before, with
parsePngTextChunks unchanged):
- readPngMetadataFile(file, {signal, prefix, yieldControl, decompress}):
a port of utils/prompt_inspector.py's _png_metadata, _PngRecords,
_png_text, _inflate and _batch_normalized. It slices only chunk headers,
80-byte keyword prefixes and the retained prompt, workflow, parameters
and comfyassets_prompt_pairs records, CRC-checks what it keeps, applies
the same budgets (2 MiB shared role budget encoded and decoded, 4 MiB
workflow that becomes limited, 8 MiB and 4 MiB duplicate budgets, 4 MiB
duplicate decoded parameters, 10,000 chunks, the batch-index rule for
parameters copies) and compares duplicate chunks byte for byte. Every
range is checked against file.size before slicing and charged against a
19,794,389-byte ceiling, so a multi-gigabyte PNG costs only its metadata.
Compressed text inflates through DecompressionStream with one 256-byte
write in flight and the output capped at the record's allowance; an
AbortSignal rejects the read with an AbortError and leaves no unhandled
rejection; yieldControl runs about every 8 ms of work.
- classifyPrefix(bytes): png, jpeg, webp, iso-bmff, matroska, avi, asf or
null from the first 32 bytes.
- readImageHeader(file, kind): JPEG (SOFn walk as Pillow) and WebP (VP8,
VP8L, VP8X) dimensions from at most 64 KiB.
tests/test_local_inspect_parity.py drives tests/js/support/local-inspect-
oracle.mjs over the committed and test-time inspection parity corpus plus
compressed, oversized, Latin-1, batch-duplicate and all-four-record shapes,
and asserts records, record formats, dimensions and readability equal
read_inspection_file on each file; the two JPEG/WebP fixtures are checked
against Pillow. tests/js/png-local-reader.test.js pins each server rule,
the sparse 3 GiB read, the prefix reuse, inflation back-pressure and abort.
tests/fixtures/local-inspect/tiny.{jpg,webp} are CC0 FFmpeg test patterns.
* feat(inspect): bounded browser video tag readers
Add web/js/workspace/local-video.mjs: ISO-BMFF (MP4/MOV/M4V) and Matroska
(MKV/WebM) walkers that read only the container tags the server asks ffprobe
for, slicing box and element headers, carrier names and retained values within
the server's budgets and never a byte inside mdat or a Cluster; a port of
utils/video_metadata.select_video_carrier with the record budgets of
read_inspection_file; and a byte-exact json.loads/json.dumps round trip so
envelope objects reserialize as the server spells them.
Every naming rule mirrors ffprobe 9.0.2 on hand-built files: legacy ilst atoms,
mdta keys, joined repeats, data types, Matroska language and default handling,
and Tags behind a Cluster found only through a SeekHead.
Tests: node units over byte-exact container builders and the committed
fixtures; a Python parity test comparing the local carrier with the server's
selection from the same tag maps (mocked probe) and from the same files (real
ffprobe, including 1 MiB + 2.5 MiB per-key records); an ffprobe
characterization of the recorded fixture maps.
* feat(inspect): server-identical local projection and payload
Port utils/inspection_response.py to the browser: a strict parse with Python's
Decimal spelling, duplicate-key rejection, NaN/Infinity as null and integers
past 2^53-1 as text; the prompt, workflow and carrier projection with the
depth-64 and 20,000-node ceilings; _metadata; and the 8 MiB omission steps
(workflow graph, then prompt graph, then all raw) measured as UTF-8 bytes.
local-inspection.mjs builds the inspection_version 1 wire for one chosen File
from the Task 1 and Task 2 readers (observation selected-local-file, interim
unresolved roles, unavailable library), fits it to the response budget, and
exposes localFileAdapter for the shared controller, inspector and viewer:
handles are session-owned by a private token, so a tampered or hand-made
descriptor is rejected and blob: URLs are never resolved through the API
client. projectInspectionPayload admits selected-local-file only for a
local-file identity.
A Python oracle test compares projectMetadata with _metadata byte for byte over
the parity PNG corpus, the committed video fixtures and tag maps, 75 committed
adversarial record sets (projection-cases.json) and node-ceiling sets built at
test time.
* test(inspect): import oracle modules by file URL for Windows
The local-inspect oracle imported product modules with path.join(...), which
Node parses as a URL scheme on Windows (D:\... -> ERR_UNSUPPORTED_ESM_URL_SCHEME),
failing every tests/test_local_inspect_parity.py oracle test on the Windows CI
job. Every dynamic import in tests/js/support now goes through moduleUrl, a
pathToFileURL wrapper, and a node unit proves the Windows form on any host and
scans the oracles so a path import cannot return.
* fix(inspect): read video tags as ffprobe reports them
Same bytes now give the server's record in five shapes where the local
reader was stricter or kept different tags (PR #277 final review I1, I2,
M1, M2, M4):
- Invalid UTF-8 in a tag value is replaced as ffprobe's default
string_validation=replace does (av_utf8_decode, one U+FFFD per failing
decode step), instead of failing the read. Key names and TagNames that
do not decode are no carrier and are skipped.
- An ilst item whose keys index is 0 or out of range is skipped, as
ffmpeg does, instead of failing the read.
- Only the first keys box is read, as ffmpeg does.
- A first per-key workflow over 4 MiB keeps its place in the carrier
choice as an unread marker, so the server's carrier (per-key, workflow
limited) is chosen; when the server's first probe would overflow, the
re-probe leaves it out and its own bound applies.
- An envelope integer of more than 4300 digits fails the Python json
port, so that envelope is skipped as CPython's json.loads skips it.
Each shape was checked against ffprobe 9.0.2 and read_inspection_file;
the real-ffprobe parity test now builds them, and a 600-case random-byte
comparison of the replacement against ffprobe matched.
* perf(inspect): measure the local response size natively and yield between steps
fitInspectionPayload counted the UTF-8 size of an 8-12 MB wire with a
per-character loop and ran every omission step on the main thread in one
turn (PR #277 final review M7). The size is now counted by TextEncoder
encodeInto through one bounded scratch buffer, and each re-measure
follows a yieldControl turn, with an abort signal checked between steps.
The fit is async; the fitted payload is the same as before.
* fix(inspect): hold video tags as ffmpeg's dictionary does, decode Mac Roman text
Three more same-bytes divergences, found while verifying the review
fixes against ffprobe 9.0.2 and read_inspection_file:
- ffmpeg keeps one tag per ASCII-folded name under the spelling set
last (ISO-BMFF joins the values with ";", Matroska keeps the last), so
PROMPT and Prompt are one record on the server; the local reader kept
both and failed the read as a duplicate. A legacy comment atom and a
later COMMENT key likewise leave no exact-case envelope.
- ffprobe's format_tags filter folds ASCII case only, so a name that is
"workflow" only under Unicode case rules is never reported; the local
reader took it.
- ISO-BMFF data types 0 and 3 are Mac Roman text to ffmpeg, written into
twice the input size (a long value is cut); the local reader decoded
type 0 as UTF-8 and skipped type 3.
The real-ffprobe parity test builds each shape.
* docs(testing): list the local Inspect divergences; raise touched coverage floors
TESTING.md now tracks the local Inspect (#249 PR L1) divergences that
remain after the review fixes, each checked on a host with ffprobe 9.0.2:
media kind by bytes versus extension (parity is for the same bytes with a
matching extension), JPEG dimensions past the 64 KiB header read, classic
QuickTime udta text atoms, raw versus escaped tag bounds, the 10,000
element ceiling, unknown EBML DocTypes, the JSON depth approximation and
the C _decimal assumption.
coverage:update raised the branch floors of the three local Inspect
modules this PR touches; the unrelated activity-scan raise it proposed is
left at main's values.
|
||
|
Vito
|
6c82098207 |
feat(images): data for the concept cards — linked titles, seed, search and filters, counts, favorites (#257)
* feat(images): link summary on workspace image items
* fix(images): carry title source and cut first-line titles like the Library
Summaries gain title_source (stored or text). The inspector always shows a
stored title and confirms only a first-line title against the file, so a
stored title equal to the first line is no longer withheld. A link-title read
failure no longer marks a readable link unavailable. First-line titles over
80 characters end with an ellipsis, as the Library's title() does. The card
gate for unconfirmed first-line titles is pinned as an expected failure for B2.
* feat(images): unambiguous seed on workspace items
Workspace items gain `seed`, read with the shared generation-parameter
parser from the prompt/workflow records the header open already loaded
(each at most 2 MiB; no extra file reads, videos give None). A
placeholder or non-integer seed gives None. JSON integers beyond 2**53
are recovered exactly, so the card prints the digits the inspector shows.
The same parse confirms first-line card titles: a `text` title shows only
when the file's parsed positive gives the same title. One helper applies
that rule on the card and in the inspector, which now reads the positive
from its own inspection records with the same parser.
* fix(images): a deeply nested record no longer fails the workspace listing
The Python parser recurses over nested arrays where JavaScript String()
does not, so a crafted 2 KB PNG turned every workspace page holding it
(and its inspect route) into a 500. Card facts now treat RecursionError
like any record the parser cannot use: no seed and no confirmed title.
Also: Library previews skip the card parse they never show; the parse
runs after the file handle closes; the exact-seed read uses the one JSON
reader (parse_js_json(..., exact_integers=True)) and the public
number_string; record size is decided from the text length unless only
the UTF-8 byte count can tell.
* feat(images): search and filter workspace images by linked prompts
The workspace listing now takes category (exact), tags (comma list, all
required) and sort=rating, and q also matches the linked positive's text,
Library title, pair title and tag names. One bounded, parameterized SELECT
(get_image_link_matches / get_image_paths_matching) returns normalized
paths of unambiguously linked images; the listing is intersected with it.
Unlinked and ambiguous files are excluded by a category or tag filter and
sort last by rating, with a newest-first tie-break. The cursor scope covers
the new filters and sort; unknown values are still rejected.
The derived listing values and sort orders are cached per gallery scan so a
50,000-file listing with 20,000 links pages well under 150 ms at p95.
* test(images): count the filter budget everywhere, time it in one job
The 150 ms wall-clock budget failed under the CI coverage tracer, so it
is split. test_filter_work_is_bounded runs on every job and under
coverage: one filter SELECT per request, the ambiguity probe is an index
search (EXPLAIN QUERY PLAN), the per-scan values are built on the cold
request only, and a cursor page calls the sort key at most
ceil(log2 n)+3 times. test_filter_budget runs only with
PROMPTMANAGER_PERF_BUDGET=1 and no tracer, asserts the best of three
rounds' p95, and has its own non-required "perf budget" job. TESTING.md
records the convention.
* fix(images): rating sort works however many link rows the database holds
The filter SELECT counted every generated_images row against a 500,000
row cap and materialized all of them first, so a large history made
sort=rating answer 400 with nothing to narrow. The rows are now streamed
and only paths in the listing's normalized set are kept, so memory is
bounded by the listing; the cap and LinkFilterTooBroad are gone.
* fix(images): rebuild workspace listing values only where the scan changed
A TTL rescan that finds the same files reuses the previous values and
sort orders; one that finds a few changes rebuilds only those files.
Gallery invalidation and a gallery-roots change drop the derived values,
so old roots no longer hold memory. category is trimmed like tags.
* fix(images): a monitor invalidation rebuilds only the new files' values
The image monitor invalidates the gallery whenever it sees a new file,
and that dropped every derived workspace value, so the next request paid
a full cold build. Invalidation now keeps the per-file values and drops
only the sort orders; the rescan reuses every unchanged file and builds
only new or changed ones. A gallery-roots change still drops everything.
* feat(library): expose run count, last use and variant totals
* fix(library): an out-of-range last use reads as unknown, not a failed list
* perf(images): read positive counts only on a plain listing's first page
* feat(images): persist image favorites
Favorites are stored in image_favorites keyed by the gallery root label
and the relative path, set and cleared through PUT/DELETE
/prompt_manager/images/favorite with a {root_index, path} body validated
against the rooted listing. Workspace image cards carry favorite: bool.
The table is created by migration, travels with backups, is optional in
old backups and required after a restore's migration.
* fix(images): key favorites by the real root and case-normalized path
Folder names are not unique across gallery roots, so two roots shown as
"output" shared each other's stars. Favorites are now keyed by the
root's real, case-normalized path (server-side only) and a
case-normalized relative path, so a Windows case-only rename keeps its
star while distinct Linux files stay distinct. Toggles look the file up
in a per-scan dictionary instead of walking the listing.
* fix(images): favorites lookup follows monitor invalidation and the stored key bound
A monitor invalidation now also drops the favorites lookup, which held the
previous scan list until the next toggle. PUT refuses a favorite whose root
or relative key would exceed the schema's 4096-character bound, so the live
database keeps passing startup and restore validation.
* docs(testing): workspace data fields for the concept pages
* fix(images): card facts read the inspector's projected graphs
The inspector hands its browser parser only graphs the inspection
projection parsed (strict JSON, depth 64, node ceiling); a deeper record
is "limited" there, so the legacy fill never runs and no seed shows. The
card parsed the raw records instead and showed a seed the inspector did
not, e.g. for a 900-deep class_type beside a normal sampler.
project_graph() is now the one projection step, used by the inspection
response and by the card. The card runs the shared parser on the
projected graphs, the same values the browser receives: big integers
arrive as their exact digits, so the separate exact-integer re-parse and
the parse_js_json flag it needed are gone.
* fix(images): send the card seed as a decimal string
JSON.parse rounds a number past 2**53, so a listing seed such as
9223372036854788153 reached the page as 9223372036854788000 while the
inspector showed the file's digits. Every non-null card seed is now its
decimal string.
The Node parity harness now JSON.parses the real listing response and
compares String(seed) with the seed the inspector shows, including a
seed past 2**53. TESTING.md describes the string contract and drops the
removed exact re-read.
* fix(images): bound digit-string seeds and isolate each card's parse
A seed recorded as a 5000-digit string reached int(), which raises
ValueError past Python's integer string limit, and the listing answered
500. A digit-string seed now has at most 20 digits (2**64 fits), and the
conversion falls back to no seed on ValueError.
Each card's parse is also guarded in the page builder: an unexpected
error in one file's records logs a warning and gives that card no seed
and no first-line title, and the rest of the page still lists.
* fix(images): filename search survives a failed prompt-link read
On main a workspace q matched only file paths, so search worked with a
broken database. Since search also matches linked prompts, any q turned
a failed link read into a 500. A q without category, tags or the rating
sort now falls back to filename matches with a null positive_count and
logs the failure with its traceback; those linked filters still fail.
* fix(images): card reads PNG records with the inspector's reader
The card read PNG text through Pillow's header info, where a later
duplicate prompt chunk silently wins and text after IDAT is missed. The
inspector's bounded reader refuses conflicting duplicates and reads text
after IDAT, so a file with two different prompt chunks showed a card
seed the inspector refused.
PNG cards now read their records with read_inspection_file, the reader
the inspector uses. When it refuses a file the card keeps its
dimensions from Pillow and shows no seed. Other formats still use
Pillow's header info, as the inspector's header read does.
* perf(images): read each card's header facts once per gallery scan
Every page parsed each card's prompt and workflow records again, about
3.4 ms per 50 KB record set and 25 ms per 400 KB workflow, inside
ComfyUI's server process, on every scroll-back and refresh.
Dimensions, the seed and the parsed positive's display title are now
cached per root by (path, mtime), beside the scan's per-file values and
invalidated the same way: a rescan keeps unchanged files and prunes the
rest, a roots change drops them all, and only files of the current scan
are stored. An unreadable header is not stored. Links and favorites
are still read on every page.
* ci: run the perf budget on main pushes and on demand only
The wall-clock budget has under 2x margin on shared ubuntu runners, and
a red exact-head check blocks a merge that must never be rerun to green.
The job now runs on each push to main and on workflow_dispatch, not on
pull requests. TESTING.md says a red run on main means fix the code.
* fix(library): cut display titles at 80 UTF-16 units, never inside a pair
The server counted code points while the Library's JavaScript counts UTF-16
units, so a first line of emoji kept 80 emoji on cards but showed 40 in the
Library. Both now cut at 80 UTF-16 units and drop a character that would
straddle the limit instead of leaving half a surrogate pair.
|
||
|
Vito
|
36564b4767 |
feat(parse): one generation-parameter parser in Python; scan links what the resolver cannot decide (#253)
* feat(parse): port the browser generation-parameter parser to Python Adds utils/generation_params.py, a line-for-line port of extractGenerationParams in web/js/comfy-metadata.js, the parser behind the gallery, admin and metadata pages. The scan and live monitor can use it so every surface reads prompts the same way. The JavaScript parser is unchanged. The port reproduces the JavaScript value semantics the result depends on (property order, truthiness, String(), trim(), strict equality, Map keys, JSON.parse number rounding, and the inputs on which it throws), and runs its recursive tracing on an explicit stack so crafted graphs cannot exhaust the Python stack. Tests: - behaviour tests for each tracing rule, with JavaScript results as expected values, including the visit-limit boundary; - golden vectors of the JavaScript parser's results for every named case and every PNG fixture, so the comparison runs without Node; - a live comparison against the JavaScript parser when Node is present; - a randomized differential test over 9,000 generated graphs and workflows, coherent and corrupted, including inputs where the JavaScript throws. Five deliberate porting mistakes (trim, truthiness, prepend handling, visit limit, pair slot) each fail the tests. On a 38,602-file output folder, all 18,491 files with a workflow give identical results. * fix(scan): link the traced prompt when the pair resolver cannot decide The output scan skipped every file the pair resolver called ambiguous or unresolved, so workflows where several samplers share one prompt (a base pass, an upscale pass and a detailer) never reached the Library from a scan. It now links the positive prompt the shared parser traces for those files, the prompt the inspector and the legacy pages show. Files the resolver decides are stored as before, and graphs without a recognized sampler keep the legacy extractor (#252). Read-only scans of a 38,602-file output folder into temporary databases: - links: 1,500 before, 7,954 after; none of the 1,500 dropped or changed; - of the added links that differ from the pre-redesign scan, 1,275 replace a number, a LoRA tag, a fragment or a truncated prompt that scan stored. * refactor(parse): return a uniform pair from the SameValueZero key |
||
|
Vito
|
048d599386 |
fix(inspect): bound inspection work, not graph size (#244)
* test(inspect): add synthetic inspection parity fixtures and legacy characterization * test(inspect): pin test-time parity vectors and keep fixtures fully synthetic * fix(inspect): bound resolver work instead of graph size * fix(inspect): bound shared-node and stripped-text work in the inspection profile Keep one memo per resolution whose entries carry their unfolded visit cost; a memo hit charges that cost to the current candidate's allowance only. Cache node validation and charge input keys, stripped characters and copied provenance entries to a whole-resolution work budget before doing the work. The default profile is unchanged. * fix(inspect): keep memoized depth honest across inspection candidates A memo entry now records its height below the reusing depth; a hit raises depth-limit exactly where a fresh walk would, so each candidate's outcome matches its anchored result and does not depend on trace or node order. The default profile is unchanged. * test(inspect): pin memo height propagation Three samplers share one chain, each later one hanging below a memo hit. The third candidate must report depth-limit in both trace orders and when anchored; removing the memo-hit height propagation fails this test. The provenance test now asserts a plain subset. * fix(inspect): tolerate ComfyUI node metadata quirks in projections Projections now take a kind. Each prompt node keeps exactly class_type and inputs; _meta, is_changed and other node keys are validated and dropped, so "is_changed": [NaN] no longer marks the whole prompt graph malformed. A nonfinite value inside inputs, anywhere in a workflow, or in a retained carrier member projects to null and is never reported as a number. The carrier applies the prompt/workflow kinds to its members. Exact numbers are unchanged (Decimal spelling, unsafe integers as strings) and duplicate keys are still malformed. The 500-node gate is replaced by a 20,000-node backstop (workflow count includes subgraph definitions). Raw records are untouched. The f14 inspection vector's facts become known; default vectors are unchanged. * fix(inspect): budget metadata per record so large workflows never hide prompts PNG inspection no longer counts every text chunk against one shared 2 MiB budget. Role records (prompt, parameters, the pair execution record) keep a 2 MiB budget together, encoded and decoded, and exceeding it still fails the whole read. A workflow has its own 4 MiB budget: a chunk over it is skipped by seek, and a compressed one stops inflating at budget + 1. Only the workflow is then reported as limited (record format "limited", no raw text, workflow_graph limited); prompts, facts and the prompt graph are unaffected. Each text chunk's keyword is read first (at most 80 bytes). Chunks that are not retained are skipped by length without being read or checksummed, so memory stays bounded. Byte-identical duplicates of a retained record are accepted and do not consume budget; differing duplicates still raise "Duplicate prompt metadata", and a duplicate of a limited workflow stays limited. The chunk-count limit is unchanged. InspectionRead gains limited_records (empty by default). The reader is only used by the inspection routes, so ingestion, scan and import are unchanged. Vectors: f04-inspection (2.5 MB workflow, within the new workflow budget) and f15-inspection (identical duplicate chunks) now read and resolve, with the same outcome and facts as f05-inspection. Default vectors are unchanged. * fix(inspect): keep prompt evidence longest under the response budget * fix(inspect): bound identical duplicate chunk reads * fix(inspect): fail closed when role duplicates exceed their budget * fix(inspect): use the bounded inspection profile end to end Inspection routes resolve with INSPECTION_PROFILE and expose reason_code and node on the resolution; scan, live monitor and admin import keep the default profile (pinned by spies over the real resolver on f03). JS facts drop the entries and workflow node-count gates; per-lookup visit sets and the server's node ceiling bound the work. The validator accepts the textless `limited` workflow format, follows the workflow-first response omission order, and checks the closed reason-code vocabulary. The file inspector labels limited records and disables their workflow download. vectors.json: 12 inspection vectors gain their reason codes; f03 facts become known and f01 facts ambiguous. Default vectors are unchanged. A new parity test asserts vectors.json facts through the workspace JS. * chore(tools): add read-only inspection corpus tool tools/inspection_corpus.py runs the inspection route pipeline and the workspace JS facts of a chosen code tree over a media folder, read-only, and reports the #240 invariants H1 to H8 with aggregate counts. Rows hold statuses and SHA-256 digests of resolved role texts, never file names or prompt text unless --with-paths is given. Outputs inside the root or a code tree are refused. The facts-over-vectors test now fails instead of skipping when CI is set and Node.js is missing. * docs(testing): document bounded inspection work * fix(tools): fail the corpus run when facts or inspection did not run Rows whose facts were never computed (no or exiting Node.js) now fail H3, and a new harness line fails on any tool exception or missing facts, so a run where nothing was measured can no longer exit 0. Unreadable files stay inspector outcomes. A missing --node binary is refused, an exited harness is reaped, H8 has a 50 ms floor for tiny baselines, and JSONL rows are sorted by id. * docs(inspect): correct ceiling wording and bandit annotations Describe the two 20,000-node ceilings (prompt entries, workflow UI plus subgraph nodes) in TESTING.md, and use bare nosec ids so Bandit stops warning about unparsed test names. * test(inspect): make parity observation tuples explicit |