From 0095db8cbcd15f4fdb4d4c2686776e595f4687b4 Mon Sep 17 00:00:00 2001 From: Vito Date: Thu, 22 Oct 2026 10:21:02 -0700 Subject: [PATCH] feat(inspect): local evidence readers for images and videos (#277) * feat(inspect): bounded browser PNG reader matching the server's The Inspect page will read a chosen file entirely in the browser, and the record it shows must equal what Images shows for the same bytes. This adds to web/js/png-metadata.js (loaded by the legacy pages as before, with parsePngTextChunks unchanged): - readPngMetadataFile(file, {signal, prefix, yieldControl, decompress}): a port of utils/prompt_inspector.py's _png_metadata, _PngRecords, _png_text, _inflate and _batch_normalized. It slices only chunk headers, 80-byte keyword prefixes and the retained prompt, workflow, parameters and comfyassets_prompt_pairs records, CRC-checks what it keeps, applies the same budgets (2 MiB shared role budget encoded and decoded, 4 MiB workflow that becomes limited, 8 MiB and 4 MiB duplicate budgets, 4 MiB duplicate decoded parameters, 10,000 chunks, the batch-index rule for parameters copies) and compares duplicate chunks byte for byte. Every range is checked against file.size before slicing and charged against a 19,794,389-byte ceiling, so a multi-gigabyte PNG costs only its metadata. Compressed text inflates through DecompressionStream with one 256-byte write in flight and the output capped at the record's allowance; an AbortSignal rejects the read with an AbortError and leaves no unhandled rejection; yieldControl runs about every 8 ms of work. - classifyPrefix(bytes): png, jpeg, webp, iso-bmff, matroska, avi, asf or null from the first 32 bytes. - readImageHeader(file, kind): JPEG (SOFn walk as Pillow) and WebP (VP8, VP8L, VP8X) dimensions from at most 64 KiB. tests/test_local_inspect_parity.py drives tests/js/support/local-inspect- oracle.mjs over the committed and test-time inspection parity corpus plus compressed, oversized, Latin-1, batch-duplicate and all-four-record shapes, and asserts records, record formats, dimensions and readability equal read_inspection_file on each file; the two JPEG/WebP fixtures are checked against Pillow. tests/js/png-local-reader.test.js pins each server rule, the sparse 3 GiB read, the prefix reuse, inflation back-pressure and abort. tests/fixtures/local-inspect/tiny.{jpg,webp} are CC0 FFmpeg test patterns. * feat(inspect): bounded browser video tag readers Add web/js/workspace/local-video.mjs: ISO-BMFF (MP4/MOV/M4V) and Matroska (MKV/WebM) walkers that read only the container tags the server asks ffprobe for, slicing box and element headers, carrier names and retained values within the server's budgets and never a byte inside mdat or a Cluster; a port of utils/video_metadata.select_video_carrier with the record budgets of read_inspection_file; and a byte-exact json.loads/json.dumps round trip so envelope objects reserialize as the server spells them. Every naming rule mirrors ffprobe 9.0.2 on hand-built files: legacy ilst atoms, mdta keys, joined repeats, data types, Matroska language and default handling, and Tags behind a Cluster found only through a SeekHead. Tests: node units over byte-exact container builders and the committed fixtures; a Python parity test comparing the local carrier with the server's selection from the same tag maps (mocked probe) and from the same files (real ffprobe, including 1 MiB + 2.5 MiB per-key records); an ffprobe characterization of the recorded fixture maps. * feat(inspect): server-identical local projection and payload Port utils/inspection_response.py to the browser: a strict parse with Python's Decimal spelling, duplicate-key rejection, NaN/Infinity as null and integers past 2^53-1 as text; the prompt, workflow and carrier projection with the depth-64 and 20,000-node ceilings; _metadata; and the 8 MiB omission steps (workflow graph, then prompt graph, then all raw) measured as UTF-8 bytes. local-inspection.mjs builds the inspection_version 1 wire for one chosen File from the Task 1 and Task 2 readers (observation selected-local-file, interim unresolved roles, unavailable library), fits it to the response budget, and exposes localFileAdapter for the shared controller, inspector and viewer: handles are session-owned by a private token, so a tampered or hand-made descriptor is rejected and blob: URLs are never resolved through the API client. projectInspectionPayload admits selected-local-file only for a local-file identity. A Python oracle test compares projectMetadata with _metadata byte for byte over the parity PNG corpus, the committed video fixtures and tag maps, 75 committed adversarial record sets (projection-cases.json) and node-ceiling sets built at test time. * test(inspect): import oracle modules by file URL for Windows The local-inspect oracle imported product modules with path.join(...), which Node parses as a URL scheme on Windows (D:\... -> ERR_UNSUPPORTED_ESM_URL_SCHEME), failing every tests/test_local_inspect_parity.py oracle test on the Windows CI job. Every dynamic import in tests/js/support now goes through moduleUrl, a pathToFileURL wrapper, and a node unit proves the Windows form on any host and scans the oracles so a path import cannot return. * fix(inspect): read video tags as ffprobe reports them Same bytes now give the server's record in five shapes where the local reader was stricter or kept different tags (PR #277 final review I1, I2, M1, M2, M4): - Invalid UTF-8 in a tag value is replaced as ffprobe's default string_validation=replace does (av_utf8_decode, one U+FFFD per failing decode step), instead of failing the read. Key names and TagNames that do not decode are no carrier and are skipped. - An ilst item whose keys index is 0 or out of range is skipped, as ffmpeg does, instead of failing the read. - Only the first keys box is read, as ffmpeg does. - A first per-key workflow over 4 MiB keeps its place in the carrier choice as an unread marker, so the server's carrier (per-key, workflow limited) is chosen; when the server's first probe would overflow, the re-probe leaves it out and its own bound applies. - An envelope integer of more than 4300 digits fails the Python json port, so that envelope is skipped as CPython's json.loads skips it. Each shape was checked against ffprobe 9.0.2 and read_inspection_file; the real-ffprobe parity test now builds them, and a 600-case random-byte comparison of the replacement against ffprobe matched. * perf(inspect): measure the local response size natively and yield between steps fitInspectionPayload counted the UTF-8 size of an 8-12 MB wire with a per-character loop and ran every omission step on the main thread in one turn (PR #277 final review M7). The size is now counted by TextEncoder encodeInto through one bounded scratch buffer, and each re-measure follows a yieldControl turn, with an abort signal checked between steps. The fit is async; the fitted payload is the same as before. * fix(inspect): hold video tags as ffmpeg's dictionary does, decode Mac Roman text Three more same-bytes divergences, found while verifying the review fixes against ffprobe 9.0.2 and read_inspection_file: - ffmpeg keeps one tag per ASCII-folded name under the spelling set last (ISO-BMFF joins the values with ";", Matroska keeps the last), so PROMPT and Prompt are one record on the server; the local reader kept both and failed the read as a duplicate. A legacy comment atom and a later COMMENT key likewise leave no exact-case envelope. - ffprobe's format_tags filter folds ASCII case only, so a name that is "workflow" only under Unicode case rules is never reported; the local reader took it. - ISO-BMFF data types 0 and 3 are Mac Roman text to ffmpeg, written into twice the input size (a long value is cut); the local reader decoded type 0 as UTF-8 and skipped type 3. The real-ffprobe parity test builds each shape. * docs(testing): list the local Inspect divergences; raise touched coverage floors TESTING.md now tracks the local Inspect (#249 PR L1) divergences that remain after the review fixes, each checked on a host with ffprobe 9.0.2: media kind by bytes versus extension (parity is for the same bytes with a matching extension), JPEG dimensions past the 64 KiB header read, classic QuickTime udta text atoms, raw versus escaped tag bounds, the 10,000 element ceiling, unknown EBML DocTypes, the JSON depth approximation and the C _decimal assumption. coverage:update raised the branch floors of the three local Inspect modules this PR touches; the unrelated activity-scan raise it proposed is left at main's values. --- TESTING.md | 35 + tests/fixtures/README.md | 69 + .../fixtures/local-inspect/core-faststart.mp4 | Bin 0 -> 3268 bytes .../fixtures/local-inspect/core-moov-end.mp4 | Bin 0 -> 2837 bytes tests/fixtures/local-inspect/core-tags.webm | Bin 0 -> 1718 bytes tests/fixtures/local-inspect/core.m4v | Bin 0 -> 2845 bytes tests/fixtures/local-inspect/core.mkv | Bin 0 -> 2489 bytes tests/fixtures/local-inspect/core.mov | Bin 0 -> 2849 bytes .../fixtures/local-inspect/legacy-comment.mp4 | Bin 0 -> 2940 bytes .../local-inspect/legacy-comment.webm | Bin 0 -> 1888 bytes .../local-inspect/projection-cases.json | 849 +++++++++++++ tests/fixtures/local-inspect/tiny.jpg | Bin 0 -> 1071 bytes tests/fixtures/local-inspect/tiny.webp | Bin 0 -> 470 bytes .../fixtures/local-inspect/video-tags-v1.json | 60 + tests/js/helpers/png-bytes.mjs | 148 +++ tests/js/helpers/video-bytes.mjs | 277 ++++ tests/js/png-local-reader.test.js | 549 ++++++++ tests/js/support-module-urls.test.js | 39 + tests/js/support/local-inspect-oracle.mjs | 59 + tests/js/support/module-url.mjs | 9 + tests/js/workspace-local-inspection.test.js | 308 +++++ tests/js/workspace-local-projection.test.js | 249 ++++ tests/js/workspace-local-video.test.js | 622 +++++++++ tests/test_local_inspect_parity.py | 1115 +++++++++++++++++ tests/test_utils_video_metadata.py | 39 + tools/vendor/frontend-coverage-baseline.json | 20 +- web/js/png-metadata.js | 611 ++++++++- web/js/workspace/inspection-state.mjs | 8 +- web/js/workspace/local-inspection.mjs | 175 +++ web/js/workspace/local-projection.mjs | 404 ++++++ web/js/workspace/local-video.mjs | 1052 ++++++++++++++++ 31 files changed, 6689 insertions(+), 8 deletions(-) create mode 100644 tests/fixtures/local-inspect/core-faststart.mp4 create mode 100644 tests/fixtures/local-inspect/core-moov-end.mp4 create mode 100644 tests/fixtures/local-inspect/core-tags.webm create mode 100644 tests/fixtures/local-inspect/core.m4v create mode 100644 tests/fixtures/local-inspect/core.mkv create mode 100644 tests/fixtures/local-inspect/core.mov create mode 100644 tests/fixtures/local-inspect/legacy-comment.mp4 create mode 100644 tests/fixtures/local-inspect/legacy-comment.webm create mode 100644 tests/fixtures/local-inspect/projection-cases.json create mode 100644 tests/fixtures/local-inspect/tiny.jpg create mode 100644 tests/fixtures/local-inspect/tiny.webp create mode 100644 tests/fixtures/local-inspect/video-tags-v1.json create mode 100644 tests/js/helpers/png-bytes.mjs create mode 100644 tests/js/helpers/video-bytes.mjs create mode 100644 tests/js/png-local-reader.test.js create mode 100644 tests/js/support-module-urls.test.js create mode 100644 tests/js/support/local-inspect-oracle.mjs create mode 100644 tests/js/support/module-url.mjs create mode 100644 tests/js/workspace-local-inspection.test.js create mode 100644 tests/js/workspace-local-projection.test.js create mode 100644 tests/js/workspace-local-video.test.js create mode 100644 tests/test_local_inspect_parity.py create mode 100644 web/js/workspace/local-inspection.mjs create mode 100644 web/js/workspace/local-projection.mjs create mode 100644 web/js/workspace/local-video.mjs diff --git a/TESTING.md b/TESTING.md index 14a10c0..e0cf324 100644 --- a/TESTING.md +++ b/TESTING.md @@ -1740,6 +1740,41 @@ ComfyUI savers reach the Library through the admin scan, which reads the shared carrier. Accepting `VIDEO_EXTENSIONS` in the monitor and routing them through `read_video_metadata` is a separate, issue-linked change. +Follow-up: local Inspect parity (#249 PR L1). The browser readers +(`web/js/png-metadata.js`, `web/js/workspace/local-video.mjs`, +`web/js/workspace/local-projection.mjs`) answer as the server does for the same bytes +and a matching extension (`tests/test_local_inspect_parity.py`; the real-ffprobe cases +skip without ffmpeg). The remaining known divergences, checked against ffprobe 9.0.2, +`read_inspection_file` and `_metadata`, are tracked here, not accepted as parity: + +- Media kind comes from the bytes locally and from the extension on the server: a PNG + named `.mp4` is read as a PNG locally (format and size shown) while the server probes + it with ffprobe (no format, no size); MP4 bytes named `.png` read locally as video + while Pillow fails them on the server (`unreadable`). Parity is stated for the same + bytes with a matching extension. +- JPEG dimensions: the local header read stops at 64 KiB, so a JPEG whose SOF follows + more than 64 KiB of APP segments (camera, Photoshop and EXIF/XMP workflow savers) + shows no dimensions, while Pillow reads them within the server's 8 MiB header bound. + The fix is a segment-header walk up to 8 MiB, which needs a plan amendment. +- Classic QuickTime `moov/udta` text atoms (`©cmt`/`desc` with a 16-bit size and + language, outside `meta`) are not read locally; ffprobe reports them as + `comment`/`description`, so the server can select that envelope while the local + read is `absent`. No ComfyUI saver writes them. +- Tag-size bounds count raw bytes locally and ffprobe's escaped JSON on the server + (first probe 12 MiB + 64 KiB, role-only re-probe 4 MiB + 64 KiB): only tag sets near + those bounds and dense with quotes or control characters can make one side re-probe + or fail while the other reads. +- More than 10,000 container elements stop the local read (`limited`, metadata + `unreadable`); ffprobe reads such files. +- An EBML DocType other than `matroska`/`webm` is an invalid container locally; ffmpeg + only warns and reads it. +- Envelope JSON deeper than 1,000 levels is refused locally as Python's default + recursion limit refuses it; the server's real limit depends on the interpreter and + call stack. A duplicate key in an object enclosing a nest deeper than the running + Python's json limit is `malformed` locally and `limited` on that Python. +- Decimal exponent limits assume CPython's C `_decimal`; on a `_pydecimal` interpreter + the server would read the over-limit exponents the local projection refuses. + Preserve raw logs, failed traces, paired PNGs, geometry, runtime versions and source hashes before later managed commands clear their output directories. Frontend baseline updates are additive through `coverage:update` only after diff --git a/tests/fixtures/README.md b/tests/fixtures/README.md index 9cf262b..25b61bc 100644 --- a/tests/fixtures/README.md +++ b/tests/fixtures/README.md @@ -13,6 +13,75 @@ ffmpeg -f lavfi -i testsrc=size=32x32:rate=4:duration=1 \ Browser tests use the committed file; FFmpeg is not required to run them. +# Local Inspect header fixtures + +`local-inspect/tiny.jpg` (48x32) and `local-inspect/tiny.webp` (40x24) are original +synthetic FFmpeg color test patterns, under 4 KB, dedicated to the public domain +under CC0-1.0. `tests/js/png-local-reader.test.js` reads their dimensions with the +browser header reader and `tests/test_local_inspect_parity.py` checks the same two +files against Pillow. + +Reproduce with FFmpeg: + +```sh +ffmpeg -f lavfi -i testsrc=size=48x32:rate=1 -frames:v 1 -q:v 10 tiny.jpg +ffmpeg -f lavfi -i testsrc=size=40x24:rate=1 -frames:v 1 -c:v libwebp -quality 40 tiny.webp +``` + +# Local Inspect video fixtures + +`local-inspect/core-faststart.mp4`, `core-moov-end.mp4`, `core.mov`, `core.m4v`, +`core.mkv`, `core-tags.webm`, `legacy-comment.mp4` and `legacy-comment.webm` are +0.2-second 16x16 black clips (each under 4 KB) carrying an original synthetic +ComfyUI graph in their container tags, dedicated to the public domain under +CC0-1.0. The `core-*` files carry one tag per key as core SaveVideo/SaveWEBM write +them (MP4/MOV/M4V through `-movflags use_metadata_tags`; Matroska upper-cases the +names); `core-faststart.mp4` also carries `parameters` and +`comfyassets_prompt_pairs`. The `legacy-comment.*` files carry the old +VideoHelperSuite `comment` JSON envelope (an ilst `©cmt` atom in MP4, a `COMMENT` +tag in WebM). `video-tags-v1.json` records `ffprobe -show_entries format_tags -of +json` for each file (ffmpeg 9.0.2); `tests/test_utils_video_metadata.py` +`VideoFixtureFfprobeTests` re-checks it when ffprobe is installed, and +`tests/js/workspace-local-video.test.js` plus `tests/test_local_inspect_parity.py` +hold the browser video reader to the same maps. + +Reproduce with FFmpeg, from `tests/fixtures/local-inspect`: + +```sh +PROMPT='{"3":{"class_type":"KSampler","inputs":{"seed":7,"steps":20,"cfg":8.0,"sampler_name":"euler","scheduler":"normal","denoise":1.0,"model":["4",0],"positive":["6",0],"negative":["7",0],"latent_image":["5",0]}},"4":{"class_type":"CheckpointLoaderSimple","inputs":{"ckpt_name":"synthetic-v1.safetensors"}},"5":{"class_type":"EmptyLatentImage","inputs":{"width":512,"height":512,"batch_size":1}},"6":{"class_type":"CLIPTextEncode","inputs":{"text":"a red fox in snow, detailed fur","clip":["4",1]}},"7":{"class_type":"CLIPTextEncode","inputs":{"text":"blurry, lowres","clip":["4",1]}},"8":{"class_type":"VAEDecode","inputs":{"samples":["3",0],"vae":["4",2]}},"9":{"class_type":"SaveVideo","inputs":{"filename_prefix":"video/synthetic","video":["8",0]}}}' +WORKFLOW='{"last_node_id":9,"nodes":[{"id":3,"type":"KSampler","widgets_values":[7,"fixed",20,8.0,"euler","normal",1.0]},{"id":6,"type":"CLIPTextEncode","widgets_values":["a red fox in snow, detailed fur"]},{"id":7,"type":"CLIPTextEncode","widgets_values":["blurry, lowres"]}],"links":[],"version":0.4}' +PARAMETERS=$'a red fox in snow, detailed fur\nNegative prompt: blurry, lowres\nSteps: 20, Sampler: Euler, CFG scale: 8.0, Seed: 7, Size: 512x512' +PAIRS='{"version":1,"nodes":{"6":{"records":[{"positive_text":"a red fox in snow, detailed fur","negative_text":"blurry, lowres","negative_effective_text":"blurry, lowres","prepend_text":"","append_text":""}]}}}' +COMMENT=$(PROMPT="$PROMPT" WORKFLOW="$WORKFLOW" python3 -c 'import json,os; print(json.dumps({"prompt": os.environ["PROMPT"], "workflow": json.loads(os.environ["WORKFLOW"])}))') +SRC=(-v quiet -y -f lavfi -i "color=c=black:s=16x16:d=0.2") +ffmpeg "${SRC[@]}" -movflags use_metadata_tags+faststart -metadata prompt="$PROMPT" -metadata workflow="$WORKFLOW" -metadata parameters="$PARAMETERS" -metadata comfyassets_prompt_pairs="$PAIRS" core-faststart.mp4 +ffmpeg "${SRC[@]}" -movflags use_metadata_tags -metadata prompt="$PROMPT" -metadata workflow="$WORKFLOW" core-moov-end.mp4 +ffmpeg "${SRC[@]}" -movflags use_metadata_tags -metadata prompt="$PROMPT" -metadata workflow="$WORKFLOW" core.mov +ffmpeg "${SRC[@]}" -movflags use_metadata_tags -metadata prompt="$PROMPT" -metadata workflow="$WORKFLOW" core.m4v +ffmpeg "${SRC[@]}" -metadata prompt="$PROMPT" -metadata workflow="$WORKFLOW" core.mkv +ffmpeg "${SRC[@]}" -c:v libvpx -metadata prompt="$PROMPT" -metadata workflow="$WORKFLOW" core-tags.webm +ffmpeg "${SRC[@]}" -metadata comment="$COMMENT" legacy-comment.mp4 +ffmpeg "${SRC[@]}" -c:v libvpx -metadata comment="$COMMENT" legacy-comment.webm +for f in core-faststart.mp4 core-moov-end.mp4 core.mov core.m4v core.mkv core-tags.webm legacy-comment.mp4 legacy-comment.webm; do + ffprobe -v quiet -show_entries format_tags -of json "$f" # -> video-tags-v1.json "fixtures"[f] = .format.tags +done +``` + +# Local Inspect projection cases + +`local-inspect/projection-cases.json` holds synthetic record sets (prompt, workflow, +parameters and pair-record texts, optional `record_formats`, a video `carrier_text` +and `limited` records) that exercise every branch of the server's inspection +projection: Python's Decimal spellings and exact-construction limits, the 4300-digit +integer limit, duplicate keys, NaN/Infinity, the depth-64 and node ceilings, prompt +node key filtering and the carrier rule. `records: null` stands for an unreadable +file. `expect` pins both graph statuses; `needs` names an interpreter feature a case +depends on. `tests/js/workspace-local-projection.test.js` runs them through +`web/js/workspace/local-projection.mjs`, and `tests/test_local_inspect_parity.py` +asserts that its answer equals `utils.inspection_response._metadata` byte for byte. +All texts are original synthetic content, dedicated to the public domain under +CC0-1.0. + # Inspection parity fixtures `inspection_parity/` holds small synthetic PNGs (`f01b.png` ... `f24.png`, each diff --git a/tests/fixtures/local-inspect/core-faststart.mp4 b/tests/fixtures/local-inspect/core-faststart.mp4 new file mode 100644 index 0000000000000000000000000000000000000000..5ba7aa9cd1c42e1a49161e54bbfb71c0833e53cb GIT binary patch literal 3268 zcmbVOO^6)F74FqWB3g_sjKDhRQ0eI4!|ZJL%k(W zPhB;$Gg=nxV1p07_#&_m(me!SbBZk)1q43$8i6myAvh!?w_phN=6l`KyED5AGAX8} zUcJA1_5P|2!(gGF17Qeyfv9Vy# z?{Lg*fIhN+^^o&v&369>q1@w~fi$_=tV>t@qqJ;1_l7XmUTCZcDZSgdS z1@SC=mdG>&aCVfGN@vZy5LC4DK_Q5T2GO4z7BQ>fh2JXOivfS4w5kI_JohPRSZU1a ztT1PA^(}xjX)K(*2L5ilX*X@F#bVhTx4NB*R{fvb-v`hE*TzC%EbV_~ywu0~`zODC z@S0&Pl9v{Z$FFSN`Q?IfD*F4^pPLW9`}SLsJ(F{lVf%v-{yC*f_k&#W zoTT(4UfhbpLZg;n_YZl&M|d1wu1nLk-qV)orLObP<(HU4T~m4EWzcR^%YtGPa zCo2YFob6!$VoBu%#}T3+Ipm<`ewKt2u9VQqt7N^L%aZ;A(M9TK0j1f=Z)BO)i&Q$~ zkN*U`v=y*n?3!)U*){zbsyvjxFdg%aZJy*vQL|ynG#_e397>45bk-WC(jo_@Wi?Dc z?3>Pd6OgJTycAYKgBVug%8x`)!j73{MZ#m~3`Ckq1vZ-`6IdCW&JD9;HmsWsGtZRN zas=`Xvs84QM%fn~g~LgFKU|zLs%#WLwhZ=I#c% zu1!~yR7XNfe|cm#6%XO`RAq%SN&D(C?VAYNRP+>&f5QROfMc5#iYTIjfGH8)4G04>-9^UuL;V(IlHSQLGIi@(v`s;@iKY{ zeXF$Z;xSFzd?d=YIj1SaB`MXA`685KID=YkwsH_WaF-Z5YrV>iJ!JnY$awUFh0c5l zAo&r0nCnU8_{9ctHNaJ!8@nbU+70t~WgxbFs=X19hb1pc1E!!z8YqpjDC#PxGavbW zbFWe9>>lVm7U*M|pBd=`t*~Kg=?~C)B(OL4P<^pX2Xt|OBQzVCrKV#wJMf+H5hk^e z4`+S%j2vA0`LuLd)uJ3Wf9TRSH4}%yV{F=V9JWafqro;_zrvKyW8pCJCfh~>a@ZOG zl#RnsRAWpS^cPHO`d`i(wtHy5eGt#xs?G|eW)W2Jt=od6AUXqdvo*3 za)({Iz5&ia_#kX#`6L#hMzE~*a?5J10vTzYJ1Z+YJ3Gy?-eR6Mv!cI3r!*s-#Nf$t zjXXk>ect1~Yol$2ZX3O-7ia#!wH?c`7*BaTQNp#xZO0nhcAF)_jl`I#VGqz6%hd#; z`1gvyMI}K9e=MPr^GKr&39sO3U$}Oc`4Qez?m^UMx)5*?}iW6fp-A@RT-u;^7(3xyIgyi}0;0Q>XxA!HFlNS6{(dJujaCmVt{{c|71b z-J<99Bv*8ZL{1ZCr+j64fHLHf*UxfLD`XB)e0V^&+gi1zpocF|Y*A<;75-4W9gEB? zIJp$A9D<8Yvi$KtKgWF*; zN05*Zh#_?GTZlhJ(|8*h{DOG@U8BAK{lEY5$;s~7#}6r~FMWI=zWnL;4@&s%QSBk83BZPQ?i5en5J#5$ZibLP%?=ao6< zzFGoR6#NUia8dUn-MX?5sSt22SkQ$CqA2Q03ev{kH*;^E1;s<=e&6|C=X;&+L{Svg zm!m8cX{;zm6oU9H)_2V~tDB0Vyx`}YgZCu%m;}7@+_CNLPrqCF<%^#lt$uz>|5knU z^6!5Pt!7=FSKTz{s@ZI)dCjscwOwm8jRpf_wF-fS*DtTUI$u}MUtNZz$6YX%(`*!R zU#ga2)aEV2YJeC>nY9)c27^JhA9_5ESW->%&H|lM4P+caCe37+CZeUftj%28RCDgz zHPz$oD0REG*)m#&$`Tfh1h13iCW+YJ>*vOpJ_XU^syrs$@hXxsn(zMG0Y@A)1h(Z^>ObCXW>02~0pRXC&k2Ean2qYCG8oc<9-vmBT!iQ50>b9Wp_O zggGzb8t||a$ToOs#uKNLX5f~@460o|f^FMs7-Q6l!vtGZxIE!*Pug{Z+{_uJH0L6K zdhR+q;SxW0Z8!JTxQ)D0SlHNBt!k(n6pU?Dn?N%%TtGIon`b}}l4o{Z4Kox84S^cM z7TZAmshY}7bnr?3(FaOx`{`f5-96Tv*?K@teg5OQ=)%3H_Y3^?h2e+W58flZa+}Eu z_a5FXara%|pg;d-oTh!yqPQP;fP2=|&)94!6@~tG*S-zN2lnT7C7*>I!A1K7`RaoC zw(J&XPVEt#Slik|E71vO!Z@%$NA&6OrzmsWlLEV09tm9TVvz>y9v%C9!Djn z8H@NQdU>Rlc;$T|+Y#`)LW)U2NahYrERkLaumKud^ znlL4={5`$B4WtgK<%*{q-~K^4)dBqM(L47qD#{dPX-e5TyL#)bit=3W>yKY+_n$m| zOuEw<6B!_RLEj7-dz|9_7OkU2&(m}C(Zm4((sf63jIg&fgwcma=(;bc!+ z;up4NVaqKjySB#q$}8YqAM*&%`CkE^IPk$AHzffq^2fm0-}gMp0KbVj!D79%w*}&Y zKjw^qjZrZPjQaUS63Q@BGfK@g?FMPy^`mrv{Yl#Mgq{jC&CzaHH0cy6QgnagL86m^ zE6NS6rnPQpZp4Ie@S<_8rM-5A#Tgn-)3q?kdQy;t;M~(%i@HXW0~E{9HP`QGt)(h3 zQEE5|CL}EIUa2nJfO`er(vmcfSp=INPts6843kFJVt5&8t#z%g>Bco(%Tf`_un+dS z)+|X0?_friq{WgHVTLEt!7T3-k_{qnZtAddFq-9nyWK3s99>HpGIAxP;P%A^zMQx& zMu`l#4Bh#@SryFZI7pGo7V9pa5pMUoD5JfV_>5m9?oJvR!o zaca#XJB$D07k&Z`pfG6l#R+kQ0y?TD-BB%b_=Xcp8=$&;mT~X{lE2@XmTZMmT z4q3Xw`n(u|`z-kgl3LBlFzUh~l2QD>v#?VM_iLMa zX|uUw^H8A=S^n2b?^uNoV^9Bs)dPjSwu##pg-MqJ2ioVkz<-Jw{cL MbBA}ppS%PA0Qq6w=>Px# literal 0 HcmV?d00001 diff --git a/tests/fixtures/local-inspect/core-tags.webm b/tests/fixtures/local-inspect/core-tags.webm new file mode 100644 index 0000000000000000000000000000000000000000..c6a709d581883c2512b97cce919ad3e5d23705c5 GIT binary patch literal 1718 zcmcIl-)kd99G^t*=!0Sulp=y*^g)g_m!#Jpw;(j>Mk$vxA?Xjbl(^ZQY=&-jmz_mFHN=S9bA~a09EbEL@ zMTUt9GmG$@Wm82u6qn%{D1=u8Xqf~%xXPB7;e-bnNR#`}qzCD;gA6B+%d+NLj&rzP z7Z}UE-prJcvTB*waFV91B0GnrlGB?MWJf9G34WDFffG**5zi!NGxMnCBn*W` zXh5#RU>0OKXd^XPG;2X@1c_m;2|*P(GS)d-;n`DkZ1qALaicT&AC4*L7294Hg1Wh8 zBXS(;)DY`pEFqyG-CmKf+F52r=O()MM1@m8((7ApVGhnf;ui#t(talm1RL%$j0Vp({ob$QaCKSzfd&Xd%%-3 zA6KTE-%`-+QUDqOKX~7FDgXiiyfD&gbz04i-wz&m@D=dKpVu;t?%NbR1FrkPiNFpR s1*0#X{uPV>z~RjCaP4^b$}r5o9B=Jk{D^{2-e6y-pEgf?^CP|QFP$kN8UO$Q literal 0 HcmV?d00001 diff --git a/tests/fixtures/local-inspect/core.m4v b/tests/fixtures/local-inspect/core.m4v new file mode 100644 index 0000000000000000000000000000000000000000..6c59992280136064ac43e9fe12bb629062eb04b2 GIT binary patch literal 2845 zcmbVO-D@0G6rVJ0l~QaT3Kk4FPV?ZyW-~MUk!0yW+ccPBiiK(rVjb?z+}Rm-zB2dj zS4*IRg7^pY!H2$yAmXEfplzX*pbtI?7W6>`Q4|FQDM%ZCXLdK)Yzm4uxpU9g{oQlV zIrrXG6h-x9FTGG+Qx)ZaLSrD32;F3D$50gIaWCT>x(AWVB*qU9AKKo2|J&uCKmGCE z%E#9W->COa{`Py!nInuP`W9iZGbFaB=y$*|K_OZ3&hx zcc55G(q70tshYZ8nl*K^0--NuTAQ2ebUMX$;PND7aWTmnbF@m)mr)3tB$YuDi<;`N zI&&;T&A4ZkRF~Jo#A#YaO|R)Hi&@wc+|s+Hn%*^xk{WT#=Ur8_>Y(P;RP-js_OVz5-}T74OM153Zzz_pMoo4G8d zUDWM*zyvK4WIW$jhX)N`)}c#M9@~v1g|;tDF}2Biz_!eaJ|OKVh{3AD;W2kw(kkoZ zX2vL_85cgLXO6wIUD9Wc0LLU}^3*D; zL5d`yAdo{~u?^&(vZ-7{1s~&Yy`hx0AO7|0XNRiCw{BBWpLu5{JbUxut(?Abw)^Jx z?bnH(xWVMvn|H4DY3DW2kUw@mN|H8YVbu0rj3aIGCoo&eghKzjZ4U$S7=L=#@=??Q z9JKckuO^hQ$Y#FgztxYQj{i@ty3%4U!G77`@j6~rFQGV{riM6-D(AH z(?9)(J@dldPZr;O>Z{33}~*Jdf$gK=B-G26?eg z(nQ+b-*}j4WzdRpRV!(=tC|xsA#A*CT&ro%Ut&>;!qW;`5T`9E$U<=LYPI=-MmGo~ zre4q-uc6fzil9V)h8^Rk1OjjMrwhmDZcf*##wlnKybQJ4x>nW- z`sIR_CL)kQ8|rnf+84#VfxD_Nn(vE3+~u*fai=$O%L<7%Hw!=+-_4TGoo1TgK3z>1 zB62C9&F$G6^m6FB=*80KGH_CbKJ;azptqWPb3S)wG-jQ^mA+Q1 z7-m89^EY9@)R}aATLf1q3Itr;2fVa;{^EUAz^lh?7nM zPpM=b)G<5SH=dIrZ literal 0 HcmV?d00001 diff --git a/tests/fixtures/local-inspect/core.mkv b/tests/fixtures/local-inspect/core.mkv new file mode 100644 index 0000000000000000000000000000000000000000..3cb0e60eee7f2d537395ae4c0fc1b5145703b274 GIT binary patch literal 2489 zcmb_eZEO=|9KVj2Fu#Dr7h*7;2PVdJ-Cett(Rc~#T3}&g%eoEY#LLsWr|q%p-R+*I zUCUgu+X|aP0tp5pA`#~obs-o7s8crLOGfmA7jX zlW;g(tzRp%aS4y~|9vofE=-lGkL-HxiHmQKtc-4I9opZ5@~LnrAO$2bxTA5=mb`xO zz>2mnmJXj;vgWZ@c3s+1NX4EX+oYGRV*YG8JXL!1nd_@g&RIQr(e;r}#tI*N`rDEd z=iWW$a>D6jBRNwWbGf|-x|P=U_?Eib=0sasxnaq*RrBn)~M9DR$=O$$D)ek)Ym3_8h6j^x7>qi_@21@>~xIdmT*hiW9G@NhoACs%QIK6 zd?rl)dGjX6q8mFsRa$$arG}7LhdMo!o{8R!kMC{oXlrfnjEzRG4hSJ3G9ai1B1EA@ z8w(L(TLNWm13Q9W&`mo>36mfgYeHm$Umz6QfCNRqpr+D7q#*z+Qr1vRlw}M!S5_x# z25a6rBAAwwMFwzc*tB#4ViMELS{gQlNRLn__{E)m!M2D_^*kK+2;nkh;xw9vHk2U) zQEXC0&!V&^S#ob8eAua8BnK;-L5mt04?~Q&aQw_X~^b80L2>FG|rhv4jF(pbK897fDx&o|#{_sOrc>OTn0P{r; zrM`2}U*-%ia8~5Fg83hlyub=J%vt&`SntZ|&OvBALpOVw!fy}5ej_bxXcz2I`(5XV)WT)u2}_RX*2KYsY#<*jEA z`!DgA*ImECX!#mmwH(Y#;dPRYFLGXCd)1w zI7NB5I#d%BgY^JrD77QCwf+750k4P*WCkoJUCV9>WN6j^8Ox>+Arj$Lltij5K`*A{ z5Hw)Yu+&~zihxf%GLcavSQZPRh**%M5TC_z1{ZjeOTuacZ<8XZ!c%cH8EOQS;K65E zF)55u0u9AMW*W;4uN<4M2Vdw-_4TT0xrLa)@va$#!AZ@p0VmT=JC5cb5 z0=G%8ZZnB8L7%NysT9Fft_kuq6=Lhp4*%aoL}xBU7a+`}4ZV1Mj%p*%#nX+RwwwH~-{T F{{>4ZIzRvb literal 0 HcmV?d00001 diff --git a/tests/fixtures/local-inspect/core.mov b/tests/fixtures/local-inspect/core.mov new file mode 100644 index 0000000000000000000000000000000000000000..a44a6123f27b004f1b72e783df9c2ff8b19d2aea GIT binary patch literal 2849 zcmbVO&1)T16rVJ$gj#GC0lVm89J6pSd6}8_k@V4lwrOgLDHJM2h;_Jc=Ds)Mo3G5h z@2e$PLBYSEi!S;H6mi>?wgrWNYr%>xL=Z*MofM>vzcceDc}WV2!<)J1>;CS!=bU@) zQxs*cEl265R8>Veq7(uCBhhP*cxeUxB5^T_{$PbQJQoR4v1( zE?9G*y?im}{GA#@lvP^>`~x z+^%glji#Zpn1v(3ZDUw%8bi~psu8z+K2$}o1!_r6#RyaJbu!O3D+VUvqej7ydk!I* znBinB?r_^|sIH$S5pyurRAt7)Fc6@Yhs&NTLAaZdZNLTQ-ArO`Tc&QBYMTk^q@o+7 zWKjTYrcTms3oh*iOO<|x83Yt2NtgLxoIRNc0~fxO0z-|nd<{29&@)R?V3SuW{g6bap7Zn<~qCE zC4J`FZq`<#7UD|V0%Kd%ilJ`OW^AL<0GSeSKC+?RI0J!@Jhf|TkRnMa2;>l0Y!msX zY$~@EMLEeo`ADhmJpb$WFOD_lwjV<@bN=ppc;VsmM>&1(!tmpr#~%_sb&tsl55K%! z(C+)7A)lFvlB5q=81;P*zB%!rEoFr#CeOUIuR*XbOuvDgbajHbRrBHPf-^`jhg(oMYt@&MG!LUj`8 z<%>+RT^wcJD{xMGsI0LaVVxMy13iTk&cm^-!&)ZD3>l>xk{Zce`gV< zHIVeuGc~sk`$n~5R!qZE!=OdosL&PPQ#%xh8mv~PJmvV#kIJhZjGsPv@6km?nWl(N zE8Az+?z}UlyyXA()7RRgXHTEf+}V_g6eD>>?wg96#DN$7xXIX#Fy^uL4?RyZ zjNe3@#{7A$%qVeDt}hhj*G3B|H%4{-3K^y7wJBBQEOCP*>$byW0RANLJfB3Mf&`aAI^$An;x>UAR8?a=NL-NfxmXm>!RlK)?(gNY^5G8EVZ9 zt)}V54P8qU5y+qq^@i3cL^1E6cNL2n#l3w^U9Ses)Zkz_(6^!g$6Rit3Fo_ljScV;qX_-i12 ztywoMUGwuhV$8IdbbUtzHz^7P+&BchvVQsM+k7Zj^E~czS3-;+GhFGO+D?YJJ5&)T z107eZWI;&M9%auBgLE8PGmp;F|5%2u%xt9N@*Uu6dJ*dKA=K;VS1)p^zkPleMS|Md zK+)yN?z235NWGQ&cjge%HP+{OZ`_B}#wMxMoD^*>7$O+?|DeU)nsfEqueqgFj^wYhOiBSBTy4xbFZ z%+6$4h@#+Mkb{SKj!3Sa%tIn9_zxKLA^}kpy-6;~`un=4vy;t$g2ncH??-*#S5<2m z25akKCRG|6#vX$pK8N>xJI>~8!!QoEb0Hvm76)7d-aW8?d;80umVf)^m&Yq#-}a?^DM7Bj>B5@`Gz&mAy}zFVfO9I%WuxivGZ4#pcx1sf~7PY zMxw2mV_EeX$8zSu47JV{XJ^;f*K2(lh&19!EzLW#6r~pGID$@^X_+Q!k@>vEeb;8W zXuEY5h*p&PUDsZ;7A?jT9u1XntwDX!8rXK7#lj86fT>;!*lCujAx!b><$-J0EEwU3 z#d075k38D2@N%AXgljjLALePyJ($`|=OT)v0=6(%2z(91zaG05QsBXjG!d?2Pun(Y zbEUmZb!A2xrGx94m$utVXm`e8I?Q1~j-oW}@(_f#!&8y;kxRv}Sdtepd>M01dB{YI zJmOFSzZLazKJ@%F&bS8cqeMFAGJy*y=A3leIgf=xv07d>1TF&?z4Ca#GulPVYe}vs zh|EQ?uXQ0ip>9EzW+L%AX$EP@%wXCTL-=-`d257vu}nZR<%>l4J?+j}lxEJUq`6Qb z%yZv+v0akqzU$|07PnAW+7<}csn;xaoi^iIwFc0P92bxe+{PI&lopvg$7F^kVIa^$ z_~O^mf9j@j6BB$yeD;Y^-+uo0ANTe*j%_`pp+5ilsp!J}=MM_}_JzTx+Ydh?eCjsW z7w$j0S>pbOz`;NGUYw?V@S?aM27qI0@2?=Xj46ZucGgP-*t_U+JDQWQJveAzpk7@F zo4Q-X+&iXNdD|M}m1rfY1iR>S#6LRv4C9mtv;uWfL<)zySY!de`$jqMIUWa+gU8}v z_ud__4;k-1D(q&$Aj%62KXhy>oZwYD#w$DtB0)M+-=dLiAjkS~;dDIBQ$Z;^8DLt- z7>neGdwIl4eClJRTM_VkN~@|Nq;r>UEItwO(1uh})*a#$o= zbi7x7)7drdt4klgfirPY}vS09bbfHIl1TWyB5v*t; zNnZFm315D}+3_{9H{O8k#z;nh;(rHtc-Mz~{FD^1C?5-|7kh!`fM;Vtuvo8+j{wFd z=UGNws@DD^vYfM1cXUb)PvT7>V{&Ss7wxJS*xZvlevE?c`*B=n!_064XWCq3=0?-3 zH_b%=KjKPxc*g|jruo(t9%mS|ra9d-Ws>!@D)p2Q0kG3x)4f4l#{%KEJ3tp|gsall zOK?Ym*)&D3a;5xG1SJK2lIAgwiWq@NQmIHCmkxZyc-$aayw)`5pkS?mm!(Q-*(W8i z8x=bd9o$PiM zqjxr^hKUY^mi|oNt|{IY2&>W@l_?@2=!8hiXuxo_$na8`;I0_!GSDGR=WPd)uy{RQ zkS(tL&{Oh8QLiElsxcwV((0wl?}&k3PUxS7S2m&{f?d<(c=`j@P6v2nm`c+1X*|K2 z%ZRwWati!NW>6kA#9k}HqT2r>EIqaPa2k*HI-ay5y@^p4CPuk>e)*!9xUX^mX*Q8< zz34iM*yp1;pn-DA{Lf7Ee1-Q#aoxUBu{KIVW9?g3Ntd!kw6yzD5MjY*UBm|kAAAw<4}1~z+?k}iaR(H{AwT#0 zec!p~o^$7z_;vsN#04ji_zk{4I}av)amErG?HVqW6I~~m=s3agY)cIw_8f^mk$Ts@ zo7%aUz0rPh{)1w6%en4JPc7YC1L=#@&LyxNUHU}LZfykOp7?Zj2aFp>j(zXppmQ$V z|MA1J+CIK_YxD8MY^rr>L8=xfCnB*(EE<1lZbTGb>&+4`1(y8Y%|UMIJtsgv?QK7D zw3%*vmAUF5{-dj#6XGRmgS&v1Mr$4}(=8TgYm ziPF5iZeoyg{EJzsY--rb@!=d#bhBbp&yQlPz-kKA)W#;$#-kv}MHQCQ5w@mYY+jej z9vZF;p{QKKiYEb!ZdheWb2${O8-z0Vn2S+16bzP`^Ev(`I7C+g*EA@xNtHQ)dUB}N zu`2DWr-o`xvaxRGNm)_{5vN>Jvk6du>9gx(3CnAyL3De;kQ8iX2`d6bjn)8s?ETHr zb=@vuo5;^sV-YG9G0;-Og37o|K$a%NP|rgfvB3m!IbPx{tZ+r6&JmrXx={;r3brMp zF>S@`0$C#_I1dgaw(8PCwf_TcL91BSdYIFUnuVcYuv1@_X5umX{1m!>+0@xvELJx&&(~&FQ$d=m1uOv{Y2u?!GoSjeq5Ptf9Ifn z-$AGq3ZnM{$3rND&c8cn=g_SHdMLDo#?koM6ThM{ lgqS+xude&6uN_vicifvhZ+z&W+pojB^rwx7FaPK~@E1%!QfvSK literal 0 HcmV?d00001 diff --git a/tests/fixtures/local-inspect/projection-cases.json b/tests/fixtures/local-inspect/projection-cases.json new file mode 100644 index 0000000..657f64d --- /dev/null +++ b/tests/fixtures/local-inspect/projection-cases.json @@ -0,0 +1,849 @@ +{ + "version": 1, + "about": "Synthetic record sets for utils/inspection_response.py _metadata parity with web/js/workspace/local-projection.mjs projectMetadata. records null = an unreadable file. record_formats default to original-text for each record. expect pins both graph statuses; needs names an interpreter feature the Python side must have.", + "cases": [ + { + "name": "unreadable_read", + "records": null, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "absent" + } + }, + { + "name": "absent_records", + "records": {}, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "absent" + } + }, + { + "name": "prompt_and_workflow_parsed", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}", + "workflow": "{\"nodes\":[],\"version\":0.4}" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "parsed" + } + }, + { + "name": "all_four_records", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}", + "workflow": "{\"nodes\":[],\"version\":0.4}", + "parameters": "cat\nNegative prompt: dog\nSteps: 20, Sampler: Euler, Seed: 7", + "comfyassets_prompt_pairs": "{\"version\":1,\"nodes\":{}}" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "parsed" + } + }, + { + "name": "parameters_only", + "records": { + "parameters": "cat\nSteps: 20" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "absent" + } + }, + { + "name": "number_spellings", + "records": { + "workflow": "{\"nodes\":[],\"a\":[1e3,1e0,-0.0,6.50,0.0100,1.0e2,0e5,0.0000001,0.000001,1E+3,1e-0,0.5e1,0.05e1,0.0e0,1.5e+300,123e-8,2.5e-3,1.0,-1.50,0.0,100.0,1234567e-6,0.5E-6,0.5E-5,1e22,0.000000,-0.000000e-3,7.0e-6,7.0e-7,12345678901234567890.123,9007199254740992.0]}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "parsed" + } + }, + { + "name": "integer_spellings", + "records": { + "workflow": "{\"a\":[9007199254740991,-9007199254740991,9007199254740992,-9007199254740992,10000000000000000,123456789012345678901234567890,0,-0,42,-7,100]}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "parsed" + } + }, + { + "name": "nonfinite_literals_are_null", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"n\":NaN,\"i\":Infinity,\"m\":-Infinity},\"_meta\":{\"n\":NaN}}}", + "workflow": "{\"a\":[NaN,Infinity,-Infinity],\"b\":NaN}" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "parsed" + } + }, + { + "name": "exponent_edges_valid", + "records": { + "workflow": "{\"a\":[1e999999999999999999,1e-1999999999999999997,0.0e1000000000000000000,1.5e999999999999999999,0.1e-1999999999999999996]}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "parsed" + } + }, + { + "name": "exponent_edge_invalid_1e1000000000000000000", + "records": { + "workflow": "{\"a\":1e1000000000000000000}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "exponent_edge_invalid_15e999999999999999999", + "records": { + "workflow": "{\"a\":15e999999999999999999}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "exponent_edge_invalid_1e_1999999999999999998", + "records": { + "workflow": "{\"a\":1e-1999999999999999998}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "exponent_edge_invalid_1_5e_1999999999999999997", + "records": { + "workflow": "{\"a\":1.5e-1999999999999999997}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "exponent_edge_invalid_0e1000000000000000000", + "records": { + "workflow": "{\"a\":0e1000000000000000000}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "exponent_edge_invalid_1e9223372036854775808", + "records": { + "workflow": "{\"a\":1e9223372036854775808}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "exponent_edge_invalid_0_0e1000000000000000001", + "records": { + "workflow": "{\"a\":0.0e1000000000000000001}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "exponent_edge_invalid_1e99999999999999999999999999", + "records": { + "workflow": "{\"a\":1e99999999999999999999999999}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "int_4300_digits_kept", + "records": { + "workflow": "{\"a\":7777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "parsed" + } + }, + { + "name": "int_4301_digits_malformed", + "records": { + "workflow": "{\"a\":77777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777}" + }, + "needs": "int-max-str-digits", + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "int_negative_4301_digits_malformed", + "records": { + "workflow": "{\"a\":-77777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777777}" + }, + "needs": "int-max-str-digits", + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "string_escapes", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"t\":\"é😀\\u0000\\u001f/\\\\\\\"\\b\\f\\n\\r\\t\",\"raw\":\"é😀 — 
\"}}}", + "workflow": "{\"s\":\"\\u00E9\\uD83D\\uDE00\\uD800\\uDC00x\\u0000\\u001f\\/\\\\\\\"\\b\\f\\n\\r\\t\\uD800\\u0041\\uDC00\"}" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "parsed" + } + }, + { + "name": "lone_surrogate_in_raw_text", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"t\":\"𐏿\\udc00\"}}}", + "parameters": "\ud800 lone\nSteps: 1" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "absent" + } + }, + { + "name": "keys_with_escapes_proto_and_order", + "records": { + "prompt": "{\"10\":{\"class_type\":\"B\",\"inputs\":{}},\"3\":{\"class_type\":\"A\",\"inputs\":{}},\"__proto__\":{\"class_type\":\"P\",\"inputs\":{\"constructor\":1}},\"constructor\":{\"class_type\":\"C\",\"inputs\":{}},\"\\u0000\":5,\"\":[]}", + "workflow": "{\"10\":1,\"3\":2,\"b\":3,\"a\":4,\"__proto__\":{\"x\":1},\"\":{}}" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "parsed" + } + }, + { + "name": "duplicate_key_top_level", + "records": { + "workflow": "{\"a\":1,\"a\":2}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "duplicate_key_nested", + "records": { + "workflow": "{\"a\":{\"b\":[{\"c\":1,\"c\":1}]}}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "duplicate_key_in_dropped_prompt_meta", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"a\":1},\"_meta\":{\"t\":1,\"t\":2}}}" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "duplicate_key_in_prompt_inputs", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"a\":1,\"a\":1}}}" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "duplicate_key_after_deep_nest", + "records": { + "workflow": "{\"x\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]],\"a\":1,\"a\":2}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "workflow_depth_64_kept", + "records": { + "workflow": "{\"a\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "parsed" + } + }, + { + "name": "workflow_depth_65_limited", + "records": { + "workflow": "{\"a\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "limited" + } + }, + { + "name": "prompt_inputs_depth_64_kept", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"v\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]}}}" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "absent" + } + }, + { + "name": "prompt_inputs_depth_65_limited", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"v\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]}}}" + }, + "expect": { + "prompt_graph": "limited", + "workflow_graph": "absent" + } + }, + { + "name": "prompt_dropped_meta_depth_65_limited", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{},\"_meta\":{\"v\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]}}}" + }, + "expect": { + "prompt_graph": "limited", + "workflow_graph": "absent" + } + }, + { + "name": "prompt_non_object_nodes_kept", + "records": { + "prompt": "{\"1\":5,\"2\":[1,2.50,\"x\",null,true],\"3\":\"s\",\"4\":null,\"5\":{\"inputs\":[1],\"class_type\":null,\"other\":{\"deep\":[NaN]}}}" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "absent" + } + }, + { + "name": "prompt_root_array_malformed", + "records": { + "prompt": "[]" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "prompt_root_string_malformed", + "records": { + "prompt": "\"{}\"" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "prompt_root_number_malformed", + "records": { + "prompt": "5" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "prompt_root_null_malformed", + "records": { + "prompt": "null" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "workflow_root_array_malformed", + "records": { + "workflow": "[1,2]" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "workflow_root_nan_malformed", + "records": { + "workflow": "NaN" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "empty_text_malformed", + "records": { + "prompt": "", + "workflow": " " + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "malformed" + } + }, + { + "name": "bom_malformed", + "records": { + "workflow": "{}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "malformed" + } + }, + { + "name": "trailing_data_malformed", + "records": { + "workflow": "{\"a\":1} x", + "prompt": "{}\u0000" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "malformed" + } + }, + { + "name": "python_whitespace_only", + "records": { + "workflow": " \t\r\n{\n\t\"a\" :\r 1 }\n", + "prompt": "\u000b{}" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "parsed" + } + }, + { + "name": "syntax_errors", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{},}}", + "workflow": "[1,]" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "malformed" + } + }, + { + "name": "bad_escape_and_control_char", + "records": { + "prompt": "{\"1\":\"\\x\"}", + "workflow": "{\"a\":\"\u0001\"}" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "malformed" + } + }, + { + "name": "number_grammar", + "records": { + "prompt": "{\"1\":01}", + "workflow": "{\"a\":1.,\"b\":.5}" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "malformed" + } + }, + { + "name": "nodes_not_list_counts_zero", + "records": { + "workflow": "{\"nodes\":{\"a\":1},\"definitions\":{\"subgraphs\":[{\"nodes\":5},7,{\"nodes\":[1,2]}]}}" + }, + "expect": { + "prompt_graph": "absent", + "workflow_graph": "parsed" + } + }, + { + "name": "limited_workflow_record", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}", + "workflow": "{\"nodes\":[],\"version\":0.4}" + }, + "limited": [ + "workflow" + ], + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "limited" + } + }, + { + "name": "limited_workflow_without_text", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "limited": [ + "workflow" + ], + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "limited" + } + }, + { + "name": "invalid_workflow_format", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "record_formats": { + "prompt": "original-text", + "workflow": "invalid" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "malformed" + } + }, + { + "name": "reserialized_without_carrier_is_limited", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}", + "workflow": "{\"nodes\":[],\"version\":0.4}" + }, + "record_formats": { + "prompt": "reserialized-json", + "workflow": "reserialized-json" + }, + "expect": { + "prompt_graph": "limited", + "workflow_graph": "limited" + } + }, + { + "name": "unknown_format_dropped_but_projected", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "record_formats": { + "prompt": "weird" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "absent" + } + }, + { + "name": "non_text_record_dropped", + "records": { + "prompt": 5, + "workflow": "{\"nodes\":[],\"version\":0.4}" + }, + "record_formats": { + "prompt": "original-text", + "workflow": "original-text" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "parsed" + } + }, + { + "name": "format_without_text_is_malformed", + "records": {}, + "record_formats": { + "prompt": "original-text" + }, + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "unknown_record_keys_ignored", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}", + "Software": "ComfyUI", + "WORKFLOW": "{\"nodes\":[],\"version\":0.4}" + }, + "record_formats": { + "prompt": "original-text", + "Software": "original-text", + "WORKFLOW": "original-text" + }, + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_prompt_string_workflow_object", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}", + "workflow": "{\"nodes\": [], \"v\": 1}" + }, + "record_formats": { + "prompt": "original-text", + "workflow": "reserialized-json" + }, + "carrier_text": "{\"prompt\":\"{\\\"1\\\":{\\\"class_type\\\":\\\"A\\\",\\\"inputs\\\":{\\\"x\\\":1}}}\",\"workflow\":{\"nodes\":[],\"v\":1}}", + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "parsed" + } + }, + { + "name": "carrier_prompt_object_projected", + "records": { + "prompt": "{\"1\": {\"class_type\": \"A\", \"inputs\": {\"x\": 8.0}, \"_meta\": {\"t\": \"x\"}}}", + "workflow": "{\"nodes\":[],\"version\":0.4}" + }, + "record_formats": { + "prompt": "reserialized-json", + "workflow": "original-text" + }, + "carrier_text": "{\"prompt\":{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":8.0},\"_meta\":{\"t\":\"x\"}}},\"workflow\":{\"nodes\":[],\"version\":0.4},\"title\":\"n\",\"extra\":[1e3]}", + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "parsed" + } + }, + { + "name": "carrier_root_array_fails_all", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}", + "workflow": "{\"nodes\":[],\"version\":0.4}" + }, + "carrier_text": "[1]", + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "malformed" + } + }, + { + "name": "carrier_duplicate_key_fails_all", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "carrier_text": "{\"prompt\":\"{}\",\"prompt\":\"{}\"}", + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_other_member_too_deep_limited", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "carrier_text": "{\"title\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]],\"prompt\":\"{\\\"1\\\":{\\\"class_type\\\":\\\"A\\\",\\\"inputs\\\":{\\\"x\\\":1}}}\"}", + "expect": { + "prompt_graph": "limited", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_other_member_depth_64_kept", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "carrier_text": "{\"title\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]],\"prompt\":\"{\\\"1\\\":{\\\"class_type\\\":\\\"A\\\",\\\"inputs\\\":{\\\"x\\\":1}}}\"}", + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_prompt_object_depth_shift", + "records": { + "prompt": "x" + }, + "record_formats": { + "prompt": "reserialized-json" + }, + "carrier_text": "{\"prompt\":{\"1\":{\"class_type\":\"A\",\"inputs\":{\"v\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]}}}}", + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_prompt_object_depth_shift_limited", + "records": { + "prompt": "x" + }, + "record_formats": { + "prompt": "reserialized-json" + }, + "carrier_text": "{\"prompt\":{\"1\":{\"class_type\":\"A\",\"inputs\":{\"v\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]}}}}", + "expect": { + "prompt_graph": "limited", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_prompt_string_invalid_json", + "records": { + "prompt": "{", + "workflow": "{\"nodes\":[],\"version\":0.4}" + }, + "carrier_text": "{\"prompt\":\"{\",\"workflow\":\"{\\\"nodes\\\":[],\\\"version\\\":0.4}\"}", + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "parsed" + } + }, + { + "name": "carrier_prompt_string_too_deep", + "records": { + "prompt": "x" + }, + "carrier_text": "{\"prompt\":\"{\\\"1\\\":{\\\"class_type\\\":\\\"A\\\",\\\"inputs\\\":{\\\"v\\\":[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]]}}}\"}", + "expect": { + "prompt_graph": "limited", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_workflow_number_malformed", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}", + "workflow": "5" + }, + "record_formats": { + "prompt": "original-text", + "workflow": "original-text" + }, + "carrier_text": "{\"prompt\":\"{\\\"1\\\":{\\\"class_type\\\":\\\"A\\\",\\\"inputs\\\":{\\\"x\\\":1}}}\",\"workflow\":5}", + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "malformed" + } + }, + { + "name": "carrier_prompt_nonfinite_malformed", + "records": { + "prompt": "NaN" + }, + "carrier_text": "{\"prompt\":NaN}", + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_prompt_bigint_malformed", + "records": { + "prompt": "123456789012345678901" + }, + "carrier_text": "{\"prompt\":123456789012345678901}", + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_prompt_decimal_malformed", + "records": { + "prompt": "1.5" + }, + "carrier_text": "{\"prompt\":1.5}", + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_missing_member_malformed", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "carrier_text": "{\"workflow\":{}}", + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_without_graph_records", + "records": { + "parameters": "cat" + }, + "carrier_text": "{\"parameters\":\"cat\"}", + "expect": { + "prompt_graph": "absent", + "workflow_graph": "absent" + } + }, + { + "name": "carrier_with_limited_workflow", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "limited": [ + "workflow" + ], + "carrier_text": "{\"prompt\":\"{\\\"1\\\":{\\\"class_type\\\":\\\"A\\\",\\\"inputs\\\":{\\\"x\\\":1}}}\"}", + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "limited" + } + }, + { + "name": "carrier_invalid_workflow_format", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "record_formats": { + "prompt": "original-text", + "workflow": "invalid" + }, + "carrier_text": "{\"prompt\":\"{\\\"1\\\":{\\\"class_type\\\":\\\"A\\\",\\\"inputs\\\":{\\\"x\\\":1}}}\",\"workflow\":null}", + "expect": { + "prompt_graph": "parsed", + "workflow_graph": "malformed" + } + }, + { + "name": "carrier_non_string_text_malformed", + "records": { + "prompt": "{\"1\":{\"class_type\":\"A\",\"inputs\":{\"x\":1}}}" + }, + "carrier_text": "", + "expect": { + "prompt_graph": "malformed", + "workflow_graph": "absent" + } + } + ] +} diff --git a/tests/fixtures/local-inspect/tiny.jpg b/tests/fixtures/local-inspect/tiny.jpg new file mode 100644 index 0000000000000000000000000000000000000000..da7ad9c4b02eca90f1d873e521a9cddf0bf8d25c GIT binary patch literal 1071 zcmex=@qlR8&*|0wATLqN1gx1tdXC4Gk?VJ$*fGZ7pqmeIp|y8yibYH#d*4u=M{&7$!0> zLI48{u(Po;a4@qlGBATAV4@5RKyenJga88*0}B%;3llR`j**F(L6F5zG0@StX=7s{ z>p>?rA>m6(n-UjJ6a;#JnT3sm4N0va1B)VSqmZGINzEh-82N(o77!()`m>Go_n1mRZg&6+dV&DN9z$C~l$Y9SPx+Uo1!^Tb(z9o|D-xyqd zx5$`lMnU9snc2^}qJ$ zK=k5Uf$w>zp1o&iE|}24$?K-Jv~TkCAEti2?-y4e*?3_?TjPebhOS+?ml!W|oA|0( z->5kK^>W68gZpUaMu4xYV-sp`Y6Id>ljzx=(nyXcYnx^23rJLYqnt}}dUb8W}* zDJS-rg_~C#`TB6v&fUjUHa*fln33vhxZ+B|5&tK9zvNd;f8`!_QB_S!=Y>^C=b7Wx z{E|QJn3#M$d(G?<)0zy`0LhCo-xdaJxM9a8pwPMVYTWe~KPNp;7EHSBTzcU8Pf62n zPuz5?`S!2cBF3yU^GciPaf8+W8B{u!w(}(}I8uaU&sYjW<(db&U zUrB107RQz14++eu9_d$m6MOU=m(Z}0sLKEH_}*EeG_HrrjTo_ z@hR{2m@PN2VClZUYkulcMX!vBysQCR+@A`qFWs0I<8mmz?8mw_2Pe(-+k5kKH!!&b zRWEAh9!IMNo&vElN8FTJSBJPnk6lVEIrjO^G#4I`hIcmqe7ueI%2LS zNjF&6ty|#nwC|goQNHhY|A3-}+aLc}{C*ODXm$MJXa1MotV+$@^D_LkGoxx&?5v9| zibB2<+9r801=wy|`z>xO|B)FCQYju`E7(0&tPiv|y!lRPMZ$#@CDZ<>iq3WHw@h3e hQ7}Qd=MuypcanZCuiUa@iWOtugc&yyKhOMs695vFwzvQQ literal 0 HcmV?d00001 diff --git a/tests/fixtures/local-inspect/tiny.webp b/tests/fixtures/local-inspect/tiny.webp new file mode 100644 index 0000000000000000000000000000000000000000..3e266983ce3fbb6b1766609f87446e6347cc7c8e GIT binary patch literal 470 zcmV;{0V)1cNk&G_0RRA3MM6+kP&gpM0RR9n3jmz~DkuOL06y(%tWc??q#~-Q0Wckj zYyh1zZ8%S$aUyVrEcwHW7f^hv@8)?G|1;cn)Ee|g{ebkl{VVuw{iF1^DUA6sAKQ)P zOkEv&SHj^|HK>{q@K-i4>&+eupaA~#Crro2TWwu?0HKC`iVg9&#Qm&E#6RbJx}viB zMT%Lo2~NC+!tlFwPlVj{ni9YCcMwU_Rd(n7BK#$hk3vDlKdPJ}qFEpP1XultKf|a- zzy5+N{=^-Osm4rOZgaIF{tQ3#$04!8`rC;}pQ_crBwXBJFm&u6HCE3CbOlNz{i?l2 zv7#vqe9dMsXYqt4-YeiA9lxFo~8;{NZ86);foecRidg1jRDt4 z;(tH(&`>`k(FSY6FhPGjzL;M1& literal 0 HcmV?d00001 diff --git a/tests/fixtures/local-inspect/video-tags-v1.json b/tests/fixtures/local-inspect/video-tags-v1.json new file mode 100644 index 0000000..964a1ed --- /dev/null +++ b/tests/fixtures/local-inspect/video-tags-v1.json @@ -0,0 +1,60 @@ +{ + "generator": "ffmpeg -show_entries format_tags -of json over tests/fixtures/local-inspect, see tests/fixtures/README.md", + "fixtures": { + "core-faststart.mp4": { + "major_brand": "isom", + "minor_version": "512", + "compatible_brands": "isomiso2avc1mp41", + "prompt": "{\"3\":{\"class_type\":\"KSampler\",\"inputs\":{\"seed\":7,\"steps\":20,\"cfg\":8.0,\"sampler_name\":\"euler\",\"scheduler\":\"normal\",\"denoise\":1.0,\"model\":[\"4\",0],\"positive\":[\"6\",0],\"negative\":[\"7\",0],\"latent_image\":[\"5\",0]}},\"4\":{\"class_type\":\"CheckpointLoaderSimple\",\"inputs\":{\"ckpt_name\":\"synthetic-v1.safetensors\"}},\"5\":{\"class_type\":\"EmptyLatentImage\",\"inputs\":{\"width\":512,\"height\":512,\"batch_size\":1}},\"6\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"a red fox in snow, detailed fur\",\"clip\":[\"4\",1]}},\"7\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"blurry, lowres\",\"clip\":[\"4\",1]}},\"8\":{\"class_type\":\"VAEDecode\",\"inputs\":{\"samples\":[\"3\",0],\"vae\":[\"4\",2]}},\"9\":{\"class_type\":\"SaveVideo\",\"inputs\":{\"filename_prefix\":\"video/synthetic\",\"video\":[\"8\",0]}}}", + "workflow": "{\"last_node_id\":9,\"nodes\":[{\"id\":3,\"type\":\"KSampler\",\"widgets_values\":[7,\"fixed\",20,8.0,\"euler\",\"normal\",1.0]},{\"id\":6,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"a red fox in snow, detailed fur\"]},{\"id\":7,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"blurry, lowres\"]}],\"links\":[],\"version\":0.4}", + "parameters": "a red fox in snow, detailed fur\nNegative prompt: blurry, lowres\nSteps: 20, Sampler: Euler, CFG scale: 8.0, Seed: 7, Size: 512x512", + "comfyassets_prompt_pairs": "{\"version\":1,\"nodes\":{\"6\":{\"records\":[{\"positive_text\":\"a red fox in snow, detailed fur\",\"negative_text\":\"blurry, lowres\",\"negative_effective_text\":\"blurry, lowres\",\"prepend_text\":\"\",\"append_text\":\"\"}]}}}", + "encoder": "Lavf63.1.102" + }, + "core-moov-end.mp4": { + "major_brand": "isom", + "minor_version": "512", + "compatible_brands": "isomiso2avc1mp41", + "prompt": "{\"3\":{\"class_type\":\"KSampler\",\"inputs\":{\"seed\":7,\"steps\":20,\"cfg\":8.0,\"sampler_name\":\"euler\",\"scheduler\":\"normal\",\"denoise\":1.0,\"model\":[\"4\",0],\"positive\":[\"6\",0],\"negative\":[\"7\",0],\"latent_image\":[\"5\",0]}},\"4\":{\"class_type\":\"CheckpointLoaderSimple\",\"inputs\":{\"ckpt_name\":\"synthetic-v1.safetensors\"}},\"5\":{\"class_type\":\"EmptyLatentImage\",\"inputs\":{\"width\":512,\"height\":512,\"batch_size\":1}},\"6\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"a red fox in snow, detailed fur\",\"clip\":[\"4\",1]}},\"7\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"blurry, lowres\",\"clip\":[\"4\",1]}},\"8\":{\"class_type\":\"VAEDecode\",\"inputs\":{\"samples\":[\"3\",0],\"vae\":[\"4\",2]}},\"9\":{\"class_type\":\"SaveVideo\",\"inputs\":{\"filename_prefix\":\"video/synthetic\",\"video\":[\"8\",0]}}}", + "workflow": "{\"last_node_id\":9,\"nodes\":[{\"id\":3,\"type\":\"KSampler\",\"widgets_values\":[7,\"fixed\",20,8.0,\"euler\",\"normal\",1.0]},{\"id\":6,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"a red fox in snow, detailed fur\"]},{\"id\":7,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"blurry, lowres\"]}],\"links\":[],\"version\":0.4}", + "encoder": "Lavf63.1.102" + }, + "core.mov": { + "major_brand": "qt ", + "minor_version": "512", + "compatible_brands": "qt ", + "prompt": "{\"3\":{\"class_type\":\"KSampler\",\"inputs\":{\"seed\":7,\"steps\":20,\"cfg\":8.0,\"sampler_name\":\"euler\",\"scheduler\":\"normal\",\"denoise\":1.0,\"model\":[\"4\",0],\"positive\":[\"6\",0],\"negative\":[\"7\",0],\"latent_image\":[\"5\",0]}},\"4\":{\"class_type\":\"CheckpointLoaderSimple\",\"inputs\":{\"ckpt_name\":\"synthetic-v1.safetensors\"}},\"5\":{\"class_type\":\"EmptyLatentImage\",\"inputs\":{\"width\":512,\"height\":512,\"batch_size\":1}},\"6\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"a red fox in snow, detailed fur\",\"clip\":[\"4\",1]}},\"7\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"blurry, lowres\",\"clip\":[\"4\",1]}},\"8\":{\"class_type\":\"VAEDecode\",\"inputs\":{\"samples\":[\"3\",0],\"vae\":[\"4\",2]}},\"9\":{\"class_type\":\"SaveVideo\",\"inputs\":{\"filename_prefix\":\"video/synthetic\",\"video\":[\"8\",0]}}}", + "workflow": "{\"last_node_id\":9,\"nodes\":[{\"id\":3,\"type\":\"KSampler\",\"widgets_values\":[7,\"fixed\",20,8.0,\"euler\",\"normal\",1.0]},{\"id\":6,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"a red fox in snow, detailed fur\"]},{\"id\":7,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"blurry, lowres\"]}],\"links\":[],\"version\":0.4}", + "encoder": "Lavf63.1.102" + }, + "core.m4v": { + "major_brand": "M4V ", + "minor_version": "512", + "compatible_brands": "M4V isomiso2avc1", + "prompt": "{\"3\":{\"class_type\":\"KSampler\",\"inputs\":{\"seed\":7,\"steps\":20,\"cfg\":8.0,\"sampler_name\":\"euler\",\"scheduler\":\"normal\",\"denoise\":1.0,\"model\":[\"4\",0],\"positive\":[\"6\",0],\"negative\":[\"7\",0],\"latent_image\":[\"5\",0]}},\"4\":{\"class_type\":\"CheckpointLoaderSimple\",\"inputs\":{\"ckpt_name\":\"synthetic-v1.safetensors\"}},\"5\":{\"class_type\":\"EmptyLatentImage\",\"inputs\":{\"width\":512,\"height\":512,\"batch_size\":1}},\"6\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"a red fox in snow, detailed fur\",\"clip\":[\"4\",1]}},\"7\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"blurry, lowres\",\"clip\":[\"4\",1]}},\"8\":{\"class_type\":\"VAEDecode\",\"inputs\":{\"samples\":[\"3\",0],\"vae\":[\"4\",2]}},\"9\":{\"class_type\":\"SaveVideo\",\"inputs\":{\"filename_prefix\":\"video/synthetic\",\"video\":[\"8\",0]}}}", + "workflow": "{\"last_node_id\":9,\"nodes\":[{\"id\":3,\"type\":\"KSampler\",\"widgets_values\":[7,\"fixed\",20,8.0,\"euler\",\"normal\",1.0]},{\"id\":6,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"a red fox in snow, detailed fur\"]},{\"id\":7,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"blurry, lowres\"]}],\"links\":[],\"version\":0.4}", + "encoder": "Lavf63.1.102" + }, + "core.mkv": { + "WORKFLOW": "{\"last_node_id\":9,\"nodes\":[{\"id\":3,\"type\":\"KSampler\",\"widgets_values\":[7,\"fixed\",20,8.0,\"euler\",\"normal\",1.0]},{\"id\":6,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"a red fox in snow, detailed fur\"]},{\"id\":7,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"blurry, lowres\"]}],\"links\":[],\"version\":0.4}", + "PROMPT": "{\"3\":{\"class_type\":\"KSampler\",\"inputs\":{\"seed\":7,\"steps\":20,\"cfg\":8.0,\"sampler_name\":\"euler\",\"scheduler\":\"normal\",\"denoise\":1.0,\"model\":[\"4\",0],\"positive\":[\"6\",0],\"negative\":[\"7\",0],\"latent_image\":[\"5\",0]}},\"4\":{\"class_type\":\"CheckpointLoaderSimple\",\"inputs\":{\"ckpt_name\":\"synthetic-v1.safetensors\"}},\"5\":{\"class_type\":\"EmptyLatentImage\",\"inputs\":{\"width\":512,\"height\":512,\"batch_size\":1}},\"6\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"a red fox in snow, detailed fur\",\"clip\":[\"4\",1]}},\"7\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"blurry, lowres\",\"clip\":[\"4\",1]}},\"8\":{\"class_type\":\"VAEDecode\",\"inputs\":{\"samples\":[\"3\",0],\"vae\":[\"4\",2]}},\"9\":{\"class_type\":\"SaveVideo\",\"inputs\":{\"filename_prefix\":\"video/synthetic\",\"video\":[\"8\",0]}}}", + "ENCODER": "Lavf63.1.102" + }, + "core-tags.webm": { + "WORKFLOW": "{\"last_node_id\":9,\"nodes\":[{\"id\":3,\"type\":\"KSampler\",\"widgets_values\":[7,\"fixed\",20,8.0,\"euler\",\"normal\",1.0]},{\"id\":6,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"a red fox in snow, detailed fur\"]},{\"id\":7,\"type\":\"CLIPTextEncode\",\"widgets_values\":[\"blurry, lowres\"]}],\"links\":[],\"version\":0.4}", + "PROMPT": "{\"3\":{\"class_type\":\"KSampler\",\"inputs\":{\"seed\":7,\"steps\":20,\"cfg\":8.0,\"sampler_name\":\"euler\",\"scheduler\":\"normal\",\"denoise\":1.0,\"model\":[\"4\",0],\"positive\":[\"6\",0],\"negative\":[\"7\",0],\"latent_image\":[\"5\",0]}},\"4\":{\"class_type\":\"CheckpointLoaderSimple\",\"inputs\":{\"ckpt_name\":\"synthetic-v1.safetensors\"}},\"5\":{\"class_type\":\"EmptyLatentImage\",\"inputs\":{\"width\":512,\"height\":512,\"batch_size\":1}},\"6\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"a red fox in snow, detailed fur\",\"clip\":[\"4\",1]}},\"7\":{\"class_type\":\"CLIPTextEncode\",\"inputs\":{\"text\":\"blurry, lowres\",\"clip\":[\"4\",1]}},\"8\":{\"class_type\":\"VAEDecode\",\"inputs\":{\"samples\":[\"3\",0],\"vae\":[\"4\",2]}},\"9\":{\"class_type\":\"SaveVideo\",\"inputs\":{\"filename_prefix\":\"video/synthetic\",\"video\":[\"8\",0]}}}", + "ENCODER": "Lavf63.1.102" + }, + "legacy-comment.mp4": { + "major_brand": "isom", + "minor_version": "512", + "compatible_brands": "isomiso2avc1mp41", + "encoder": "Lavf63.1.102", + "comment": "{\"prompt\": \"{\\\"3\\\":{\\\"class_type\\\":\\\"KSampler\\\",\\\"inputs\\\":{\\\"seed\\\":7,\\\"steps\\\":20,\\\"cfg\\\":8.0,\\\"sampler_name\\\":\\\"euler\\\",\\\"scheduler\\\":\\\"normal\\\",\\\"denoise\\\":1.0,\\\"model\\\":[\\\"4\\\",0],\\\"positive\\\":[\\\"6\\\",0],\\\"negative\\\":[\\\"7\\\",0],\\\"latent_image\\\":[\\\"5\\\",0]}},\\\"4\\\":{\\\"class_type\\\":\\\"CheckpointLoaderSimple\\\",\\\"inputs\\\":{\\\"ckpt_name\\\":\\\"synthetic-v1.safetensors\\\"}},\\\"5\\\":{\\\"class_type\\\":\\\"EmptyLatentImage\\\",\\\"inputs\\\":{\\\"width\\\":512,\\\"height\\\":512,\\\"batch_size\\\":1}},\\\"6\\\":{\\\"class_type\\\":\\\"CLIPTextEncode\\\",\\\"inputs\\\":{\\\"text\\\":\\\"a red fox in snow, detailed fur\\\",\\\"clip\\\":[\\\"4\\\",1]}},\\\"7\\\":{\\\"class_type\\\":\\\"CLIPTextEncode\\\",\\\"inputs\\\":{\\\"text\\\":\\\"blurry, lowres\\\",\\\"clip\\\":[\\\"4\\\",1]}},\\\"8\\\":{\\\"class_type\\\":\\\"VAEDecode\\\",\\\"inputs\\\":{\\\"samples\\\":[\\\"3\\\",0],\\\"vae\\\":[\\\"4\\\",2]}},\\\"9\\\":{\\\"class_type\\\":\\\"SaveVideo\\\",\\\"inputs\\\":{\\\"filename_prefix\\\":\\\"video/synthetic\\\",\\\"video\\\":[\\\"8\\\",0]}}}\", \"workflow\": {\"last_node_id\": 9, \"nodes\": [{\"id\": 3, \"type\": \"KSampler\", \"widgets_values\": [7, \"fixed\", 20, 8.0, \"euler\", \"normal\", 1.0]}, {\"id\": 6, \"type\": \"CLIPTextEncode\", \"widgets_values\": [\"a red fox in snow, detailed fur\"]}, {\"id\": 7, \"type\": \"CLIPTextEncode\", \"widgets_values\": [\"blurry, lowres\"]}], \"links\": [], \"version\": 0.4}}" + }, + "legacy-comment.webm": { + "COMMENT": "{\"prompt\": \"{\\\"3\\\":{\\\"class_type\\\":\\\"KSampler\\\",\\\"inputs\\\":{\\\"seed\\\":7,\\\"steps\\\":20,\\\"cfg\\\":8.0,\\\"sampler_name\\\":\\\"euler\\\",\\\"scheduler\\\":\\\"normal\\\",\\\"denoise\\\":1.0,\\\"model\\\":[\\\"4\\\",0],\\\"positive\\\":[\\\"6\\\",0],\\\"negative\\\":[\\\"7\\\",0],\\\"latent_image\\\":[\\\"5\\\",0]}},\\\"4\\\":{\\\"class_type\\\":\\\"CheckpointLoaderSimple\\\",\\\"inputs\\\":{\\\"ckpt_name\\\":\\\"synthetic-v1.safetensors\\\"}},\\\"5\\\":{\\\"class_type\\\":\\\"EmptyLatentImage\\\",\\\"inputs\\\":{\\\"width\\\":512,\\\"height\\\":512,\\\"batch_size\\\":1}},\\\"6\\\":{\\\"class_type\\\":\\\"CLIPTextEncode\\\",\\\"inputs\\\":{\\\"text\\\":\\\"a red fox in snow, detailed fur\\\",\\\"clip\\\":[\\\"4\\\",1]}},\\\"7\\\":{\\\"class_type\\\":\\\"CLIPTextEncode\\\",\\\"inputs\\\":{\\\"text\\\":\\\"blurry, lowres\\\",\\\"clip\\\":[\\\"4\\\",1]}},\\\"8\\\":{\\\"class_type\\\":\\\"VAEDecode\\\",\\\"inputs\\\":{\\\"samples\\\":[\\\"3\\\",0],\\\"vae\\\":[\\\"4\\\",2]}},\\\"9\\\":{\\\"class_type\\\":\\\"SaveVideo\\\",\\\"inputs\\\":{\\\"filename_prefix\\\":\\\"video/synthetic\\\",\\\"video\\\":[\\\"8\\\",0]}}}\", \"workflow\": {\"last_node_id\": 9, \"nodes\": [{\"id\": 3, \"type\": \"KSampler\", \"widgets_values\": [7, \"fixed\", 20, 8.0, \"euler\", \"normal\", 1.0]}, {\"id\": 6, \"type\": \"CLIPTextEncode\", \"widgets_values\": [\"a red fox in snow, detailed fur\"]}, {\"id\": 7, \"type\": \"CLIPTextEncode\", \"widgets_values\": [\"blurry, lowres\"]}], \"links\": [], \"version\": 0.4}}", + "ENCODER": "Lavf63.1.102" + } + } +} diff --git a/tests/js/helpers/png-bytes.mjs b/tests/js/helpers/png-bytes.mjs new file mode 100644 index 0000000..ca5f581 --- /dev/null +++ b/tests/js/helpers/png-bytes.mjs @@ -0,0 +1,148 @@ +// Byte-exact PNG builders for the local reader tests (real CRC32 and zlib), and +// File-like blobs: one that records every [start, end) slice over a sparse layout +// that is never allocated, one that answers slowly, one that fails or shrinks. +import zlib from "node:zlib"; + +export const SIGNATURE = Uint8Array.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]); + +export function u32(n) { + return Uint8Array.from([(n >>> 24) & 0xff, (n >>> 16) & 0xff, (n >>> 8) & 0xff, n & 0xff]); +} + +export function latin1(text) { + return Buffer.from(text, "latin1"); +} + +export function utf8(text) { + return Buffer.from(text, "utf8"); +} + +export function concat(...parts) { + return Uint8Array.from(Buffer.concat(parts.map((part) => Buffer.from(part)))); +} + +/** + * One chunk with a real CRC32 over type + data. `crc` overrides the checksum and + * `declaredLength` the length field (to declare more data than is present). + */ +export function chunk(type, data, { crc, declaredLength } = {}) { + const typeBytes = latin1(type); + const body = Buffer.from(data); + const sum = crc ?? zlib.crc32(Buffer.concat([typeBytes, body])); + return concat(u32(declaredLength ?? body.length), typeBytes, body, u32(sum)); +} + +export function ihdr(width = 1, height = 1, options) { + return chunk("IHDR", concat(u32(width), u32(height), [8, 6, 0, 0, 0]), options); +} + +export function idat(data = [0]) { + return chunk("IDAT", data); +} + +export function iend(options) { + return chunk("IEND", [], options); +} + +/** tEXt: keyword NUL text, Latin-1 (text may be a string or raw bytes). */ +export function tEXt(keyword, text, options) { + const body = typeof text === "string" ? latin1(text) : Buffer.from(text); + return chunk("tEXt", concat(latin1(keyword), [0], body), options); +} + +/** zTXt: keyword NUL method zlib(text). `compressed` replaces the deflated body. */ +export function zTXt(keyword, text, { method = 0, compressed, level = 6 } = {}) { + const body = compressed ?? zlib.deflateSync(typeof text === "string" ? latin1(text) : Buffer.from(text), { level }); + return chunk("zTXt", concat(latin1(keyword), [0, method], body)); +} + +/** iTXt: keyword NUL flag method lang NUL translated NUL text (UTF-8, deflated when flag). */ +export function iTXt(keyword, text, { flag = 0, method = 0, lang = "", translated = "", compressed } = {}) { + const raw = typeof text === "string" ? utf8(text) : Buffer.from(text); + const body = compressed ?? (flag ? zlib.deflateSync(raw) : raw); + const langBytes = typeof lang === "string" ? latin1(lang) : Buffer.from(lang); + const translatedBytes = typeof translated === "string" ? utf8(translated) : Buffer.from(translated); + return concat(chunk("iTXt", concat(latin1(keyword), [0, flag, method], langBytes, [0], translatedBytes, [0], body))); +} + +/** A whole PNG: signature, IHDR, IDAT, the given chunks, IEND. */ +export function png(...chunks) { + return concat(SIGNATURE, ihdr(), idat(), ...chunks, iend()); +} + +/** Signature plus exactly the given chunks (no implied IHDR/IDAT/IEND). */ +export function raw(...chunks) { + return concat(SIGNATURE, ...chunks); +} + +export function blobOf(bytes) { + return new Blob([bytes]); +} + +/** + * A File-like object over a sparse layout: `parts` are [{offset, bytes}] placed in a + * file of `size` bytes, zero everywhere else; nothing of `size` is allocated. Every + * slice is recorded in `ranges` as [start, end). + */ +export function sparseBlob(parts, size) { + const ranges = []; + function bytesOf(start, end) { + const out = new Uint8Array(end - start); + for (const part of parts) { + const from = Math.max(start, part.offset); + const to = Math.min(end, part.offset + part.bytes.length); + if (from < to) out.set(part.bytes.subarray(from - part.offset, to - part.offset), from - start); + } + return out; + } + return { + size, + ranges, + slice(start, end) { + ranges.push([start, end]); + return { arrayBuffer: async () => bytesOf(start, end).buffer }; + }, + }; +} + +/** A recording blob over real bytes. */ +export function recordingBlob(bytes) { + return sparseBlob([{ offset: 0, bytes }], bytes.length); +} + +/** Every slice resolves after `delayMs`. */ +export function slowBlob(bytes, delayMs) { + return { + size: bytes.length, + slice(start, end) { + return { + arrayBuffer: () => new Promise((resolve) => setTimeout(() => resolve(bytes.slice(start, end).buffer), delayMs)), + }; + }, + }; +} + +/** Slices fail (as a browser does when the file vanished). */ +export function failingBlob(size) { + return { + size, + slice() { + return { arrayBuffer: () => Promise.reject(new Error("NotReadableError")) }; + }, + }; +} + +/** Reports `size` but every slice beyond `actual` bytes comes back short. */ +export function shrunkBlob(bytes, size) { + return { + size, + slice(start, end) { + return { arrayBuffer: async () => bytes.slice(start, Math.min(end, bytes.length)).buffer }; + }, + }; +} + +/** An A1111 `parameters` record with one settings footer carrying a batch index. */ +export function parametersText(positive, batchIndex, { seedField = "Seed: 1" } = {}) { + return `${positive}\nNegative prompt: blurry\nSteps: 20, Sampler: Euler, Batch index: ${batchIndex}, ${seedField}`; +} diff --git a/tests/js/helpers/video-bytes.mjs b/tests/js/helpers/video-bytes.mjs new file mode 100644 index 0000000..7b60b42 --- /dev/null +++ b/tests/js/helpers/video-bytes.mjs @@ -0,0 +1,277 @@ +// Byte-exact ISO-BMFF (MP4/MOV/M4V) and Matroska (MKV/WebM) builders for the local +// video tag reader tests, laid out as ffmpeg's muxers write them (see the dumps in +// tests/fixtures/local-inspect). The File-like blobs come from png-bytes.mjs. +import { concat, u32, utf8, latin1 } from "./png-bytes.mjs"; + +export { concat, u32, utf8, latin1 }; +export { sparseBlob, recordingBlob, slowBlob, failingBlob, blobOf } from "./png-bytes.mjs"; + +const toBytes = (part) => (typeof part === "string" ? utf8(part) : Buffer.from(part)); + +function u64(n) { + const out = new Uint8Array(8); + new DataView(out.buffer).setBigUint64(0, BigInt(n)); + return out; +} + +// ---------------------------------------------------------------- ISO-BMFF + +/** A box: 32-bit size, 4-byte type (string or raw bytes), payload. `size` overrides the field. */ +export function box(type, ...parts) { + const options = parts.length && parts[parts.length - 1] && parts[parts.length - 1].__options ? parts.pop() : {}; + const payload = concat(...parts.map(toBytes)); + const typeBytes = typeof type === "string" ? latin1(type) : Buffer.from(type); + const total = 8 + payload.length; + if (options.largesize) return concat(u32(1), typeBytes, u64(options.largesize === true ? total + 8 : options.largesize), payload); + return concat(u32(options.size ?? total), typeBytes, payload); +} + +/** Marker for `box` options: box('ftyp', ..., opts({size: 3})). */ +export function opts(options) { + return { __options: true, ...options }; +} + +export function fullBox(type, ...parts) { + return box(type, u32(0), ...parts); +} + +export function ftyp(brand = "isom", compatible = "isomiso2avc1mp41") { + return box("ftyp", latin1(brand), u32(512), latin1(compatible)); +} + +export function mdat(payload = [0, 0, 0, 0]) { + return box("mdat", payload); +} + +export const moov = (...children) => box("moov", ...children); +export const udta = (...children) => box("udta", ...children); +export const free = (count = 0) => box("free", new Uint8Array(count)); + +/** `meta` as ffmpeg writes it: a FullBox (version/flags) whose first child is hdlr. */ +export function meta(...children) { + return fullBox("meta", ...children); +} + +export function hdlr(handler = "mdta") { + return fullBox("hdlr", u32(0), latin1(handler), new Uint8Array(12), [0]); +} + +/** `keys`: one mdta entry per name (an entry may be [name, namespace]); `count` overrides the count field. */ +export function keys(names, { count } = {}) { + const entries = names.map((entry) => { + const [name, namespace] = Array.isArray(entry) ? entry : [entry, "mdta"]; + const bytes = toBytes(name); + return concat(u32(8 + bytes.length), latin1(namespace), bytes); + }); + return fullBox("keys", u32(count ?? names.length), ...entries); +} + +export const ilst = (...items) => box("ilst", ...items); + +/** A `data` atom: type indicator (1 = UTF-8), locale, payload. */ +export function data(payload, { type = 1, locale = 0, size } = {}) { + return box("data", u32(type), u32(locale), toBytes(payload), ...(size === undefined ? [] : [opts({ size })])); +} + +/** An ilst item addressed by its 1-based `keys` index. */ +export function item(index, ...children) { + return box(u32(index), ...children); +} + +/** An ilst item addressed by a legacy fourcc such as "©cmt" (Latin-1 bytes). */ +export function legacyItem(fourcc, ...children) { + return box(latin1(fourcc), ...children); +} + +/** `moov/udta/meta` carrying per-key mdta tags, as core SaveVideo writes through ffmpeg. */ +export function mdtaUdta(tags) { + const names = Object.keys(tags); + return udta(meta(hdlr("mdta"), keys(names), ilst(...names.map((name, index) => item(index + 1, data(tags[name])))))); +} + +/** `moov/udta/meta` with legacy ilst atoms: {comment, description} → ©cmt / desc. */ +export function legacyUdta({ comment, description, tool = "Lavf" } = {}) { + const items = [legacyItem("©too", data(tool))]; + if (comment !== undefined) items.push(legacyItem("©cmt", data(comment))); + if (description !== undefined) items.push(legacyItem("desc", data(description))); + return udta(meta(hdlr("mdir"), ilst(...items))); +} + +/** A whole MP4: ftyp, (moov first when faststart), mdat, moov with the given udta. */ +export function mp4(udtaBox, { brand = "isom", faststart = false, extraMoov = [], payload } = {}) { + const movie = moov(...extraMoov, udtaBox); + const media = mdat(payload); + return faststart ? concat(ftyp(brand), movie, media) : concat(ftyp(brand), free(), media, movie); +} + +export function perKeyMp4(tags, options) { + return mp4(mdtaUdta(tags), options); +} + +/** Top-level boxes of `bytes` as [{type, start, end}]. */ +export function topLevelBoxes(bytes) { + const out = []; + let offset = 0; + while (offset + 8 <= bytes.length) { + const view = new DataView(bytes.buffer, bytes.byteOffset + offset); + let size = view.getUint32(0); + if (size === 1) size = Number(view.getBigUint64(8)); + if (size === 0) size = bytes.length - offset; + out.push({ type: latin1Of(bytes.subarray(offset + 4, offset + 8)), start: offset, end: offset + size }); + offset += size; + } + return out; +} + +function latin1Of(bytes) { + return Buffer.from(bytes).toString("latin1"); +} + +// ---------------------------------------------------------------- Matroska + +export const ID = { + EBML: 0x1a45dfa3, + EBMLVersion: 0x4286, + DocType: 0x4282, + DocTypeVersion: 0x4287, + Segment: 0x18538067, + SeekHead: 0x114d9b74, + Seek: 0x4dbb, + SeekID: 0x53ab, + SeekPosition: 0x53ac, + Void: 0xec, + CRC32: 0xbf, + Info: 0x1549a966, + Tracks: 0x1654ae6b, + Tags: 0x1254c367, + Tag: 0x7373, + Targets: 0x63c0, + TargetTypeValue: 0x68ca, + TagTrackUID: 0x63c5, + SimpleTag: 0x67c8, + TagName: 0x45a3, + TagString: 0x4487, + TagLanguage: 0x447a, + TagDefault: 0x4484, + TagBinary: 0x4485, + Cluster: 0x1f43b675, + Timestamp: 0xe7, + Cues: 0x1c53bb6b, +}; + +/** The ID's bytes (EBML IDs carry their own length marker). */ +export function idBytes(id) { + const out = []; + let value = id; + while (value > 0) { + out.unshift(value & 0xff); + value = Math.floor(value / 256); + } + return Uint8Array.from(out); +} + +/** A size VINT: minimal width unless `width` is given; `unknown` writes all ones. */ +export function vint(value, { width, unknown = false } = {}) { + let length = width; + if (!length) { + length = 1; + while (length < 8 && value >= 2 ** (7 * length) - 1) length += 1; + } + const out = new Uint8Array(length); + if (unknown) { + out.fill(0xff); + out[0] = 0xff >> (length - 1); + return out; + } + let rest = BigInt(value); + for (let index = length - 1; index >= 0; index -= 1) { + out[index] = Number(rest & 0xffn); + rest >>= 8n; + } + out[0] |= 0x80 >> (length - 1); + return out; +} + +/** An element: ID, size VINT, payload. `size`/`width`/`unknown` shape the size field. */ +export function el(id, ...parts) { + const options = parts.length && parts[parts.length - 1] && parts[parts.length - 1].__options ? parts.pop() : {}; + const payload = concat(...parts.map(toBytes)); + const size = vint(options.size ?? payload.length, { width: options.width, unknown: options.unknown }); + return concat(idBytes(id), size, payload); +} + +export function uint(value, width) { + const out = []; + let rest = BigInt(value); + do { + out.unshift(Number(rest & 0xffn)); + rest >>= 8n; + } while (rest > 0n); + while (width && out.length < width) out.unshift(0); + return Uint8Array.from(out); +} + +export function ebmlHeader(docType = "matroska") { + return el(ID.EBML, el(ID.EBMLVersion, [1]), el(ID.DocType, docType), el(ID.DocTypeVersion, [4])); +} + +/** Segment with an 8-byte size field as ffmpeg writes it; `unknown` makes it unknown-sized. */ +export function segment(children, { unknown = false } = {}) { + return el(ID.Segment, ...children, opts({ width: 8, unknown })); +} + +/** SeekHead entries: [[id, position]] with positions relative to the Segment payload. */ +export function seekHead(entries) { + return el(ID.SeekHead, ...entries.map(([id, position]) => el(ID.Seek, el(ID.SeekID, idBytes(id)), el(ID.SeekPosition, uint(position))))); +} + +export function voidEl(count = 4) { + return el(ID.Void, new Uint8Array(count)); +} + +/** A SimpleTag with TagName and TagString (`value` undefined omits the string). */ +export function simpleTag(name, value, { lang, def, binary, nested = [], extra = [] } = {}) { + const children = [el(ID.TagName, name)]; + if (lang !== undefined) children.push(el(ID.TagLanguage, lang)); + if (def !== undefined) children.push(el(ID.TagDefault, [def])); + if (value !== undefined) children.push(el(ID.TagString, value)); + if (binary !== undefined) children.push(el(ID.TagBinary, binary)); + return el(ID.SimpleTag, ...children, ...nested, ...extra); +} + +/** A Tag whose Targets carry the given track UID (none → global). */ +export function tag(simpleTags, { trackUid, targetType } = {}) { + const targets = []; + if (targetType !== undefined) targets.push(el(ID.TargetTypeValue, uint(targetType))); + if (trackUid !== undefined) targets.push(el(ID.TagTrackUID, uint(trackUid, 8))); + return el(ID.Tag, el(ID.Targets, ...targets), ...simpleTags); +} + +export function tagsEl(...tagElements) { + return el(ID.Tags, ...tagElements); +} + +/** A Cluster of `size` payload bytes (zeros); `unknown` makes its size unknown. */ +export function cluster(size = 8, { unknown = false, payload } = {}) { + return el(ID.Cluster, payload ?? new Uint8Array(size), opts({ unknown })); +} + +/** A Cluster header only, declaring `size` payload bytes (for sparse layouts). */ +export function clusterHeader(size) { + return concat(idBytes(ID.Cluster), vint(size, { width: 8 })); +} + +/** A whole Matroska file: EBML header + Segment(children). */ +export function mkv(children, { docType = "matroska", unknown = false } = {}) { + return concat(ebmlHeader(docType), segment(children, { unknown })); +} + +/** A global Tags element from {name: value} (ffmpeg upper-cases names at mux time). */ +export function globalTags(tags, options) { + return tagsEl(tag(Object.entries(tags).map(([name, value]) => simpleTag(name, value)), options)); +} + +/** A Matroska file whose global Tags hold {name: value}, ffmpeg layout. */ +export function perKeyMkv(tags, { docType = "matroska" } = {}) { + return mkv([el(ID.Info, new Uint8Array(4)), globalTags(tags), cluster()], { docType }); +} diff --git a/tests/js/png-local-reader.test.js b/tests/js/png-local-reader.test.js new file mode 100644 index 0000000..dc0aedb --- /dev/null +++ b/tests/js/png-local-reader.test.js @@ -0,0 +1,549 @@ +// Run with: node --test tests/js/ +// The browser PNG File reader is a port of utils/prompt_inspector.py's bounded +// reader (_png_metadata, _PngRecords, _png_text, _inflate, _batch_normalized). Each +// rule below names the Python branch it mirrors; tests/test_local_inspect_parity.py +// proves the two agree on the committed parity corpus. +const test = require("node:test"); +const assert = require("node:assert/strict"); +const fs = require("node:fs"); +const path = require("node:path"); +const zlib = require("node:zlib"); +const { classifyPrefix, readPngMetadataFile, readImageHeader } = require("../../web/js/png-metadata.js"); +const bytes = require("./helpers/png-bytes.mjs"); + +const { png, raw, ihdr, idat, iend, tEXt, zTXt, iTXt, chunk, blobOf, sparseBlob, recordingBlob, slowBlob, failingBlob, shrunkBlob, concat, u32, latin1, parametersText } = bytes; + +const MiB = 1024 * 1024; +const ROLE_BUDGET = 2 * MiB; +const WORKFLOW_BUDGET = 4 * MiB; +const PNG_READ_CEILING = 19794389; +const FIXTURES = path.join(__dirname, "..", "fixtures", "local-inspect"); + +async function read(fileBytes, options) { + return readPngMetadataFile(blobOf(fileBytes), options); +} + +function fill(byte, count) { + return Buffer.alloc(count, byte); +} + +test("contract: the reader, classifier and header reader are exported", () => { + assert.equal(typeof readPngMetadataFile, "function"); + assert.equal(typeof classifyPrefix, "function"); + assert.equal(typeof readImageHeader, "function"); +}); + +test("reads the four retained records and the IHDR size from a core SaveImage PNG", async () => { + const result = await read(raw( + ihdr(640, 480), + idat(), + tEXt("prompt", '{"1":{"class_type":"KSampler"}}'), + iTXt("workflow", '{"nodes":[]}'), + tEXt("parameters", "cat\nSteps: 20, Seed: 1"), + zTXt("comfyassets_prompt_pairs", '{"pairs":[]}'), + tEXt("Software", "ComfyUI"), + iend(), + )); + assert.deepEqual(result, { + status: "complete", + records: { + prompt: '{"1":{"class_type":"KSampler"}}', + workflow: '{"nodes":[]}', + parameters: "cat\nSteps: 20, Seed: 1", + comfyassets_prompt_pairs: '{"pairs":[]}', + }, + limited: [], + width: 640, + height: 480, + }); +}); + +test("local PNG skips sparse pixels and reads post-IDAT text", async () => { + // Two 1.5 GiB IDAT chunks (each under the 0x7FFFFFFF chunk ceiling) with the + // prompt after them: nothing of the 3 GiB is read or allocated. + const half = 1.5 * 1024 * 1024 * 1024; + const head = concat(bytes.SIGNATURE, ihdr(8192, 8192), u32(half), latin1("IDAT")); + const idat1 = { start: bytes.SIGNATURE.length + ihdr().length, end: bytes.SIGNATURE.length + ihdr().length + 8 + half + 4 }; + const idat2 = { start: idat1.end, end: idat1.end + 8 + half + 4 }; + const between = concat(u32(0), u32(half), latin1("IDAT")); + const tail = concat(u32(0), tEXt("prompt", '{"3":{"class_type":"KSampler"}}'), iend()); + const size = idat2.end - 4 + tail.length; + const file = sparseBlob([ + { offset: 0, bytes: head }, + { offset: idat1.end - 4, bytes: between }, + { offset: idat2.end - 4, bytes: tail }, + ], size); + + const result = await readPngMetadataFile(file); + + assert.equal(result.status, "complete"); + assert.deepEqual(result.records, { prompt: '{"3":{"class_type":"KSampler"}}' }); + assert.deepEqual([result.width, result.height], [8192, 8192]); + const insideIdat = file.ranges.filter(([start, end]) => [idat1, idat2].some((region) => start >= region.start && end <= region.end)); + assert.ok(insideIdat.length >= 2, "the IDAT headers were read"); + for (const [start, end] of insideIdat) assert.ok(end - start <= 12, `range inside IDAT longer than 12: [${start}, ${end})`); + const total = file.ranges.reduce((sum, [start, end]) => sum + (end - start), 0); + assert.ok(total <= PNG_READ_CEILING, `read ${total} bytes`); +}); + +test("serves the retained prefix without slicing it again", async () => { + const whole = png(tEXt("prompt", "p")); + const file = recordingBlob(whole); + const result = await readPngMetadataFile(file, { prefix: whole.subarray(0, 32) }); + assert.equal(result.status, "complete"); + assert.ok(file.ranges.every(([start]) => start >= 32), JSON.stringify(file.ranges)); + assert.deepEqual(result.records, { prompt: "p" }); +}); + +test("unknown text keys are skipped by length without reading their body", async () => { + const body = fill(0x41, 1000); + const whole = png(tEXt("Software", body), tEXt("prompt", "p")); + const softwareStart = bytes.SIGNATURE.length + ihdr().length + idat().length; + const bodyStart = softwareStart + 8 + "Software".length + 1; + const bodyEnd = bodyStart + body.length + 4; + const file = recordingBlob(whole); + const result = await readPngMetadataFile(file); + assert.equal(result.status, "complete"); + const insideBody = file.ranges.filter(([start, end]) => start < bodyEnd && end > softwareStart + 8 + 80); + assert.deepEqual(insideBody, [], "the body past the 80-byte keyword prefix was sliced"); +}); + +const WF_OVER = WORKFLOW_BUDGET + 1; +const bigWorkflow = () => tEXt("workflow", fill(0x7b, WF_OVER - "workflow".length - 1)); + +// One row per Python branch (prompt_inspector.py:91-260). +const RULES = [ + { name: "bad IHDR CRC", file: () => raw(ihdr(1, 1, { crc: 1 }), idat(), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "bad text CRC", file: () => png(tEXt("prompt", "p", { crc: 1 })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "bad IEND CRC", file: () => raw(ihdr(), idat(), iend({ crc: 1 })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "first chunk not IHDR", file: () => raw(idat(), ihdr(), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "IHDR not 13 bytes", file: () => raw(chunk("IHDR", concat(u32(1), u32(1), [8, 6, 0, 0])), idat(), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "second IHDR", file: () => raw(ihdr(), idat(), ihdr(), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "zero width", file: () => raw(ihdr(0, 1), idat(), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "zero height", file: () => raw(ihdr(1, 0), idat(), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "length 0x80000000", file: () => raw(ihdr(), idat(), chunk("tEXt", latin1("k\0v"), { declaredLength: 0x80000000 }), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "end past size", file: () => raw(ihdr(), idat(), chunk("tEXt", latin1("k\0v"), { declaredLength: 100 }), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "file ends inside a chunk header", file: () => raw(ihdr(), idat(), u32(5)), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "file ends before any chunk", file: () => raw(), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "too short for a signature", file: () => bytes.SIGNATURE.subarray(0, 5), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "not a PNG", file: () => latin1("GIF89a................................."), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "IEND without IDAT", file: () => raw(ihdr(), tEXt("prompt", "p"), iend()), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "nonzero IEND length", file: () => raw(ihdr(), idat(), chunk("IEND", [1])), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "duplicate key with a different length", file: () => png(tEXt("prompt", "one"), tEXt("prompt", "other")), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "duplicate key, same length, different bytes", file: () => png(tEXt("prompt", "one"), tEXt("prompt", "two")), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "duplicate key, same bytes, different chunk type", file: () => png(tEXt("prompt", "one"), chunk("zTXt", latin1("prompt\0one"))), expect: { status: "unreadable", reason: "invalid-png" } }, + { + name: "byte-identical duplicate kept once", + file: () => png(tEXt("prompt", "same"), tEXt("prompt", "same"), tEXt("prompt", "same")), + expect: { status: "complete", records: { prompt: "same" }, limited: [], width: 1, height: 1 }, + }, + { + name: "parameters copies differing only in Batch index keep the last", + file: () => png(tEXt("parameters", parametersText("cat", 0)), tEXt("parameters", parametersText("cat", 1)), tEXt("parameters", parametersText("cat", 12))), + expect: { status: "complete", records: { parameters: parametersText("cat", 12) }, limited: [], width: 1, height: 1 }, + }, + { name: "parameters copies with any other difference", file: () => png(tEXt("parameters", parametersText("cat", 0)), tEXt("parameters", parametersText("dog", 1))), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "parameters copies without a settings footer", file: () => png(tEXt("parameters", "cat"), tEXt("parameters", "dog")), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "parameters copies with two footers", file: () => png(tEXt("parameters", `${parametersText("cat", 0)}\nSteps: 1, x`), tEXt("parameters", `${parametersText("cat", 1)}\nSteps: 1, x`)), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "parameters copies with two batch indexes", file: () => png(tEXt("parameters", parametersText("cat", 0, { seedField: "Batch index: 0" })), tEXt("parameters", parametersText("cat", 1, { seedField: "Batch index: 1" }))), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "parameters copies whose batch index is not a whole settings field", file: () => png(tEXt("parameters", "Steps: 20, Batch index: 0 trailing"), tEXt("parameters", "Steps: 20, Batch index: 1 trailing")), expect: { status: "unreadable", reason: "invalid-png" } }, + { + name: "workflow 4 MiB + 1 encoded is limited, other records intact", + file: () => png(tEXt("prompt", "p"), bigWorkflow(), tEXt("parameters", "cat\nSteps: 1, x")), + expect: { status: "complete", records: { prompt: "p", parameters: "cat\nSteps: 1, x" }, limited: ["workflow"], width: 1, height: 1 }, + }, + { + name: "a duplicate of a limited workflow is skipped", + file: () => png(bigWorkflow(), bigWorkflow(), tEXt("prompt", "p")), + expect: { status: "complete", records: { prompt: "p" }, limited: ["workflow"], width: 1, height: 1 }, + }, + { + name: "workflow over 4 MiB decoded (zTXt) is limited", + file: () => png(zTXt("workflow", fill(0x7b, WORKFLOW_BUDGET + 1), { level: 9 }), tEXt("prompt", "p")), + expect: { status: "complete", records: { prompt: "p" }, limited: ["workflow"], width: 1, height: 1 }, + }, + { + name: "workflow at exactly 4 MiB decoded is kept", + file: () => png(zTXt("workflow", fill(0x7b, WORKFLOW_BUDGET), { level: 9 })), + check: (result) => { + assert.equal(result.status, "complete"); + assert.equal(result.records.workflow.length, WORKFLOW_BUDGET); + assert.deepEqual(result.limited, []); + }, + }, + { + name: "workflow under 4 MiB inflated but over 4 MiB as UTF-8 is limited", + file: () => png(zTXt("workflow", fill(0xe9, WORKFLOW_BUDGET - 10), { level: 9 }), tEXt("prompt", "p")), + expect: { status: "complete", records: { prompt: "p" }, limited: ["workflow"], width: 1, height: 1 }, + }, + { name: "roles 2 MiB + 1 encoded", file: () => png(tEXt("prompt", fill(0x41, ROLE_BUDGET + 1 - "prompt".length - 1))), expect: { status: "unreadable", reason: "metadata-limit" } }, + { name: "roles summed over 2 MiB encoded", file: () => png(tEXt("prompt", fill(0x41, 1.5 * MiB)), tEXt("parameters", fill(0x41, 0.5 * MiB))), expect: { status: "unreadable", reason: "metadata-limit" } }, + { name: "roles 2 MiB + 1 decoded via zTXt", file: () => png(zTXt("prompt", fill(0x41, ROLE_BUDGET + 1), { level: 9 })), expect: { status: "unreadable", reason: "metadata-limit" } }, + { name: "roles over 2 MiB as UTF-8 from Latin-1 high bytes", file: () => png(tEXt("prompt", fill(0xe9, MiB + 1))), expect: { status: "unreadable", reason: "metadata-limit" } }, + { + name: "roles share the decoded budget across records", + file: () => png(zTXt("prompt", fill(0x41, 1.5 * MiB), { level: 9 }), zTXt("parameters", fill(0x41, 0.5 * MiB + 1), { level: 9 })), + expect: { status: "unreadable", reason: "metadata-limit" }, + }, + { + name: "roles at exactly 2 MiB decoded are kept", + file: () => png(zTXt("prompt", fill(0x41, 1.5 * MiB), { level: 9 }), zTXt("parameters", fill(0x41, 0.5 * MiB), { level: 9 })), + check: (result) => { + assert.equal(result.status, "complete"); + assert.equal(result.records.prompt.length + result.records.parameters.length, ROLE_BUDGET); + }, + }, + { + name: "duplicate workflow bytes capped at 8 MiB: the workflow becomes limited", + file: () => { + const workflow = tEXt("workflow", fill(0x7b, 3 * MiB)); + return png(workflow, workflow, workflow, workflow, tEXt("prompt", "p")); + }, + expect: { status: "complete", records: { prompt: "p" }, limited: ["workflow"], width: 1, height: 1 }, + }, + { + name: "duplicate workflow bytes within 8 MiB keep the workflow", + file: () => { + const workflow = tEXt("workflow", fill(0x7b, 3 * MiB)); + return png(workflow, workflow, workflow); + }, + check: (result) => { + assert.equal(result.status, "complete"); + assert.equal(result.records.workflow.length, 3 * MiB); + assert.deepEqual(result.limited, []); + }, + }, + { + name: "duplicate role bytes capped at 4 MiB", + file: () => { + const prompt = tEXt("prompt", fill(0x41, 1.5 * MiB)); + return png(prompt, prompt, prompt, prompt); + }, + expect: { status: "unreadable", reason: "metadata-limit" }, + }, + { + name: "duplicate decoded parameters capped at 4 MiB", + file: () => { + const copy = (index) => zTXt("parameters", parametersText(fill(0x41, 1.5 * MiB).toString("latin1"), index), { level: 9 }); + return png(copy(0), copy(1), copy(2), copy(3)); + }, + expect: { status: "unreadable", reason: "metadata-limit" }, + }, + { + name: "a longer batch copy draws the growth from the role budget", + file: () => { + const base = parametersText("", 9); + const positive = fill(0x41, ROLE_BUDGET - base.length - 1).toString("latin1"); + return png(zTXt("parameters", parametersText(positive, 9), { level: 9 }), zTXt("parameters", parametersText(positive, 100), { level: 9 })); + }, + expect: { status: "unreadable", reason: "metadata-limit" }, + }, + { + name: "a longer batch copy within the role budget is kept", + file: () => { + const base = parametersText("", 9); + const positive = fill(0x41, ROLE_BUDGET - base.length - 2).toString("latin1"); + return png(zTXt("parameters", parametersText(positive, 9), { level: 9 }), zTXt("parameters", parametersText(positive, 10), { level: 9 })); + }, + check: (result) => { + assert.equal(result.status, "complete"); + assert.ok(result.records.parameters.endsWith("Batch index: 10, Seed: 1")); + }, + }, + { + name: "a batch copy inflating past the role budget", + file: () => png(tEXt("parameters", parametersText("cat", 0)), zTXt("parameters", parametersText(fill(0x41, ROLE_BUDGET).toString("latin1"), 1), { level: 9 })), + expect: { status: "unreadable", reason: "metadata-limit" }, + }, + { + name: "10,000 chunks without IEND", + file: () => concat(bytes.SIGNATURE, ihdr(), idat(), ...Array.from({ length: 9999 }, () => chunk("xxXx", [])), iend()), + expect: { status: "unreadable", reason: "metadata-limit" }, + }, + { + name: "IEND as the 10,000th chunk is read", + file: () => concat(bytes.SIGNATURE, ihdr(), idat(), ...Array.from({ length: 9997 }, () => chunk("xxXx", [])), iend()), + expect: { status: "complete", records: {}, limited: [], width: 1, height: 1 }, + }, + { name: "zTXt method not 0", file: () => png(zTXt("prompt", "p", { method: 1 })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "zTXt without a method byte", file: () => png(chunk("zTXt", latin1("prompt\0"))), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "iTXt flag 2", file: () => png(iTXt("prompt", "p", { flag: 2 })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "iTXt compressed with method 1", file: () => png(iTXt("prompt", "p", { flag: 1, method: 1 })), expect: { status: "unreadable", reason: "invalid-png" } }, + { + name: "iTXt uncompressed ignores the method byte", + file: () => png(iTXt("prompt", "p", { flag: 0, method: 7 })), + expect: { status: "complete", records: { prompt: "p" }, limited: [], width: 1, height: 1 }, + }, + { name: "iTXt shorter than flag and method", file: () => png(chunk("iTXt", latin1("prompt\0\0"))), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "iTXt missing the translated keyword separator", file: () => png(chunk("iTXt", concat(latin1("prompt\0"), [0, 0], latin1("en\0text")))), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "iTXt non-ASCII language tag", file: () => png(iTXt("prompt", "p", { lang: [0xc3, 0xa9] })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "iTXt invalid UTF-8 translated keyword", file: () => png(iTXt("prompt", "p", { translated: [0xff] })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "iTXt invalid UTF-8 text", file: () => png(iTXt("prompt", [0xc3, 0x28])), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "iTXt encoded surrogate", file: () => png(iTXt("prompt", [0xed, 0xa0, 0x80])), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "NUL in tEXt text", file: () => png(tEXt("prompt", "a\0b")), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "NUL in inflated zTXt text", file: () => png(zTXt("prompt", "a\0b")), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "NUL in iTXt text", file: () => png(iTXt("prompt", "a\0b")), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "keyword of 0 bytes", file: () => png(chunk("tEXt", latin1("\0v"))), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "keyword of 80 bytes", file: () => png(tEXt("k".repeat(80), "v")), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "keyword without a NUL", file: () => png(chunk("tEXt", latin1("nonul"))), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "empty text chunk", file: () => png(chunk("tEXt", [])), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "malformed keyword on an unknown key still fails", file: () => png(tEXt("prompt", "p"), chunk("tEXt", latin1("x".repeat(100)))), expect: { status: "unreadable", reason: "invalid-png" } }, + { + name: "keyword of 79 bytes is read", + file: () => png(tEXt("k".repeat(79), "v"), tEXt("prompt", "p")), + expect: { status: "complete", records: { prompt: "p" }, limited: [], width: 1, height: 1 }, + }, + { + name: "Latin-1 0xE9 is an e-acute in tEXt and zTXt", + file: () => png(tEXt("prompt", [0xe9]), zTXt("parameters", [0xe9, 0x0a, 0x53, 0x74, 0x65, 0x70, 0x73, 0x3a, 0x20, 0x31])), + expect: { status: "complete", records: { prompt: "é", parameters: "é\nSteps: 1" }, limited: [], width: 1, height: 1 }, + }, + { + name: "iTXt BOM is preserved, compressed and not", + file: () => png(iTXt("prompt", "{}"), iTXt("workflow", "[]", { flag: 1, lang: "en", translated: "wörkflow" })), + expect: { status: "complete", records: { prompt: "{}", workflow: "[]" }, limited: [], width: 1, height: 1 }, + }, + { name: "truncated zlib data", file: () => png(zTXt("prompt", "x".repeat(500), { compressed: zlib.deflateSync(latin1("x".repeat(500))).subarray(0, 8) })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "trailing bytes after the zlib stream", file: () => png(zTXt("prompt", "p", { compressed: concat(zlib.deflateSync(latin1("p")), [1, 2, 3]) })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "empty zlib data", file: () => png(zTXt("prompt", "p", { compressed: [] })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "garbage zlib data", file: () => png(zTXt("prompt", "p", { compressed: [1, 2, 3, 4, 5, 6, 7, 8] })), expect: { status: "unreadable", reason: "invalid-png" } }, + { name: "truncated compressed iTXt", file: () => png(iTXt("prompt", "p", { flag: 1, compressed: zlib.deflateSync(latin1("x".repeat(500))).subarray(0, 8) })), expect: { status: "unreadable", reason: "invalid-png" } }, + { + name: "chunks after IEND are ignored", + file: () => concat(png(tEXt("prompt", "p")), tEXt("workflow", "w")), + expect: { status: "complete", records: { prompt: "p" }, limited: [], width: 1, height: 1 }, + }, + { + name: "non-ASCII chunk types are skipped", + file: () => png(chunk("\xff\xfe\x80\x81", [1, 2, 3]), tEXt("prompt", "p")), + expect: { status: "complete", records: { prompt: "p" }, limited: [], width: 1, height: 1 }, + }, +]; + +for (const rule of RULES) { + test(`server rules, one by one: ${rule.name}`, async () => { + const result = await read(rule.file()); + if (rule.check) rule.check(result); + else assert.deepEqual(result, rule.expect); + }); +} + +test("decompress: undefined with zTXt is decompression-unavailable and yields no records", async () => { + const result = await read(png(tEXt("prompt", "p"), zTXt("workflow", "w")), { decompress: undefined }); + assert.deepEqual(result, { status: "unreadable", reason: "decompression-unavailable" }); + const plain = await read(png(tEXt("prompt", "p"), zTXt("Comment", "skipped")), { decompress: undefined }); + assert.deepEqual(plain.records, { prompt: "p" }); +}); + +function instrumentedStream(transform) { + const stats = { writes: [], inFlight: 0, maxInFlight: 0 }; + return { + stats, + Stream: class { + constructor() { + const stream = new TransformStream({ + async transform(chunk, controller) { + stats.writes.push(chunk.byteLength); + stats.inFlight += 1; + stats.maxInFlight = Math.max(stats.maxInFlight, stats.inFlight); + await transform(chunk, controller); + stats.inFlight -= 1; + }, + }); + this.writable = stream.writable; + this.readable = stream.readable; + } + }, + }; +} + +test("inflation keeps one <=256-byte write in flight", async () => { + const body = fill(0x41, 3000); + const { stats, Stream } = instrumentedStream(async (chunk, controller) => { + await new Promise((resolve) => setTimeout(resolve, 1)); + controller.enqueue(chunk); + }); + // The fake "inflates" by passing bytes through, so the stored body is the text. + const result = await read(png(chunk("zTXt", concat(latin1("prompt\0"), [0], body))), { decompress: Stream }); + assert.equal(result.status, "complete"); + assert.equal(result.records.prompt, body.toString("latin1")); + assert.ok(stats.writes.every((size) => size <= 256), JSON.stringify(stats.writes)); + assert.equal(stats.writes.reduce((sum, size) => sum + size, 0), body.length); + assert.equal(stats.maxInFlight, 1); +}); + +test("inflation settles once on abort with an AbortError and no unhandled rejection", async () => { + const unhandled = []; + const onUnhandled = (reason) => unhandled.push(reason); + process.on("unhandledRejection", onUnhandled); + try { + const { stats, Stream } = instrumentedStream(() => new Promise(() => {})); + const controller = new AbortController(); + const pending = readPngMetadataFile(blobOf(png(zTXt("prompt", "p".repeat(1000)))), { decompress: Stream, signal: controller.signal }); + let settled = 0; + pending.then(() => settled++, () => settled++); + await new Promise((resolve) => setTimeout(resolve, 5)); + assert.equal(settled, 0, "the read waits on the stream"); + controller.abort(); + await assert.rejects(pending, (error) => error.name === "AbortError"); + await new Promise((resolve) => setTimeout(resolve, 10)); + assert.equal(settled, 1); + assert.equal(stats.maxInFlight, 1); + assert.deepEqual(unhandled, []); + } finally { + process.off("unhandledRejection", onUnhandled); + } +}); + +test("a stream error under a live signal is still invalid-png, not an abort", async () => { + const controller = new AbortController(); + const result = await read(png(zTXt("prompt", "p", { compressed: [1, 2, 3, 4, 5, 6, 7, 8] })), { signal: controller.signal }); + assert.deepEqual(result, { status: "unreadable", reason: "invalid-png" }); + assert.equal(controller.signal.aborted, false); +}); + +test("an already aborted signal rejects before any slice", async () => { + const controller = new AbortController(); + controller.abort(); + const file = recordingBlob(png(tEXt("prompt", "p"))); + await assert.rejects(readPngMetadataFile(file, { signal: controller.signal }), (error) => error.name === "AbortError"); + assert.deepEqual(file.ranges, []); +}); + +test("aborting between slices rejects with an AbortError", async () => { + const controller = new AbortController(); + const whole = png(tEXt("prompt", "p")); + let slices = 0; + const file = { + size: whole.length, + slice(start, end) { + return { + arrayBuffer: async () => { + if (++slices === 3) controller.abort(); + return whole.slice(start, end).buffer; + }, + }; + }, + }; + await assert.rejects(readPngMetadataFile(file, { signal: controller.signal }), (error) => error.name === "AbortError"); +}); + +test("aborting while inflating a real deflate stream rejects with an AbortError", async () => { + const controller = new AbortController(); + let inflating = false; + class Tracked extends DecompressionStream { + constructor(format) { + super(format); + inflating = true; + } + } + // The slow slices make the pacer yield inside the inflater's feed loop; that + // yield aborts, so the real stream is torn down mid-inflation. + const yieldControl = async () => { + if (inflating) controller.abort(); + }; + const slow = slowBlob(png(zTXt("prompt", "p", { compressed: require("node:crypto").randomBytes(5000) })), 10); + await assert.rejects(readPngMetadataFile(slow, { signal: controller.signal, yieldControl, decompress: Tracked }), (error) => error.name === "AbortError"); + assert.equal(inflating, true); +}); + +test("yields control about every 8 ms while the file answers slowly", async () => { + let yields = 0; + const yieldControl = async () => { + yields += 1; + }; + const result = await readPngMetadataFile(slowBlob(png(tEXt("prompt", "p")), 10), { yieldControl }); + assert.equal(result.status, "complete"); + assert.ok(yields >= 1, `yielded ${yields} times`); +}); + +test("the default yield lets the event loop turn", async () => { + const result = await readPngMetadataFile(slowBlob(png(tEXt("prompt", "p")), 10)); + assert.deepEqual(result.records, { prompt: "p" }); +}); + +test("a slice that fails is a read failure", async () => { + assert.deepEqual(await readPngMetadataFile(failingBlob(100)), { status: "unreadable", reason: "read-failure" }); + assert.deepEqual(await readPngMetadataFile(null), { status: "unreadable", reason: "read-failure" }); +}); + +test("a file that shrank under the reader is a read failure", async () => { + const whole = png(tEXt("prompt", "p")); + assert.deepEqual(await readPngMetadataFile(shrunkBlob(whole.subarray(0, 40), whole.length)), { status: "unreadable", reason: "read-failure" }); +}); + +test("classifier", () => { + assert.deepEqual(classifyPrefix(bytes.SIGNATURE), { media: "image", kind: "png" }); + assert.deepEqual(classifyPrefix(Uint8Array.from([0xff, 0xd8, 0xff, 0xe0])), { media: "image", kind: "jpeg" }); + assert.deepEqual(classifyPrefix(latin1("RIFF\0\0\0\0WEBPVP8 ")), { media: "image", kind: "webp" }); + assert.deepEqual(classifyPrefix(latin1("\0\0\0\x18ftypisom\0\0\0\0")), { media: "video", kind: "iso-bmff" }); + assert.deepEqual(classifyPrefix(latin1("\0\0\0\x14ftypqt ")), { media: "video", kind: "iso-bmff" }); + assert.deepEqual(classifyPrefix(Uint8Array.from([0x1a, 0x45, 0xdf, 0xa3, 0x01])), { media: "video", kind: "matroska" }); + assert.deepEqual(classifyPrefix(latin1("RIFF\0\0\0\0AVI LIST")), { media: "video", kind: "avi" }); + const asf = Uint8Array.from([0x30, 0x26, 0xb2, 0x75, 0x8e, 0x66, 0xcf, 0x11, 0xa6, 0xd9, 0x00, 0xaa, 0x00, 0x62, 0xce, 0x6c, 0, 0]); + assert.deepEqual(classifyPrefix(asf), { media: "video", kind: "asf" }); + assert.deepEqual(classifyPrefix(asf.buffer), { media: "video", kind: "asf" }); + for (const text of ["plain text file", "", ' { + // tests/test_local_inspect_parity.py asserts the same two files against Pillow. + const jpeg = blobOf(fs.readFileSync(path.join(FIXTURES, "tiny.jpg"))); + assert.deepEqual(await readImageHeader(jpeg, "jpeg"), { format: "JPEG", width: 48, height: 32 }); + const webp = blobOf(fs.readFileSync(path.join(FIXTURES, "tiny.webp"))); + assert.deepEqual(await readImageHeader(webp, "webp"), { format: "WEBP", width: 40, height: 24 }); +}); + +test("image header: JPEG marker walk skips padding, junk, standalone markers and tables", async () => { + const segment = (marker, body) => concat([0xff, marker], u32(body.length + 2).subarray(2), body); + const sof = (marker, width, height) => segment(marker, concat([8], u32(height).subarray(2), u32(width).subarray(2), [3])); + const sized = concat([0xff, 0xd8], segment(0xe1, fill(0, 20)), [0xff, 0xff, 0xff, 0x00, 0x00, 0x11], [0xff, 0xd0], [0xff, 0x01], segment(0xc4, fill(0, 4)), segment(0xc8, []), segment(0xcc, [1]), sof(0xc2, 300, 200)); + assert.deepEqual(await readImageHeader(blobOf(sized), "jpeg"), { format: "JPEG", width: 300, height: 200 }); + for (const marker of [0xc0, 0xc1, 0xc3, 0xc5, 0xc6, 0xc7, 0xc9, 0xca, 0xcb, 0xcd, 0xce, 0xcf]) { + assert.deepEqual(await readImageHeader(blobOf(concat([0xff, 0xd8], sof(marker, 7, 9))), "jpeg"), { format: "JPEG", width: 7, height: 9 }, marker.toString(16)); + } + const none = { format: "JPEG", width: null, height: null }; + assert.deepEqual(await readImageHeader(blobOf(concat([0xff, 0xd8], segment(0xda, [1]))), "jpeg"), none, "SOS before SOF"); + assert.deepEqual(await readImageHeader(blobOf(concat([0xff, 0xd8], [0xff])), "jpeg"), none, "ends on a marker byte"); + assert.deepEqual(await readImageHeader(blobOf(concat([0xff, 0xd8], [0xff, 0xe1, 0x00])), "jpeg"), none, "ends inside a length"); + assert.deepEqual(await readImageHeader(blobOf(concat([0xff, 0xd8], [0xff, 0xc0, 0x00, 0x08, 8, 0])), "jpeg"), none, "ends inside SOF"); + assert.deepEqual(await readImageHeader(blobOf(concat([0xff, 0xd8], segment(0xe1, []))), "jpeg"), none, "no SOF at all"); + assert.deepEqual(await readImageHeader(blobOf(Uint8Array.from([0xff])), "jpeg"), none, "too short"); + assert.deepEqual(await readImageHeader(blobOf(latin1("RIFF")), "jpeg"), none, "not a JPEG"); + // A SOF past the 64 KiB read gives no dimensions rather than reading on. + assert.deepEqual(await readImageHeader(blobOf(concat([0xff, 0xd8], segment(0xe1, fill(0, 65000)), segment(0xe2, fill(0, 1000)), sof(0xc0, 5, 5))), "jpeg"), none, "SOF beyond 64 KiB"); +}); + +test("image header: WebP VP8, VP8L and VP8X headers", async () => { + const riff = (...rest) => concat(latin1("RIFF"), u32(0), latin1("WEBP"), ...rest); + const chunkHeader = (type, size) => concat(latin1(type), Uint8Array.from([size & 0xff, (size >>> 8) & 0xff, (size >>> 16) & 0xff, 0])); + const vp8 = riff(chunkHeader("VP8 ", 100), [0x30, 0x0b, 0x00, 0x9d, 0x01, 0x2a], [40, 0x40], [24, 0x80], fill(0, 20)); + assert.deepEqual(await readImageHeader(blobOf(vp8), "webp"), { format: "WEBP", width: 40, height: 24 }, "scale bits are masked"); + // 14-bit width-1 = 63 and height-1 = 47, little-endian bit-packed after the 0x2f signature. + const vp8l = riff(chunkHeader("VP8L", 100), [0x2f], [0x3f, 0xc0, 0x0b, 0x00, 0x00], fill(0, 20)); + assert.deepEqual(await readImageHeader(blobOf(vp8l), "webp"), { format: "WEBP", width: 64, height: 48 }); + const vp8x = riff(chunkHeader("VP8X", 10), [0x10, 0, 0, 0], [0xff, 0x00, 0x01], [0x2f, 0x00, 0x00], fill(0, 20)); + assert.deepEqual(await readImageHeader(blobOf(vp8x), "webp"), { format: "WEBP", width: 65792, height: 48 }); + const none = { format: "WEBP", width: null, height: null }; + assert.deepEqual(await readImageHeader(blobOf(riff(chunkHeader("VP8 ", 100), [0x30, 0x0b, 0x00, 0x00, 0x01, 0x2a], fill(0, 24))), "webp"), none, "bad start code"); + assert.deepEqual(await readImageHeader(blobOf(riff(chunkHeader("VP8L", 100), [0x00], fill(0, 24))), "webp"), none, "bad VP8L signature"); + assert.deepEqual(await readImageHeader(blobOf(riff(chunkHeader("ALPH", 100), fill(0, 24))), "webp"), none, "unknown first chunk"); + assert.deepEqual(await readImageHeader(blobOf(riff(chunkHeader("VP8 ", 100))), "webp"), none, "too short"); + assert.deepEqual(await readImageHeader(blobOf(latin1("RIFF\0\0\0\0WAVEfmt ".padEnd(40, "\0"))), "webp"), none, "not WebP"); +}); + +test("image header: other kinds give no format and the read stays within 64 KiB", async () => { + assert.deepEqual(await readImageHeader(blobOf(bytes.SIGNATURE), "png"), { format: null, width: null, height: null }); + const file = recordingBlob(concat([0xff, 0xd8], fill(0, 200000))); + await readImageHeader(file, "jpeg"); + assert.deepEqual(file.ranges, [[0, 65536]]); + const small = recordingBlob(concat([0xff, 0xd8, 0xff])); + await readImageHeader(small, "jpeg"); + assert.deepEqual(small.ranges, [[0, 3]]); +}); diff --git a/tests/js/support-module-urls.test.js b/tests/js/support-module-urls.test.js new file mode 100644 index 0000000..7a96132 --- /dev/null +++ b/tests/js/support-module-urls.test.js @@ -0,0 +1,39 @@ +// The Node oracles that Python drives (tests/js/support/*.mjs) import product +// modules dynamically. A dynamic import takes a URL or a bare/relative specifier, +// never a filesystem path: on Windows `D:\...\x.mjs` parses as the `d:` scheme +// (ERR_UNSUPPORTED_ESM_URL_SCHEME). Every such import goes through moduleUrl, and +// this test proves the Windows form on any host so Linux catches a regression. +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const here = path.dirname(fileURLToPath(import.meta.url)); +const support = path.join(here, "support"); +const m = await import("./support/module-url.mjs").catch((error) => { + if (error.code !== "ERR_MODULE_NOT_FOUND") throw error; + return {}; +}); + +test("moduleUrl turns a filesystem path into a file: URL on every platform", async () => { + assert.equal(typeof m.moduleUrl, "function", "moduleUrl is exported"); + assert.equal(m.moduleUrl("D:\\x\\y z.mjs", { windows: true }), "file:///D:/x/y%20z.mjs"); + assert.equal(m.moduleUrl("C:\\repo\\web\\js\\workspace\\local-video.mjs", { windows: true }), "file:///C:/repo/web/js/workspace/local-video.mjs"); + assert.equal(m.moduleUrl("/repo/web/js/a b.mjs", { windows: false }), "file:///repo/web/js/a%20b.mjs"); + const own = m.moduleUrl(path.join(here, "support", "module-url.mjs")); + assert.ok(own.startsWith("file:///"), own); + assert.equal((await import(own)).moduleUrl, m.moduleUrl, "the URL imports the same module instance"); +}); + +test("every dynamic import in the Node oracles is a file URL, never a path", () => { + const scripts = fs.readdirSync(support).filter((name) => name.endsWith(".mjs")); + assert.ok(scripts.includes("local-inspect-oracle.mjs")); + for (const name of scripts) { + const source = fs.readFileSync(path.join(support, name), "utf8").split("\n").filter((line) => !line.trim().startsWith("//")).join("\n"); + for (const match of source.matchAll(/\bimport\(\s*([^)]*?)[,)]/g)) { + const argument = match[1].trim(); + assert.ok(/^(moduleUrl\(|pathToFileURL\(|["'`]\.{1,2}\/)/.test(argument), `${name}: import(${argument}) must be a file URL or a relative specifier`); + } + } +}); diff --git a/tests/js/support/local-inspect-oracle.mjs b/tests/js/support/local-inspect-oracle.mjs new file mode 100644 index 0000000..61711ce --- /dev/null +++ b/tests/js/support/local-inspect-oracle.mjs @@ -0,0 +1,59 @@ +// The browser's local readers and projection run over files on disk, for +// tests/test_local_inspect_parity.py, which compares every answer with the server's +// utils.prompt_inspector.read_inspection_file and utils.inspection_response._metadata +// on the same file. Each file is opened as a Blob (fs.openAsBlob) so the reader +// slices it exactly as it slices a File. +// stdin: [{name, path} | {name, tags, limited?} | {name, records, record_formats?, carrier_text?, limited?}] +// stdout: {name: {classified, result, metadata}} where result is readPngMetadataFile's +// answer for a PNG, readImageHeader's for another image, for a video +// {read: readVideoTags(...), selection: selectVideoCarrier(read.tags, {limited})} +// (selection null unless the read completed), else null; metadata is +// projectMetadata(...) of the local read (the server's _metadata) for every +// classified file, else null. +// A {tags} entry answers {classified: null, result: {selection}, metadata} for +// the carrier rule alone, as the server selects from ffprobe's tag map. +// A {records} entry (records null = an unreadable file) answers +// {classified: null, result: null, metadata} for the projection alone. +import fs from 'node:fs'; +import path from 'node:path'; +import { createRequire } from 'node:module'; +import { fileURLToPath } from 'node:url'; +import { moduleUrl } from './module-url.mjs'; + +const here = path.dirname(fileURLToPath(import.meta.url)); +const require = createRequire(import.meta.url); +const pngMetadata = require(path.join(here, '..', '..', '..', 'web', 'js', 'png-metadata.js')); +const web = path.join(here, '..', '..', '..', 'web', 'js', 'workspace'); +// ES modules import by file URL (a Windows path would parse as a URL scheme). +const { selectVideoCarrier } = await import(moduleUrl(path.join(web, 'local-video.mjs'))); +const { projectMetadata } = await import(moduleUrl(path.join(web, 'local-projection.mjs'))); +const { readLocalFile, toInspectionRead } = await import(moduleUrl(path.join(web, 'local-inspection.mjs'))); + +const selectionRead = (selection) => selection.status === 'complete' ? + { records: selection.records, record_formats: selection.record_formats, carrierText: selection.carrierText, limited: selection.limited } : null; + +const chunks = []; +for await (const chunk of process.stdin) chunks.push(chunk); +const output = {}; +for (const entry of JSON.parse(Buffer.concat(chunks).toString('utf8'))) { + const { name } = entry; + if (entry.records !== undefined) { + const read = entry.records === null ? null : { + records: entry.records, + record_formats: entry.record_formats ?? Object.fromEntries(Object.keys(entry.records).map((key) => [key, 'original-text'])), + carrierText: entry.carrier_text ?? null, limited: entry.limited ?? [], + }; + output[name] = { classified: null, result: null, metadata: projectMetadata(read) }; + continue; + } + if (entry.path === undefined) { + const selection = selectVideoCarrier(entry.tags, { limited: entry.limited || [] }); + output[name] = { classified: null, result: { selection }, metadata: projectMetadata(selectionRead(selection)) }; + continue; + } + const file = await fs.openAsBlob(entry.path); + const { classification, result } = await readLocalFile(file, { pngMetadata }); + const metadata = classification === null ? null : projectMetadata(toInspectionRead(classification, result).read); + output[name] = { classified: classification, result, metadata }; +} +process.stdout.write(JSON.stringify(output)); diff --git a/tests/js/support/module-url.mjs b/tests/js/support/module-url.mjs new file mode 100644 index 0000000..1c8c93f --- /dev/null +++ b/tests/js/support/module-url.mjs @@ -0,0 +1,9 @@ +// A dynamic import() takes a URL, never a filesystem path: on Windows a path such +// as D:\repo\x.mjs parses as the "d:" scheme (ERR_UNSUPPORTED_ESM_URL_SCHEME). +// Every oracle in this directory imports product modules through moduleUrl. +import { pathToFileURL } from 'node:url'; + +/** The file: URL for a filesystem path; `windows` forces the Windows form on any host (for tests). */ +export function moduleUrl(filePath, { windows = process.platform === 'win32' } = {}) { + return pathToFileURL(filePath, { windows }).href; +} diff --git a/tests/js/workspace-local-inspection.test.js b/tests/js/workspace-local-inspection.test.js new file mode 100644 index 0000000..f20400d --- /dev/null +++ b/tests/js/workspace-local-inspection.test.js @@ -0,0 +1,308 @@ +// web/js/workspace/local-inspection.mjs turns one chosen File's local reads into +// the inspection wire the workspace already renders, fits it into the 8 MiB +// response budget exactly as the server does, and plugs into the shared +// inspection controller, inspector and viewer through localFileAdapter. +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import path from "node:path"; +import { createRequire } from "node:module"; +import { fileURLToPath } from "node:url"; +import { raw, ihdr, idat, iend, tEXt, iTXt, utf8 } from "./helpers/png-bytes.mjs"; +import { createInspectionController, toInspectionRecord, projectInspectionPayload } from "../../web/js/workspace/inspection-state.mjs"; +import { inspectionDownloads } from "../../web/js/workspace/file-inspector.mjs"; + +const here = path.dirname(fileURLToPath(import.meta.url)); +const require = createRequire(import.meta.url); +const PngMetadata = require("../../web/js/png-metadata.js"); +const ComfyMetadata = require("../../web/js/comfy-metadata.js"); +const { extractGenerationFacts } = ComfyMetadata; +// The page loads both as scripts; the legacy fill and the PNG reader find them on the global. +globalThis.PngMetadata = PngMetadata; +globalThis.ComfyMetadata = ComfyMetadata; +const m = await import("../../web/js/workspace/local-inspection.mjs").catch((error) => { + if (error.code !== "ERR_MODULE_NOT_FOUND") throw error; + return {}; +}); + +const FIXTURES = path.join(here, "..", "fixtures", "local-inspect"); +const PARITY = path.join(here, "..", "fixtures", "inspection_parity"); +const PROMPT = '{"3":{"class_type":"KSampler","inputs":{"seed":7,"steps":20,"cfg":8.0,"sampler_name":"euler","scheduler":"normal","denoise":1.0,"model":["4",0],"positive":["6",0],"negative":["7",0],"latent_image":["5",0]}},"4":{"class_type":"CheckpointLoaderSimple","inputs":{"ckpt_name":"synthetic-v1.safetensors"}},"5":{"class_type":"EmptyLatentImage","inputs":{"width":512,"height":512,"batch_size":1}},"6":{"class_type":"CLIPTextEncode","inputs":{"text":"a red fox in snow, detailed fur","clip":["4",1]}},"7":{"class_type":"CLIPTextEncode","inputs":{"text":"blurry, lowres","clip":["4",1]}}}'; +const WORKFLOW = '{"last_node_id":7,"nodes":[{"id":3,"type":"KSampler","widgets_values":[7,"fixed",20,8.0,"euler","normal",1.0]}],"links":[],"version":0.4}'; +const CONTAINER_WORKFLOW = "Original workflow JSON text is unavailable for this container record. Download metadata or the original file."; +const LIMIT = 8388608; +const plain = (value) => JSON.parse(JSON.stringify(value)); +const bytes = (value) => Buffer.byteLength(JSON.stringify(value), "utf8"); + +function pngFile(name, chunks, { lastModified = 1_700_000_000_123 } = {}) { + return new File([raw(ihdr(640, 480), idat(), ...chunks, iend())], name, { type: "image/png", lastModified }); +} +const fixtureFile = (name, directory = FIXTURES) => new File([fs.readFileSync(path.join(directory, name))], name, { lastModified: 1_700_000_000_000 }); +const classify = async (file) => PngMetadata.classifyPrefix(new Uint8Array(await file.slice(0, 32).arrayBuffer())); +async function handleFor(file, { url = "blob:http://host.test/11111111-2222-3333-4444-555555555555", filename } = {}) { + return m.createLocalFileHandle({ file, url, classification: await classify(file), filename }); +} +async function recordFor(file, options) { + const handle = await handleFor(file, options); + const wire = await m.localFileAdapter.read(handle); + return { handle, wire, record: m.localFileAdapter.toRecord(wire, handle, extractGenerationFacts) }; +} + +test("contract", () => { + for (const name of ["buildLocalPayload", "fitLocalPayload", "createLocalFileHandle", "readLocalFile", "toInspectionRead", "sanitizeLocalFilename"]) { + assert.equal(typeof m[name], "function", `${name} is exported`); + } + assert.equal(typeof m.localFileAdapter, "object"); + for (const name of ["validate", "key", "scope", "inspectionPath", "read", "toRecord", "originalDownload", "mediaUrl"]) { + assert.equal(typeof m.localFileAdapter[name], "function", `adapter.${name}`); + } + assert.ok(Object.isFrozen(m.localFileAdapter)); +}); + +test("a PNG File becomes the server-shaped wire with local provenance", async () => { + const file = pngFile("Fox Render.PNG", [tEXt("prompt", PROMPT), tEXt("workflow", WORKFLOW)]); + const { wire, record, handle } = await recordFor(file); + assert.deepEqual(wire.file, { observation: "selected-local-file", source_label: "Selected local file", media_type: "image", extension: ".png", format: "PNG", + size: file.size, width: 640, height: 480, modified_time: 1_700_000_000.123 }); + assert.equal(wire.success, true); + assert.equal(wire.inspection_version, 1); + assert.deepEqual(wire.prompts, { positive: { status: "unresolved" }, negative: { status: "unresolved" } }); + assert.deepEqual(wire.resolution, { state: "unresolved", sampler_id: null, reason: "Resolved in this browser from the workflow.", reason_code: null, node: null }); + assert.deepEqual(wire.library, { status: "unavailable", scope: "saved-library", tags: [], category_scope: null, tags_scope: null }); + assert.equal(wire.metadata.status, "available"); + assert.deepEqual(wire.metadata.raw, { prompt: PROMPT, workflow: WORKFLOW }); + assert.deepEqual(wire.metadata.record_formats, { prompt: "original-text", workflow: "original-text" }); + assert.equal(wire.metadata.prompt_graph.status, "parsed"); + assert.equal(plain(wire.metadata.prompt_graph.value)["3"].inputs.cfg, "8.0"); + assert.equal(wire.metadata.workflow_graph.status, "parsed"); + assert.equal(wire.metadata.parameters, null); + for (const key of Object.keys(wire)) assert.ok(["success", "inspection_version", "file", "prompts", "resolution", "metadata", "library"].includes(key), key); + // The existing projection renders it: facts from the API graph, legacy fill for the roles, no library link. + assert.deepEqual(record.identity, { kind: "local-file", key: handle.key, path: "Fox Render.PNG" }); + assert.equal(record.file.observation, "selected-local-file"); + assert.equal(record.generation.seed.value, "7"); + assert.equal(record.generation.cfg.value, "8.0"); + assert.equal(record.prompts.positive.text, "a red fox in snow, detailed fur"); + assert.equal(record.prompts.positive.source, "graph-inferred"); + assert.equal(record.prompts.negative.text, "blurry, lowres"); + assert.equal(record.historyTarget, null); + assert.deepEqual(record.download, { url: "blob:http://host.test/11111111-2222-3333-4444-555555555555", filename: "Fox Render.PNG" }); + const downloads = inspectionDownloads(record); + assert.equal(downloads.metadata.available, true); + assert.equal(downloads.workflow.bodyText, WORKFLOW); + assert.equal(downloads.metadata.filename, "Fox Render.metadata.json"); + const exported = JSON.parse(downloads.metadata.bodyText); + assert.equal(exported.identity.kind, "local-file"); + assert.equal(exported.file.source_label, "Selected local file"); +}); + +test("file facts follow the server: extension like Path.suffix, null when unknown", async () => { + const cases = [["a.b.PnG", ".png"], [".hidden", ""], ["noext", ""], ["dot.", ""], ["x.tar.gz", ".gz"]]; + for (const [name, extension] of cases) { + const wire = m.buildLocalPayload({ read: { status: "complete", records: {}, limited: [], width: 2, height: 3 }, file: { name, size: 10, lastModified: 0 }, classification: { media: "image", kind: "png" } }); + assert.equal(wire.file.extension, extension, name); + assert.equal(wire.file.modified_time, 0); + } + const sparse = m.buildLocalPayload({ read: { status: "complete", records: {}, limited: [], width: 2, height: 3 }, file: { name: "a.png" }, classification: { media: "image", kind: "png" } }); + assert.deepEqual([sparse.file.size, sparse.file.modified_time], [null, null]); + assert.equal(m.buildLocalPayload({ read: { status: "complete", records: {}, limited: [], width: 2, height: 3 }, file: { name: "a.png", size: -1, lastModified: NaN }, classification: { media: "image", kind: "png" } }).file.size, null); + assert.throws(() => m.buildLocalPayload({ read: null, file: { name: "a" }, classification: null }), TypeError); + assert.throws(() => m.buildLocalPayload({ read: null, file: { name: "a" }, classification: { media: "text", kind: "svg" } }), TypeError); +}); + +test("sanitized filenames carry no separators or control characters", () => { + assert.equal(m.sanitizeLocalFilename("../a/b\\c\u0000\u001f\u007f.png"), ".._a_b_c.png"); + assert.equal(m.sanitizeLocalFilename(" spaced .png "), "spaced .png"); + assert.equal(m.sanitizeLocalFilename(""), "file"); + assert.equal(m.sanitizeLocalFilename(undefined), "file"); + assert.equal(m.sanitizeLocalFilename("\u0000"), "file"); + assert.equal([...m.sanitizeLocalFilename("😀".repeat(300))].length, 255); +}); + +test("unreadable PNG metadata is unreadable, not absent", async () => { + const { wire, record } = await recordFor(pngFile("bad.png", [tEXt("prompt", PROMPT, { crc: 0 })])); + assert.equal(wire.metadata.status, "unreadable"); + assert.deepEqual([wire.file.format, wire.file.width, wire.file.height], [null, null, null]); + assert.equal(record.generation.seed.reason, "metadata-unreadable"); + assert.equal(record.prompts.positive.status, "unresolved"); + assert.equal(JSON.parse(inspectionDownloads(record).metadata.bodyText).metadata.status, "unreadable"); + const limited = await recordFor(pngFile("big.png", [tEXt("prompt", PROMPT), tEXt("workflow", "[" + "0,".repeat(2_097_153) + "]")])); + assert.deepEqual(limited.wire.metadata.record_formats, { prompt: "original-text", workflow: "limited" }); + assert.equal(limited.wire.metadata.workflow_graph.status, "limited"); + assert.equal(limited.record.metadata.record_formats.workflow, "limited"); +}); + +test("JPEG and WebP previews have header facts and absent metadata", async () => { + for (const [name, format, width, height] of [["tiny.jpg", "JPEG", 48, 32], ["tiny.webp", "WEBP", 40, 24]]) { + const { wire, record } = await recordFor(fixtureFile(name)); + assert.deepEqual([wire.file.format, wire.file.width, wire.file.height, wire.file.media_type], [format, width, height, "image"], name); + assert.equal(wire.metadata.status, "absent"); + assert.equal(record.generation.seed.status, "not-recorded"); + } +}); + +test("video payloads", async () => { + const legacy = await recordFor(fixtureFile("legacy-comment.mp4")); + assert.deepEqual([legacy.wire.file.media_type, legacy.wire.file.format, legacy.wire.file.width, legacy.wire.file.extension], ["video", null, null, ".mp4"]); + assert.equal(legacy.wire.metadata.record_formats.workflow, "reserialized-json"); + assert.equal(legacy.wire.metadata.record_formats.prompt, "original-text"); + assert.equal(legacy.wire.metadata.workflow_graph.status, "parsed"); + assert.equal(inspectionDownloads(legacy.record).workflow.available, false); + assert.equal(inspectionDownloads(legacy.record).workflow.reason, CONTAINER_WORKFLOW); + assert.equal(legacy.record.prompts.positive.text, "a red fox in snow, detailed fur"); + // Ruling M1: a Matroska COMMENT tag is not the exact-case `comment` envelope, so it is absent, as Images reports it. + const webm = await recordFor(fixtureFile("legacy-comment.webm")); + assert.equal(webm.wire.metadata.status, "absent"); + const tags = await recordFor(fixtureFile("core-tags.webm")); + assert.deepEqual(tags.wire.metadata.record_formats, { prompt: "original-text", workflow: "original-text" }); + assert.equal(tags.record.generation.seed.value, "7"); + // AVI and ASF are recognised, previewable, and never read. + const avi = new File([utf8("RIFF"), new Uint8Array([0, 0, 0, 0]), utf8("AVI LIST"), new Uint8Array(64)], "clip.avi"); + const aviRecord = await recordFor(avi); + assert.equal(aviRecord.wire.metadata.status, "unreadable"); + assert.equal(aviRecord.wire.file.media_type, "video"); + assert.equal(aviRecord.record.generation.seed.reason, "metadata-unreadable"); + const asf = new File([new Uint8Array([0x30, 0x26, 0xb2, 0x75, 0x8e, 0x66, 0xcf, 0x11, 0xa6, 0xd9, 0x00, 0xaa, 0x00, 0x62, 0xce, 0x6c]), new Uint8Array(64)], "clip.wmv"); + assert.equal((await recordFor(asf)).wire.metadata.status, "unreadable"); + // A pre-classified video needs no PNG reader; an image without one is an error, not a silent absence. + const clip = fixtureFile("core.mkv"); + assert.equal((await m.readLocalFile(clip, { classification: { media: "video", kind: "matroska" }, pngMetadata: null })).result.read.status, "complete"); + await assert.rejects(m.readLocalFile(clip, { pngMetadata: null }), /PNG reader/); + await assert.rejects(m.readLocalFile(fixtureFile("tiny.jpg"), { classification: { media: "image", kind: "jpeg" }, pngMetadata: null }), /PNG reader/); + await assert.rejects(m.readLocalFile("not a blob"), TypeError); + assert.deepEqual(await m.readLocalFile(new File([utf8("")], "x.png"), { pngMetadata: PngMetadata }), { classification: null, result: null }); + // A container that fails to parse is unreadable too. + const broken = new File([utf8("\0\0\0\x08ftypisom"), utf8("\0\0\0\x01moov")], "broken.mp4"); + assert.equal((await recordFor(broken)).wire.metadata.status, "unreadable"); +}); + +test("local cap measures escaped UTF-8 atomically", async () => { + // A 1 MiB Latin-1 prompt of 'é' (2 UTF-8 bytes, 1 code unit each) and a 4 MiB UTF-8 workflow of + // emoji (4 bytes, 2 units): the UTF-16 length of the wire is well under the limit while its UTF-8 + // size is not, so a cap counting code units would keep too much. + const prompt = `{"1":{"class_type":"A","inputs":{"t":"${"é".repeat(1_000_000)}"}}}`; + const workflow = `{"nodes":[],"u":"${"😀".repeat(1_000_000)}"}`; + const file = pngFile("wide.png", [iTXt("workflow", workflow), tEXt("prompt", prompt)]); + const handle = await handleFor(file); + const raw = await m.readLocalFile(handle.file, { classification: handle.classification, pngMetadata: PngMetadata }); + assert.equal(raw.result.status, "complete"); + const wire = m.buildLocalPayload({ read: raw.result, file: { name: file.name, size: file.size, lastModified: file.lastModified }, classification: handle.classification }); + const text = JSON.stringify(wire); + assert.ok(text.length < LIMIT && bytes(wire) > LIMIT, `${text.length} code units, ${bytes(wire)} bytes`); + const fitted = await m.fitLocalPayload(wire); + assert.ok(bytes(fitted) <= LIMIT); + assert.equal(fitted.metadata.workflow_graph.status, "limited", "the workflow graph goes first"); + assert.equal(fitted.metadata.prompt_graph.status, "parsed"); + assert.deepEqual(Object.keys(fitted.metadata.raw).sort(), ["prompt", "workflow"], "raw is kept"); + const record = m.localFileAdapter.toRecord(fitted, handle, extractGenerationFacts); + assert.equal(inspectionDownloads(record).metadata.available, true); + assert.equal(inspectionDownloads(record).workflow.available, false); + // Whole-raw omission is atomic and marks the response-size limit; the inspector offers no metadata download. + const squeezed = await m.fitLocalPayload(wire, bytes(fitted) - 1); + assert.deepEqual([squeezed.metadata.workflow_graph.status, squeezed.metadata.prompt_graph.status], ["limited", "limited"]); + const tiny = await m.fitLocalPayload(wire, 2000); + assert.equal(tiny.metadata.status, "limited"); + assert.equal(tiny.metadata.omitted_reason, "response-size-limit"); + assert.deepEqual(tiny.metadata.raw, {}); + assert.equal(tiny.metadata.parameters, null); + assert.ok(bytes(tiny) <= 2000); + const omitted = m.localFileAdapter.toRecord(tiny, handle, extractGenerationFacts); + assert.equal(inspectionDownloads(omitted).metadata.available, false); + assert.equal(omitted.generation.seed.reason, "projection-limited"); + await assert.rejects(m.fitLocalPayload(wire, 100), /exceeds/); + assert.equal(wire.metadata.status, "available", "the input wire is not mutated"); + assert.equal((await m.localFileAdapter.read(handle)).metadata.workflow_graph.status, "limited", "adapter.read returns the fitted wire"); +}); + +test("adapter rejects foreign blob URLs and never resolves owned ones", async () => { + const file = pngFile("a.png", [tEXt("prompt", PROMPT)]); + const classification = await classify(file); + const owned = m.createLocalFileHandle({ file, url: "blob:http://host.test/0b0e", classification }); + assert.ok(Object.isFrozen(owned)); + assert.deepEqual(m.localFileAdapter.originalDownload(owned), { url: "blob:http://host.test/0b0e", filename: "a.png" }); + assert.equal(m.localFileAdapter.mediaUrl(owned), "blob:http://host.test/0b0e"); + // The controller and the viewer session spread-copy descriptors: a copy is the same selection. + const copy = { ...owned, occurrence: {}, reference: {} }; + assert.equal(m.localFileAdapter.key(copy), owned.key); + assert.equal(m.localFileAdapter.mediaUrl(copy), "blob:http://host.test/0b0e"); + // A tampered URL, key, File or name, or a descriptor this module never made, is foreign. + const tampered = [{ ...owned, url: "blob:http://host.test/other" }, { ...owned, key: "local:1" }, { ...owned, file: new File([], "x.png") }, { ...owned, filename: "other.png" }, + { kind: "local-file", key: owned.key, filename: owned.filename, url: owned.url, file, is_video: false, classification }, null, "local:1"]; + for (const foreign of tampered) { + for (const method of ["validate", "originalDownload", "mediaUrl", "key", "scope", "inspectionPath"]) assert.throws(() => m.localFileAdapter[method](foreign), TypeError, method); + await assert.rejects(m.localFileAdapter.read(foreign), TypeError); + assert.throws(() => m.localFileAdapter.toRecord({ success: true, inspection_version: 1 }, foreign, extractGenerationFacts), TypeError); + } + for (const url of ["http://host.test/x.png", "/prompt_manager/images/serve/a.png", "javascript:alert(1)", "data:image/png;base64,AA==", "", 5]) { + assert.throws(() => m.createLocalFileHandle({ file, url, classification }), TypeError, String(url)); + } + assert.throws(() => m.createLocalFileHandle({ file: "not a blob", url: "blob:http://host.test/1", classification }), TypeError); + assert.throws(() => m.createLocalFileHandle({ file, url: "blob:http://host.test/1", classification: null }), TypeError); + assert.throws(() => m.createLocalFileHandle({ file, url: "blob:http://host.test/1", classification: { media: "text", kind: "svg" } }), TypeError); + // A proxy-mounted host resolves API URLs; blob URLs are session-owned and bypass it. + const before = Object.getOwnPropertyDescriptor(globalThis, "window"); + Object.defineProperty(globalThis, "window", { configurable: true, value: { location: { href: "https://example.test/proxy/prompt_manager/workspace" }, ApiClient: { resolveUrl: (url) => `/proxy${url}` } } }); + try { + assert.equal(m.localFileAdapter.mediaUrl(owned), "blob:http://host.test/0b0e"); + assert.equal(m.localFileAdapter.originalDownload(owned).url, "blob:http://host.test/0b0e"); + } finally { + if (before) Object.defineProperty(globalThis, "window", before); else delete globalThis.window; + } + // Without an object URL the record has no download and the viewer cannot show the original. + const previewless = m.createLocalFileHandle({ file, url: null, classification }); + assert.equal(m.localFileAdapter.mediaUrl(previewless), null); + assert.throws(() => m.localFileAdapter.originalDownload(previewless), TypeError); + const record = m.localFileAdapter.toRecord(await m.localFileAdapter.read(previewless), previewless, extractGenerationFacts); + assert.equal(record.download, null); + assert.equal(inspectionDownloads(record).metadata.filename, "a.metadata.json"); +}); + +test("select reuses the same local key", async () => { + const file = pngFile("a.png", [tEXt("prompt", PROMPT)]); + const classification = await classify(file); + const first = m.createLocalFileHandle({ file, url: "blob:http://host.test/1", classification }); + const second = m.createLocalFileHandle({ file, url: "blob:http://host.test/2", classification }); + assert.notEqual(first.key, second.key, "the same bytes chosen twice are two selections"); + assert.equal(m.localFileAdapter.scope(first), "local"); + assert.equal(m.localFileAdapter.inspectionPath(first), `local-file:${first.key}`); + assert.equal(m.localFileAdapter.key(first), first.key); + const reads = []; + // Until the loader seam lands (PR L2 Task 4) the controller reaches the adapter through api.get. + const api = { get: (sentinel) => { reads.push(sentinel); return m.localFileAdapter.read(reads.length === 1 || sentinel === `local-file:${first.key}` ? first : second); } }; + const controller = createInspectionController({ api, extractFacts: extractGenerationFacts, adapter: m.localFileAdapter }); + const pending = controller.select(first); + assert.equal(controller.select(first), pending, "the same handle returns the pending read"); + await pending; + assert.deepEqual(reads, [`local-file:${first.key}`]); + assert.equal(controller.state.record.identity.key, first.key); + assert.equal(controller.state.record.prompts.positive.text, "a red fox in snow, detailed fur"); + await controller.select(second); + assert.equal(reads.length, 2); + assert.equal(controller.state.record.identity.key, second.key); + await controller.select({ ...second, occurrence: {}, reference: {} }); + assert.equal(reads.length, 2, "the controller's own copy is the same selection"); + await controller.select({ ...second, url: "blob:http://host.test/evil" }); + assert.equal(controller.state.error, true, "a foreign handle is published as an error, never read"); + assert.equal(reads.length, 2); + controller.dispose(); +}); + +test("fileFor admits selected-local-file only for a local identity", async () => { + const file = pngFile("a.png", [tEXt("prompt", PROMPT)]); + const { wire, handle } = await recordFor(file); + const rooted = { root_index: 0, path: "folder/a.png", filename: "a.png", url: "/prompt_manager/images/serve/folder/a.png?root=0" }; + assert.throws(() => toInspectionRecord({ ...wire, image: { root_index: 0, path: "folder/a.png", filename: "a.png" } }, rooted, extractGenerationFacts), /Invalid inspection file/); + assert.throws(() => projectInspectionPayload({ ...wire, file: { ...wire.file, observation: "current-file" } }, { kind: "local-file", key: handle.key, path: "a.png" }, null, extractGenerationFacts), /Invalid inspection file/); + assert.throws(() => m.localFileAdapter.toRecord({ ...wire, success: false }, handle, extractGenerationFacts), TypeError); + assert.throws(() => m.localFileAdapter.toRecord({ ...wire, inspection_version: 2 }, handle, extractGenerationFacts), TypeError); + assert.throws(() => m.localFileAdapter.toRecord(null, handle, extractGenerationFacts), TypeError); +}); + +test("the parity corpus PNGs read into renderable records", async () => { + for (const name of ["f02.png", "f06a.png", "f23.png"]) { + const { wire, record } = await recordFor(fixtureFile(name, PARITY)); + assert.equal(wire.metadata.status, "available", name); + assert.equal(record.file.format, "PNG"); + assert.ok(["parsed", "limited", "malformed"].includes(wire.metadata.prompt_graph.status)); + } +}); diff --git a/tests/js/workspace-local-projection.test.js b/tests/js/workspace-local-projection.test.js new file mode 100644 index 0000000..faa3e5c --- /dev/null +++ b/tests/js/workspace-local-projection.test.js @@ -0,0 +1,249 @@ +// web/js/workspace/local-projection.mjs ports utils/inspection_response.py: the +// strict Decimal/duplicate-key parse, the prompt/workflow/carrier projection, the +// depth and node ceilings, _metadata and the 8 MiB omission steps. These units pin +// the rules the plan names; tests/test_local_inspect_parity.py proves byte parity +// with the server over the committed corpus. +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; + +const here = path.dirname(fileURLToPath(import.meta.url)); +const m = await import("../../web/js/workspace/local-projection.mjs").catch((error) => { + if (error.code !== "ERR_MODULE_NOT_FOUND") throw error; + return {}; +}); +const CASES = JSON.parse(fs.readFileSync(path.join(here, "..", "fixtures", "local-inspect", "projection-cases.json"), "utf8")).cases; + +const plain = (value) => JSON.parse(JSON.stringify(value)); +const bytes = (value) => Buffer.byteLength(JSON.stringify(value), "utf8"); +const graphOf = (text, kind = "workflow") => plain(m.projectGraph(text, kind)); +const nested = (depth, close = "]", open = "[") => open.repeat(depth) + close.repeat(depth); +const nodes = (count) => `{${Array.from({ length: count }, (_, i) => `"${i}":{"class_type":"A","inputs":{}}`).join(",")}}`; + +test("contract", () => { + for (const name of ["projectGraph", "projectMetadata", "fitInspectionPayload", "inspectionBytes", "parseStrictJson"]) { + assert.equal(typeof m[name], "function", `${name} is exported`); + } + assert.equal(m.MAX_RESPONSE_BYTES, 8388608); + assert.equal(m.MAX_PROJECTION_DEPTH, 64); + assert.equal(m.GRAPH_NODE_CEILING, 20000); +}); + +test("projection spells numbers like the server", () => { + // literal -> Python str(Decimal(literal)) (or the int when it is a safe integer) + const table = [ + ["1e3", "1E+3"], ["1e0", "1"], ["-0.0", "-0.0"], ["6.50", "6.50"], ["0.0100", "0.0100"], ["1.0e2", "1.0E+2"], + ["0e5", "0E+5"], ["0.0000001", "1E-7"], ["0.000001", "0.000001"], ["1E+3", "1E+3"], ["1e-0", "1"], ["0.5e1", "5"], + ["0.05e1", "0.5"], ["0.0e0", "0.0"], ["1.5e+300", "1.5E+300"], ["123e-8", "0.00000123"], ["2.5e-3", "0.0025"], + ["1.0", "1.0"], ["-1.50", "-1.50"], ["0.0", "0.0"], ["100.0", "100.0"], ["1234567e-6", "1.234567"], ["0.5E-6", "5E-7"], + ["0.5E-5", "0.000005"], ["1e22", "1E+22"], ["0.000000", "0.000000"], ["-0.000000e-3", "-0E-9"], ["7.0e-6", "0.0000070"], + ["7.0e-7", "7.0E-7"], ["12345678901234567890.123", "12345678901234567890.123"], ["9007199254740992.0", "9007199254740992.0"], + ["1e999999999999999999", "1E+999999999999999999"], ["1e-1999999999999999997", "1E-1999999999999999997"], + ["0.0e1000000000000000000", "0E+999999999999999999"], ["1.5e999999999999999999", "1.5E+999999999999999999"], + ["0.1e-1999999999999999996", "1E-1999999999999999997"], + ["9007199254740991", 9007199254740991], ["-9007199254740991", -9007199254740991], ["9007199254740992", "9007199254740992"], + ["-9007199254740992", "-9007199254740992"], ["10000000000000000", "10000000000000000"], ["123456789012345678901234567890", "123456789012345678901234567890"], + ["0", 0], ["-0", 0], ["42", 42], ["-7", -7], + ]; + for (const [literal, expected] of table) { + const graph = graphOf(`{"n":${literal}}`); + assert.equal(graph.status, "parsed", literal); + assert.deepEqual(graph.value, { n: expected }, literal); + } + assert.deepEqual(graphOf('{"n":[NaN,Infinity,-Infinity]}').value, { n: [null, null, null] }); + // A 4300-digit integer is Python's int() limit; one more digit fails the parse. + assert.deepEqual(graphOf(`{"n":${"1".repeat(4300)}}`).value, { n: "1".repeat(4300) }); + assert.equal(graphOf(`{"n":${"1".repeat(4301)}}`).status, "malformed"); + assert.equal(graphOf(`{"n":-${"1".repeat(4301)}}`).status, "malformed"); + // Exact Decimal construction fails past the C library's exponent range. + for (const literal of ["1e1000000000000000000", "15e999999999999999999", "1e-1999999999999999998", "1.5e-1999999999999999997", + "0e1000000000000000000", "1e9223372036854775808", "0.0e1000000000000000001", "1e99999999999999999999999999"]) { + assert.equal(graphOf(`{"n":${literal}}`).status, "malformed", literal); + } + // Escapes: surrogate pairs join, lone surrogates stay, control characters decode. + const text = graphOf('{"s":"\\u00E9\\uD83D\\uDE00\\uD800\\u0000\\u001f\\/\\\\\\"\\b\\f\\n\\r\\t\\uD800\\u0041"}').value.s; + assert.equal(text, "é\u{1f600}\ud800\u0000\u001f/\\\"\b\f\n\r\t\ud800A"); + assert.equal(JSON.stringify(m.projectGraph('{"s":"\\ud800"}', "workflow").value), '{"s":"\\ud800"}'); +}); + +test("projection rejects what Python's json rejects and nothing else", () => { + const rejected = ["{}", "\f1", "  1", '{"a":1,}', "[1,]", "{'a':1}", '{"a"}', "[1 2]", "tru", "nul", "-NaN", "1.", ".5", "01", "+1", + "1e", "-", '"\\x"', '"abc', "", " ", '"\u0001"', '{"a":1}x', '"\\u12"', '{"a":1,"a":2}', '{"a":{"b":1,"b":1}}', "[1,[2,{\"k\":1,\"k\":2}]]", + '{"a":1 "b":2}', "[,1]", "{,}", '{"a":}', "[1}", "{1:2}", "
[]", "[]\u000b", "0x10", "1_000", "--1", "1e+", "1E-", '"\\uD800\\uDC"']; + for (const text of rejected) assert.equal(graphOf(text).status, "malformed", JSON.stringify(text)); + for (const text of [" \t\r\n{}\n\t ", '{"a":[1,{"b":null}],"c":"d"}', '{"\\u0000":1}', '{"a":1,"A":1}', '{"":[]}']) { + assert.equal(graphOf(text).status, "parsed", JSON.stringify(text)); + } + // Top-level values that are not objects parse but are not graphs. + for (const text of ["[]", '"x"', "1", "null", "true", "NaN", "1.5", "-Infinity"]) assert.equal(graphOf(text).status, "malformed", text); + assert.equal(graphOf(undefined).status, "malformed"); + assert.equal(graphOf(null).status, "malformed"); +}); + +test("projection limits", () => { + // A workflow's root is depth 1; the 64th nested container is kept, the 65th is limited. + assert.equal(graphOf(`{"a":${nested(63)}}`).status, "parsed"); + assert.equal(graphOf(`{"a":${nested(64)}}`).status, "limited"); + // Prompt: root 1, node 2, inputs 3; a value chain reaching depth 64 is kept, 65 is limited. + assert.equal(graphOf(`{"1":{"class_type":"A","inputs":{"v":${nested(61)}}}}`, "prompt").status, "parsed"); + assert.equal(graphOf(`{"1":{"class_type":"A","inputs":{"v":${nested(62)}}}}`, "prompt").status, "limited"); + // Dropped node keys are still validated for depth. + assert.equal(graphOf(`{"1":{"class_type":"A","inputs":{},"_meta":{"v":${nested(61)}}}}`, "prompt").status, "parsed"); + assert.equal(graphOf(`{"1":{"class_type":"A","inputs":{},"_meta":{"v":${nested(62)}}}}`, "prompt").status, "limited"); + // A workflow is normalized before its shape is checked, so a deep non-object is limited; a + // prompt checks its root first. Depth never overflows the parser (100,000 nested arrays). + assert.equal(graphOf(nested(10)).status, "malformed"); + assert.equal(graphOf(nested(5000)).status, "limited"); + assert.equal(graphOf(nested(5000), "prompt").status, "malformed"); + assert.equal(graphOf(`{"a":${nested(100000)}}`).status, "limited"); + // Node ceilings: 20,000 API nodes parse, 20,001 are limited. + assert.equal(graphOf(nodes(20000), "prompt").status, "parsed"); + assert.equal(graphOf(nodes(20001), "prompt").status, "limited"); + const list = (count) => `[${Array.from({ length: count }, () => "{}").join(",")}]`; + assert.equal(graphOf(`{"nodes":${list(20000)}}`).status, "parsed"); + assert.equal(graphOf(`{"nodes":${list(20001)}}`).status, "limited"); + assert.equal(graphOf(`{"nodes":${list(10001)},"definitions":{"subgraphs":[{"nodes":${list(10000)}},5,{"nodes":3}]}}`).status, "limited"); + assert.equal(graphOf(`{"nodes":${list(10000)},"definitions":{"subgraphs":[{"nodes":${list(10000)}}]}}`).status, "parsed"); + assert.equal(graphOf(`{"nodes":{"${"x".repeat(5)}":1},"definitions":[]}`).status, "parsed"); + // Duplicate keys anywhere are malformed, including inside dropped node keys. + assert.equal(graphOf('{"1":{"class_type":"A","inputs":{},"_meta":{"t":1,"t":2}}}', "prompt").status, "malformed"); + assert.equal(graphOf('{"1":{"class_type":"A","inputs":{"a":1,"a":1}}}', "prompt").status, "malformed"); +}); + +test("prompt nodes keep only class_type and inputs on null-prototype objects", () => { + const text = '{"3":{"class_type":"KSampler","inputs":{"seed":9007199254740993,"cfg":8.0,"n":NaN},"_meta":{"title":"x"},"is_changed":[1]},"4":5,"5":[1,2.50],"__proto__":{"class_type":"P","inputs":{}},"constructor":"c"}'; + const graph = m.projectGraph(text, "prompt"); + assert.equal(graph.status, "parsed"); + assert.equal(Object.getPrototypeOf(graph.value), null); + assert.equal(Object.getPrototypeOf(graph.value["3"]), null); + assert.equal(Object.getPrototypeOf(graph.value["3"].inputs), null); + // JSON.parse, like the projection, makes "__proto__" an own key; an object literal would not. + assert.deepEqual(plain(graph.value), JSON.parse('{"3":{"class_type":"KSampler","inputs":{"seed":"9007199254740993","cfg":"8.0","n":null}},"4":5,"5":[1,"2.50"],"__proto__":{"class_type":"P","inputs":{}},"constructor":"c"}')); + assert.ok(Object.hasOwn(graph.value, "__proto__")); + assert.deepEqual(plain(graph.value["__proto__"]), { class_type: "P", inputs: {} }); + assert.equal(graph.value.constructor, "c"); + for (const text of ["[]", '"3"', "5", "null", '{"nodes":[]}x']) assert.equal(m.projectGraph(text, "prompt").status, text === '{"nodes":[]}x' ? "malformed" : "malformed", text); + // A workflow keeps everything, normalized. + assert.deepEqual(graphOf('{"nodes":[{"id":1,"widgets_values":[7,"fixed",8.0,1e3]}],"links":[],"version":0.4}').value, + { nodes: [{ id: 1, widgets_values: [7, "fixed", "8.0", "1E+3"] }], links: [], version: "0.4" }); +}); + +test("projectMetadata mirrors _metadata for records, formats and carriers", () => { + const prompt = '{"1":{"class_type":"A","inputs":{"x":1.0}}}', workflow = '{"nodes":[],"version":0.4}'; + const read = (records, record_formats, carrierText = null, limited = []) => ({ records, record_formats, carrierText, limited }); + assert.deepEqual(plain(m.projectMetadata(null)), { status: "unreadable", omitted_reason: null, raw: {}, record_formats: {}, parameters: null, + prompt_graph: { status: "absent", value: null }, workflow_graph: { status: "absent", value: null } }); + assert.deepEqual(plain(m.projectMetadata(read({}, {}))), { status: "absent", omitted_reason: null, raw: {}, record_formats: {}, parameters: null, + prompt_graph: { status: "absent", value: null }, workflow_graph: { status: "absent", value: null } }); + const full = plain(m.projectMetadata(read({ prompt, workflow, parameters: "cat\nSteps: 1", comfyassets_prompt_pairs: "{}", other: "x" }, + { prompt: "original-text", workflow: "original-text", parameters: "original-text", comfyassets_prompt_pairs: "original-text", other: "original-text" }))); + assert.deepEqual(full, { status: "available", omitted_reason: null, + raw: { prompt, workflow, parameters: "cat\nSteps: 1", comfyassets_prompt_pairs: "{}" }, + record_formats: { prompt: "original-text", workflow: "original-text", parameters: "original-text", comfyassets_prompt_pairs: "original-text" }, + parameters: "cat\nSteps: 1", prompt_graph: { status: "parsed", value: { 1: { class_type: "A", inputs: { x: "1.0" } } } }, + workflow_graph: { status: "parsed", value: { nodes: [], version: "0.4" } } }); + // limited records lose their text; invalid is malformed; reserialized without a carrier is limited. + const limited = plain(m.projectMetadata(read({ prompt, workflow }, { prompt: "original-text" }, null, ["workflow"]))); + assert.deepEqual(limited.raw, { prompt }); + assert.deepEqual(limited.record_formats, { prompt: "original-text", workflow: "limited" }); + assert.equal(limited.workflow_graph.status, "limited"); + assert.equal(plain(m.projectMetadata(read({ prompt }, { prompt: "original-text", workflow: "invalid" }))).workflow_graph.status, "malformed"); + assert.equal(plain(m.projectMetadata(read({ workflow }, { workflow: "reserialized-json" }))).workflow_graph.status, "limited"); + assert.equal(plain(m.projectMetadata(read({ prompt: "{" }, { prompt: "original-text" }))).prompt_graph.status, "malformed"); + // Unknown formats are dropped from record_formats but the raw text still projects. + const odd = plain(m.projectMetadata(read({ prompt }, { prompt: "weird" }))); + assert.deepEqual(odd.record_formats, {}); + assert.equal(odd.prompt_graph.status, "parsed"); + // A non-text record vanishes from raw and its named graph is malformed. + assert.equal(plain(m.projectMetadata(read({ prompt: 5 }, { prompt: "original-text" }))).prompt_graph.status, "malformed"); + // Carriers: strings re-parse by kind, objects were projected at depth 2; a bad carrier fails every present graph. + const envelope = JSON.stringify({ prompt, workflow: { nodes: [], v: 1 } }); + const carried = plain(m.projectMetadata(read({ prompt, workflow: '{"nodes": [], "v": 1}' }, { prompt: "original-text", workflow: "reserialized-json" }, envelope))); + assert.deepEqual(carried.prompt_graph, { status: "parsed", value: { 1: { class_type: "A", inputs: { x: "1.0" } } } }); + assert.deepEqual(carried.workflow_graph, { status: "parsed", value: { nodes: [], v: 1 } }); + const broken = plain(m.projectMetadata(read({ prompt, workflow }, { prompt: "original-text", workflow: "original-text" }, "[1]"))); + assert.deepEqual([broken.prompt_graph.status, broken.workflow_graph.status], ["malformed", "malformed"]); + const deep = plain(m.projectMetadata(read({ prompt }, { prompt: "original-text" }, `{"title":${nested(70)},"prompt":${JSON.stringify(prompt)}}`))); + assert.equal(deep.prompt_graph.status, "limited"); + const partial = plain(m.projectMetadata(read({ prompt, workflow }, { prompt: "original-text", workflow: "original-text" }, '{"prompt":"{","workflow":5}'))); + assert.deepEqual([partial.prompt_graph.status, partial.workflow_graph.status], ["malformed", "malformed"]); + const missing = plain(m.projectMetadata(read({ prompt }, { prompt: "original-text" }, '{"workflow":{}}'))); + assert.deepEqual([missing.prompt_graph.status, missing.workflow_graph.status], ["malformed", "absent"]); + // The carrier's own prompt object is projected: depth 2, node keys filtered. + const objectPrompt = plain(m.projectMetadata(read({ prompt: "x" }, { prompt: "reserialized-json" }, '{"prompt":{"1":{"class_type":"A","inputs":{},"_meta":{}}}}'))); + assert.deepEqual(objectPrompt.prompt_graph, { status: "parsed", value: { 1: { class_type: "A", inputs: {} } } }); +}); + +test("fitInspectionPayload omits the workflow graph, then the prompt graph, then all raw atomically", async () => { + const payload = { + success: true, inspection_version: 1, metadata: { + status: "available", omitted_reason: null, raw: { prompt: "é".repeat(100), workflow: "{}" }, record_formats: { prompt: "original-text", workflow: "original-text" }, + parameters: null, prompt_graph: { status: "parsed", value: { 1: { class_type: "A", inputs: { t: "\ud800".repeat(50) } } } }, + workflow_graph: { status: "parsed", value: { nodes: Array.from({ length: 50 }, (_, i) => ({ id: i })) } }, + }, + }; + const text = JSON.stringify(payload); + // UTF-8 bytes, not UTF-16 code units: 'é' is 2 bytes, an escaped lone surrogate 6. + assert.equal(m.inspectionBytes(payload), Buffer.byteLength(text, "utf8")); + assert.ok(m.inspectionBytes(payload) > text.length); + // Measured in bounded chunks: pairs and 2-byte characters across every chunk edge count once. + for (const shift of [0, 1, 2, 3]) { + const wide = { s: "x".repeat(shift) + "é😀".repeat(200000) + "\ud800", t: "a😀".repeat(70000) }; + assert.equal(m.inspectionBytes(wide), Buffer.byteLength(JSON.stringify(wide), "utf8"), `shift ${shift}`); + } + const whole = await m.fitInspectionPayload(payload, bytes(payload)); + assert.equal(whole, payload, "a payload within the limit is returned as is"); + const first = await m.fitInspectionPayload(payload, bytes(payload) - 1); + assert.equal(first.metadata.workflow_graph.status, "limited"); + assert.equal(first.metadata.prompt_graph.status, "parsed"); + assert.deepEqual(first.metadata.raw, payload.metadata.raw); + assert.ok(bytes(first) <= bytes(payload) - 1); + const second = await m.fitInspectionPayload(payload, bytes(first) - 1); + assert.deepEqual([second.metadata.workflow_graph.status, second.metadata.prompt_graph.status], ["limited", "limited"]); + assert.equal(second.metadata.status, "available"); + assert.deepEqual(second.metadata.raw, payload.metadata.raw); + const third = await m.fitInspectionPayload(payload, bytes(second) - 1); + assert.deepEqual(third.metadata, { ...second.metadata, raw: {}, parameters: null, status: "limited", omitted_reason: "response-size-limit" }); + assert.ok(bytes(third) <= bytes(second) - 1); + await assert.rejects(m.fitInspectionPayload(payload, bytes(third) - 1), /exceeds/); + // The caller's payload is never mutated and absent graphs are left alone. + assert.equal(JSON.stringify(payload), text); + const absent = { ...payload, metadata: { ...payload.metadata, workflow_graph: { status: "absent", value: null } } }; + const fitted = await m.fitInspectionPayload(absent, bytes(absent) - 1); + assert.equal(fitted.metadata.workflow_graph.status, "absent"); + assert.equal(fitted.metadata.prompt_graph.status, "limited"); + assert.equal((await m.fitInspectionPayload(payload)).metadata.status, "available", "the default limit is 8 MiB"); + // Every omission step after the first measure hands the event loop a turn, and an + // abort between steps rejects with AbortError; the answer itself is unchanged. + let turns = 0; + const counted = await m.fitInspectionPayload(payload, bytes(second) - 1, { yieldControl: async () => { turns += 1; } }); + assert.equal(JSON.stringify(counted), JSON.stringify(third)); + assert.equal(turns, 3); + assert.equal(await m.fitInspectionPayload(payload, bytes(payload), { yieldControl: async () => { turns += 1; } }), payload); + assert.equal(turns, 3, "a payload within the limit is measured once, without a turn"); + const controller = new AbortController(); + await assert.rejects(m.fitInspectionPayload(payload, bytes(second) - 1, { signal: controller.signal, yieldControl: async () => controller.abort() }), { name: "AbortError" }); +}); + +test("the committed projection corpus has every outcome and parses with the oracle entry shape", () => { + assert.ok(CASES.length >= 40); + const names = new Set(), statuses = new Set(), graphs = new Set(); + for (const entry of CASES) { + assert.equal(typeof entry.name, "string"); + assert.ok(!names.has(entry.name), `duplicate case ${entry.name}`); + names.add(entry.name); + const read = entry.records === null ? null : { records: entry.records, record_formats: entry.record_formats ?? Object.fromEntries(Object.keys(entry.records).map((key) => [key, "original-text"])), + carrierText: entry.carrier_text ?? null, limited: entry.limited ?? [] }; + const metadata = m.projectMetadata(read); + statuses.add(metadata.status); + graphs.add(metadata.prompt_graph.status); + graphs.add(metadata.workflow_graph.status); + if (entry.expect) { + for (const [field, status] of Object.entries(entry.expect)) assert.equal(metadata[field].status, status, `${entry.name} ${field}`); + } + } + assert.deepEqual([...statuses].sort(), ["absent", "available", "unreadable"]); + assert.deepEqual([...graphs].sort(), ["absent", "limited", "malformed", "parsed"]); +}); diff --git a/tests/js/workspace-local-video.test.js b/tests/js/workspace-local-video.test.js new file mode 100644 index 0000000..c1f3cd3 --- /dev/null +++ b/tests/js/workspace-local-video.test.js @@ -0,0 +1,622 @@ +// Run with: node --test tests/js/ +// The browser video tag readers mirror what the server reads through ffprobe +// (utils/prompt_inspector.py _video_metadata) and the carrier rule of +// utils/video_metadata.py select_video_carrier. Each rule names its server twin; +// tests/test_local_inspect_parity.py proves the two agree on the committed fixtures +// and on recorded tag maps. +import test from "node:test"; +import assert from "node:assert/strict"; +import fs from "node:fs"; +import path from "node:path"; +import { fileURLToPath } from "node:url"; +import * as v from "./helpers/video-bytes.mjs"; + +const here = path.dirname(fileURLToPath(import.meta.url)); +const FIXTURES = path.join(here, "..", "fixtures", "local-inspect"); +const RECORDED = JSON.parse(fs.readFileSync(path.join(FIXTURES, "video-tags-v1.json"), "utf8")).fixtures; +const m = await import("../../web/js/workspace/local-video.mjs").catch((error) => { + if (error.code !== "ERR_MODULE_NOT_FOUND") throw error; + return {}; +}); +const { readVideoTags, selectVideoCarrier, pythonJsonDumps, pythonJsonLoads } = m; + +const MiB = 1024 * 1024; +const GiB = 1024 * MiB; +const ROLE_BUDGET = 2 * MiB; +const WORKFLOW_BUDGET = 4 * MiB; +const TAG_READ_BYTES = 12 * MiB + 64 * 1024; +const PER_KEY = new Set(["prompt", "workflow", "parameters", "comfyassets_prompt_pairs"]); +const GRAPH = '{"3":{"class_type":"KSampler","inputs":{"seed":7}}}'; +const WORKFLOW = '{"nodes":[],"version":0.4}'; + +// The tags ffprobe reports to the server: the six probed names, ASCII case folded. +function knownTags(map) { + return Object.fromEntries(Object.entries(map).filter(([name]) => PER_KEY.has(name.toLowerCase()) || ["comment", "description"].includes(name.toLowerCase()))); +} + +function fixture(name) { + return new Uint8Array(fs.readFileSync(path.join(FIXTURES, name))); +} + +async function read(bytes, options = {}) { + return readVideoTags(v.blobOf(bytes), options); +} + +function fill(byte, count) { + return Buffer.alloc(count, byte); +} + +function complete(tags, extra = {}) { + return { status: "complete", tags, limited: [], ...extra }; +} + +test("contract: the readers, the carrier rule and the json port are exported", () => { + assert.equal(typeof readVideoTags, "function"); + assert.equal(typeof selectVideoCarrier, "function"); + assert.equal(typeof pythonJsonDumps, "function"); + assert.equal(typeof pythonJsonLoads, "function"); +}); + +test("iso-bmff reader finds per-key tags with moov after mdat", async () => { + for (const [name, recorded] of Object.entries(RECORDED)) { + const blob = v.recordingBlob(fixture(name)); + const result = await readVideoTags(blob, { prefix: fixture(name).subarray(0, 32) }); + assert.equal(result.status, "complete", name); + assert.deepEqual(result.tags, knownTags(recorded), name); + assert.equal(result.container, name.endsWith(".mkv") || name.endsWith(".webm") ? "matroska" : "iso-bmff", name); + assert.ok(["isom", "qt", "M4V", "matroska", "webm"].includes(result.format), `${name}: ${result.format}`); + assert.deepEqual(result.limited, [], name); + assert.ok(blob.ranges.length > 0, name); + } + // core-moov-end.mp4 behind a 3 GiB mdat: only the box headers and the tag bytes are sliced. + const bytes = fixture("core-moov-end.mp4"); + const boxes = v.topLevelBoxes(bytes); + const mdat = boxes.find((box) => box.type === "mdat"); + const moov = boxes.find((box) => box.type === "moov"); + const mdatSize = 3 * GiB + 16; + const header = v.concat(v.u32(1), v.latin1("mdat"), new Uint8Array(8)); + new DataView(header.buffer).setBigUint64(8, BigInt(mdatSize)); + const moovOffset = mdat.start + mdatSize; + const blob = v.sparseBlob( + [ + { offset: 0, bytes: bytes.subarray(0, mdat.start) }, + { offset: mdat.start, bytes: header }, + { offset: moovOffset, bytes: bytes.subarray(moov.start, moov.end) }, + ], + moovOffset + (moov.end - moov.start), + ); + const result = await readVideoTags(blob, { prefix: bytes.subarray(0, 32) }); + assert.deepEqual(result.tags, knownTags(RECORDED["core-moov-end.mp4"])); + const inside = blob.ranges.filter(([start]) => start >= mdat.start && start < moovOffset); + assert.ok(inside.every(([start, end]) => start === mdat.start && end - start <= 16), JSON.stringify(inside)); + const total = blob.ranges.reduce((sum, [start, end]) => sum + (end - start), 0); + assert.ok(total <= m.VIDEO_READ_CEILING && total < 64 * 1024, `${total}`); +}); + +test("iso-bmff: legacy ilst atoms, moov/meta, sticky mdta handler and data types", async () => { + // ©cmt and desc map to comment and description, as ffprobe reports them. + const legacy = await read(v.mp4(v.legacyUdta({ comment: '{"prompt":"{}"}', description: "note" }))); + assert.deepEqual(legacy, complete({ comment: '{"prompt":"{}"}', description: "note" }, { container: "iso-bmff", format: "isom" })); + // moov/meta (ISO style) is read like moov/udta/meta. + const isoMeta = await read(v.mp4(v.box("meta", v.u32(0), v.hdlr("mdta"), v.keys(["prompt"]), v.ilst(v.item(1, v.data(GRAPH)))))); + assert.deepEqual(isoMeta.tags, { prompt: GRAPH }); + // meta without the version/flags word (QuickTime) still finds hdlr. + const qtMeta = await read(v.mp4(v.udta(v.box("meta", v.hdlr("mdta"), v.keys(["prompt"]), v.ilst(v.item(1, v.data(GRAPH))))))); + assert.deepEqual(qtMeta.tags, { prompt: GRAPH }); + // Index items need an mdta handler and a keys box; with a legacy handler they are skipped. + const noKeys = await read(v.mp4(v.udta(v.meta(v.hdlr("mdir"), v.ilst(v.item(1, v.data(GRAPH))))))); + assert.deepEqual(noKeys.tags, {}); + // The mdta handler is sticky across the file, as ffmpeg's found_hdlr_mdta. + const sticky = await read(v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt"])), v.meta(v.hdlr("mdir"), v.ilst(v.item(1, v.data(GRAPH))))))); + assert.deepEqual(sticky.tags, { prompt: GRAPH }); + // As ffprobe reports them: consecutive data children join with ";", integer data + // types print as decimals, other types are not tags, text types 0/1/4 are read, NUL + // truncates, and the run of data children stops at the first other child. + const mixed = await read(v.mp4(v.udta(v.meta( + v.hdlr("mdta"), + v.keys(["prompt", "workflow", "parameters", "comfyassets_prompt_pairs", "comment", "description"]), + v.ilst( + v.item(1, v.data("first"), v.data("second"), v.box("name", "x"), v.data("not read")), + v.item(2, v.data([0, 0, 0, 7], { type: 21 }), v.data([0xff, 0xfe], { type: 21 }), v.data([0xff, 0xfe], { type: 22 })), + v.item(3, v.data("cut\0off")), + v.item(4, v.data("sort", { type: 4 }), v.data("mac", { type: 0 })), + v.item(5, v.data([0, 0, 0, 0, 0, 0, 0], { type: 21 })), + v.item(6, v.data("float", { type: 23 })), + ), + )))); + assert.deepEqual(mixed.tags, { prompt: "first;second", workflow: "7;-2;65534", parameters: "cut", comfyassets_prompt_pairs: "sort;mac" }); + // Keys outside the mdta namespace address nothing; unknown names are skipped unread. + const blob = v.recordingBlob(v.mp4(v.udta(v.meta( + v.hdlr("mdta"), + v.keys([["prompt", "udta"], "encoder", "workflow"]), + v.ilst(v.item(1, v.data(GRAPH)), v.item(2, v.data(fill(0x41, 5000))), v.item(3, v.data(WORKFLOW))), + )))); + const result = await readVideoTags(blob); + assert.deepEqual(result.tags, { workflow: WORKFLOW }); + assert.ok(blob.ranges.every(([start, end]) => end - start < 5000), "the encoder value was never sliced"); + // Legacy fourccs are mapped even in mdta mode, and a repeated name joins with ";". + const repeated = await read(v.mp4(v.udta(v.meta( + v.hdlr("mdta"), + v.keys(["prompt", "prompt", "comment"]), + v.ilst(v.item(1, v.data("one")), v.item(2, v.data("two")), v.item(3, v.data("mdta")), v.legacyItem("©cmt", v.data("c"))), + )))); + assert.deepEqual(repeated.tags, { prompt: "one;two", comment: "mdta;c" }); + // Apple's namespaced keys are reported verbatim, so they are never an envelope. + const apple = await read(v.perKeyMp4({ "com.apple.quicktime.description": '{"prompt":"{}"}', prompt: GRAPH })); + assert.deepEqual(apple.tags, { prompt: GRAPH }); + // An item whose first child is not a data atom, or a data atom shorter than its + // 16-byte header, is skipped as ffmpeg skips it. + const junk = await read(v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt"]), v.ilst(v.item(1, v.box("name", "x"), v.data(GRAPH))))))); + assert.deepEqual(junk.tags, {}); + const short = await read(v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt"]), v.ilst(v.item(1, v.data(GRAPH, { size: 12 }))))))); + assert.deepEqual(short.tags, {}); + // A box declared with size 0 runs to the end of its parent; faststart puts moov first. + const fast = await read(v.perKeyMp4({ prompt: GRAPH }, { faststart: true, brand: "mp42" })); + assert.deepEqual(fast, complete({ prompt: GRAPH }, { container: "iso-bmff", format: "mp42" })); + const toEnd = await read(v.concat(v.ftyp(), v.mdat(), v.moov(v.mdtaUdta({ prompt: GRAPH }), v.opts({ size: 0 })))); + assert.deepEqual(toEnd.tags, { prompt: GRAPH }); +}); + +test("matroska reads global Tags and skips clusters", async () => { + const { ID } = v; + // ffmpeg layout: SeekHead, Info, Tracks, Tags, Cluster, Cues; names upper-cased. + const plain = await read(v.perKeyMkv({ PROMPT: GRAPH, WORKFLOW, ENCODER: "Lavf" }, { docType: "webm" })); + assert.deepEqual(plain, complete({ PROMPT: GRAPH, WORKFLOW }, { container: "matroska", format: "webm" })); + // Tags after a 3 GiB Cluster, located through the SeekHead: no slice inside the cluster. + const tags = v.globalTags({ PROMPT: GRAPH }); + const clusterSize = 3 * GiB; + // The Tags position (relative to the Segment payload) is a 4-byte uint either way, + // so the SeekHead's length can be measured with a placeholder first. + const seekLength = v.seekHead([[ID.Tags, clusterSize]]).length; + const seekHead = v.seekHead([[ID.Tags, seekLength + 12 + clusterSize]]); + assert.equal(seekHead.length, seekLength); + const head = v.concat(v.ebmlHeader(), v.idBytes(ID.Segment), v.vint(seekHead.length + 12 + clusterSize + tags.length, { width: 8 }), seekHead, v.clusterHeader(clusterSize)); + const clusterStart = v.ebmlHeader().length + 12 + seekHead.length; + const tagsOffset = head.length + clusterSize; + const blob = v.sparseBlob([{ offset: 0, bytes: head }, { offset: tagsOffset, bytes: tags }], tagsOffset + tags.length); + const seeked = await readVideoTags(blob, { prefix: head.subarray(0, 32) }); + assert.deepEqual(seeked.tags, { PROMPT: GRAPH }); + const inside = blob.ranges.filter(([start]) => start >= clusterStart && start < tagsOffset); + assert.ok(inside.every(([start, end]) => start === clusterStart && end - start <= 16), JSON.stringify(inside)); + assert.ok(blob.ranges.reduce((sum, [start, end]) => sum + (end - start), 0) <= m.VIDEO_READ_CEILING); + // ffprobe reads level-1 elements up to the first Cluster: Tags behind it are found + // only through a SeekHead, so without one they are not read (nor sliced). + const behind = v.recordingBlob(v.mkv([v.cluster(64), v.globalTags({ PROMPT: GRAPH }), el(ID.Cues, new Uint8Array(2))])); + const sequential = await readVideoTags(behind); + assert.deepEqual(sequential.tags, {}); + assert.ok(behind.ranges.every(([, end]) => end <= v.ebmlHeader().length + 12 + 12 + 32), JSON.stringify(behind.ranges)); + // Track-targeted Tags are ignored; chapter and attachment targets too. + const targeted = await read(v.mkv([ + v.tagsEl( + v.tag([v.simpleTag("PROMPT", "track")], { trackUid: 1 }), + v.tag([v.simpleTag("WORKFLOW", WORKFLOW)], { targetType: 50 }), + v.tag([v.simpleTag("PROMPT", "zero uid is global")], { trackUid: 0 }), + ), + v.cluster(), + ])); + assert.deepEqual(targeted.tags, { WORKFLOW, PROMPT: "zero uid is global" }); + // An unknown-size Cluster (live recordings) is the same stop; a SeekHead still works. + const hidden = await read(v.mkv([v.cluster(8, { unknown: true }), v.globalTags({ PROMPT: GRAPH })])); + assert.deepEqual(hidden, complete({}, { container: "matroska", format: "matroska" })); + const before = await read(v.mkv([v.globalTags({ PROMPT: GRAPH }), v.cluster(8, { unknown: true })])); + assert.deepEqual(before.tags, { PROMPT: GRAPH }); + const info = el(ID.Info, new Uint8Array(4)); + const clusterUnknown = v.cluster(8, { unknown: true }); + const seekToTags = v.seekHead([[ID.Tags, 0x7f]]); // a one-byte position, like the real one + const viaSeek = await read(v.mkv([v.seekHead([[ID.Tags, seekToTags.length + info.length + clusterUnknown.length]]), info, clusterUnknown, v.globalTags({ PROMPT: GRAPH })])); + assert.deepEqual(viaSeek.tags, { PROMPT: GRAPH }); + // An unknown-size Segment is fine (live recordings); other unknown sizes are invalid. + const liveSegment = await read(v.mkv([v.globalTags({ PROMPT: GRAPH }), v.cluster()], { unknown: true })); + assert.deepEqual(liveSegment.tags, { PROMPT: GRAPH }); + const unknownTags = await read(v.mkv([el(ID.Tags, v.tag([v.simpleTag("PROMPT", GRAPH)]), v.opts({ unknown: true }))])); + assert.deepEqual(unknownTags, { status: "unreadable", reason: "invalid-container" }); + // ffmpeg's tag naming (checked against ffprobe 9.0.2): a language other than und + // files the tag under NAME-lang only, unless TagDefault is set; nested SimpleTags + // become PARENT/CHILD (never ours); a string-less tag deletes the key; a repeated + // name keeps the last value; NUL ends the string; Void and CRC-32 are skipped. + const naming = await read(v.mkv([ + v.voidEl(8), + v.tagsEl( + el(ID.CRC32, new Uint8Array(4)), + v.tag([ + v.simpleTag("PROMPT", "eng only", { lang: "eng" }), + v.simpleTag("WORKFLOW", "eng only", { lang: "eng", def: 0 }), + v.simpleTag("workflow", "default and eng", { lang: "eng", def: 1 }), + v.simpleTag("PARAMETERS", "und", { lang: "und", nested: [v.simpleTag("prompt", "nested")] }), + v.simpleTag("comfyassets_prompt_pairs", "cut\0here"), + v.simpleTag("description", undefined, { binary: [1, 2, 3] }), + v.simpleTag("comment", "note"), + v.simpleTag("comment", undefined), + v.simpleTag("prompt", "first"), + v.simpleTag("prompt", "last", { binary: [1] }), + ]), + ), + v.cluster(), + ])); + assert.deepEqual(naming.tags, { workflow: "default and eng", PARAMETERS: "und", comfyassets_prompt_pairs: "cut", prompt: "last" }); + // Long names are never ours and are skipped without reading their strings. + const longName = v.recordingBlob(v.mkv([v.tagsEl(v.tag([v.simpleTag("N".repeat(300), fill(0x41, 4000)), v.simpleTag("PROMPT", GRAPH)])), v.cluster()])); + const skipped = await readVideoTags(longName); + assert.deepEqual(skipped.tags, { PROMPT: GRAPH }); + assert.ok(longName.ranges.every(([start, end]) => end - start < 300)); + function el(id, ...parts) { + return v.el(id, ...parts); + } +}); + +test("containers fail closed", async () => { + const { ID } = v; + const invalid = { status: "unreadable", reason: "invalid-container" }; + const rows = [ + ["box size below its header", v.concat(v.ftyp(), v.box("free", v.opts({ size: 4 })), v.mdat(), v.moov(v.mdtaUdta({ prompt: GRAPH })))], + ["child box past its parent", v.concat(v.ftyp(), v.mdat(), v.moov(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt"]), v.ilst(v.item(1, v.data(GRAPH), v.opts({ size: 5000 })))))))], + ["top-level box past the end of the file", v.concat(v.ftyp(), v.mdat(), v.box("moov", v.opts({ size: 4000 })))], + ["64-bit size above MAX_SAFE_INTEGER", v.concat(v.ftyp(), v.box("mdat", v.opts({ largesize: 2 ** 60 })), v.moov(v.mdtaUdta({ prompt: GRAPH })))], + ["64-bit size below its header", v.concat(v.ftyp(), v.box("mdat", v.opts({ largesize: 12 })), v.moov(v.mdtaUdta({ prompt: GRAPH })))], + ["keys count larger than the entries", v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt"], { count: 3 }), v.ilst(v.item(1, v.data(GRAPH))))))], + ["keys entry overrunning the box", v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.box("keys", v.u32(0), v.u32(1), v.u32(90), v.latin1("mdta"), v.utf8("prompt")), v.ilst(v.item(1, v.data(GRAPH))))))], + ["no moov box", v.concat(v.ftyp(), v.mdat(), v.free(32))], + ["not a container at all", v.utf8("#EXTM3U\n#EXT-X-VERSION:3\n".padEnd(64, "x"))], + ["bad VINT: zero ID byte", v.concat(v.ebmlHeader(), v.idBytes(ID.Segment), v.vint(20, { width: 8 }), [0x00, 0x81, 0x00], new Uint8Array(17))], + ["bad VINT: zero size byte", v.concat(v.ebmlHeader(), v.idBytes(ID.Segment), v.vint(20, { width: 8 }), v.idBytes(ID.Info), [0x00], new Uint8Array(17))], + ["element past its parent", v.mkv([v.el(ID.Tags, v.tag([v.simpleTag("PROMPT", GRAPH)]), v.opts({ size: 5000 }))])], + ["element past the end of the file", v.concat(v.ebmlHeader(), v.idBytes(ID.Segment), v.vint(5000, { width: 8 }), v.globalTags({ PROMPT: GRAPH }))], + ["unknown DocType", v.perKeyMkv({ PROMPT: GRAPH }, { docType: "avi" })], + ["EBML header without a DocType", v.concat(v.el(ID.EBML, v.el(ID.EBMLVersion, [1])), v.segment([v.globalTags({ PROMPT: GRAPH })]))], + ["Matroska bytes after a stray size", v.concat(v.ebmlHeader(), v.idBytes(ID.Segment), v.vint(3, { width: 8 }), [0xec, 0x82, 0x00])], + ]; + for (const [name, bytes] of rows) { + assert.deepEqual(await read(bytes), invalid, name); + } + // Trailing bytes shorter than a box header are ignored, as ffmpeg does. + assert.deepEqual((await read(v.concat(v.perKeyMp4({ prompt: GRAPH }), [1, 2, 3]))).tags, { prompt: GRAPH }); + // 10,001 boxes stop the read. + const many = v.concat(v.ftyp(), ...Array.from({ length: 10001 }, () => v.free()), v.mdat(), v.moov(v.mdtaUdta({ prompt: GRAPH }))); + assert.deepEqual(await read(many), { status: "limited", reason: "chunk-limit" }); + const manyElements = v.mkv([...Array.from({ length: 10001 }, () => v.voidEl(1)), v.globalTags({ PROMPT: GRAPH })]); + assert.deepEqual(await read(manyElements), { status: "limited", reason: "chunk-limit" }); + // Exactly 10,000 elements still read (ftyp, mdat, moov, udta, meta, hdlr, keys, its entry, ilst, the item and its data atom). + const exact = v.concat(v.ftyp(), ...Array.from({ length: 10000 - 11 }, () => v.free()), v.mdat(), v.moov(v.mdtaUdta({ prompt: GRAPH }))); + assert.deepEqual((await read(exact)).tags, { prompt: GRAPH }); + // A vanished file is a read failure, not a container error. + assert.deepEqual(await readVideoTags(v.failingBlob(4096), { prefix: v.ftyp() }), { status: "unreadable", reason: "read-failure" }); + assert.deepEqual(await readVideoTags({}), { status: "unreadable", reason: "read-failure" }); +}); + +// ffprobe's string_validation=replace (av_utf8_decode): one U+FFFD per failing +// decode step. A bad continuation or a truncated sequence fails one byte at a time; +// an overlong form, a surrogate, a code above U+10FFFF, U+FFFE/U+FFFF and 5/6-byte +// forms fail as one whole sequence. Each row was read through ffprobe 9.0.2. +const FFPROBE_REPLACEMENTS = [ + ["one Latin-1 byte", [0x63, 0x61, 0x66, 0xe9], "caf�"], + ["truncated 4-byte sequence", [0x61, 0xf0, 0x9f, 0x22, 0x62], 'a��"b'], + ["overlong 2-byte form", [0x61, 0xc0, 0xaf, 0x62], "a�b"], + ["C1 overlong lead", [0x61, 0xc1, 0xbf, 0x62], "a�b"], + ["overlong 3-byte form", [0x61, 0xe0, 0x80, 0x80, 0x62], "a�b"], + ["overlong 4-byte form", [0x61, 0xf0, 0x80, 0x80, 0x80, 0x62], "a�b"], + ["UTF-16 surrogate", [0x61, 0xed, 0xa0, 0x80, 0x62], "a�b"], + ["U+FFFE", [0x61, 0xef, 0xbf, 0xbe, 0x62], "a�b"], + ["U+FFFF", [0x61, 0xef, 0xbf, 0xbf, 0x62], "a�b"], + ["U+FDD0 is kept", [0x61, 0xef, 0xb7, 0x90, 0x62], "a﷐b"], + ["5-byte form", [0x61, 0xf8, 0x88, 0x80, 0x80, 0x80, 0x62], "a�b"], + ["6-byte form", [0x61, 0xfc, 0x84, 0x80, 0x80, 0x80, 0x80, 0x62], "a�b"], + ["truncated 5-byte form", [0x61, 0xf8, 0x88, 0x62], "a��b"], + ["above U+10FFFF", [0x61, 0xf4, 0x90, 0x80, 0x80, 0x62], "a�b"], + ["0xFE", [0x61, 0xfe, 0x62], "a�b"], + ["0xFF", [0x61, 0xff, 0x62], "a�b"], + ["0xFE 0xFF", [0xfe, 0xff], "��"], + ["lone continuation byte", [0x61, 0x80, 0x62], "a�b"], + ["truncated at the end", [0x61, 0xe2, 0x82], "a��"], + ["lead byte before a whole sequence", [0x61, 0xc3, 0xc3, 0xa9, 0x62], "a�éb"], + ["valid emoji", [0x61, 0xf0, 0x9f, 0x98, 0x80, 0x62], "a\u{1f600}b"], + ["control characters", [0x61, 0x01, 0x1f, 0x62], "a\u0001\u001fb"], +]; + +test("invalid UTF-8 reads as ffprobe replaces it; names that do not decode are not carriers", async () => { + for (const [name, bytes, expected] of FFPROBE_REPLACEMENTS) { + assert.deepEqual((await read(v.perKeyMp4({ prompt: bytes }))).tags, { prompt: expected }, name); + assert.deepEqual((await read(v.perKeyMkv({ PROMPT: bytes }))).tags, { PROMPT: expected }, `${name} (Matroska)`); + } + // A long value with one bad byte keeps every other byte. + const long = Buffer.concat([Buffer.from('{"a":"'), Buffer.alloc(100000, 0x78), Buffer.from([0xe9]), Buffer.from('"}')]); + assert.equal((await read(v.perKeyMp4({ workflow: long }))).tags.workflow, `{"a":"${"x".repeat(100000)}�"}`); + assert.deepEqual((await read(v.perKeyMp4({ prompt: [0x7b, 0xff, 0xfe, 0x7d] }))).tags, { prompt: "{��}" }); + // A keys name with a bad byte is reported as "prom�pt", which is no carrier. + const badKey = v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys([Uint8Array.from([0x70, 0x72, 0x6f, 0x6d, 0xff, 0x70, 0x74]), "prompt"]), v.ilst(v.item(1, v.data("junk")), v.item(2, v.data(GRAPH)))))); + assert.deepEqual(await read(badKey), complete({ prompt: GRAPH }, { container: "iso-bmff", format: "isom" })); + const onlyBadKey = v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys([Uint8Array.from([0xc3, 0x28])]), v.ilst(v.item(1, v.data(GRAPH)))))); + assert.deepEqual((await read(onlyBadKey)).tags, {}); + assert.deepEqual((await read(v.mkv([v.tagsEl(v.tag([v.simpleTag([0xff], GRAPH), v.simpleTag("WORKFLOW", WORKFLOW)]))]))).tags, { WORKFLOW: WORKFLOW }); + assert.deepEqual((await read(v.mkv([v.tagsEl(v.tag([v.simpleTag("PROMPT", [0xff, 0xfe])]))]))).tags, { PROMPT: "��" }); +}); + +test("ilst items with index 0 or past the keys are skipped; the first keys box wins", async () => { + const wf = '{"nodes":[]}'; + const stray = v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt", "workflow"]), v.ilst(v.item(1, v.data(GRAPH)), v.item(5, v.data("junk")), v.item(2, v.data(wf)))))); + assert.deepEqual((await read(stray)).tags, { prompt: GRAPH, workflow: wf }); + const zero = v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt", "workflow"]), v.ilst(v.item(0, v.data("junk")), v.item(1, v.data(GRAPH)), v.item(2, v.data(wf)))))); + assert.deepEqual((await read(zero)).tags, { prompt: GRAPH, workflow: wf }); + assert.deepEqual((await read(v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt"]), v.ilst(v.item(2, v.data(GRAPH)))))))).tags, {}); + // ffmpeg reads one keys box: a second one (even malformed) addresses nothing. + const twoKeys = v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt"]), v.keys(["workflow"]), v.ilst(v.item(1, v.data(GRAPH)))))); + assert.deepEqual((await read(twoKeys)).tags, { prompt: GRAPH }); + const brokenSecond = v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["prompt"]), v.keys(["workflow"], { count: 9 }), v.ilst(v.item(1, v.data(GRAPH)))))); + assert.deepEqual((await read(brokenSecond)).tags, { prompt: GRAPH }); + const emptyFirst = v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys([]), v.keys(["prompt"]), v.ilst(v.item(1, v.data(GRAPH)))))); + assert.deepEqual((await read(emptyFirst)).tags, {}); + // A keys box too short for its count field is not read, so the next one is. + const shortFirst = v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.fullBox("keys"), v.keys(["prompt"]), v.ilst(v.item(1, v.data(GRAPH)))))); + assert.deepEqual((await read(shortFirst)).tags, { prompt: GRAPH }); +}); + +test("one tag per ASCII-folded name and Mac Roman text, as ffmpeg's tag dictionary holds them", async () => { + const mdta = (names, items, legacy = []) => v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(names), v.ilst(...items.map(([index, value, type = 1]) => v.item(index, v.data(value, { type }))), ...legacy)))); + // ISO-BMFF joins case variants with ";" under the spelling set last (checked through ffprobe). + assert.deepEqual((await read(mdta(["workflow", "Workflow"], [[1, "A"], [2, "B"]]))).tags, { Workflow: "A;B" }); + // A legacy comment atom and a later mdta COMMENT key become one COMMENT tag: no exact-case envelope. + const merged = await read(mdta(["COMMENT"], [[1, "y"]], [v.legacyItem("©cmt", v.data("x"))])); + assert.deepEqual(merged.tags, { comment: "y;x" }); + const mergedUpper = await read(v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["COMMENT"]), v.ilst(v.legacyItem("©cmt", v.data("x")), v.item(1, v.data("y"))))))); + assert.deepEqual(mergedUpper.tags, { COMMENT: "x;y" }); + assert.deepEqual(selectVideoCarrier(mergedUpper.tags).records, {}); + // Matroska keeps the last spelling and value; a string-less SimpleTag deletes any spelling. + assert.deepEqual((await read(v.mkv([v.tagsEl(v.tag([v.simpleTag("PROMPT", GRAPH), v.simpleTag("Prompt", "B")]))]))).tags, { Prompt: "B" }); + assert.deepEqual((await read(v.mkv([v.tagsEl(v.tag([v.simpleTag("PROMPT", GRAPH), v.simpleTag("Prompt", undefined)]))]))).tags, {}); + // ffprobe's format_tags filter folds ASCII only: a Kelvin-sign "workflow" is never reported. + assert.deepEqual((await read(mdta(["prompt", "worKflow"], [[1, GRAPH], [2, WORKFLOW]]))).tags, { prompt: GRAPH }); + // data types 0 and 3 are Mac Roman to ffmpeg (types 1 and 4 UTF-8, type 2 not text). + const cafe = [0x63, 0x61, 0x66, 0x8e]; + for (const [type, expected] of [[0, "café"], [3, "café"], [1, "caf�"], [4, "caf�"]]) { + assert.deepEqual((await read(mdta(["prompt"], [[1, cafe, type]]))).tags, { prompt: expected }, `type ${type}`); + } + assert.deepEqual((await read(mdta(["prompt"], [[1, cafe, 2]]))).tags, {}); + const high = Uint8Array.from({ length: 128 }, (_, index) => 0x80 + index); + const roomy = Buffer.concat([high, Buffer.alloc(400, 0x7c)]); + assert.equal((await read(mdta(["prompt"], [[1, roomy, 0]]))).tags.prompt, new TextDecoder("macintosh").decode(roomy)); + // ffmpeg writes the UTF-8 into twice the input size: a long non-ASCII value is cut, + // here inside U+00DA, which ffprobe then replaces (its own output for these bytes). + const tight = Buffer.concat([high, Buffer.from([0x7c, 0x01, 0x7f])]); + assert.equal((await read(mdta(["prompt"], [[1, tight, 0]]))).tags.prompt, new TextDecoder("macintosh").decode(high).slice(0, 114) + "�"); +}); + +test("budgets: roles share 2 MiB, an oversized workflow is left unread, duplicates and the total are bounded", async () => { + const limited = { status: "limited", reason: "metadata-limit" }; + // Role tags over 2 MiB together stop the read (the server fails the whole read). + assert.deepEqual(await read(v.perKeyMp4({ prompt: fill(0x41, ROLE_BUDGET + 1) })), limited); + assert.deepEqual(await read(v.perKeyMkv({ PROMPT: fill(0x41, ROLE_BUDGET - 10), PARAMETERS: fill(0x42, 11) })), limited); + const atRole = await read(v.perKeyMp4({ prompt: fill(0x41, ROLE_BUDGET - 10), comfyassets_prompt_pairs: fill(0x42, 10) })); + assert.equal(atRole.status, "complete"); + assert.equal(atRole.tags.prompt.length, ROLE_BUDGET - 10); + // A per-key workflow over 4 MiB is not sliced; the result says the workflow is limited. + const big = v.recordingBlob(v.perKeyMp4({ prompt: GRAPH, workflow: fill(0x7b, WORKFLOW_BUDGET + 1) })); + const over = await readVideoTags(big); + // It stays in the tag map as a marker, so the carrier choice still sees a per-key workflow. + assert.deepEqual(over, { status: "complete", container: "iso-bmff", format: "isom", tags: { prompt: GRAPH, workflow: m.OVERSIZED_WORKFLOW }, limited: ["workflow"] }); + assert.ok(big.ranges.every(([start, end]) => end - start < WORKFLOW_BUDGET), "the oversized workflow was never sliced"); + const bigMkv = await read(v.perKeyMkv({ WORKFLOW: fill(0x7b, WORKFLOW_BUDGET + 1), PROMPT: GRAPH })); + assert.deepEqual([bigMkv.limited, Object.keys(bigMkv.tags)], [["workflow"], ["WORKFLOW", "PROMPT"]]); + const atWorkflow = await read(v.perKeyMp4({ workflow: fill(0x7b, WORKFLOW_BUDGET) })); + assert.deepEqual([atWorkflow.limited, atWorkflow.tags.workflow.length], [[], WORKFLOW_BUDGET]); + // Fold-duplicates draw from the duplicate budgets (2x each class), then stop. + // (ffmpeg keeps one tag per ASCII-folded name: Matroska keeps the last spelling and value.) + const twice = await read(v.perKeyMkv({ PROMPT: fill(0x41, ROLE_BUDGET), prompt: fill(0x42, ROLE_BUDGET) })); + assert.deepEqual(Object.keys(twice.tags), ["prompt"]); + assert.equal(twice.tags.prompt[0], "B"); + const thrice = await read(v.perKeyMkv({ PROMPT: fill(0x41, ROLE_BUDGET), prompt: fill(0x41, ROLE_BUDGET), Prompt: fill(0x41, ROLE_BUDGET + 1) })); + assert.deepEqual(thrice, limited); + const workflowTwice = await read(v.perKeyMkv({ WORKFLOW: fill(0x7b, WORKFLOW_BUDGET), workflow: fill(0x7b, WORKFLOW_BUDGET) })); + assert.equal(workflowTwice.status, "complete"); + // Envelopes count against the total only; the total is the server's probe bound. + const envelope = await read(v.mp4(v.legacyUdta({ comment: fill(0x41, 5 * MiB) }))); + assert.equal(envelope.tags.comment.length, 5 * MiB); + const tooMuch = await read(v.mp4(v.legacyUdta({ comment: fill(0x41, TAG_READ_BYTES - 10), description: fill(0x42, 11) }))); + assert.deepEqual(tooMuch, limited); + assert.equal(m.VIDEO_TAG_READ_BYTES, TAG_READ_BYTES); +}); + +test("an oversized per-key workflow keeps its place in the carrier choice, as in the server's first probe", async () => { + const big = fill(0x7b, WORKFLOW_BUDGET + 1); + const envelope = '{"workflow": {"nodes": [1]}}'; + const beside = (comment) => v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["workflow"]), v.ilst(v.item(1, v.data(big)), v.legacyItem("©cmt", v.data(comment)))))); + const read1 = await read(beside(envelope)); + assert.deepEqual([read1.tags, read1.limited], [{ workflow: m.OVERSIZED_WORKFLOW, comment: envelope }, ["workflow"]]); + // The server picks the per-key carrier (no role on either side) and marks the workflow limited. + assert.deepEqual(selectVideoCarrier(read1.tags, { limited: read1.limited }), { status: "complete", records: {}, record_formats: { workflow: "limited" }, carrierText: null, limited: ["workflow"] }); + // A role envelope still wins and clears it. + const roleEnvelope = '{"prompt": "{}", "workflow": {"a": 1}}'; + const read2 = await read(beside(roleEnvelope)); + assert.deepEqual(selectVideoCarrier(read2.tags, { limited: read2.limited }), { status: "complete", records: { prompt: "{}", workflow: '{"a": 1}' }, record_formats: { prompt: "original-text", workflow: "reserialized-json" }, carrierText: roleEnvelope, limited: [] }); + // ISO-BMFF joins a repeat onto the oversized value, which stays over budget and unread. + const repeat = (extra) => v.mp4(v.udta(v.meta(v.hdlr("mdta"), v.keys(["workflow", "prompt"]), v.ilst(v.item(1, v.data(big)), v.item(1, v.data(extra)), v.item(2, v.data(GRAPH)))))); + const repeatedBlob = v.recordingBlob(repeat(WORKFLOW)); + const repeated = await readVideoTags(repeatedBlob); + assert.deepEqual([repeated.tags, repeated.limited], [{ workflow: m.OVERSIZED_WORKFLOW, prompt: GRAPH }, ["workflow"]]); + assert.ok(repeatedBlob.ranges.every(([start, end]) => end - start < 1024), "neither workflow part was sliced"); + // When the first probe would overflow its bound, the server re-probes without the + // per-key workflow: then it takes no part in the choice ... + const dropped = await read(repeat(fill(0x41, TAG_READ_BYTES - WORKFLOW_BUDGET))); + assert.deepEqual([dropped.tags, dropped.limited], [{ prompt: GRAPH }, ["workflow"]]); + // ... and the re-probe's own bound (twice the role budget plus 64 KiB) applies to the rest. + const wide = '{"workflow": {"n": "' + "x".repeat(TAG_READ_BYTES - WORKFLOW_BUDGET) + '"}}'; + assert.deepEqual(await read(beside(wide)), { status: "limited", reason: "metadata-limit" }); + assert.equal(m.VIDEO_REPROBE_BYTES, 2 * ROLE_BUDGET + 64 * 1024); + // Matroska keeps the last value of a name, and a string-less SimpleTag deletes it. + const replaced = await read(v.mkv([v.tagsEl(v.tag([v.simpleTag("WORKFLOW", big), v.simpleTag("WORKFLOW", WORKFLOW)]))])); + assert.deepEqual([replaced.tags, replaced.limited], [{ WORKFLOW: WORKFLOW }, []]); + const deleted = await read(v.mkv([v.tagsEl(v.tag([v.simpleTag("WORKFLOW", big), v.simpleTag("WORKFLOW", undefined), v.simpleTag("PROMPT", GRAPH)]))])); + assert.deepEqual([deleted.tags, deleted.limited], [{ PROMPT: GRAPH }, []]); +}); + +test("avi and asf are recognised but not read", async () => { + const avi = v.concat(v.latin1("RIFF"), v.u32(1000), v.latin1("AVI LIST"), new Uint8Array(16)); + const asf = v.concat([0x30, 0x26, 0xb2, 0x75, 0x8e, 0x66, 0xcf, 0x11, 0xa6, 0xd9, 0x00, 0xaa, 0x00, 0x62, 0xce, 0x6c], new Uint8Array(16)); + for (const bytes of [avi, asf]) { + const blob = v.recordingBlob(v.concat(bytes, new Uint8Array(4096))); + assert.deepEqual(await readVideoTags(blob, { prefix: bytes }), { status: "unsupported", reason: "video-container-unsupported" }); + assert.deepEqual(blob.ranges, []); + } + // The classifier's kind can be handed in instead of a prefix. + const blob = v.recordingBlob(new Uint8Array(64)); + assert.deepEqual(await readVideoTags(blob, { kind: "avi" }), { status: "unsupported", reason: "video-container-unsupported" }); + assert.deepEqual(blob.ranges, []); + // Without a prefix the reader slices the first 32 bytes itself. + const mp4 = v.recordingBlob(v.perKeyMp4({ prompt: GRAPH })); + assert.deepEqual((await readVideoTags(mp4)).tags, { prompt: GRAPH }); + assert.deepEqual(mp4.ranges[0], [0, 32]); +}); + +test("abort during a moov walk settles once", async () => { + const unhandled = []; + const onUnhandled = (reason) => unhandled.push(reason); + process.on("unhandledRejection", onUnhandled); + try { + const bytes = v.concat(v.ftyp(), ...Array.from({ length: 50 }, () => v.free()), v.mdat(), v.moov(v.mdtaUdta({ prompt: GRAPH, workflow: WORKFLOW }))); + const controller = new AbortController(); + let slices = 0; + const blob = { + size: bytes.length, + slice(start, end) { + slices += 1; + if (slices === 3) controller.abort(); + return { arrayBuffer: () => new Promise((resolve) => setTimeout(() => resolve(bytes.slice(start, end).buffer), 2)) }; + }, + }; + await assert.rejects(readVideoTags(blob, { signal: controller.signal }), { name: "AbortError" }); + const already = new AbortController(); + already.abort(); + await assert.rejects(readVideoTags(v.blobOf(bytes), { signal: already.signal }), { name: "AbortError" }); + const slow = v.slowBlob(bytes, 1); + const during = new AbortController(); + const pending = readVideoTags(slow, { signal: during.signal, yieldControl: async () => during.abort() }); + await assert.rejects(pending, { name: "AbortError" }); + await new Promise((resolve) => setTimeout(resolve, 10)); + assert.deepEqual(unhandled, []); + } finally { + process.off("unhandledRejection", onUnhandled); + } +}); + +test("yieldControl is awaited while walking many boxes", async () => { + let yields = 0; + const many = v.concat(v.ftyp(), ...Array.from({ length: 3000 }, () => v.free()), v.mdat(), v.moov(v.mdtaUdta({ prompt: GRAPH }))); + const result = await readVideoTags(v.slowBlob(many, 0), { yieldControl: async () => { yields += 1; } }); + assert.equal(result.status, "complete"); + assert.ok(yields > 0, "the reader handed control back"); +}); + +test("carrier selection: per-key tags, then the comment envelope, then description, never merged", () => { + const ENVELOPE_WORKFLOW = '{"last_node_id": 9, "nodes": [{"id": 3, "type": "KSampler", "widgets_values": [7, "fixed", 20, 8.0, "euler", "normal", 1.0]}, {"id": 6, "type": "CLIPTextEncode", "widgets_values": ["a red fox in snow, detailed fur"]}, {"id": 7, "type": "CLIPTextEncode", "widgets_values": ["blurry, lowres"]}], "links": [], "version": 0.4}'; + const original = (keys) => Object.fromEntries(keys.map((key) => [key, "original-text"])); + const text = (tags, keys) => Object.fromEntries(keys.map((key) => [key, tags[key] ?? tags[key.toUpperCase()]])); + // The recorded fixtures. + for (const name of ["core-faststart.mp4", "core-moov-end.mp4", "core.mov", "core.m4v", "core.mkv", "core-tags.webm"]) { + const tags = RECORDED[name]; + const keys = Object.keys(tags).map((key) => key.toLowerCase()).filter((key) => PER_KEY.has(key)); + assert.deepEqual(selectVideoCarrier(tags), { status: "complete", records: text(tags, keys), record_formats: original(keys), carrierText: null, limited: [] }, name); + } + const legacy = RECORDED["legacy-comment.mp4"]; + const comment = pythonJsonLoads(legacy.comment); + assert.deepEqual(selectVideoCarrier(legacy), { + status: "complete", + records: { prompt: comment.get("prompt"), workflow: ENVELOPE_WORKFLOW }, + record_formats: { prompt: "original-text", workflow: "reserialized-json" }, + carrierText: legacy.comment, + limited: [], + }); + assert.ok(comment.get("prompt").startsWith('{"3":{"class_type":"KSampler"')); + // Matroska upper-cases COMMENT, and envelope names are exact-case (ruling M1): nothing is read. + assert.deepEqual(selectVideoCarrier(RECORDED["legacy-comment.webm"]), { status: "complete", records: {}, record_formats: {}, carrierText: null, limited: [] }); + // Synthetic maps. + const envelope = JSON.stringify({ prompt: '{"9":{}}', workflow: { v: 1 } }); + const cases = [ + ["per-key wins over comment", { prompt: GRAPH, workflow: WORKFLOW, comment: envelope }, { records: { prompt: GRAPH, workflow: WORKFLOW }, record_formats: original(["prompt", "workflow"]), carrierText: null }], + ["Matroska names fold", { PROMPT: GRAPH, Workflow: WORKFLOW, ENCODER: "Lavf" }, { records: { prompt: GRAPH, workflow: WORKFLOW }, record_formats: original(["prompt", "workflow"]), carrierText: null }], + ["identical fold-duplicates are one record", { PROMPT: GRAPH, prompt: GRAPH }, { records: { prompt: GRAPH }, record_formats: original(["prompt"]), carrierText: null }], + ["differing fold-duplicates fail", { PROMPT: GRAPH, prompt: '{"9":{}}' }, { status: "unreadable", reason: "duplicate-metadata" }], + ["differing workflow duplicates fail even with a prompt", { Workflow: WORKFLOW, WORKFLOW: "{}", prompt: GRAPH }, { status: "unreadable", reason: "duplicate-metadata" }], + ["envelope names never fold", { comment: envelope, COMMENT: "a free-text note" }, { records: { prompt: '{"9":{}}', workflow: '{"v": 1}' }, record_formats: { prompt: "original-text", workflow: "reserialized-json" }, carrierText: envelope }], + ["non-JSON comment falls through to description", { comment: "not json", description: envelope }, { records: { prompt: '{"9":{}}', workflow: '{"v": 1}' }, record_formats: { prompt: "original-text", workflow: "reserialized-json" }, carrierText: envelope }], + ["a JSON array comment is not an envelope", { comment: '["prompt"]', description: '{"workflow": {}}' }, { records: { workflow: "{}" }, record_formats: { workflow: "reserialized-json" }, carrierText: '{"workflow": {}}' }], + ["a role envelope beats a workflow-only per-key carrier", { workflow: WORKFLOW, comment: envelope }, { records: { prompt: '{"9":{}}', workflow: '{"v": 1}' }, record_formats: { prompt: "original-text", workflow: "reserialized-json" }, carrierText: envelope }], + ["role description beats workflow-only comment", { comment: '{"workflow": {"a": 1}}', description: '{"parameters": "cat"}' }, { records: { parameters: "cat" }, record_formats: { parameters: "original-text" }, carrierText: '{"parameters": "cat"}' }], + ["workflow-only carriers: the first wins, never merged", { comment: '{"workflow": {"a": 1}}', description: '{"workflow": {"b": 2}}' }, { records: { workflow: '{"a": 1}' }, record_formats: { workflow: "reserialized-json" }, carrierText: '{"workflow": {"a": 1}}' }], + ["description only", { description: envelope }, { records: { prompt: '{"9":{}}', workflow: '{"v": 1}' }, record_formats: { prompt: "original-text", workflow: "reserialized-json" }, carrierText: envelope }], + ["no known key", { encoder: "Lavf", major_brand: "isom" }, { records: {}, record_formats: {}, carrierText: null }], + ["envelope with only unknown keys is no carrier", { comment: '{"title": "x"}' }, { records: {}, record_formats: {}, carrierText: null }], + ["null role marker is refused", { comment: '{"prompt": null, "workflow": {}}' }, { status: "unreadable", reason: "invalid-metadata" }], + ["numeric role is refused", { description: '{"parameters": 5}' }, { status: "unreadable", reason: "invalid-metadata" }], + ["a non-text non-object workflow is invalid but the roles stay", { comment: '{"prompt": "{}", "workflow": null}' }, { records: { prompt: "{}" }, record_formats: { prompt: "original-text", workflow: "invalid" }, carrierText: '{"prompt": "{}", "workflow": null}' }], + ["an object prompt is reserialized", { comment: '{"prompt": {"1": {"class_type": "A"}}}' }, { records: { prompt: '{"1": {"class_type": "A"}}' }, record_formats: { prompt: "reserialized-json" }, carrierText: '{"prompt": {"1": {"class_type": "A"}}}' }], + ["a non-text per-key role marker is refused", { prompt: 5, workflow: WORKFLOW }, { status: "unreadable", reason: "invalid-metadata" }], + ["a non-text per-key workflow is invalid", { prompt: GRAPH, workflow: 5 }, { records: { prompt: GRAPH }, record_formats: { prompt: "original-text", workflow: "invalid" }, carrierText: null }], + ["a workflow-only envelope with an invalid workflow is still the carrier", { comment: '{"workflow": 5}' }, { records: {}, record_formats: { workflow: "invalid" }, carrierText: '{"workflow": 5}' }], + ["not a map", null, { records: {}, record_formats: {}, carrierText: null }], + ]; + for (const [name, tags, expected] of cases) { + const want = expected.status ? expected : { status: "complete", limited: [], ...expected }; + assert.deepEqual(selectVideoCarrier(tags), want, name); + } + // The server's budget loop: a 4 MiB workflow is limited, roles over 2 MiB fail, exact sizes pass. + const bigWorkflow = "{" + "0".repeat(WORKFLOW_BUDGET); + assert.deepEqual(selectVideoCarrier({ prompt: GRAPH, workflow: bigWorkflow }), { status: "complete", records: { prompt: GRAPH }, record_formats: { prompt: "original-text", workflow: "limited" }, carrierText: null, limited: ["workflow"] }); + assert.deepEqual(selectVideoCarrier({ workflow: bigWorkflow.slice(1) }).records, { workflow: bigWorkflow.slice(1) }); + assert.deepEqual(selectVideoCarrier({ comment: JSON.stringify({ prompt: "x", workflow: { k: "0".repeat(WORKFLOW_BUDGET) } }) }).record_formats, { prompt: "original-text", workflow: "limited" }); + assert.deepEqual(selectVideoCarrier({ prompt: "a".repeat(ROLE_BUDGET), parameters: "b" }), { status: "limited", reason: "metadata-limit" }); + assert.deepEqual(selectVideoCarrier({ prompt: "é".repeat(ROLE_BUDGET / 2) }).records.prompt.length, ROLE_BUDGET / 2); + assert.deepEqual(selectVideoCarrier({ prompt: "é".repeat(ROLE_BUDGET / 2) + "a" }), { status: "limited", reason: "metadata-limit" }); + assert.deepEqual(selectVideoCarrier({ prompt: "\ud800".repeat(ROLE_BUDGET / 3) + "aaa" }), { status: "limited", reason: "metadata-limit" }); + assert.deepEqual(selectVideoCarrier({ prompt: "\u{1f600}".repeat(ROLE_BUDGET / 4) }).records.prompt.length, ROLE_BUDGET / 2); + assert.deepEqual(selectVideoCarrier({ prompt: "\u{1f600}".repeat(ROLE_BUDGET / 4) + "a" }), { status: "limited", reason: "metadata-limit" }); + // The reader's limited workflow (the server's role-only re-probe): kept with the per-key + // carrier or no carrier, cleared when an envelope wins. + assert.deepEqual(selectVideoCarrier({ prompt: GRAPH }, { limited: ["workflow"] }).record_formats, { prompt: "original-text", workflow: "limited" }); + assert.deepEqual(selectVideoCarrier({ encoder: "x" }, { limited: ["workflow"] }), { status: "complete", records: {}, record_formats: { workflow: "limited" }, carrierText: null, limited: ["workflow"] }); + assert.deepEqual(selectVideoCarrier({ comment: envelope }, { limited: ["workflow"] }).limited, []); + assert.deepEqual(selectVideoCarrier({ comment: envelope }, { limited: ["workflow"] }).record_formats, { prompt: "original-text", workflow: "reserialized-json" }); +}); + +test("pythonJsonLoads and pythonJsonDumps spell values as Python's json module does", () => { + const rows = [ + ["1.0", "1.0"], ["8.0", "8.0"], ["0.1", "0.1"], ["1e16", "1e+16"], ["1e15", "1000000000000000.0"], + ["123456789012345678", "123456789012345678"], ["0.0001", "0.0001"], ["0.00001", "1e-05"], ["1e-7", "1e-07"], + ["1.5e300", "1.5e+300"], ["1e400", "Infinity"], ["-0", "0"], ["-0.0", "-0.0"], ["1E2", "100.0"], ["2.5", "2.5"], + ["100.0", "100.0"], ["1e21", "1e+21"], ["123456.789", "123456.789"], ["0.000123", "0.000123"], ["-1e-05", "-1e-05"], + ["9007199254740993", "9007199254740993"], ["1.7976931348623157e308", "1.7976931348623157e+308"], ["5e-324", "5e-324"], + ["NaN", "NaN"], ["-Infinity", "-Infinity"], ["Infinity", "Infinity"], ["[1.0e+2]", "[100.0]"], ["-1e400", "-Infinity"], + ["[]", "[]"], ["{}", "{}"], ["[true, false, null]", "[true, false, null]"], ['{"k":[1,{"z":1}]}', '{"k": [1, {"z": 1}]}'], + [' \t\n\r{"a" : 1 } ', '{"a": 1}'], + ['{"a":"é\\u0001\u007f\\"\\\\\\n\\t\\b\\f\\r\\/ 😀 \\ud800"}', '{"a": "\\u00e9\\u0001\\u007f\\"\\\\\\n\\t\\b\\f\\r/ \\ud83d\\ude00 \\ud800"}'], + ['{"a":"\\ud83d\\ude00"}', '{"a": "\\ud83d\\ude00"}'], + ['{"10":1,"3":2,"b":3,"a":4}', '{"10": 1, "3": 2, "b": 3, "a": 4}'], + ['{"a":1,"b":2,"a":3}', '{"a": 3, "b": 2}'], + ]; + for (const [input, output] of rows) { + assert.equal(pythonJsonDumps(pythonJsonLoads(input)), output, input); + } + const bad = ['{"a":1} x', '{"a":"\u0001"}', '{"a":"\\x"}', "01", "1.", ".5", "+1", "[1,]", '{"a":1,}', "{a:1}", "'a'", "", " ", "[", "nul", "-", "1e", '"\\u12"', "[1 2]", "tru", "Infinit"]; + for (const input of bad) { + assert.throws(() => pythonJsonLoads(input), { name: "PythonJsonError" }, JSON.stringify(input)); + } + assert.throws(() => pythonJsonLoads("[".repeat(2000) + "]".repeat(2000)), { name: "PythonJsonError", recursion: true }); + // CPython's int() refuses more than 4300 digits (sign excluded); floats have no such limit. + const digits = (count) => "1" + "0".repeat(count - 1); + assert.equal(pythonJsonDumps(pythonJsonLoads(digits(4300))), digits(4300)); + assert.equal(pythonJsonDumps(pythonJsonLoads("-" + digits(4300))), "-" + digits(4300)); + for (const input of [digits(4301), "-" + digits(4301), `[${digits(5000)}]`]) { + assert.throws(() => pythonJsonLoads(input), (error) => error.name === "PythonJsonError" && error.recursion === false, input.slice(0, 8)); + } + assert.equal(pythonJsonLoads(digits(4301) + ".0").isFloat, true); + // So an envelope holding such an integer is skipped, as the server's _carriers does. + const hugeInt = `{"workflow": {"a": ${digits(4400)}}}`; + assert.deepEqual(selectVideoCarrier({ comment: hugeInt, description: '{"workflow": {"b": 2}}' }).carrierText, '{"workflow": {"b": 2}}'); + assert.throws(() => pythonJsonDumps(undefined), { name: "PythonJsonError" }); + assert.throws(() => pythonJsonLoads(5), { name: "PythonJsonError" }); + assert.equal(pythonJsonDumps(pythonJsonLoads("[".repeat(500) + "]".repeat(500))), "[".repeat(500) + "]".repeat(500)); + const parsed = pythonJsonLoads('{"s":"x","n":1,"f":1.5,"b":true,"z":null,"l":[1]}'); + assert.ok(parsed instanceof Map); + assert.deepEqual([...parsed.keys()], ["s", "n", "f", "b", "z", "l"]); + assert.equal(parsed.get("s"), "x"); + assert.equal(parsed.get("b"), true); + assert.equal(parsed.get("z"), null); + assert.ok(Array.isArray(parsed.get("l"))); +}); diff --git a/tests/test_local_inspect_parity.py b/tests/test_local_inspect_parity.py new file mode 100644 index 0000000..9fdfdd9 --- /dev/null +++ b/tests/test_local_inspect_parity.py @@ -0,0 +1,1115 @@ +"""The browser's local readers answer exactly as the server's on the same file. + +``web/js/png-metadata.js`` ``readPngMetadataFile`` ports the bounded PNG reader in +``utils/prompt_inspector.py``. This test hands the inspection parity corpus (the +committed ``f*.png`` and the test-time shapes), extra compressed, oversized, +Latin-1 and batch-duplicate PNGs, and the committed JPEG/WebP header fixtures to a +Node oracle, and asserts per file that records, record formats, dimensions and +readability equal ``read_inspection_file``. The oracle reads each file through +``fs.openAsBlob`` slices, as the page reads a chosen ``File``. + +``web/js/workspace/local-video.mjs`` reads the container tags the server asks +ffprobe for and ports ``utils/video_metadata.select_video_carrier`` with the +record budgets of ``read_inspection_file``. The video tests hand the oracle the +committed container fixtures (``tests/fixtures/local-inspect/*.{mp4,mov,m4v,mkv, +webm}`` with their recorded ffprobe tag maps in ``video-tags-v1.json``), recorded +and synthetic tag maps, and multi-megabyte per-key containers built at test time, +and assert that the local carrier equals the server's, which selects from the +same tag map through a mocked probe, or from the same file through real ffprobe. + +``web/js/workspace/local-projection.mjs`` ports ``utils/inspection_response.py``: +the strict Decimal/duplicate-key parse, the prompt, workflow and carrier +projection, the depth and node ceilings and ``_metadata``. The projection tests +hand the oracle every file above plus the committed adversarial record sets in +``tests/fixtures/local-inspect/projection-cases.json`` and node-ceiling sets built +at test time, and assert that ``projectMetadata`` equals ``_metadata`` byte for +byte, as the server's ``_bytes`` writes it with sorted keys. +""" + +import io +import json +import os +import shutil +import struct +import subprocess # nosec B404 +import sys +import tempfile +import unittest +import zlib +from pathlib import Path +from unittest.mock import patch + +from PIL import Image +from PIL.PngImagePlugin import PngInfo + +from utils.inspection_response import GRAPH_NODE_CEILING, _bytes, _metadata +from utils.prompt_inspector import ( + ROLE_METADATA_BYTES, + WORKFLOW_METADATA_BYTES, + InspectionRead, + MetadataReadError, + read_inspection_file, +) + +try: + from tests import inspection_parity as parity + from tests.open_handles import assert_closed + from tests.test_utils_video_metadata import HAVE_FFMPEG, write_video +except ImportError: # pragma: no cover - discover -s tests puts tests/ first + import inspection_parity as parity + from open_handles import assert_closed + from test_utils_video_metadata import HAVE_FFMPEG, write_video + +HERE = Path(__file__).resolve().parent +ORACLE = HERE / "js" / "support" / "local-inspect-oracle.mjs" +HEADER_FIXTURES = HERE / "fixtures" / "local-inspect" +VIDEO_FIXTURES = HEADER_FIXTURES +RECORDED_TAGS = json.loads( + (VIDEO_FIXTURES / "video-tags-v1.json").read_text(encoding="utf-8") +)["fixtures"] +PROJECTION_CASES = json.loads( + (HEADER_FIXTURES / "projection-cases.json").read_text(encoding="utf-8") +)["cases"] +NODE_TIMEOUT_SECONDS = 180 +PNG_CLASS = {"media": "image", "kind": "png"} +_FOOTER = "Steps: 20, Sampler: Euler, Batch index: {}, Seed: 7" +MiB = 1024 * 1024 +GRAPH = '{"3":{"class_type":"KSampler","inputs":{"seed":7}}}' +WORKFLOW = '{"nodes":[],"version":0.4}' +ENVELOPE = json.dumps({"prompt": '{"9":{}}', "workflow": {"v": 1}}) +VIDEO_CONTAINERS = { + ".mp4": "iso-bmff", + ".mov": "iso-bmff", + ".m4v": "iso-bmff", + ".mkv": "matroska", + ".webm": "matroska", +} +PER_KEY_NAMES = {"prompt", "workflow", "parameters", "comfyassets_prompt_pairs"} + +# Tag maps as ffprobe would report them, covering every branch of the carrier rule: +# (name, tags, records the reader left out). The server answer is computed, not +# written down, so a change in either side fails here. +TAG_MAP_VECTORS = [ + ( + "per_key_wins_over_comment", + {"prompt": GRAPH, "workflow": WORKFLOW, "comment": ENVELOPE}, + [], + ), + ( + "matroska_names_fold", + {"PROMPT": GRAPH, "Workflow": WORKFLOW, "ENCODER": "Lavf"}, + [], + ), + ("identical_fold_duplicates", {"PROMPT": GRAPH, "prompt": GRAPH}, []), + ("differing_fold_duplicates_fail", {"PROMPT": GRAPH, "prompt": '{"9":{}}'}, []), + ( + "differing_workflow_duplicates_fail", + {"Workflow": WORKFLOW, "WORKFLOW": "{}", "prompt": GRAPH}, + [], + ), + ( + "envelope_names_never_fold", + {"comment": ENVELOPE, "COMMENT": "a free-text note"}, + [], + ), + ( + "non_json_comment_falls_to_description", + {"comment": "not json", "description": ENVELOPE}, + [], + ), + ( + "array_comment_is_not_an_envelope", + {"comment": '["prompt"]', "description": '{"workflow": {}}'}, + [], + ), + ( + "role_envelope_beats_workflow_only_per_key", + {"workflow": WORKFLOW, "comment": ENVELOPE}, + [], + ), + ( + "role_description_beats_workflow_comment", + {"comment": '{"workflow": {"a": 1}}', "description": '{"parameters": "cat"}'}, + [], + ), + ( + "workflow_only_carriers_never_merge", + {"comment": '{"workflow": {"a": 1}}', "description": '{"workflow": {"b": 2}}'}, + [], + ), + ("description_only", {"description": ENVELOPE}, []), + ("no_known_key", {"encoder": "Lavf", "major_brand": "isom"}, []), + ("envelope_without_known_keys", {"comment": '{"title": "x"}'}, []), + ("null_role_marker_refused", {"comment": '{"prompt": null, "workflow": {}}'}, []), + ("numeric_role_refused", {"description": '{"parameters": 5}'}, []), + ( + "invalid_envelope_workflow_keeps_roles", + {"comment": '{"prompt": "{}", "workflow": null}'}, + [], + ), + ( + "object_prompt_reserialized", + { + "comment": '{"prompt": {"1": {"class_type": "A", "f": 8.0, "n": 1e16, ' + '"u": "é😀"}}}' + }, + [], + ), + ("workflow_only_invalid_envelope", {"comment": '{"workflow": 5}'}, []), + ("non_text_per_key_role_refused", {"prompt": 5, "workflow": WORKFLOW}, []), + ("non_text_per_key_workflow_invalid", {"prompt": GRAPH, "workflow": 5}, []), + ( + "python_number_spellings", + { + "description": '{"workflow": {"a": [1.0, 0.1, 1E2, 1e-7, -0.0, -0, ' + "123456789012345678, 1e400, NaN, 0.00001, 1e15], " + '"b": "\\u0001\\u007f/"}}' + }, + [], + ), + ( + "integer_like_keys_keep_order", + {"description": '{"workflow": {"10": 1, "3": 2, "b": 3, "a": 4, "3": 5}}'}, + [], + ), + ( + "workflow_over_budget_limited", + {"prompt": GRAPH, "workflow": "{" + "0" * WORKFLOW_METADATA_BYTES}, + [], + ), + ("workflow_at_budget_kept", {"workflow": "0" * WORKFLOW_METADATA_BYTES}, []), + ( + "roles_over_budget_fail", + {"prompt": "a" * ROLE_METADATA_BYTES, "parameters": "b"}, + [], + ), + ("roles_at_budget_kept", {"prompt": "é" * (ROLE_METADATA_BYTES // 2)}, []), + ( + "lone_surrogates_count_three_bytes", + {"prompt": "\ud800" * (ROLE_METADATA_BYTES // 3) + "aaa"}, + [], + ), + ( + "envelope_workflow_over_budget_limited", + { + "comment": json.dumps( + {"prompt": "x", "workflow": {"k": "0" * WORKFLOW_METADATA_BYTES}} + ) + }, + [], + ), + ("reader_limited_workflow_with_per_key", {"prompt": GRAPH}, ["workflow"]), + ("reader_limited_workflow_without_carrier", {"encoder": "x"}, ["workflow"]), + ( + "reader_limited_workflow_cleared_by_envelope", + {"comment": ENVELOPE}, + ["workflow"], + ), + # CPython's json.loads refuses an integer of more than 4300 digits, so the + # server skips that envelope for the next one. + ( + "envelope_integer_over_digit_limit_skipped", + { + "comment": '{"workflow": {"a": 1' + "0" * 4400 + "}}", + "description": '{"workflow": {"b": 2}}', + }, + [], + ), +] + + +def _raw_chunk(kind, data): + return ( + struct.pack(">I", len(data)) + + kind + + data + + struct.pack(">I", zlib.crc32(kind + data) & 0xFFFFFFFF) + ) + + +def _text_chunk(key, text): + return _raw_chunk(b"tEXt", key.encode("latin-1") + b"\0" + text.encode("latin-1")) + + +def _write_png(path, info, extra_chunks=()): + """A 16x16 PNG with Pillow-written text records and spliced raw chunks.""" + buffer = io.BytesIO() + with Image.new("RGB", (16, 16), (10, 20, 30)) as image: + image.save(buffer, format="PNG", pnginfo=info, compress_level=4) + data = buffer.getvalue() + iend = data.rindex(b"IEND") - 4 + with open(path, "wb") as stream: + stream.write(data[:iend] + b"".join(extra_chunks) + data[iend:]) + + +def _info(**texts): + info = PngInfo() + for key, value in texts.items(): + if isinstance(value, tuple): + text, options = value + if options.get("itxt"): + info.add_itxt( + key, + text, + lang=options.get("lang", ""), + tkey=options.get("tkey", ""), + zip=options.get("zip", False), + ) + else: + info.add_text(key, text, zip=options.get("zip", False)) + else: + info.add_text(key, value) + return info + + +PROMPT = json.dumps(parity.build("f02")["prompt"]) +WORKFLOW = json.dumps(parity.build("f02")["workflow"]) + +# Shapes the committed corpus lacks: compressed chunks, the budgets, Latin-1 text, +# the batch-index rule and the pair execution record. +EXTRA_SHAPES = { + "ztxt_prompt_itxt_workflow": lambda path: _write_png( + path, + _info( + prompt=(PROMPT, {"zip": True}), + workflow=(WORKFLOW, {"itxt": True, "zip": True}), + ), + ), + "itxt_bom_lang": lambda path: _write_png( + path, + _info( + prompt=("" + PROMPT, {"itxt": True, "lang": "en", "tkey": "prompt"}), + workflow=(WORKFLOW, {"itxt": True}), + ), + ), + "latin1_prompt": lambda path: _write_png( + path, _info(prompt='{"1":{"class_type":"Note","inputs":{"text":"café ÿ"}}}') + ), + "all_four_records": lambda path: _write_png( + path, + _info( + prompt=PROMPT, + workflow=WORKFLOW, + parameters="cat\nNegative prompt: dog\n" + _FOOTER.format(0), + comfyassets_prompt_pairs='{"pairs":[]}', + ), + ), + "workflow_over_budget_limited": lambda path: _write_png( + path, + _info( + prompt=PROMPT, + workflow="[" + "0," * ((WORKFLOW_METADATA_BYTES + 1) // 2) + "]", + ), + ), + "workflow_at_budget_kept": lambda path: _write_png( + path, _info(workflow=("{" * WORKFLOW_METADATA_BYTES, {"zip": True})) + ), + "roles_over_budget_unreadable": lambda path: _write_png( + path, _info(prompt="A" * (ROLE_METADATA_BYTES + 1)) + ), + "roles_over_budget_decoded_unreadable": lambda path: _write_png( + path, _info(prompt=("A" * (ROLE_METADATA_BYTES + 1), {"zip": True})) + ), + "roles_at_budget_kept": lambda path: _write_png( + path, + _info( + prompt=("A" * (ROLE_METADATA_BYTES // 2), {"zip": True}), + parameters=("B" * (ROLE_METADATA_BYTES // 2), {"zip": True}), + ), + ), + "batch_parameters_last_kept": lambda path: _write_png( + path, + _info(prompt=PROMPT), + [ + _text_chunk("parameters", "cat\n" + _FOOTER.format(index)) + for index in (0, 1, 2, 10) + ], + ), + "batch_parameters_conflict_unreadable": lambda path: _write_png( + path, + PngInfo(), + [ + _text_chunk("parameters", "cat\n" + _FOOTER.format(0)), + _text_chunk("parameters", "dog\n" + _FOOTER.format(1)), + ], + ), + "duplicate_workflow_identical": lambda path: _write_png( + path, + _info(prompt=PROMPT), + [_text_chunk("workflow", WORKFLOW), _text_chunk("workflow", WORKFLOW)], + ), + "unknown_keys_only": lambda path: _write_png( + path, _info(Software="ComfyUI", Comment=("café", {"zip": True})) + ), +} + + +def _classes(results): + return {name: entry["classified"] for name, entry in results.items()} + + +def _run_oracle(testcase, entries): + node = shutil.which("node") + if node is None: # pragma: no cover - CI runners and dev hosts ship Node + if os.environ.get("CI"): + testcase.fail("Node.js is required in CI for the local reader parity test") + testcase.skipTest("Node.js is not installed") + # Fixed local Node harness, no shell. + completed = subprocess.run( # nosec B603 + [node, str(ORACLE)], + input=json.dumps(entries).encode("utf-8"), + capture_output=True, + timeout=NODE_TIMEOUT_SECONDS, + check=False, + ) + testcase.assertEqual(completed.returncode, 0, completed.stderr.decode("utf-8")) + return json.loads(completed.stdout) + + +def _known_tags(tags): + """The tags ffprobe reports to the server: the six probed names, ASCII folded.""" + return { + key: value + for key, value in tags.items() + if key.lower() in PER_KEY_NAMES or key.lower() in ("comment", "description") + } + + +def _fake_probe(tags, drop_workflow=False): + """``_probe_video_tags`` answering from a tag map as ffprobe would. + + ffprobe's ``format_tags=`` filter is case-insensitive and the stdout bound is + the only reason a probe answers ``None``; ``drop_workflow`` makes the first + probe (the one asking for ``workflow``) overflow, as a huge per-key workflow + tag does, so the server takes its role-only re-probe. + """ + + def probe(binary, path, keys, limit): + if drop_workflow and "workflow" in keys: + return None + selected = {key: value for key, value in tags.items() if key.lower() in keys} + raw = json.dumps({"format": {"tags": selected}}, ensure_ascii=False).encode( + "utf-8", "surrogatepass" + ) + return None if len(raw) > limit else raw + + return probe + + +def _server_selection(tags, limited=()): + """The server's carrier for a tag map, in the oracle's shape; None when it fails.""" + with ( + patch("utils.prompt_inspector.ffprobe_path", return_value="/tool/ffprobe"), + patch( + "utils.prompt_inspector._probe_video_tags", + side_effect=_fake_probe(tags, drop_workflow="workflow" in limited), + ), + ): + try: + read = read_inspection_file(Path("vector.mp4")) + except MetadataReadError: + return None + return _read_shape(read) + + +def _read_shape(read): + return { + "records": read.metadata, + "record_formats": read.record_formats, + "carrierText": read.video_carrier_text, + "limited": sorted(read.limited_records), + } + + +def _local_shape(selection): + if selection["status"] != "complete": + return None + return { + "records": selection["records"], + "record_formats": selection["record_formats"], + "carrierText": selection["carrierText"], + "limited": sorted(selection["limited"]), + } + + +def _box(kind, *parts): + payload = b"".join(parts) + return struct.pack(">I", 8 + len(payload)) + kind + payload + + +def _full_box(kind, *parts): + return _box(kind, b"\0\0\0\0", *parts) + + +def _mdta_udta(tags): + """``moov/udta/meta`` with one mdta key per tag, laid out as ffmpeg writes it.""" + names = [name.encode("utf-8") for name in tags] + values = [value.encode("utf-8") for value in tags.values()] + return _mdta_items(names, list(enumerate(values, 1))) + + +def _mdta_items(names, items, keys_boxes=(), legacy=()): + """``moov/udta/meta`` from raw bytes: mdta key ``names``, ``(index, value)`` + items, extra ``keys`` boxes after the first (each a list of names) and + ``(fourcc, value)`` legacy atoms in the same ``ilst``. An item may carry a + third member, its ``data`` type (default 1, UTF-8).""" + + def keys(entries): + return _full_box( + b"keys", + struct.pack(">I", len(entries)), + *(struct.pack(">I", 8 + len(name)) + b"mdta" + name for name in entries), + ) + + def data(value, kind=1): + return _box(b"data", struct.pack(">II", kind, 0), value) + + ilst = _box( + b"ilst", + *(_box(struct.pack(">I", item[0]), data(*item[1:])) for item in items), + *(_box(fourcc.encode("latin-1"), data(value)) for fourcc, value in legacy), + ) + hdlr = _full_box(b"hdlr", b"\0\0\0\0", b"mdta", bytes(12), b"\0") + boxes = [keys(names), *(keys(entries) for entries in keys_boxes)] + return _box(b"udta", _full_box(b"meta", hdlr, *boxes, ilst)) + + +def _with_udta(data, udta): + """``data`` (an MP4 with moov after mdat) with its moov's udta replaced.""" + out = bytearray() + offset = 0 + while offset < len(data): + size, kind = struct.unpack(">I4s", data[offset : offset + 8]) + if kind != b"moov": + out += data[offset : offset + size] + offset += size + continue + body = data[offset + 8 : offset + size] + children = bytearray() + inner = 0 + while inner < len(body): + child_size, child_kind = struct.unpack(">I4s", body[inner : inner + 8]) + if child_kind != b"udta": + children += body[inner : inner + child_size] + inner += child_size + children += udta + out += struct.pack(">I", 8 + len(children)) + b"moov" + bytes(children) + offset += size + return bytes(out) + + +def _text_of(size, prefix): + """A valid JSON object of exactly ``size`` UTF-8 bytes with quotes throughout.""" + unit = '"k":"v",' + body = (unit * (size // len(unit) + 1))[: size - len(prefix) - 2] + body = body[: body.rfind(",")] if body.endswith(",") else body + text = prefix + body + "}" + return text + " " * (size - len(text)) + + +class LocalPngReaderParityTests(unittest.TestCase): + def setUp(self): + temporary = tempfile.TemporaryDirectory() + self.addCleanup(temporary.cleanup) + self.addCleanup(assert_closed, self, temporary.name) + self.directory = Path(temporary.name) + + def _files(self): + files = {} + for name in parity.COMMITTED: + files[name] = parity.fixture_path(name) + for name in parity.TEST_TIME: + path = parity.fixture_path(name, self.directory) + parity.write_png(path, parity.build(name)) + files[name] = path + for name, write in EXTRA_SHAPES.items(): + path = self.directory / f"{name}.png" + write(path) + files[name] = path + for header in ("tiny.jpg", "tiny.webp"): + files[header] = HEADER_FIXTURES / header + return files + + def _oracle(self, files): + return _run_oracle( + self, [{"name": name, "path": str(path)} for name, path in files.items()] + ) + + def test_local_png_reader_matches_read_inspection_file(self): + files = self._files() + observed = self._oracle(files) + self.assertEqual(set(observed), set(files)) + outcomes = set() + for name, path in files.items(): + if path.suffix != ".png": + continue + with self.subTest(fixture=name): + self.assertEqual(observed[name]["classified"], PNG_CLASS) + result = observed[name]["result"] + try: + read = read_inspection_file(path) + except MetadataReadError: + outcomes.add("unreadable") + self.assertEqual(result["status"], "unreadable", result) + continue + self.assertEqual(result["status"], "complete", result) + self.assertEqual(result["records"], read.metadata) + self.assertEqual( + sorted(result["limited"]), sorted(read.limited_records) + ) + formats = {key: "original-text" for key in result["records"]} + formats.update({key: "limited" for key in result["limited"]}) + self.assertEqual(formats, read.record_formats) + self.assertEqual( + (result["width"], result["height"]), (read.width, read.height) + ) + self.assertEqual(read.format, "PNG") + outcomes.add("limited" if read.limited_records else "complete") + # The corpus must exercise every outcome, or agreement proves little. + self.assertEqual(outcomes, {"complete", "limited", "unreadable"}) + + def test_local_image_header_matches_pillow(self): + files = {name: HEADER_FIXTURES / name for name in ("tiny.jpg", "tiny.webp")} + observed = self._oracle(files) + self.assertEqual( + _classes(observed), + { + "tiny.jpg": {"media": "image", "kind": "jpeg"}, + "tiny.webp": {"media": "image", "kind": "webp"}, + }, + ) + for name, path in files.items(): + with self.subTest(fixture=name): + read = read_inspection_file(path) + self.assertEqual(read.metadata, {}) + self.assertEqual( + observed[name]["result"], + {"format": read.format, "width": read.width, "height": read.height}, + ) + self.assertLess(path.stat().st_size, 4096) + + +class LocalVideoCarrierParityTests(unittest.TestCase): + """The local video readers and carrier rule answer as the server does.""" + + def setUp(self): + temporary = tempfile.TemporaryDirectory() + self.addCleanup(temporary.cleanup) + self.addCleanup(assert_closed, self, temporary.name) + self.directory = Path(temporary.name) + + def test_local_carrier_matches_server_selection(self): + vectors = [(name, tags, []) for name, tags in RECORDED_TAGS.items()] + vectors += TAG_MAP_VECTORS + observed = _run_oracle( + self, + [ + {"name": name, "tags": tags, "limited": limited} + for name, tags, limited in vectors + ], + ) + outcomes = set() + for name, tags, limited in vectors: + with self.subTest(vector=name): + selection = observed[name]["result"]["selection"] + server = _server_selection(tags, limited) + self.assertEqual( + _local_shape(selection), server, selection.get("reason") + ) + if server is None: + outcomes.add(selection["status"] + ":" + selection["reason"]) + elif not server["records"] and not server["record_formats"]: + outcomes.add("no-carrier") + else: + outcomes.update(server["record_formats"].values()) + # Every outcome of the rule must occur, or agreement proves little. + self.assertEqual( + outcomes, + { + "original-text", + "reserialized-json", + "invalid", + "limited", + "no-carrier", + "unreadable:duplicate-metadata", + "unreadable:invalid-metadata", + "limited:metadata-limit", + }, + ) + + def test_local_video_readers_match_recorded_tags_and_server(self): + files = {name: VIDEO_FIXTURES / name for name in RECORDED_TAGS} + observed = _run_oracle( + self, [{"name": name, "path": str(path)} for name, path in files.items()] + ) + for name, path in files.items(): + with self.subTest(fixture=name): + self.assertLess(path.stat().st_size, 16 * 1024) + container = VIDEO_CONTAINERS[path.suffix] + self.assertEqual( + observed[name]["classified"], {"media": "video", "kind": container} + ) + read = observed[name]["result"]["read"] + self.assertEqual(read["status"], "complete", read) + self.assertEqual(read["container"], container) + self.assertEqual(read["tags"], _known_tags(RECORDED_TAGS[name])) + self.assertEqual(read["limited"], []) + local = _local_shape(observed[name]["result"]["selection"]) + self.assertEqual(local, _server_selection(RECORDED_TAGS[name])) + if HAVE_FFMPEG: + self.assertEqual(local, _read_shape(read_inspection_file(path))) + # The corpus covers per-key carriers in every container, an envelope and a + # Matroska COMMENT that the exact-case envelope rule leaves unread. + self.assertTrue( + _server_selection(RECORDED_TAGS["legacy-comment.mp4"])["carrierText"] + ) + self.assertEqual( + _server_selection(RECORDED_TAGS["legacy-comment.webm"])["records"], {} + ) + + @unittest.skipUnless(HAVE_FFMPEG, "ffmpeg/ffprobe not installed") + def test_large_per_key_videos_read_the_same_as_the_server(self): + base = self.directory / "base.mp4" + write_video(base) + source = base.read_bytes() + shapes = { + "available": { + "prompt": _text_of( + MiB, '{"1":{"class_type":"Note","inputs":{"text":"a"}},' + ), + "workflow": _text_of(5 * MiB // 2, '{"nodes":[],"version":0.4,'), + }, + "workflow_limited": { + "prompt": GRAPH, + "workflow": "[" + "0," * (WORKFLOW_METADATA_BYTES // 2) + "]", + }, + "roles_over_budget": {"prompt": "A" * (ROLE_METADATA_BYTES + 1)}, + } + files = {} + for name, tags in shapes.items(): + path = self.directory / f"{name}.mp4" + path.write_bytes(_with_udta(source, _mdta_udta(tags))) + files[name] = path + observed = _run_oracle( + self, [{"name": name, "path": str(path)} for name, path in files.items()] + ) + available = observed["available"]["result"] + self.assertEqual(available["read"]["status"], "complete") + self.assertEqual(available["read"]["limited"], []) + self.assertEqual( + available["selection"]["records"], shapes["available"], "1 MiB + 2.5 MiB" + ) + self.assertEqual( + _local_shape(available["selection"]), + _read_shape(read_inspection_file(files["available"])), + ) + limited = observed["workflow_limited"]["result"] + self.assertEqual(limited["read"]["limited"], ["workflow"]) + # The unread workflow keeps its place in the carrier choice as a marker. + self.assertEqual(limited["read"]["tags"]["workflow"], {"oversized": "workflow"}) + self.assertEqual( + _local_shape(limited["selection"]), + _read_shape(read_inspection_file(files["workflow_limited"])), + ) + self.assertEqual( + limited["selection"]["record_formats"], + {"prompt": "original-text", "workflow": "limited"}, + ) + over = observed["roles_over_budget"]["result"] + self.assertEqual( + over["read"], {"status": "limited", "reason": "metadata-limit"} + ) + self.assertIsNone(over["selection"]) + with self.assertRaises(MetadataReadError): + read_inspection_file(files["roles_over_budget"]) + + @unittest.skipUnless(HAVE_FFMPEG, "ffmpeg/ffprobe not installed") + def test_byte_level_container_shapes_read_the_same_as_the_server(self): + """Shapes where ffprobe is lenient: invalid UTF-8 is replaced (one U+FFFD + per failing decode step), names that do not decode are no carrier, stray + ``ilst`` indexes are skipped, the first ``keys`` box wins, and an + oversized per-key workflow still takes part in the carrier choice.""" + base = self.directory / "base.mp4" + write_video(base) + source = base.read_bytes() + graph, workflow = GRAPH.encode(), WORKFLOW.encode() + envelope = json.dumps({"workflow": {"nodes": [1]}}).encode() + oversized = b'{"nodes":[' + b"0," * (5 * MiB // 2) + b"0]}" + shapes = { + "latin1_byte_in_workflow": _mdta_items( + [b"prompt", b"workflow"], [(1, graph), (2, b'{"a":"caf\xe9"}')] + ), + "undecodable_key_name": _mdta_items( + [b"prom\xffpt", b"prompt"], [(1, b"junk"), (2, graph)] + ), + "stray_ilst_indexes": _mdta_items( + [b"prompt", b"workflow"], + [(1, graph), (5, b"junk"), (0, b"junk"), (2, workflow)], + ), + "second_keys_box": _mdta_items( + [b"prompt"], [(1, graph)], keys_boxes=[[b"workflow"]] + ), + "oversized_workflow_beside_envelope": _mdta_items( + [b"workflow"], [(1, oversized)], legacy=[("\u00a9cmt", envelope)] + ), + # ffmpeg holds one tag per ASCII-folded name (MP4 joins with ";" + # under the last spelling), its format_tags filter folds ASCII only, + # and data types 0 and 3 are Mac Roman text. + "case_variant_keys": _mdta_items( + [b"prompt", b"Prompt", b"workflow"], + [(1, graph), (2, b"x"), (3, workflow)], + ), + "legacy_comment_then_upper_key": _mdta_items( + [b"COMMENT"], + [(1, b"y")], + legacy=[("\u00a9cmt", envelope)], + ), + "kelvin_sign_workflow": _mdta_items( + [b"prompt", "wor\u212aflow".encode()], [(1, graph), (2, workflow)] + ), + "mac_roman_types": _mdta_items( + [b"prompt", b"workflow"], + [ + (1, b'{"1":{"class_type":"caf\x8e","inputs":{}}}', 0), + (2, b'{"n":"\x80\x81' + bytes(range(0x80, 0x100)) + b'|"}', 3), + ], + ), + # A repeat joined onto it makes the first probe overflow: the server's + # role-only re-probe reports no per-key workflow. + "first_probe_overflows": _mdta_items( + [b"workflow", b"prompt"], + [(1, oversized), (1, b"A" * (8 * MiB)), (2, graph)], + ), + } + files = {} + for name, udta in shapes.items(): + files[name] = self.directory / f"{name}.mp4" + files[name].write_bytes(_with_udta(source, udta)) + # One byte of core.mkv's PROMPT TagString patched to 0xFF. + mkv = bytearray((VIDEO_FIXTURES / "core.mkv").read_bytes()) + at = mkv.index(b'"KSampler"', mkv.index(b"PROMPT")) + 1 + mkv[at] = 0xFF + files["matroska_invalid_byte"] = self.directory / "invalid-byte.mkv" + files["matroska_invalid_byte"].write_bytes(bytes(mkv)) + # Its WORKFLOW TagName renamed to "Prompt": ffmpeg keeps the last of the + # two spellings, so the server reads one prompt, not a duplicate. + case_mkv = (VIDEO_FIXTURES / "core.mkv").read_bytes() + files["matroska_case_variants"] = self.directory / "case-variants.mkv" + files["matroska_case_variants"].write_bytes( + case_mkv.replace(b"WORKFLOW", b"Prompt\0\0") + ) + observed = _run_oracle( + self, [{"name": name, "path": str(path)} for name, path in files.items()] + ) + for name, path in files.items(): + with self.subTest(shape=name): + server = read_inspection_file(path) + self.assertEqual( + _local_shape(observed[name]["result"]["selection"]), + _read_shape(server), + ) + metadata = _metadata(server) + self.assertEqual( + _canonical_bytes(observed[name]["metadata"]), + _canonical_bytes(metadata), + ) + self.assertEqual( + metadata["status"], + ( + "absent" + if name == "legacy_comment_then_upper_key" + else "available" + ), + ) + # The re-probe has its own bound: an 8 MiB envelope beside the oversized + # workflow fails the server's read and stops the local one. + wide = self.directory / "reprobe_overflows.mp4" + wide.write_bytes( + _with_udta( + source, + _mdta_items( + [b"workflow"], + [(1, oversized)], + legacy=[("\u00a9cmt", b'{"n":"' + b"x" * (8 * MiB) + b'"}')], + ), + ) + ) + local = _run_oracle(self, [{"name": "wide", "path": str(wide)}])["wide"] + self.assertEqual( + local["result"]["read"], {"status": "limited", "reason": "metadata-limit"} + ) + with self.assertRaises(MetadataReadError): + read_inspection_file(wide) + # The server outcomes these shapes pin. + variants = read_inspection_file(files["case_variant_keys"]) + self.assertEqual(variants.metadata["prompt"], GRAPH + ";x") + upper = read_inspection_file(files["legacy_comment_then_upper_key"]) + self.assertEqual((upper.metadata, upper.video_carrier_text), ({}, None)) + self.assertNotIn( + "workflow", read_inspection_file(files["kelvin_sign_workflow"]).metadata + ) + mac = read_inspection_file(files["mac_roman_types"]) + self.assertIn("caf\u00e9", mac.metadata["prompt"]) + self.assertIn("\u00c4\u00c5", mac.metadata["workflow"]) + self.assertIn( + '"last_node_id"', + read_inspection_file(files["matroska_case_variants"]).metadata["prompt"], + ) + latin1 = read_inspection_file(files["latin1_byte_in_workflow"]) + self.assertEqual(latin1.metadata["workflow"], '{"a":"caf\ufffd"}') + self.assertEqual( + read_inspection_file(files["stray_ilst_indexes"]).metadata, + {"prompt": GRAPH, "workflow": WORKFLOW}, + ) + self.assertEqual( + read_inspection_file(files["second_keys_box"]).metadata, {"prompt": GRAPH} + ) + patched = read_inspection_file(files["matroska_invalid_byte"]) + self.assertIn('"\ufffdSampler"', patched.metadata["prompt"]) + over = read_inspection_file(files["oversized_workflow_beside_envelope"]) + self.assertEqual( + (over.record_formats, over.video_carrier_text), + ({"workflow": "limited"}, None), + ) + dropped = read_inspection_file(files["first_probe_overflows"]) + self.assertEqual( + (dropped.metadata, dropped.record_formats), + ({"prompt": GRAPH}, {"prompt": "original-text", "workflow": "limited"}), + ) + + +def _canonical(value): + """``value`` with every dict's keys sorted, so two orders compare as one.""" + if isinstance(value, dict): + return {key: _canonical(value[key]) for key in sorted(value)} + if isinstance(value, list): + return [_canonical(item) for item in value] + return value + + +def _canonical_bytes(value): + return _bytes(_canonical(value)) + + +def _server_metadata(path): + try: + return _metadata(read_inspection_file(path)) + except MetadataReadError: + return _metadata(None) + + +def _synthetic_read(case): + if case["records"] is None: + return None + formats = case.get("record_formats") + if formats is None: + formats = {key: "original-text" for key in case["records"]} + return InspectionRead( + case["records"], + formats, + case.get("carrier_text"), + None, + None, + None, + frozenset(case.get("limited", ())), + ) + + +def _case_supported(case): + """Whether this interpreter has the feature a case depends on.""" + if case.get("needs") == "int-max-str-digits": + limit = getattr(sys, "get_int_max_str_digits", None) + return limit is not None and limit() == 4300 + return True + + +def _node_ceiling_cases(): + """Record sets at and past the 20,000-node ceiling, too large to commit.""" + api = lambda count: "{%s}" % ",".join( # noqa: E731 + '"%d":{"class_type":"A","inputs":{}}' % index for index in range(count) + ) + ui = lambda count: "[%s]" % ",".join("{}" for _ in range(count)) # noqa: E731 + half = GRAPH_NODE_CEILING // 2 + return [ + {"name": "prompt_at_ceiling", "records": {"prompt": api(GRAPH_NODE_CEILING)}}, + { + "name": "prompt_over_ceiling", + "records": {"prompt": api(GRAPH_NODE_CEILING + 1)}, + }, + { + "name": "workflow_at_ceiling", + "records": {"workflow": '{"nodes":%s}' % ui(GRAPH_NODE_CEILING)}, + }, + { + "name": "workflow_over_ceiling", + "records": {"workflow": '{"nodes":%s}' % ui(GRAPH_NODE_CEILING + 1)}, + }, + { + "name": "workflow_subgraphs_over_ceiling", + "records": { + "workflow": '{"nodes":%s,"definitions":{"subgraphs":[{"nodes":%s},' + '5,{"nodes":3}]}}' % (ui(half + 1), ui(half)) + }, + }, + { + "name": "workflow_subgraphs_at_ceiling", + "records": { + "workflow": '{"nodes":%s,"definitions":{"subgraphs":[{"nodes":%s}]}}' + % (ui(half), ui(half)) + }, + }, + { + "name": "carrier_prompt_over_ceiling", + "records": {"prompt": "x"}, + "record_formats": {"prompt": "reserialized-json"}, + "carrier_text": '{"prompt":%s}' % api(GRAPH_NODE_CEILING + 1), + }, + ] + + +class LocalProjectionParityTests(unittest.TestCase): + """The local projection answers byte for byte as ``_metadata`` does.""" + + def setUp(self): + temporary = tempfile.TemporaryDirectory() + self.addCleanup(temporary.cleanup) + self.addCleanup(assert_closed, self, temporary.name) + self.directory = Path(temporary.name) + + def _files(self): + files = {} + for name in parity.COMMITTED: + files[name] = parity.fixture_path(name) + for name in parity.TEST_TIME: + path = parity.fixture_path(name, self.directory) + parity.write_png(path, parity.build(name)) + files[name] = path + for name, write in EXTRA_SHAPES.items(): + path = self.directory / f"{name}.png" + write(path) + files[name] = path + for header in ("tiny.jpg", "tiny.webp"): + files[header] = HEADER_FIXTURES / header + for name in RECORDED_TAGS: + files[name] = VIDEO_FIXTURES / name + return files + + def test_local_metadata_matches_route_for_parity_corpus(self): + files = self._files() + cases = [ + case + for case in PROJECTION_CASES + _node_ceiling_cases() + if _case_supported(case) + ] + self.assertGreater(len(cases), len(PROJECTION_CASES) - 3) + entries = [{"name": name, "path": str(path)} for name, path in files.items()] + entries += [ + { + "name": "case:" + case["name"], + "records": case["records"], + "record_formats": case.get("record_formats"), + "carrier_text": case.get("carrier_text"), + "limited": case.get("limited", []), + } + for case in cases + ] + observed = _run_oracle(self, entries) + outcomes = set() + for name, path in files.items(): + with self.subTest(fixture=name): + self.assertIsNotNone(observed[name]["classified"], name) + if path.suffix in VIDEO_CONTAINERS and not HAVE_FFMPEG: + # Without ffprobe the server cannot read the file; the tag-map + # vectors below cover the carrier projection instead. + continue + server = _server_metadata(path) + local = observed[name]["metadata"] + self.assertEqual(_canonical_bytes(local), _canonical_bytes(server)) + outcomes.add(server["status"]) + for field in ("prompt_graph", "workflow_graph"): + outcomes.add(field + ":" + server[field]["status"]) + for case in cases: + with self.subTest(case=case["name"]): + server = _metadata(_synthetic_read(case)) + local = observed["case:" + case["name"]]["metadata"] + self.assertEqual(_canonical_bytes(local), _canonical_bytes(server)) + expect = case.get("expect") or {} + for field, status in expect.items(): + self.assertEqual(server[field]["status"], status, field) + outcomes.add(server["status"]) + for field in ("prompt_graph", "workflow_graph"): + outcomes.add(field + ":" + server[field]["status"]) + # Every projection outcome must occur, or agreement proves little. + self.assertEqual( + outcomes, + { + "available", + "absent", + "unreadable", + "prompt_graph:parsed", + "prompt_graph:limited", + "prompt_graph:malformed", + "prompt_graph:absent", + "workflow_graph:parsed", + "workflow_graph:limited", + "workflow_graph:malformed", + "workflow_graph:absent", + }, + ) + + def test_recorded_video_tag_maps_project_as_the_server(self): + """The carrier projection on the recorded maps, with or without ffprobe.""" + vectors = [(name, tags, []) for name, tags in RECORDED_TAGS.items()] + vectors += [ + (name, tags, limited) + for name, tags, limited in TAG_MAP_VECTORS + if name + in { + "per_key_wins_over_comment", + "object_prompt_reserialized", + "python_number_spellings", + "integer_like_keys_keep_order", + "invalid_envelope_workflow_keeps_roles", + "workflow_only_invalid_envelope", + "reader_limited_workflow_with_per_key", + "envelope_workflow_over_budget_limited", + } + ] + observed = _run_oracle( + self, + [ + {"name": name, "tags": tags, "limited": limited} + for name, tags, limited in vectors + ], + ) + seen = set() + for name, tags, limited in vectors: + with self.subTest(vector=name): + with ( + patch( + "utils.prompt_inspector.ffprobe_path", + return_value="/tool/ffprobe", + ), + patch( + "utils.prompt_inspector._probe_video_tags", + side_effect=_fake_probe( + tags, drop_workflow="workflow" in limited + ), + ), + ): + try: + server = _metadata(read_inspection_file(Path("vector.mp4"))) + except MetadataReadError: + server = _metadata(None) + local = observed[name]["metadata"] + self.assertEqual(_canonical_bytes(local), _canonical_bytes(server)) + seen.update(server["record_formats"].values()) + seen.add(server["workflow_graph"]["status"]) + self.assertTrue( + {"reserialized-json", "original-text", "invalid", "limited"} <= seen + ) + self.assertTrue({"parsed", "malformed", "limited"} <= seen) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/test_utils_video_metadata.py b/tests/test_utils_video_metadata.py index 1cacefa..f891878 100644 --- a/tests/test_utils_video_metadata.py +++ b/tests/test_utils_video_metadata.py @@ -390,5 +390,44 @@ class TestReadVideoMetadataWithRealFfprobe(unittest.TestCase): self.assertEqual(read_video_metadata(path), {}) +@unittest.skipUnless(HAVE_FFMPEG, "ffmpeg/ffprobe not installed") +class VideoFixtureFfprobeTests(unittest.TestCase): + """Characterization: the committed local-inspect video fixtures still carry the + tag maps recorded in ``video-tags-v1.json``, which the browser reader parity + tests use in place of ffprobe. A different ffprobe that reports other names or + values for the same bytes shows up here, not as a silent parity gap.""" + + FIXTURES = os.path.join( + os.path.dirname(os.path.abspath(__file__)), "fixtures", "local-inspect" + ) + + def test_committed_fixtures_produce_their_recorded_tags(self): + with open( + os.path.join(self.FIXTURES, "video-tags-v1.json"), encoding="utf-8" + ) as stream: + recorded = json.load(stream)["fixtures"] + self.assertEqual(len(recorded), 8) + for name, tags in recorded.items(): + with self.subTest(fixture=name): + probe = subprocess.run( + [ + shutil.which("ffprobe"), + "-v", + "quiet", + "-show_entries", + "format_tags", + "-of", + "json", + os.path.join(self.FIXTURES, name), + ], + capture_output=True, + text=True, + encoding="utf-8", + check=True, + timeout=60, + ) + self.assertEqual(json.loads(probe.stdout)["format"]["tags"], tags) + + if __name__ == "__main__": unittest.main() diff --git a/tools/vendor/frontend-coverage-baseline.json b/tools/vendor/frontend-coverage-baseline.json index b4bfa2f..df2ed5d 100644 --- a/tools/vendor/frontend-coverage-baseline.json +++ b/tools/vendor/frontend-coverage-baseline.json @@ -73,7 +73,7 @@ }, "png-metadata.js": { "lines": 98, - "branches": 90, + "branches": 96, "functions": 98, "statements": 98 }, @@ -293,6 +293,24 @@ "functions": 96, "statements": 96 }, + "workspace/local-inspection.mjs": { + "lines": 98, + "branches": 89, + "functions": 98, + "statements": 98 + }, + "workspace/local-projection.mjs": { + "lines": 97, + "branches": 96, + "functions": 98, + "statements": 97 + }, + "workspace/local-video.mjs": { + "lines": 98, + "branches": 94, + "functions": 98, + "statements": 98 + }, "workspace/media-viewer.mjs": { "lines": 98, "branches": 93, diff --git a/web/js/png-metadata.js b/web/js/png-metadata.js index c813676..fef8c45 100644 --- a/web/js/png-metadata.js +++ b/web/js/png-metadata.js @@ -2,9 +2,14 @@ * PNG text-chunk reader: the one place that walks a PNG's chunks to pull out * the tEXt/iTXt entries ComfyUI writes (`prompt`, `workflow`, ...). * - * Only uncompressed text is returned. zTXt chunks and iTXt chunks with the - * compression flag set are skipped rather than decoded as garbage; ComfyUI - * never writes those. Nothing throws: unreadable input yields {}. + * `parsePngTextChunks` (the legacy pages): only uncompressed text is returned. + * zTXt chunks and iTXt chunks with the compression flag set are skipped rather + * than decoded as garbage; ComfyUI never writes those. Nothing throws: + * unreadable input yields {}. + * + * `readPngMetadataFile`, `classifyPrefix` and `readImageHeader` (the Inspect + * page, #249): bounded readers over a File that port the server's inspection + * reader so a local file shows exactly what Images shows for the same bytes. * * Loadable in the browser (window.PngMetadata) and in Node for tests. */ @@ -94,7 +99,605 @@ return result; } - const api = { parsePngTextChunks, PNG_SIGNATURE }; + // ------------------------------------------------------------------------ + // Local File readers for the Inspect page (#249): a port of the server's + // bounded PNG reader (utils/prompt_inspector.py: _png_metadata, _PngRecords, + // _png_text, _inflate, _batch_normalized), a media classifier and a JPEG/WebP + // header reader. Nothing here uploads, stores or logs the file. The reader + // slices only chunk headers, keyword prefixes and the retained records, so a + // multi-gigabyte PNG costs at most PNG_READ_CEILING bytes of reads. + // tests/test_local_inspect_parity.py proves it answers as the server does. + // ------------------------------------------------------------------------ + + // Role records (prompt, parameters, the pair execution record) share one + // budget, encoded and decoded: past it the read fails. A workflow has its own + // budget; past it only the workflow is "limited". Duplicate chunks draw from + // separate budgets per class, so repeats cost bounded reads, not file size. + const ROLE_METADATA_BYTES = 2 * 1024 * 1024; + const WORKFLOW_METADATA_BYTES = 4 * 1024 * 1024; + const DUPLICATE_WORKFLOW_BYTES = 2 * WORKFLOW_METADATA_BYTES; + const DUPLICATE_ROLE_BYTES = 2 * ROLE_METADATA_BYTES; + const MAX_KEYWORD_BYTES = 80; // a 1-79 byte keyword plus its NUL separator + const MAX_PNG_CHUNKS = 10000; + const MAX_CHUNK_LENGTH = 0x7fffffff; + const IHDR_LENGTH = 13; + // Every byte the reader can slice from a PNG of any size: the signature, each + // chunk's header, CRC and keyword prefix, the retained-data budgets and IHDR. + const PNG_READ_CEILING = + PNG_SIGNATURE.length + + MAX_PNG_CHUNKS * (CHUNK_HEADER + CHUNK_CRC + MAX_KEYWORD_BYTES) + + ROLE_METADATA_BYTES + + WORKFLOW_METADATA_BYTES + + DUPLICATE_WORKFLOW_BYTES + + DUPLICATE_ROLE_BYTES + + IHDR_LENGTH; // 19,794,389 + const HEADER_READ_BYTES = 64 * 1024; + const INFLATE_WRITE_BYTES = 256; + const YIELD_INTERVAL_MS = 8; + const LATIN1_SLICE = 8192; + const ROLE_KEYS = new Set(["prompt", "parameters", "comfyassets_prompt_pairs"]); + const METADATA_KEYS = new Set([...ROLE_KEYS, "workflow"]); + const TEXT_CHUNKS = new Set(["tEXt", "zTXt", "iTXt"]); + // The server's _FOOTER (with its MULTILINE anchors spelled out: a line start is + // the text start or after "\n"; a line end is before "\n" or the text end) and + // _BATCH_INDEX, with Python's Unicode \d. + const FOOTER = /(?:^|(?<=\n))Steps:\s*\p{Nd}+\s*,[^\n]*(?=\n|$)/gu; + const BATCH_INDEX = /(?:^|(?<=, ))Batch index: \p{Nd}+(?=, |$)/gu; + const ASF_GUID = [0x30, 0x26, 0xb2, 0x75, 0x8e, 0x66, 0xcf, 0x11, 0xa6, 0xd9, 0x00, 0xaa, 0x00, 0x62, 0xce, 0x6c]; + const JPEG_SOF = new Set([0xc0, 0xc1, 0xc2, 0xc3, 0xc5, 0xc6, 0xc7, 0xc9, 0xca, 0xcb, 0xcd, 0xce, 0xcf]); + const JPEG_SOS = 0xda; + + const CRC_TABLE = (() => { + const table = new Uint32Array(256); + for (let n = 0; n < 256; n++) { + let c = n; + for (let k = 0; k < 8; k++) c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1; + table[n] = c; + } + return table; + })(); + + function crc32(...parts) { + let c = 0xffffffff; + for (const bytes of parts) for (let i = 0; i < bytes.length; i++) c = CRC_TABLE[(c ^ bytes[i]) & 0xff] ^ (c >>> 8); + return (c ^ 0xffffffff) >>> 0; + } + + function readUint32(bytes, offset) { + return bytes[offset] * 0x1000000 + ((bytes[offset + 1] << 16) | (bytes[offset + 2] << 8) | bytes[offset + 3]); + } + + function startsWith(bytes, prefix, at = 0) { + if (bytes.length < at + prefix.length) return false; + for (let i = 0; i < prefix.length; i++) if (bytes[at + i] !== prefix[i]) return false; + return true; + } + + function sameBytes(a, b) { + if (a.length !== b.length) return false; + for (let i = 0; i < a.length; i++) if (a[i] !== b[i]) return false; + return true; + } + + function concatBytes(parts, total) { + const out = new Uint8Array(total); + let offset = 0; + for (const part of parts) { + out.set(part, offset); + offset += part.length; + } + return out; + } + + /** ISO-8859-1 in slices: TextDecoder's "latin1" label is windows-1252, not this. */ + function latin1Text(bytes) { + let out = ""; + for (let i = 0; i < bytes.length; i += LATIN1_SLICE) out += String.fromCharCode.apply(null, bytes.subarray(i, i + LATIN1_SLICE)); + return out; + } + + /** UTF-8 length of the Latin-1 decoding: each byte >= 0x80 becomes two bytes. */ + function latin1Utf8Length(bytes) { + let high = 0; + for (let i = 0; i < bytes.length; i++) if (bytes[i] >= 0x80) high++; + return bytes.length + high; + } + + function isAscii(bytes) { + for (let i = 0; i < bytes.length; i++) if (bytes[i] >= 0x80) return false; + return true; + } + + function utf8Length(text) { + return new TextEncoder().encode(text).length; + } + + class PngReadError extends Error { + constructor(reason, message) { + super(message); + this.name = "PngReadError"; + this.reason = reason; + } + } + + const invalid = (message) => new PngReadError("invalid-png", message); + const overLimit = () => new PngReadError("metadata-limit", "Metadata exceeds inspection limits"); + const readFailure = (message) => new PngReadError("read-failure", message); + + function abortError() { + return new DOMException("The file read was aborted", "AbortError"); + } + + function throwIfAborted(signal) { + if (signal && signal.aborted) throw abortError(); + } + + function quiet(promise) { + Promise.resolve(promise).catch(() => {}); + } + + /** `promise`, or an AbortError as soon as `signal` aborts (the promise is left handled). */ + function untilAborted(promise, signal) { + if (!signal) return promise; + if (signal.aborted) { + quiet(promise); + return Promise.reject(abortError()); + } + return new Promise((resolve, reject) => { + const onAbort = () => { + quiet(promise); + reject(abortError()); + }; + signal.addEventListener("abort", onAbort, { once: true }); + promise.then( + (value) => { + signal.removeEventListener("abort", onAbort); + resolve(value); + }, + (error) => { + signal.removeEventListener("abort", onAbort); + reject(error); + }, + ); + }); + } + + /** Strict UTF-8 (invalid sequences and encoded surrogates fail), BOM kept, as Python's decode. */ + function utf8Strict(bytes) { + try { + return new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }).decode(bytes); + } catch (error) { + throw invalid("Media metadata could not be read"); + } + } + + function defaultYield() { + return new Promise((resolve) => setTimeout(resolve, 0)); + } + + /** Hands the event loop a turn about every YIELD_INTERVAL_MS of work. */ + function createPacer(yieldControl) { + let last = performance.now(); + return { + async maybeYield() { + if (performance.now() - last < YIELD_INTERVAL_MS) return; + await yieldControl(); + last = performance.now(); + }, + }; + } + + /** + * Logical byte ranges of `file`, served from the retained `prefix` where it + * covers them. Every sliced byte is charged against PNG_READ_CEILING, and every + * range is checked against `file.size` before it is sliced. + */ + function createSliceReader(file, prefix, signal) { + const retained = prefix || new Uint8Array(0); + let charged = 0; + return { + async read(start, end) { + if (!Number.isSafeInteger(end) || end > file.size) throw invalid("Incomplete PNG metadata"); + if (end <= retained.length) return retained.subarray(start, end); + const head = start < retained.length ? retained.subarray(start) : null; + const sliceStart = head ? retained.length : start; + charged += end - sliceStart; + if (charged > PNG_READ_CEILING) throw readFailure("PNG read exceeds the inspection ceiling"); + const buffer = await file.slice(sliceStart, end).arrayBuffer(); + throwIfAborted(signal); + const bytes = new Uint8Array(buffer); + if (bytes.length !== end - sliceStart) throw readFailure("The file changed while it was being read"); + return head ? concatBytes([head, bytes], end - start) : bytes; + }, + }; + } + + /** + * Inflates zlib data through `decompress` (a DecompressionStream constructor) + * with one <=256-byte write in flight, keeping at most `limit` output bytes. + * Returns null when the output is over `limit`, as the server's _inflate does. + */ + function createInflater(decompress, signal, pacer) { + return async function inflate(compressed, limit) { + if (typeof decompress !== "function") { + throw new PngReadError("decompression-unavailable", "This browser cannot inflate compressed PNG text"); + } + const stream = new decompress("deflate"); + const writer = stream.writable.getWriter(); + const reader = stream.readable.getReader(); + const parts = []; + let total = 0; + let over = false; + const stop = () => { + quiet(writer.abort()); + quiet(reader.cancel()); + }; + if (signal) signal.addEventListener("abort", stop, { once: true }); + try { + const drain = (async () => { + for (;;) { + const { value, done } = await untilAborted(reader.read(), signal); + if (done) return; + total += value.byteLength; + if (total > limit) { + over = true; + stop(); + return; + } + parts.push(value); + } + })(); + const feed = (async () => { + for (let offset = 0; offset < compressed.length; offset += INFLATE_WRITE_BYTES) { + await pacer.maybeYield(); + await untilAborted(writer.ready, signal); + await untilAborted(writer.write(compressed.subarray(offset, offset + INFLATE_WRITE_BYTES)), signal); + } + await untilAborted(writer.close(), signal); + })(); + const outcomes = await Promise.allSettled([drain, feed]); + throwIfAborted(signal); + if (over) return null; + if (outcomes.some((outcome) => outcome.status === "rejected")) { + throw invalid("Compressed metadata is invalid or exceeds limits"); + } + return concatBytes(parts, total); + } finally { + if (signal) signal.removeEventListener("abort", stop); + } + }; + } + + /** The keyword before the first NUL of a <=80-byte prefix: 1-79 Latin-1 bytes. */ + function keywordOf(prefix) { + const nul = indexOfNul(prefix, 0, prefix.length); + if (nul < 1 || nul > MAX_KEYWORD_BYTES - 1) throw invalid("Malformed PNG text metadata"); + return latin1(prefix, 0, nul); + } + + /** Splits on NUL at most `limit` times (Python's bytes.split(b"\\0", limit)). */ + function splitNul(bytes, limit) { + const fields = []; + let start = 0; + while (fields.length < limit) { + const nul = indexOfNul(bytes, start, bytes.length); + if (nul === -1) break; + fields.push(bytes.subarray(start, nul)); + start = nul + 1; + } + fields.push(bytes.subarray(start)); + return fields; + } + + /** + * Decodes one retained text chunk (the server's _png_text): {text, size} with + * size the UTF-8 length of text, or null when it inflates past `limit`. + */ + async function pngText(kind, data, limit, inflate) { + let body = data.subarray(indexOfNul(data, 0, data.length) + 1); + let unicode = false; + if (kind === "zTXt") { + if (body.length === 0 || body[0] !== 0) throw invalid("Unsupported PNG text compression"); + body = await inflate(body.subarray(1), limit); + } else if (kind === "iTXt") { + if (body.length < 2 || body[0] > 1) throw invalid("Malformed international PNG metadata"); + const flag = body[0]; + const method = body[1]; + const fields = splitNul(body.subarray(2), 2); + if (fields.length !== 3 || (flag && method !== 0)) throw invalid("Malformed international PNG metadata"); + if (!isAscii(fields[0])) throw invalid("Media metadata could not be read"); + utf8Strict(fields[1]); + body = flag ? await inflate(fields[2], limit) : fields[2]; + unicode = true; + } + if (body === null) return null; + if (indexOfNul(body, 0, body.length) !== -1) throw invalid("Malformed PNG text metadata"); + if (unicode) return { text: utf8Strict(body), size: body.length }; + return { text: latin1Text(body), size: latin1Utf8Length(body) }; + } + + /** The server's _batch_normalized: the footer's single batch index blanked, else null. */ + function batchNormalized(text) { + const footers = [...text.matchAll(FOOTER)]; + if (footers.length !== 1) return null; + const [footer] = footers; + let count = 0; + const line = footer[0].replace(BATCH_INDEX, () => { + count++; + return "Batch index: #"; + }); + if (count !== 1) return null; + return text.slice(0, footer.index) + line + text.slice(footer.index + footer[0].length); + } + + /** Retained records with per-class budgets and byte-identical duplicates (the server's _PngRecords). */ + class PngRecords { + constructor() { + this.found = {}; + this.limited = new Set(); + this.retained = new Map(); // key -> {kind, data} of the kept chunk, compared byte for byte + this.roleEncoded = 0; + this.roleDecoded = 0; + this.duplicateWorkflow = 0; + this.duplicateRole = 0; + this.duplicateDecoded = 0; + } + + /** Decide from the keyword and length alone whether to seek past a chunk. */ + skip(key, length) { + if (!METADATA_KEYS.has(key) || this.limited.has(key)) return true; // never retained, or a duplicate of a limited workflow + const kept = this.retained.get(key); + if (kept) { + if (kept.data.length !== length && key !== "parameters") throw invalid("Duplicate prompt metadata"); + if (key === "workflow") { + if (this.duplicateWorkflow + length > DUPLICATE_WORKFLOW_BYTES) { + this.limited.add(key); + delete this.found[key]; + return true; + } + this.duplicateWorkflow += length; + } else { + if (this.duplicateRole + length > DUPLICATE_ROLE_BYTES) throw overLimit(); + this.duplicateRole += length; + } + return false; // same length: read and compare bytes + } + if (key === "workflow") { + if (length > WORKFLOW_METADATA_BYTES) { + this.limited.add(key); + return true; + } + return false; + } + this.roleEncoded += length; + if (this.roleEncoded > ROLE_METADATA_BYTES) throw overLimit(); + return false; + } + + async add(kind, data, inflate) { + const key = keywordOf(data.subarray(0, MAX_KEYWORD_BYTES)); + const kept = this.retained.get(key); + if (kept) { + if (kept.kind !== kind || !sameBytes(kept.data, data)) { + if (key !== "parameters") throw invalid("Duplicate prompt metadata"); + await this.batchCopy(kind, data, inflate); + // The kept copy is the last one; later repeats compare with it. + this.retained.set(key, { kind, data }); + } + return; + } + this.retained.set(key, { kind, data }); + const workflow = key === "workflow"; + const limit = workflow ? WORKFLOW_METADATA_BYTES : ROLE_METADATA_BYTES - this.roleDecoded; + const decoded = await pngText(kind, data, limit, inflate); + if (workflow) { + if (decoded === null || decoded.size > WORKFLOW_METADATA_BYTES) { + this.limited.add(key); + return; + } + } else { + this.roleDecoded += decoded === null ? limit + 1 : decoded.size; + if (this.roleDecoded > ROLE_METADATA_BYTES) throw overLimit(); + } + this.found[key] = decoded.text; + } + + /** + * Keep a later `parameters` copy that differs only in its batch index. Each + * copy must carry exactly one `Batch index: N` settings field; any other + * difference is a conflict. Decoding every copy draws from the duplicate + * role budget, so small compressed copies stay bounded. + */ + async batchCopy(kind, data, inflate) { + const decoded = await pngText(kind, data, ROLE_METADATA_BYTES, inflate); + const size = decoded === null ? ROLE_METADATA_BYTES + 1 : decoded.size; + this.duplicateDecoded += size; + if (decoded === null || this.duplicateDecoded > DUPLICATE_ROLE_BYTES) throw overLimit(); + const kept = this.found.parameters; + const normalized = batchNormalized(kept); + if (normalized === null || normalized !== batchNormalized(decoded.text)) throw invalid("Duplicate prompt metadata"); + this.roleDecoded += Math.max(0, size - utf8Length(kept)); + if (this.roleDecoded > ROLE_METADATA_BYTES) throw overLimit(); + this.found.parameters = decoded.text; + } + } + + /** + * Walks the chunk layout (the server's _png_metadata): CRC-checks IHDR, IEND + * and retained text chunks, reads keyword prefixes, and seeks past everything + * else by length, so pixel data and unknown text are never read. + */ + async function readPngChunks(reader, fileSize, { signal, inflate, pacer }) { + const records = new PngRecords(); + let sawHeader = false; + let sawPixels = false; + let width = 0; + let height = 0; + let offset = PNG_SIGNATURE.length; + for (let index = 0; index < MAX_PNG_CHUNKS; index++) { + await pacer.maybeYield(); + throwIfAborted(signal); + const header = await reader.read(offset, offset + CHUNK_HEADER); + const length = readUint32(header, 0); + const kindBytes = header.subarray(4, 8); + const kind = latin1(header, 4, 8); + const dataStart = offset + CHUNK_HEADER; + const next = dataStart + length + CHUNK_CRC; + if (length > MAX_CHUNK_LENGTH || next > fileSize) throw invalid("Incomplete PNG chunk"); + if (index === 0 && (kind !== "IHDR" || length !== IHDR_LENGTH)) throw invalid("Missing PNG header"); + let data; + let checksum; + if (TEXT_CHUNKS.has(kind)) { + const prefix = await reader.read(dataStart, dataStart + Math.min(length, MAX_KEYWORD_BYTES)); + if (records.skip(keywordOf(prefix), length)) { + offset = next; + continue; + } + const rest = await reader.read(dataStart + prefix.length, next); + data = concatBytes([prefix, rest.subarray(0, rest.length - CHUNK_CRC)], length); + checksum = rest.subarray(rest.length - CHUNK_CRC); + } else if (kind === "IHDR" || kind === "IEND") { + const bytes = await reader.read(dataStart, next); + data = bytes.subarray(0, length); + checksum = bytes.subarray(length); + } else { + sawPixels = sawPixels || kind === "IDAT"; + offset = next; + continue; + } + if (crc32(kindBytes, data) !== readUint32(checksum, 0)) throw invalid("PNG metadata checksum mismatch"); + if (kind === "IHDR") { + const w = readUint32(data, 0); + const h = readUint32(data, 4); + if (sawHeader || !w || !h) throw invalid("Malformed PNG header"); + sawHeader = true; + width = w; + height = h; + } else if (kind === "IEND") { + if (length || !sawPixels) throw invalid("Incomplete PNG image"); + return { records: { ...records.found }, limited: [...records.limited].sort(), width, height }; + } else { + await records.add(kind, data, inflate); + } + offset = next; + } + throw new PngReadError("metadata-limit", "PNG chunk count exceeds inspection limits"); + } + + /** + * Reads the retained text records and the IHDR size of a PNG File without + * reading its pixels or sending it anywhere. + * + * @param {Blob} file - the chosen File (only `size` and `slice` are used) + * @param {Object} [options] + * @param {AbortSignal} [options.signal] - rejects the read with an AbortError + * @param {Uint8Array} [options.prefix] - the file's first bytes, already read for classification + * @param {Function} [options.yieldControl] - awaited about every 8 ms of work + * @param {Function} [options.decompress] - DecompressionStream constructor; pass undefined where none exists + * @returns {Promise<{status:'complete', records:Object, limited:string[], width:number, height:number} + * | {status:'unreadable', reason:'invalid-png'|'read-failure'|'metadata-limit'|'decompression-unavailable'}>} + */ + async function readPngMetadataFile(file, options = {}) { + const { signal, prefix, yieldControl = defaultYield } = options; + const decompress = Object.hasOwn(options, "decompress") ? options.decompress : globalThis.DecompressionStream; + try { + throwIfAborted(signal); + if (!file || !Number.isSafeInteger(file.size) || typeof file.slice !== "function") throw readFailure("Not a readable file"); + const reader = createSliceReader(file, toBytes(prefix), signal); + const pacer = createPacer(yieldControl); + const signature = await reader.read(0, PNG_SIGNATURE.length); + if (!hasSignature(signature)) throw invalid("Not a PNG file"); + const inflate = createInflater(decompress, signal, pacer); + const result = await readPngChunks(reader, file.size, { signal, inflate, pacer }); + return { status: "complete", ...result }; + } catch (error) { + if (error && error.name === "AbortError") throw error; + if (error instanceof PngReadError) return { status: "unreadable", reason: error.reason }; + return { status: "unreadable", reason: "read-failure" }; + } + } + + /** + * Media kind from a file's first bytes (<= 32), or null for anything else. + * @returns {{media:'image'|'video', kind:'png'|'jpeg'|'webp'|'iso-bmff'|'matroska'|'avi'|'asf'}|null} + */ + function classifyPrefix(input) { + const bytes = toBytes(input); + if (!bytes) return null; + if (hasSignature(bytes)) return { media: "image", kind: "png" }; + if (startsWith(bytes, [0xff, 0xd8, 0xff])) return { media: "image", kind: "jpeg" }; + if (bytes.length >= 12 && latin1(bytes, 0, 4) === "RIFF") { + const form = latin1(bytes, 8, 12); + if (form === "WEBP") return { media: "image", kind: "webp" }; + if (form === "AVI ") return { media: "video", kind: "avi" }; + return null; + } + if (bytes.length >= 8 && latin1(bytes, 4, 8) === "ftyp") return { media: "video", kind: "iso-bmff" }; + if (startsWith(bytes, [0x1a, 0x45, 0xdf, 0xa3])) return { media: "video", kind: "matroska" }; + if (startsWith(bytes, ASF_GUID)) return { media: "video", kind: "asf" }; + return null; + } + + /** [width, height] from the first SOFn segment, walking markers as Pillow does. */ + function jpegSize(bytes) { + if (!startsWith(bytes, [0xff, 0xd8])) return null; + let offset = 2; + while (offset + 1 < bytes.length) { + if (bytes[offset] !== 0xff) { + offset++; // junk between segments + continue; + } + const marker = bytes[offset + 1]; + if (marker === 0xff) { + offset++; // padding + continue; + } + if (marker === 0x00 || marker === 0x01 || (marker >= 0xd0 && marker <= 0xd9)) { + offset += 2; // escaped 0xFF, TEM, RSTn, SOI, EOI: no length + continue; + } + if (marker === JPEG_SOS || offset + 4 > bytes.length) return null; + if (JPEG_SOF.has(marker)) { + if (offset + 9 > bytes.length) return null; + return [(bytes[offset + 7] << 8) | bytes[offset + 8], (bytes[offset + 5] << 8) | bytes[offset + 6]]; + } + offset += 2 + ((bytes[offset + 2] << 8) | bytes[offset + 3]); + } + return null; + } + + /** [width, height] from the first WebP chunk: VP8 frame, VP8L header or VP8X canvas. */ + function webpSize(bytes) { + if (bytes.length < 30 || latin1(bytes, 0, 4) !== "RIFF" || latin1(bytes, 8, 12) !== "WEBP") return null; + const kind = latin1(bytes, 12, 16); + if (kind === "VP8 ") { + if (!startsWith(bytes, [0x9d, 0x01, 0x2a], 23)) return null; + return [(bytes[26] | (bytes[27] << 8)) & 0x3fff, (bytes[28] | (bytes[29] << 8)) & 0x3fff]; + } + if (kind === "VP8L") { + if (bytes[20] !== 0x2f) return null; + const packed = bytes[21] | (bytes[22] << 8) | (bytes[23] << 16) | (bytes[24] << 24); + return [(packed & 0x3fff) + 1, ((packed >>> 14) & 0x3fff) + 1]; + } + if (kind === "VP8X") { + return [1 + (bytes[24] | (bytes[25] << 8) | (bytes[26] << 16)), 1 + (bytes[27] | (bytes[28] << 8) | (bytes[29] << 16))]; + } + return null; + } + + /** + * Format and dimensions of a JPEG or WebP File from at most 64 KiB of its head. + * @returns {Promise<{format:'JPEG'|'WEBP'|null, width:number|null, height:number|null}>} + */ + async function readImageHeader(file, kind) { + const format = kind === "jpeg" ? "JPEG" : kind === "webp" ? "WEBP" : null; + if (!format) return { format: null, width: null, height: null }; + const bytes = new Uint8Array(await file.slice(0, Math.min(file.size, HEADER_READ_BYTES)).arrayBuffer()); + const size = (format === "JPEG" ? jpegSize : webpSize)(bytes); + return { format, width: size ? size[0] : null, height: size ? size[1] : null }; + } + + const api = { parsePngTextChunks, PNG_SIGNATURE, classifyPrefix, readPngMetadataFile, readImageHeader }; if (typeof module !== "undefined" && module.exports) module.exports = api; else root.PngMetadata = api; })(typeof window !== "undefined" ? window : globalThis); diff --git a/web/js/workspace/inspection-state.mjs b/web/js/workspace/inspection-state.mjs index 1a2fdbe..bfb61ed 100644 --- a/web/js/workspace/inspection-state.mjs +++ b/web/js/workspace/inspection-state.mjs @@ -87,8 +87,10 @@ function promptsFor(value) { return value; } -function fileFor(value) { - if (!object(value) || value.observation !== 'current-file' || typeof value.source_label !== 'string' || +function fileFor(value, identity) { + // A local record observes the selected file itself; every other adapter observes a current server file. + const observation = identity?.kind === 'local-file' ? 'selected-local-file' : 'current-file'; + if (!object(value) || value.observation !== observation || typeof value.source_label !== 'string' || !['image', 'video'].includes(value.media_type) || typeof value.extension !== 'string' || (value.format !== null && typeof value.format !== 'string')) throw new TypeError('Invalid inspection file'); const file = { observation: value.observation, source_label: value.source_label, media_type: value.media_type, extension: value.extension, format: value.format }; @@ -281,7 +283,7 @@ export function projectInspectionPayload(result, identity, download, extractFact }; if (legacy) return record; const resolution = resolutionFor(result.resolution); - record.file = fileFor(result.file); + record.file = fileFor(result.file, identity); record.metadata = metadataFor(result.metadata); record.library = libraryFor(result.library); if (typeof extractFacts !== 'function') throw new TypeError('Inspection facts parser unavailable'); diff --git a/web/js/workspace/local-inspection.mjs b/web/js/workspace/local-inspection.mjs new file mode 100644 index 0000000..8b5c034 --- /dev/null +++ b/web/js/workspace/local-inspection.mjs @@ -0,0 +1,175 @@ +// One chosen local File, read entirely in this browser, as the inspection wire +// the workspace already renders. The readers are ports of the server's +// (png-metadata.js readPngMetadataFile/readImageHeader, local-video.mjs); the +// projection is local-projection.mjs. Nothing here builds a URL or sends bytes: +// the only URL a handle carries is the session-owned blob: URL it was created with. +import { readVideoTags, selectVideoCarrier } from './local-video.mjs'; +import { projectMetadata, fitInspectionPayload, MAX_RESPONSE_BYTES } from './local-projection.mjs'; +import { projectInspectionPayload } from './inspection-state.mjs'; + +export const LOCAL_OBSERVATION = 'selected-local-file'; +export const LOCAL_SOURCE_LABEL = 'Selected local file'; +// Interim roles until PR L3 ports the inspection resolver: every role is open, and +// the legacy fill in projectInspectionPayload shows what the legacy pages show. +const LOCAL_RESOLUTION_REASON = 'Resolved in this browser from the workflow.'; +const LOCAL_LIBRARY = Object.freeze({ status: 'unavailable', scope: 'saved-library', tags: [], category_scope: null, tags_scope: null }); +const LOCAL_SCOPE = 'local'; +const PREFIX_BYTES = 32; +const MAX_FILENAME_CHARS = 255; +const MEDIA_TYPES = new Set(['image', 'video']); + +const object = value => value !== null && typeof value === 'object' && !Array.isArray(value); +const pngReader = () => globalThis.PngMetadata || globalThis.window?.PngMetadata; + +/** A display name with no path separators or control characters; never empty. */ +export function sanitizeLocalFilename(name) { + const text = typeof name === 'string' ? name : ''; + const cleaned = text.replace(/[\x00-\x1f\x7f]/g, '').replace(/[\\/]/g, '_').trim(); + return [...cleaned].slice(0, MAX_FILENAME_CHARS).join('') || 'file'; +} + +/** `pathlib.Path(name).suffix.lower()`. */ +function localExtension(filename) { + const dot = filename.lastIndexOf('.'); + return dot > 0 && dot < filename.length - 1 ? filename.slice(dot).toLowerCase() : ''; +} + +function classificationOf(value) { + if (!object(value) || !MEDIA_TYPES.has(value.media) || typeof value.kind !== 'string' || !value.kind) throw new TypeError('Unsupported local file'); + return Object.freeze({ media: value.media, kind: value.kind }); +} + +/** + * Run the readers the server would run for these bytes: the bounded PNG reader, + * the JPEG/WebP header reader, or the container tag readers plus the carrier rule. + * Returns `{classification, result}` in the oracle's shape; `result` is null for + * bytes no reader claims. + */ +export async function readLocalFile(file, { classification, signal, pngMetadata = pngReader() } = {}) { + if (!(file instanceof Blob)) throw new TypeError('A local file is required'); + const reader = () => { + if (!pngMetadata || typeof pngMetadata.classifyPrefix !== 'function') throw new TypeError('The local PNG reader is unavailable'); + return pngMetadata; + }; + const prefix = new Uint8Array(await file.slice(0, Math.min(PREFIX_BYTES, file.size)).arrayBuffer()); + const classified = classification === undefined ? reader().classifyPrefix(prefix) : classification; + if (classified === null) return { classification: null, result: null }; + const { media, kind } = classificationOf(classified); + if (kind === 'png') return { classification: classified, result: await reader().readPngMetadataFile(file, { signal, prefix }) }; + if (media === 'image') return { classification: classified, result: await reader().readImageHeader(file, kind) }; + const read = await readVideoTags(file, { signal, prefix, kind }); + return { classification: classified, result: { read, selection: read.status === 'complete' ? selectVideoCarrier(read.tags, { limited: read.limited }) : null } }; +} + +/** + * The server's InspectionRead for a reader result: `read` is null when the server + * would have raised MetadataReadError (unreadable), else + * `{records, record_formats, carrierText, limited}`; `format`, `width` and + * `height` follow `read_inspection_file` (a video has none). + */ +export function toInspectionRead(classification, result) { + const { media, kind } = classificationOf(classification); + const none = { read: null, format: null, width: null, height: null }; + if (kind === 'png') { + if (!object(result) || result.status !== 'complete') return none; + const limited = [...(result.limited ?? [])]; + const record_formats = Object.fromEntries(Object.keys(result.records).map(key => [key, 'original-text'])); + for (const key of limited) record_formats[key] = 'limited'; + return { read: { records: { ...result.records }, record_formats, carrierText: null, limited }, format: 'PNG', width: result.width ?? null, height: result.height ?? null }; + } + if (media === 'image') { + if (!object(result)) return none; + return { read: { records: {}, record_formats: {}, carrierText: null, limited: [] }, format: result.format ?? null, width: result.width ?? null, height: result.height ?? null }; + } + const selection = result?.selection; + if (result?.read?.status !== 'complete' || !object(selection) || selection.status !== 'complete') return none; + return { read: { records: { ...selection.records }, record_formats: { ...selection.record_formats }, carrierText: selection.carrierText ?? null, limited: [...(selection.limited ?? [])] }, format: null, width: null, height: null }; +} + +const dimension = value => Number.isSafeInteger(value) && value > 0 ? value : null; + +/** The `inspection_version: 1` wire for one local read, before the response budget. */ +export function buildLocalPayload({ read, file, classification }) { + const classified = classificationOf(classification); + const inspection = toInspectionRead(classified, read); + const filename = sanitizeLocalFilename(file?.name); + const size = Number.isSafeInteger(file?.size) && file.size >= 0 ? file.size : null; + const modified = typeof file?.lastModified === 'number' && Number.isFinite(file.lastModified) && file.lastModified >= 0 ? file.lastModified / 1000 : null; + return { + success: true, inspection_version: 1, + file: { + observation: LOCAL_OBSERVATION, source_label: LOCAL_SOURCE_LABEL, media_type: classified.media, extension: localExtension(filename), + format: inspection.format, size, width: dimension(inspection.width), height: dimension(inspection.height), modified_time: modified, + }, + prompts: { positive: { status: 'unresolved' }, negative: { status: 'unresolved' } }, + resolution: { state: 'unresolved', sampler_id: null, reason: LOCAL_RESOLUTION_REASON, reason_code: null, node: null }, + metadata: projectMetadata(inspection.read), + library: { ...LOCAL_LIBRARY, tags: [] }, + }; +} + +/** The server's 8 MiB response budget, applied to the local wire with the same omission order. */ +export function fitLocalPayload(wire, limit = MAX_RESPONSE_BYTES, options = {}) { + return fitInspectionPayload(wire, limit, options); +} + +// Handles this module created, by a private token: the controller and the viewer +// session spread-copy descriptors, so a copy of an owned handle is the same +// selection, while a hand-made descriptor or a tampered URL is foreign. +const TOKEN = Symbol('local-file'); +const OWNED = new WeakMap(); +let sequence = 0; + +/** + * One selection of a local file. `url` is the session's own `blob:` URL (or null + * when none could be created); every selection gets a new key, so choosing the + * same bytes twice never reuses a stale read. + */ +export function createLocalFileHandle({ file, url = null, classification, filename = file?.name }) { + if (!(file instanceof Blob)) throw new TypeError('A local file is required'); + if (url !== null && (typeof url !== 'string' || !url.startsWith('blob:'))) throw new TypeError('Local media must be a blob: URL'); + const classified = classificationOf(classification); + const token = Object.freeze({}); + const secret = Object.freeze({ key: `local:${++sequence}`, filename: sanitizeLocalFilename(filename), url, file, classification: classified }); + OWNED.set(token, secret); + return Object.freeze({ kind: 'local-file', key: secret.key, filename: secret.filename, is_video: classified.media === 'video', url, file, classification: classified, [TOKEN]: token }); +} + +/** The owned selection behind a handle (or a copy of one); throws for anything else. */ +function owned(handle) { + const secret = object(handle) ? OWNED.get(handle[TOKEN]) : undefined; + if (!secret || secret.key !== handle.key || secret.url !== handle.url || secret.file !== handle.file || secret.filename !== handle.filename) throw new TypeError('Unknown local file'); + return secret; +} + +function validate(handle) { + owned(handle); +} + +function originalDownload(handle) { + const secret = owned(handle); + if (secret.url === null) throw new TypeError('Original media is unavailable'); + return { url: secret.url, filename: secret.filename }; +} + +export const localFileAdapter = Object.freeze({ + validate, + key(handle) { return owned(handle).key; }, + scope(handle) { owned(handle); return LOCAL_SCOPE; }, + // An inert sentinel: the controller's same-key check works and no URL is ever built. + inspectionPath(handle) { return `local-file:${owned(handle).key}`; }, + async read(handle, { signal } = {}) { + const secret = owned(handle); + const { result } = await readLocalFile(secret.file, { classification: secret.classification, signal }); + return fitLocalPayload(buildLocalPayload({ read: result, file: { name: secret.filename, size: secret.file.size, lastModified: secret.file.lastModified }, classification: secret.classification }), MAX_RESPONSE_BYTES, { signal }); + }, + toRecord(result, handle, extractFacts) { + const secret = owned(handle); + if (!object(result) || result.success !== true || result.inspection_version !== 1) throw new TypeError('Invalid local inspection'); + return projectInspectionPayload(result, { kind: 'local-file', key: secret.key, path: secret.filename }, + secret.url === null ? null : originalDownload(handle), extractFacts); + }, + originalDownload, + // Session-owned, never resolved through the API client. + mediaUrl(handle) { return owned(handle).url; }, +}); diff --git a/web/js/workspace/local-projection.mjs b/web/js/workspace/local-projection.mjs new file mode 100644 index 0000000..a8c7ffe --- /dev/null +++ b/web/js/workspace/local-projection.mjs @@ -0,0 +1,404 @@ +// Browser port of utils/inspection_response.py: the strict parse (Decimal +// spelling, no duplicate keys, NaN/Infinity as null, integers past 2^53-1 as +// text), the prompt/workflow/carrier projection with its depth and node +// ceilings, `_metadata`, and the response-size omission steps of +// `_encode_inspection`. A local file must project exactly as the server does for +// the same bytes; tests/test_local_inspect_parity.py holds both to one answer. +// Pure: no DOM, no I/O. + +export const MAX_RESPONSE_BYTES = 8 * 1024 * 1024; +export const MAX_PROJECTION_DEPTH = 64; +// Backstop, not an expected outcome: executable API nodes for a prompt; UI nodes +// including subgraph definitions for a workflow. +export const GRAPH_NODE_CEILING = 20000; +export const RECORD_KEYS = Object.freeze(['prompt', 'workflow', 'parameters', 'comfyassets_prompt_pairs']); +const FORMATS = new Set(['original-text', 'reserialized-json', 'invalid', 'limited']); +const PROMPT_NODE_KEYS = new Set(['class_type', 'inputs']); +const MAX_SAFE_DIGITS = '9007199254740991'; +// Python's int() refuses more digits than sys.get_int_max_str_digits() (4300). +export const INT_MAX_STR_DIGITS = 4300; +// decimal's exact constructor (libmpdec, 64-bit) refuses values outside its +// maximal context: adjusted exponent above MAX_EMAX or exponent below Etiny. +const DECIMAL_MAX_EMAX = 999999999999999999n; +const DECIMAL_ETINY = -1999999999999999997n; +const NUMBER = /-?(?:0|[1-9][0-9]*)(\.[0-9]+)?([eE][-+]?[0-9]+)?/y; +const STRING_STOP = /["\\\u0000-\u001f]/g; +const HEX4 = /^[0-9a-fA-F]{4}$/; +const ESCAPES = { '"': '"', '\\': '\\', '/': '/', b: '\b', f: '\f', n: '\n', r: '\r', t: '\t' }; +// Python's json: `null`/`true`/`false`, then the constants parse_constant receives. +const LITERALS = [['null', null], ['true', true], ['false', false], ['NaN', null], ['Infinity', null], ['-Infinity', null]]; + +/** A present graph cannot be projected within inspection limits (`_ProjectionLimit`). */ +export class ProjectionLimit extends Error { + constructor() { + super('Projection exceeds inspection limits'); + this.name = 'ProjectionLimit'; + } +} + +/** The server's ValueError/TypeError family: the text or tree is malformed. */ +export class ProjectionError extends Error { + constructor(message) { + super(message); + this.name = 'ProjectionError'; + } +} + +const isArray = Array.isArray; +const isObject = value => value !== null && typeof value === 'object' && !isArray(value); +const isContainer = value => value !== null && typeof value === 'object'; + +/** Python's `str(Decimal(literal))` from the literal's digits and exponent, never through a float. */ +export function spellDecimal(literal) { + const negative = literal[0] === '-'; + const body = negative ? literal.slice(1) : literal; + const exponentAt = body.search(/[eE]/); + const mantissa = exponentAt < 0 ? body : body.slice(0, exponentAt); + const dot = mantissa.indexOf('.'); + const integer = dot < 0 ? mantissa : mantissa.slice(0, dot), fraction = dot < 0 ? '' : mantissa.slice(dot + 1); + const exponent = (exponentAt < 0 ? 0n : BigInt(body.slice(exponentAt + 1))) - BigInt(fraction.length); + const digits = (integer + fraction).replace(/^0+/, '') || '0'; + const adjusted = exponent + BigInt(digits.length) - 1n; + if (adjusted > DECIMAL_MAX_EMAX || exponent < DECIMAL_ETINY) throw new ProjectionError('JSON decimal exceeds supported range'); + const leftDigits = exponent + BigInt(digits.length); + const dotPlace = exponent <= 0n && leftDigits > -6n ? leftDigits : 1n; + let intPart, fracPart; + if (dotPlace <= 0n) { + intPart = '0'; fracPart = '.' + '0'.repeat(Number(-dotPlace)) + digits; + } else if (dotPlace >= BigInt(digits.length)) { + intPart = digits + '0'.repeat(Number(dotPlace - BigInt(digits.length))); fracPart = ''; + } else { + intPart = digits.slice(0, Number(dotPlace)); fracPart = '.' + digits.slice(Number(dotPlace)); + } + const shift = leftDigits - dotPlace; + const exp = shift === 0n ? '' : `E${shift > 0n ? '+' : '-'}${shift < 0n ? -shift : shift}`; + return (negative ? '-' : '') + intPart + fracPart + exp; +} + +function numberValue(match) { + const [literal, fraction, exponent] = match; + if (fraction || exponent) return spellDecimal(literal); + const digits = literal[0] === '-' ? literal.slice(1) : literal; + if (digits.length > INT_MAX_STR_DIGITS) throw new ProjectionError('Exceeds the limit for integer string conversion'); + if (digits.length > MAX_SAFE_DIGITS.length || (digits.length === MAX_SAFE_DIGITS.length && digits > MAX_SAFE_DIGITS)) return literal; + return Number(literal) || 0; +} + +/** + * `json.loads(text, parse_float=Decimal, object_pairs_hook=_pairs, parse_constant=_nonfinite)` + * with its results already normalized: objects have a null prototype, fractions and + * exponents are Decimal text, integers past 2^53-1 are their literal text, and + * NaN/Infinity are null. Iterative, so nesting depth never overflows the stack. + */ +export function parseStrictJson(text) { + if (typeof text !== 'string') throw new ProjectionError('JSON text must be a string'); + let pos = 0; + const fail = message => new ProjectionError(`${message} at ${pos}`); + const skip = () => { + while (pos < text.length) { + const c = text.charCodeAt(pos); + if (c !== 0x20 && c !== 0x09 && c !== 0x0a && c !== 0x0d) break; + pos++; + } + }; + function string() { + pos++; + let out = ''; + for (;;) { + STRING_STOP.lastIndex = pos; + const stop = STRING_STOP.exec(text); + if (!stop) throw fail('Unterminated string'); + out += text.slice(pos, stop.index); + pos = stop.index + 1; + if (stop[0] === '"') return out; + if (stop[0] !== '\\') throw fail('Invalid control character'); + const escape = text[pos]; + if (escape === 'u') { + const hex = text.slice(pos + 1, pos + 5); + if (!HEX4.test(hex)) throw fail('Invalid \\uXXXX escape'); + pos += 5; + const code = parseInt(hex, 16); + if (code >= 0xd800 && code <= 0xdbff && text[pos] === '\\' && text[pos + 1] === 'u') { + const low = text.slice(pos + 2, pos + 6); + const lowCode = HEX4.test(low) ? parseInt(low, 16) : -1; + if (lowCode >= 0xdc00 && lowCode <= 0xdfff) { + out += String.fromCharCode(code, lowCode); + pos += 6; + continue; + } + } + out += String.fromCharCode(code); + } else if (Object.hasOwn(ESCAPES, escape)) { + out += ESCAPES[escape]; + pos++; + } else throw fail('Invalid \\escape'); + } + } + function key() { + if (text[pos] !== '"') throw fail('Expecting property name enclosed in double quotes'); + const name = string(); + skip(); + if (text[pos] !== ':') throw fail("Expecting ':' delimiter"); + pos++; + skip(); + return name; + } + function scalar() { + if (text[pos] === '"') return string(); + for (const [literal, value] of LITERALS) { + if (text.startsWith(literal, pos)) { + pos += literal.length; + return value; + } + } + NUMBER.lastIndex = pos; + const match = NUMBER.exec(text); + if (!match) throw fail('Expecting value'); + pos += match[0].length; + return numberValue(match); + } + skip(); + const stack = []; + let result; + for (;;) { + let value; + if (text[pos] === '{') { + pos++; skip(); + const frame = { value: Object.create(null), object: true, key: null, duplicate: false }; + if (text[pos] === '}') pos++; + else { frame.key = key(); stack.push(frame); continue; } + value = frame.value; + } else if (text[pos] === '[') { + pos++; skip(); + const frame = { value: [], object: false }; + if (text[pos] === ']') pos++; + else { stack.push(frame); continue; } + value = frame.value; + } else value = scalar(); + // Attach the finished value, closing containers whose last member it was. + for (;;) { + if (!stack.length) { result = value; break; } + const frame = stack[stack.length - 1]; + if (frame.object) { + if (Object.hasOwn(frame.value, frame.key)) frame.duplicate = true; + frame.value[frame.key] = value; + } else frame.value.push(value); + skip(); + if (text[pos] === ',') { + pos++; skip(); + if (frame.object) frame.key = key(); + break; + } + if (text[pos] !== (frame.object ? '}' : ']')) throw fail("Expecting ',' delimiter"); + pos++; + stack.pop(); + // `_pairs` runs when the object closes, after everything inside it parsed. + if (frame.object && frame.duplicate) throw new ProjectionError('Duplicate JSON key'); + value = frame.value; + } + if (!stack.length) break; + } + skip(); + if (pos !== text.length) throw fail('Extra data'); + return result; +} + +/** `_normalize`: walk once; unretained values are only checked against the depth bound. */ +function normalize(value, depth, retain) { + if (!isContainer(value)) return retain ? value : null; + if (depth > MAX_PROJECTION_DEPTH) throw new ProjectionLimit(); + if (isArray(value)) { + const result = retain ? [] : null; + for (const child of value) { + const normalized = normalize(child, depth + 1, retain); + if (retain) result.push(normalized); + } + return result; + } + const result = retain ? Object.create(null) : null; + for (const key of Object.keys(value)) { + const normalized = normalize(value[key], depth + 1, retain); + if (retain) result[key] = normalized; + } + return result; +} + +/** `_project_prompt`: each API node keeps exactly `class_type` and `inputs`; other keys are validated and dropped. */ +function projectPrompt(tree, depth) { + if (!isObject(tree)) throw new ProjectionError('Prompt graph must be an object of nodes'); + if (depth + 1 > MAX_PROJECTION_DEPTH) throw new ProjectionLimit(); + const result = Object.create(null); + for (const id of Object.keys(tree)) { + const node = tree[id]; + if (!isObject(node)) { + result[id] = normalize(node, depth + 1, true); + continue; + } + const projected = Object.create(null); + for (const key of Object.keys(node)) { + const keep = PROMPT_NODE_KEYS.has(key); + const normalized = normalize(node[key], depth + 2, keep); + if (keep) projected[key] = normalized; + } + result[id] = projected; + } + return result; +} + +function project(tree, kind, depth) { + if (kind === 'prompt') return projectPrompt(tree, depth); + if (kind === 'workflow') return normalize(tree, depth, true); + if (!isObject(tree)) throw new ProjectionError('Malformed carrier'); + const result = Object.create(null); + for (const key of Object.keys(tree)) { + const member = tree[key]; + if (key !== 'prompt' && key !== 'workflow') normalize(member, depth + 1, false); + else if (typeof member === 'string') result[key] = member; // projected by its own kind when read + else if (key === 'prompt' && !isObject(member)) result[key] = normalize(member, depth + 1, true); + else result[key] = project(member, key, depth + 1); + } + return result; +} + +/** `_strict_projection`: parse exactly and project by record kind ('prompt'|'workflow'|'carrier'). */ +export function strictProjection(text, kind) { + return project(parseStrictJson(text), kind, 1); +} + +function nodeCount(value, workflow) { + if (!workflow) return Object.keys(value).length; + const nodes = value.nodes; + let count = isArray(nodes) ? nodes.length : 0; + const definitions = value.definitions; + const subgraphs = isObject(definitions) ? definitions.subgraphs : null; + for (const subgraph of isArray(subgraphs) ? subgraphs : []) { + const inner = isObject(subgraph) ? subgraph.nodes : null; + count += isArray(inner) ? inner.length : 0; + } + return count; +} + +function graph(value, workflow) { + if (!isObject(value)) return { status: 'malformed', value: null }; + if (nodeCount(value, workflow) > GRAPH_NODE_CEILING) return { status: 'limited', value: null }; + return { status: 'parsed', value }; +} + +function outcome(error) { + if (error instanceof ProjectionLimit) return { status: 'limited', value: null }; + if (error instanceof ProjectionError) return { status: 'malformed', value: null }; + throw error; +} + +/** + * One record's graph as the inspector projects it: `{status, value}` with `status` + * `parsed` (the projected graph the facts parser receives), `limited` (past the + * depth or node bound) or `malformed`. + */ +export function projectGraph(text, kind) { + try { + return graph(strictProjection(text, kind), kind === 'workflow'); + } catch (error) { + return outcome(error); + } +} + +/** + * `_metadata` for one local read: `{records, record_formats, carrierText, limited}` + * (the server's InspectionRead) or null when the file's metadata is unreadable. + */ +export function projectMetadata(inspection) { + const formats = {}, raw = {}; + if (inspection) { + for (const [key, value] of Object.entries(inspection.record_formats ?? {})) { + if (RECORD_KEYS.includes(key) && FORMATS.has(value)) formats[key] = value; + } + // A record over its read budget has no text: like `invalid`, raw omits it, + // and its graph is `limited` rather than parsed from nothing. + for (const key of inspection.limited ?? []) if (RECORD_KEYS.includes(key)) formats[key] = 'limited'; + for (const [key, value] of Object.entries(inspection.records ?? {})) { + if (RECORD_KEYS.includes(key) && typeof value === 'string' && formats[key] !== 'limited') raw[key] = value; + } + } + const result = { + status: inspection === null || inspection === undefined ? 'unreadable' : Object.keys(raw).length || Object.keys(formats).length ? 'available' : 'absent', + omitted_reason: null, raw, record_formats: formats, parameters: Object.hasOwn(raw, 'parameters') ? raw.parameters : null, + }; + let carrier = null, failed = null; + if (inspection && inspection.carrierText !== null && inspection.carrierText !== undefined) { + try { carrier = strictProjection(inspection.carrierText, 'carrier'); } catch (error) { failed = outcome(error).status; } + } + for (const [key, field] of [['prompt', 'prompt_graph'], ['workflow', 'workflow_graph']]) { + let value = { status: 'absent', value: null }; + if (Object.hasOwn(raw, key) || Object.hasOwn(formats, key)) { + if (failed || formats[key] === 'limited') value.status = failed || 'limited'; + else if (formats[key] === 'invalid') value.status = 'malformed'; + else if (carrier === null && formats[key] === 'reserialized-json') value.status = 'limited'; + else if (carrier === null) value = projectGraph(raw[key], key); + else { + try { + let member = carrier[key]; + if (typeof member === 'string') member = strictProjection(member, key); + value = graph(member, key === 'workflow'); + } catch (error) { value.status = outcome(error).status; } + } + } + result[field] = value; + } + return result; +} + +const UTF8 = new TextEncoder(); +const MEASURE_CHUNK_BYTES = 256 * 1024; + +/** + * UTF-8 length of `JSON.stringify(value)`: the server's `_bytes` count. The text + * holds no lone surrogate (JSON.stringify escapes them), so the encoder's count is + * exact; it encodes into one bounded scratch buffer and never splits a pair. + */ +export function inspectionBytes(value) { + const text = JSON.stringify(value); + const scratch = new Uint8Array(Math.min(MEASURE_CHUNK_BYTES, text.length * 3 + 4)); + let count = 0; + for (let offset = 0; offset < text.length; ) { + const { read, written } = UTF8.encodeInto(offset ? text.slice(offset) : text, scratch); + offset += read; + count += written; + } + return count; +} + +const withMetadata = (payload, changes) => ({ ...payload, metadata: { ...payload.metadata, ...changes } }); + +function defaultYield() { + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +/** + * The omission steps of `_encode_inspection`: the workflow graph first, then the + * prompt graph, then all raw records atomically, each measured as encoded bytes. + * Each re-measure of a multi-megabyte payload follows a `yieldControl` turn, and + * `signal` aborts between steps. Resolves to a new payload; rejects when even the + * bare payload is over `limit`. + */ +export async function fitInspectionPayload(payload, limit = MAX_RESPONSE_BYTES, { yieldControl = defaultYield, signal } = {}) { + const measure = async (candidate) => { + await yieldControl(); + if (signal && signal.aborted) throw new DOMException('The inspection was aborted', 'AbortError'); + return inspectionBytes(candidate); + }; + let fitted = payload, body = inspectionBytes(payload); + for (const field of ['workflow_graph', 'prompt_graph']) { + if (body <= limit) break; + if (fitted.metadata[field].status !== 'absent') { + fitted = withMetadata(fitted, { [field]: { status: 'limited', value: null } }); + body = await measure(fitted); + } + } + if (body > limit) { + fitted = withMetadata(fitted, { raw: {}, parameters: null, status: 'limited', omitted_reason: 'response-size-limit' }); + body = await measure(fitted); + } + if (body > limit) throw new RangeError('Inspection response exceeds limits'); + return fitted; +} diff --git a/web/js/workspace/local-video.mjs b/web/js/workspace/local-video.mjs new file mode 100644 index 0000000..f414cd5 --- /dev/null +++ b/web/js/workspace/local-video.mjs @@ -0,0 +1,1052 @@ +// Local video tag readers for the Inspect page (#249): bounded ISO-BMFF (MP4/MOV/M4V) +// and Matroska (MKV/WebM) walkers that read only the container tags the server asks +// ffprobe for (utils/prompt_inspector.py _video_metadata), and a port of the carrier +// rule utils/video_metadata.py select_video_carrier plus the record budgets of +// read_inspection_file. Nothing here uploads, stores or logs the file. The walkers +// slice box and element headers, the tag names and the retained tag values only, so +// a multi-gigabyte video costs at most VIDEO_READ_CEILING bytes of reads and never a +// byte inside mdat or a Cluster. tests/test_local_inspect_parity.py proves the +// answers equal the server's on the committed fixtures and recorded tag maps. +// +// What ffprobe reports is the oracle: ISO-BMFF mdta keys are reported verbatim and +// the legacy ilst atoms ©cmt/desc as comment/description; Matroska TagNames are +// reported as written (ffmpeg upper-cases them at mux time). ffmpeg keeps one tag +// per ASCII-folded name (the last spelling) and ffprobe's tag filter folds ASCII +// only, so the reader does too. Envelope names are exact-case (comment, +// description) and the per-key names fold case in the selector, exactly as the +// server folds them. + +import { INT_MAX_STR_DIGITS } from "./local-projection.mjs"; + +// Role records (prompt, parameters, the pair execution record) share one budget: +// past it the read fails. A workflow has its own budget; past it only the workflow +// is "limited". Duplicate tags draw from separate budgets per class. The total of +// tag bytes read is the server's ffprobe stdout bound. +export const ROLE_METADATA_BYTES = 2 * 1024 * 1024; +export const WORKFLOW_METADATA_BYTES = 4 * 1024 * 1024; +export const DUPLICATE_WORKFLOW_BYTES = 2 * WORKFLOW_METADATA_BYTES; +export const DUPLICATE_ROLE_BYTES = 2 * ROLE_METADATA_BYTES; +export const VIDEO_TAG_READ_BYTES = 2 * (ROLE_METADATA_BYTES + WORKFLOW_METADATA_BYTES) + 64 * 1024; +// The server's role-only re-probe bound, taken when the first probe overflows. +export const VIDEO_REPROBE_BYTES = 2 * ROLE_METADATA_BYTES + 64 * 1024; +// A first per-key workflow over its budget is never sliced; this marker holds its +// place in the tag map, as the server's first probe still reports the tag. +export const OVERSIZED_WORKFLOW = Object.freeze({ oversized: "workflow" }); +export const MAX_VIDEO_ELEMENTS = 10000; +const MAX_HEADER_BYTES = 16; // a 64-bit ISO-BMFF box header; an EBML header is at most 12 +const MAX_NAME_BYTES = 64; // the longest carrier name is 24 bytes; longer names are never ours +// Every byte the readers can slice from a video of any size: each element's header +// and name plus the retained tag values. +export const VIDEO_READ_CEILING = MAX_VIDEO_ELEMENTS * (MAX_HEADER_BYTES + MAX_NAME_BYTES) + VIDEO_TAG_READ_BYTES; +const PREFIX_BYTES = 32; +const YIELD_INTERVAL_MS = 8; +const JSON_MAX_DEPTH = 1000; // Python's default recursion limit + +const ROLE_KEYS = ["prompt", "parameters", "comfyassets_prompt_pairs"]; +const METADATA_KEYS = ["prompt", "workflow", "parameters", "comfyassets_prompt_pairs"]; +const ENVELOPE_KEYS = ["comment", "description"]; +const PER_KEY_NAMES = new Set(METADATA_KEYS); +const ENVELOPE_NAMES = new Set(ENVELOPE_KEYS); +// The tags ffprobe is asked for (format_tags=...): its filter, like ffmpeg's tag +// dictionary, matches names ignoring ASCII case only. +const PROBED_NAMES = new Set([...METADATA_KEYS, ...ENVELOPE_KEYS]); +// ffmpeg's mac_to_unicode for bytes 0x80-0xFF (Mac OS Roman, checked through ffprobe). +const MAC_ROMAN = + "\u00c4\u00c5\u00c7\u00c9\u00d1\u00d6\u00dc\u00e1\u00e0\u00e2\u00e4\u00e3\u00e5\u00e7\u00e9\u00e8" + + "\u00ea\u00eb\u00ed\u00ec\u00ee\u00ef\u00f1\u00f3\u00f2\u00f4\u00f6\u00f5\u00fa\u00f9\u00fb\u00fc" + + "\u2020\u00b0\u00a2\u00a3\u00a7\u2022\u00b6\u00df\u00ae\u00a9\u2122\u00b4\u00a8\u2260\u00c6\u00d8" + + "\u221e\u00b1\u2264\u2265\u00a5\u00b5\u2202\u2211\u220f\u03c0\u222b\u00aa\u00ba\u03a9\u00e6\u00f8" + + "\u00bf\u00a1\u00ac\u221a\u0192\u2248\u2206\u00ab\u00bb\u2026\u00a0\u00c0\u00c3\u00d5\u0152\u0153" + + "\u2013\u2014\u201c\u201d\u2018\u2019\u00f7\u25ca\u00ff\u0178\u2044\u20ac\u2039\u203a\ufb01\ufb02" + + "\u2021\u00b7\u201a\u201e\u2030\u00c2\u00ca\u00c1\u00cb\u00c8\u00cd\u00ce\u00cf\u00cc\u00d3\u00d4" + + "\uf8ff\u00d2\u00da\u00db\u00d9\u0131\u02c6\u02dc\u00af\u02d8\u02d9\u02da\u00b8\u02dd\u02db\u02c7"; +const MAC_ROMAN_UTF8 = Array.from(MAC_ROMAN, (char) => new TextEncoder().encode(char)); +// ffprobe's names for the legacy ilst atoms that can carry an envelope. +const LEGACY_ATOMS = new Map([["©cmt", "comment"], ["desc", "description"], ["©des", "description"]]); +const ASF_GUID = [0x30, 0x26, 0xb2, 0x75, 0x8e, 0x66, 0xcf, 0x11, 0xa6, 0xd9, 0x00, 0xaa, 0x00, 0x62, 0xce, 0x6c]; + +const EBML = { + header: 0x1a45dfa3, + docType: 0x4282, + segment: 0x18538067, + seekHead: 0x114d9b74, + seek: 0x4dbb, + seekId: 0x53ab, + seekPosition: 0x53ac, + tags: 0x1254c367, + tag: 0x7373, + targets: 0x63c0, + trackUid: 0x63c5, + chapterUid: 0x63c4, + attachmentUid: 0x63c6, + simpleTag: 0x67c8, + tagName: 0x45a3, + tagString: 0x4487, + tagLanguage: 0x447a, + tagDefault: 0x4484, + cluster: 0x1f43b675, +}; +const TAGS_ID_BYTES = [0x12, 0x54, 0xc3, 0x67]; + +class VideoReadError extends Error { + constructor(status, reason, message) { + super(message); + this.name = "VideoReadError"; + this.status = status; + this.reason = reason; + } +} + +const invalid = (message) => new VideoReadError("unreadable", "invalid-container", message); +const limited = (reason, message) => new VideoReadError("limited", reason, message); +const readFailure = (message) => new VideoReadError("unreadable", "read-failure", message); + +function abortError() { + return new DOMException("The file read was aborted", "AbortError"); +} + +function throwIfAborted(signal) { + if (signal && signal.aborted) throw abortError(); +} + +function toBytes(input) { + if (input instanceof Uint8Array) return input; + if (input instanceof ArrayBuffer) return new Uint8Array(input); + if (Array.isArray(input)) return Uint8Array.from(input); + return null; +} + +function latin1(bytes, start, end) { + let out = ""; + for (let index = start; index < end; index += 1) out += String.fromCharCode(bytes[index]); + return out; +} + +function startsWith(bytes, prefix, at = 0) { + if (bytes.length < at + prefix.length) return false; + for (let index = 0; index < prefix.length; index += 1) if (bytes[at + index] !== prefix[index]) return false; + return true; +} + +function readUint32(bytes, offset) { + return ((bytes[offset] << 24) >>> 0) + (bytes[offset + 1] << 16) + (bytes[offset + 2] << 8) + bytes[offset + 3]; +} + +/** A big-endian unsigned integer of up to 8 bytes as a Number, or null past MAX_SAFE_INTEGER. */ +function readUintBytes(bytes, start, end) { + let value = 0n; + for (let index = start; index < end; index += 1) value = (value << 8n) | BigInt(bytes[index]); + return value <= BigInt(Number.MAX_SAFE_INTEGER) ? Number(value) : null; +} + +function concatBytes(parts, total) { + const out = new Uint8Array(total); + let offset = 0; + for (const part of parts) { + out.set(part, offset); + offset += part.length; + } + return out; +} + +/** ffmpeg stores tag text as a C string: the first NUL ends it. */ +function untilNul(bytes) { + const nul = bytes.indexOf(0); + return nul < 0 ? bytes : bytes.subarray(0, nul); +} + +const STRICT_UTF8 = new TextDecoder("utf-8", { fatal: true, ignoreBOM: true }); +const REPLACEMENT = "\ufffd"; +// av_utf8_decode's smallest code point per continuation count (overlong below it). +const OVERLONG_MINIMUM = [0, 0x80, 0x800, 0x10000]; + +/** + * One av_utf8_decode step (flags 0, as ffprobe calls it) from `start`: the next + * offset and whether the sequence is valid. A lead byte that is a continuation or + * 0xFE/0xFF, an overlong form, a surrogate, a code above U+10FFFF, U+FFFE/U+FFFF + * and every 5/6-byte form fail as the whole sequence; a missing or bad + * continuation byte fails the lead byte alone. + */ +function ffprobeStep(bytes, start) { + const lead = bytes[start]; + if ((lead & 0xc0) === 0x80 || lead >= 0xfe) return [start + 1, false]; + let tail = 0; + if (lead & 0x80) while (lead & (0x40 >> tail)) tail += 1; + let code = tail ? lead & (0x3f >> tail) : lead; + let next = start + 1; + for (let index = 0; index < tail; index += 1, next += 1) { + if (next >= bytes.length || (bytes[next] & 0xc0) !== 0x80) return [start + 1, false]; + code = code * 64 + (bytes[next] & 0x3f); + } + const valid = tail <= 3 && code >= OVERLONG_MINIMUM[tail] && code <= 0x10ffff && !(code >= 0xd800 && code <= 0xdfff) && code !== 0xfffe && code !== 0xffff; + return [next, valid]; +} + +/** + * Tag text as ffprobe prints it with its default string_validation=replace + * (fftools validate_string): each failing av_utf8_decode step becomes one U+FFFD, + * so Python's json.loads of ffprobe's output never meets invalid UTF-8. This + * differs from WHATWG decoding in how many replacements a bad sequence yields. + */ +function ffprobeText(bytes) { + let text = null; + try { + text = STRICT_UTF8.decode(bytes); + } catch (error) { + text = null; + } + if (text !== null && !/[\ufffe\uffff]/.test(text)) return text; + const parts = []; + let run = 0; + for (let offset = 0; offset < bytes.length; ) { + const [next, valid] = ffprobeStep(bytes, offset); + if (!valid) { + if (offset > run) parts.push(STRICT_UTF8.decode(bytes.subarray(run, offset))); + parts.push(REPLACEMENT); + run = next; + } + offset = next; + } + if (bytes.length > run) parts.push(STRICT_UTF8.decode(bytes.subarray(run))); + return parts.join(""); +} + +/** ASCII-only lower case, as ffmpeg's av_strcasecmp compares names. */ +function asciiLower(name) { + return name.replace(/[A-Z]+/g, (letters) => letters.toLowerCase()); +} + +/** + * ffmpeg's mov_read_mac_string: Mac Roman bytes as UTF-8 into a buffer of twice + * the input size, so a long non-ASCII value is cut, possibly inside a character + * (ffprobe then replaces the cut sequence). + */ +function macRomanBytes(bytes) { + const limit = bytes.length * 2; + const out = new Uint8Array(limit); + let length = 0; + for (const byte of bytes) { + if (length >= limit) break; + if (byte < 0x80) out[length++] = byte; + else for (const unit of MAC_ROMAN_UTF8[byte - 0x80]) if (length < limit) out[length++] = unit; + } + return out.subarray(0, length); +} + +/** UTF-8 length as Python's ``encode("utf-8", "surrogatepass")`` counts it. */ +function utf8Length(text) { + let total = 0; + for (let index = 0; index < text.length; index += 1) { + const code = text.charCodeAt(index); + if (code < 0x80) total += 1; + else if (code < 0x800) total += 2; + else if (code >= 0xd800 && code <= 0xdbff && index + 1 < text.length) { + const next = text.charCodeAt(index + 1); + if (next >= 0xdc00 && next <= 0xdfff) { + total += 4; + index += 1; + } else total += 3; + } else total += 3; + } + return total; +} + +function defaultYield() { + return new Promise((resolve) => setTimeout(resolve, 0)); +} + +/** Hands the event loop a turn about every YIELD_INTERVAL_MS of work. */ +function createPacer(yieldControl, signal) { + let last = performance.now(); + return { + async maybeYield() { + if (performance.now() - last < YIELD_INTERVAL_MS) return; + await yieldControl(); + throwIfAborted(signal); + last = performance.now(); + }, + }; +} + +/** + * Logical byte ranges of `file`, served from the retained `prefix` where it covers + * them (the twin of png-metadata.js's reader). Every sliced byte is charged against + * VIDEO_READ_CEILING, and every range is checked against `file.size` before slicing. + */ +function createSliceReader(file, prefix, signal) { + const retained = prefix || new Uint8Array(0); + let charged = 0; + return { + async read(start, end) { + throwIfAborted(signal); + if (!Number.isSafeInteger(end) || end > file.size || start > end) throw invalid("Incomplete video container"); + if (end <= retained.length) return retained.subarray(start, end); + const head = start < retained.length ? retained.subarray(start) : null; + const sliceStart = head ? retained.length : start; + charged += end - sliceStart; + if (charged > VIDEO_READ_CEILING) throw readFailure("Video read exceeds the inspection ceiling"); + const buffer = await file.slice(sliceStart, end).arrayBuffer(); + throwIfAborted(signal); + const bytes = new Uint8Array(buffer); + if (bytes.length !== end - sliceStart) throw readFailure("The file changed while it was being read"); + return head ? concatBytes([head, bytes], end - start) : bytes; + }, + }; +} + +/** Container kind from the first bytes, mirroring png-metadata.js classifyPrefix. */ +function detectKind(bytes) { + if (!bytes) return null; + if (bytes.length >= 12 && latin1(bytes, 0, 4) === "RIFF") return latin1(bytes, 8, 12) === "AVI " ? "avi" : null; + if (bytes.length >= 8 && latin1(bytes, 4, 8) === "ftyp") return "iso-bmff"; + if (startsWith(bytes, [0x1a, 0x45, 0xdf, 0xa3])) return "matroska"; + if (startsWith(bytes, ASF_GUID)) return "asf"; + return null; +} + +/** + * The shared walk state: the element count, the tag budgets and the collected tags. + * `value` is the one gate every tag value passes before it is sliced. + */ +function createWalk(reader, signal, pacer) { + const tags = {}; + const limitedRecords = []; + const seen = new Set(); + let count = 0; + let role = 0; + let workflow = 0; + let duplicateRole = 0; + let duplicateWorkflow = 0; + let total = 0; + let workflowRead = 0; // bytes of per-key workflow tags read, which the re-probe leaves out + let oversized = null; // {size}: the unread first per-key workflow + // ffmpeg keeps one tag per ASCII-folded name, under the spelling set last. + const spelling = new Map(); + const overLimit = () => limited("metadata-limit", "Video metadata exceeds inspection limits"); + const forgetOversized = () => { + oversized = null; + limitedRecords.splice(limitedRecords.indexOf("workflow"), 1); + }; + const current = (folded) => (spelling.has(folded) ? tags[spelling.get(folded)] : undefined); + const store = (name, folded, value) => { + if (spelling.has(folded)) delete tags[spelling.get(folded)]; + tags[name] = value; + spelling.set(folded, name); + }; + return { + tags, + limited: limitedRecords, + async header(offset, length) { + count += 1; + if (count > MAX_VIDEO_ELEMENTS) throw limited("chunk-limit", "Video container exceeds inspection limits"); + await pacer.maybeYield(); + return reader.read(offset, offset + length); + }, + read: (start, end) => reader.read(start, end), + /** Whether ffprobe reports `name` to the server (the probed names, ASCII case folded). */ + known(name) { + return PROBED_NAMES.has(asciiLower(name)); + }, + /** A string-less Matroska SimpleTag deletes the name, ignoring ASCII case. */ + remove(name) { + const folded = asciiLower(name); + if (current(folded) === OVERSIZED_WORKFLOW) forgetOversized(); + if (spelling.has(folded)) delete tags[spelling.get(folded)]; + spelling.delete(folded); + }, + /** + * Settles the oversized workflow as the server's probes would: when the + * first probe fits, the tag keeps its marker; when it overflows, the + * role-only re-probe reports no per-key workflow and has its own bound. + */ + finish() { + if (!oversized || total + oversized.size <= VIDEO_TAG_READ_BYTES) return; + // The re-probe leaves the per-key workflow out; it stays "limited". + if (spelling.has("workflow")) delete tags[spelling.get("workflow")]; + spelling.delete("workflow"); + if (total - workflowRead > VIDEO_REPROBE_BYTES) throw overLimit(); + }, + /** + * Reads and keeps the tag `name` (a known carrier, see `known`) whose value + * spans [start, end), within the budgets. `join` appends to an existing value + * of the same folded name with that separator (ISO-BMFF repeats, as ffprobe + * reports them; Matroska replaces); `text` stores an already decoded value + * and `mac` decodes Mac Roman. + */ + async value(name, start, end, { join = null, text = null, mac = false } = {}) { + const size = end - start; + const folded = asciiLower(name); + const previous = current(folded); + if (PER_KEY_NAMES.has(folded)) { + const first = !seen.has(folded); + seen.add(folded); + if (folded === "workflow") { + if (first && size > WORKFLOW_METADATA_BYTES) { + oversized = { size }; + store(name, folded, OVERSIZED_WORKFLOW); + limitedRecords.push("workflow"); + return; + } + if (previous === OVERSIZED_WORKFLOW) { + // An ISO-BMFF repeat joins onto the unread value, still over budget. + if (join !== null) { + oversized.size += size; + store(name, folded, OVERSIZED_WORKFLOW); + return; + } + // Matroska keeps a name's last value: the oversized one is gone. + forgetOversized(); + } + workflowRead += size; + if (first) workflow += size; + else duplicateWorkflow += size; + if (duplicateWorkflow > DUPLICATE_WORKFLOW_BYTES) throw overLimit(); + } else { + if (first) role += size; + else duplicateRole += size; + if (role > ROLE_METADATA_BYTES || duplicateRole > DUPLICATE_ROLE_BYTES) throw overLimit(); + } + } + total += size; + if (total > VIDEO_TAG_READ_BYTES) throw overLimit(); + let value = text; + if (value === null) { + const bytes = await reader.read(start, end); + value = ffprobeText(untilNul(mac ? macRomanBytes(bytes) : bytes)); + } + store(name, folded, join !== null && previous !== undefined ? `${previous}${join}${value}` : value); + }, + }; +} + +// ------------------------------------------------------------------ ISO-BMFF + +/** One box header at `offset` inside a parent ending at `parentEnd`, or null when fewer than 8 bytes remain. */ +async function readBox(walk, offset, parentEnd) { + if (parentEnd - offset < 8) return null; + const bytes = await walk.header(offset, Math.min(MAX_HEADER_BYTES, parentEnd - offset)); + let size = readUint32(bytes, 0); + let bodyStart = offset + 8; + if (size === 1) { + if (bytes.length < 16) throw invalid("Truncated 64-bit box size"); + const large = readUintBytes(bytes, 8, 16); + if (large === null || large < 16) throw invalid("Invalid 64-bit box size"); + size = large; + bodyStart = offset + 16; + } else if (size === 0) { + size = parentEnd - offset; + } else if (size < 8) { + throw invalid("Box size below its header"); + } + const end = offset + size; + if (end > parentEnd) throw invalid("Box extends past its parent"); + return { type: latin1(bytes, 4, 8), raw: bytes.subarray(4, 8), start: offset, bodyStart, end }; +} + +async function readIsoBmff(walk, fileSize) { + const state = { hdlrMdta: false, keys: null, format: null }; + let offset = 0; + let box; + while ((box = await readBox(walk, offset, fileSize))) { + if (box.type === "ftyp" && box.end - box.bodyStart >= 4) { + state.format = latin1(await walk.read(box.bodyStart, box.bodyStart + 4), 0, 4).replace(/[\s\0]+$/, ""); + } else if (box.type === "moov") { + await readMoov(walk, box, state); + return state.format; + } + offset = box.end; + } + throw invalid("No moov box"); +} + +async function readMoov(walk, moov, state) { + let offset = moov.bodyStart; + let box; + while ((box = await readBox(walk, offset, moov.end))) { + if (box.type === "meta") await readMeta(walk, box, state); + else if (box.type === "udta") { + let inner = box.bodyStart; + let child; + while ((child = await readBox(walk, inner, box.end))) { + if (child.type === "meta") await readMeta(walk, child, state); + inner = child.end; + } + } + offset = box.end; + } +} + +/** `meta` as ffmpeg reads it: scan 4-byte words for the first child's `hdlr` type. */ +async function readMeta(walk, meta, state) { + const scan = await walk.read(meta.bodyStart, Math.min(meta.end, meta.bodyStart + MAX_NAME_BYTES)); + let childrenStart = null; + for (let word = 4; word + 4 <= scan.length; word += 4) { + if (latin1(scan, word, word + 4) === "hdlr") { + childrenStart = meta.bodyStart + word - 4; + break; + } + } + if (childrenStart === null) return; + let offset = childrenStart; + let box; + while ((box = await readBox(walk, offset, meta.end))) { + if (box.type === "hdlr") { + if (box.end - box.bodyStart >= 12) { + const handler = await walk.read(box.bodyStart + 8, box.bodyStart + 12); + if (latin1(handler, 0, 4) === "mdta") state.hdlrMdta = true; + } + } else if (box.type === "keys") await readKeys(walk, box, state); + else if (box.type === "ilst") await readIlst(walk, box, state); + offset = box.end; + } +} + +/** + * The `keys` box: 1-based mdta names; other namespaces and overlong names address + * nothing. As ffmpeg, only the first keys box with a count field is read. + */ +async function readKeys(walk, box, state) { + if (state.keys || box.end - box.bodyStart < 8) return; + const head = await walk.read(box.bodyStart, box.bodyStart + 8); + const count = readUint32(head, 4); + const keys = [null]; + let offset = box.bodyStart + 8; + for (let index = 1; index <= count; index += 1) { + if (box.end - offset < 8) throw invalid("Keys box count exceeds its entries"); + const entry = await walk.header(offset, 8); + const keySize = readUint32(entry, 0); + if (keySize < 8 || offset + keySize > box.end) throw invalid("Keys entry exceeds its box"); + const nameLength = keySize - 8; + let name = null; + if (latin1(entry, 4, 8) === "mdta" && nameLength <= MAX_NAME_BYTES) { + name = ffprobeText(await walk.read(offset + 8, offset + keySize)); + } + keys.push(name); + offset += keySize; + } + state.keys = keys; +} + +/** ffprobe's text for an integer `data` atom (types 21 and 22) of 1, 2, 4 or 8 bytes, else null. */ +function integerText(bytes, signed) { + if (![1, 2, 4, 8].includes(bytes.length)) return null; + let value = 0n; + for (const byte of bytes) value = (value << 8n) | BigInt(byte); + if (signed) value = BigInt.asIntN(bytes.length * 8, value); + return value.toString(); +} + +/** + * ilst items: a legacy fourcc or a `keys` index, then the leading run of `data` + * children. As ffprobe reports them, repeated values of one name join with ";", + * types 1 and 4 are UTF-8 text, 0 and 3 Mac Roman text, 21 and 22 integers, other + * types are not tags. + */ +async function readIlst(walk, ilst, state) { + let offset = ilst.bodyStart; + let item; + while ((item = await readBox(walk, offset, ilst.end))) { + offset = item.end; + let name = LEGACY_ATOMS.get(item.type); + if (name === undefined && state.hdlrMdta && state.keys) { + const index = readUint32(item.raw, 0); + // ffmpeg warns that the index is out of range and skips the item. + if (index === 0 || index >= state.keys.length) continue; + name = state.keys[index]; + } + if (!name || !walk.known(name)) continue; + let inner = item.bodyStart; + while (item.end - inner >= 16) { + const data = await walk.header(inner, 16); + const dataSize = readUint32(data, 0); + if (latin1(data, 4, 8) !== "data" || dataSize < 16 || dataSize > item.end - inner) break; + const type = readUint32(data, 8); + const start = inner + 16; + const end = inner + dataSize; + inner = end; + // Types 0 and 3 are Mac Roman text to ffmpeg; 1 and 4 are UTF-8. + if (type === 0 || type === 3) await walk.value(name, start, end, { join: ";", mac: true }); + else if (type === 1 || type === 4) await walk.value(name, start, end, { join: ";" }); + else if ((type === 21 || type === 22) && end - start <= 8) { + const text = integerText(await walk.read(start, end), type === 21); + if (text !== null) await walk.value(name, start, end, { join: ";", text }); + } + } + } +} + +// ------------------------------------------------------------------ Matroska + +/** One EBML element header at `offset` inside a parent ending at `parentEnd`. */ +async function readElement(walk, offset, parentEnd) { + if (parentEnd - offset < 2) return null; + const bytes = await walk.header(offset, Math.min(12, parentEnd - offset)); + const idLength = Math.clz32(bytes[0]) - 23; + if (bytes[0] === 0 || idLength > 4 || bytes.length < idLength + 1) throw invalid("Invalid EBML element id"); + const id = readUintBytes(bytes, 0, idLength); + const sizeByte = bytes[idLength]; + const sizeLength = Math.clz32(sizeByte) - 23; + if (sizeByte === 0 || bytes.length < idLength + sizeLength) throw invalid("Invalid EBML element size"); + let size = BigInt(sizeByte & (0xff >> sizeLength)); + for (let index = idLength + 1; index < idLength + sizeLength; index += 1) size = (size << 8n) | BigInt(bytes[index]); + const unknown = size === (1n << BigInt(7 * sizeLength)) - 1n; + if (!unknown && size > BigInt(Number.MAX_SAFE_INTEGER)) throw invalid("EBML element size exceeds inspection limits"); + const dataStart = offset + idLength + sizeLength; + const end = unknown ? null : dataStart + Number(size); + if (end !== null && end > parentEnd) throw invalid("EBML element extends past its parent"); + return { id, dataStart, end, unknown }; +} + +/** Children of a sized master element, in order. */ +async function* children(walk, element) { + let offset = element.dataStart; + let child; + while ((child = await readElement(walk, offset, element.end))) { + if (child.unknown) throw invalid("Unknown-size element inside a master element"); + yield child; + offset = child.end; + } +} + +async function readMatroska(walk, fileSize) { + const header = await readElement(walk, 0, fileSize); + if (!header || header.id !== EBML.header || header.unknown) throw invalid("Not an EBML file"); + let docType = null; + for await (const child of children(walk, header)) { + if (child.id === EBML.docType && child.end - child.dataStart <= MAX_NAME_BYTES) { + docType = latin1(untilNul(await walk.read(child.dataStart, child.end)), 0, Math.min(child.end - child.dataStart, MAX_NAME_BYTES)); + } + } + if (docType !== "matroska" && docType !== "webm") throw invalid("Unsupported EBML DocType"); + const segment = await readElement(walk, header.end, fileSize); + if (!segment || segment.id !== EBML.segment) throw invalid("No Segment element"); + const segmentEnd = segment.unknown ? fileSize : segment.end; + // As ffmpeg's demuxer: level-1 elements are read up to the first Cluster (of known + // or unknown size), then SeekHead entries locate Tags stored behind the clusters. + const seekTargets = []; + let offset = segment.dataStart; + let element; + while ((element = await readElement(walk, offset, segmentEnd))) { + if (element.id === EBML.cluster) break; + if (element.unknown) throw invalid("Unknown-size element that is not a Segment or Cluster"); + if (element.id === EBML.seekHead) await readSeekHead(walk, element, segment.dataStart, seekTargets); + else if (element.id === EBML.tags) await readTags(walk, element); + offset = element.end; + } + for (const target of seekTargets) { + if (target < offset || target >= segmentEnd) continue; + const pointed = await readElement(walk, target, segmentEnd); + if (!pointed || pointed.id !== EBML.tags || pointed.unknown) continue; + await readTags(walk, pointed); + } + return docType; +} + +async function readSeekHead(walk, seekHead, segmentStart, targets) { + for await (const seek of children(walk, seekHead)) { + if (seek.id !== EBML.seek) continue; + let isTags = false; + let position = null; + for await (const field of children(walk, seek)) { + const length = field.end - field.dataStart; + if (field.id === EBML.seekId && length <= 4) { + isTags = startsWith(await walk.read(field.dataStart, field.end), TAGS_ID_BYTES) && length === 4; + } else if (field.id === EBML.seekPosition && length <= 8) { + position = readUintBytes(await walk.read(field.dataStart, field.end), 0, length); + } + } + if (isTags && position !== null) targets.push(segmentStart + position); + } +} + +async function readTags(walk, tags) { + for await (const tag of children(walk, tags)) { + if (tag.id !== EBML.tag) continue; + let targeted = false; + const simpleTags = []; + for await (const child of children(walk, tag)) { + if (child.id === EBML.simpleTag) simpleTags.push(child); + else if (child.id === EBML.targets) targeted = await readTargets(walk, child); + } + if (targeted) continue; + for (const simpleTag of simpleTags) await readSimpleTag(walk, simpleTag); + } +} + +/** Whether the Targets name a track, chapter or attachment (ffmpeg files those tags elsewhere). */ +async function readTargets(walk, targets) { + let targeted = false; + for await (const field of children(walk, targets)) { + const length = field.end - field.dataStart; + if ((field.id === EBML.trackUid || field.id === EBML.chapterUid || field.id === EBML.attachmentUid) && length <= 8) { + if ((await walk.read(field.dataStart, field.end)).some((byte) => byte !== 0)) targeted = true; + } + } + return targeted; +} + +/** + * One SimpleTag as ffmpeg's matroska_convert_tag files it: under its plain name when + * it has no language other than und or TagDefault is set (the "-lang" name is never + * ours), a string-less tag deletes the name, nested SimpleTags become PARENT/CHILD + * (never ours). + */ +async function readSimpleTag(walk, simpleTag) { + let name = null; + let language = null; + let isDefault = 0; + let string = null; + for await (const field of children(walk, simpleTag)) { + const length = field.end - field.dataStart; + if (field.id === EBML.tagName) { + name = length <= MAX_NAME_BYTES ? ffprobeText(untilNul(await walk.read(field.dataStart, field.end))) : ""; + } else if (field.id === EBML.tagLanguage && length <= MAX_NAME_BYTES) { + language = latin1(untilNul(await walk.read(field.dataStart, field.end)), 0, Math.min(length, MAX_NAME_BYTES)); + } else if (field.id === EBML.tagDefault && length <= 8) { + isDefault = readUintBytes(await walk.read(field.dataStart, field.end), 0, length) ?? 0; + } else if (field.id === EBML.tagString) { + string = field; + } + } + if (name === null || !walk.known(name)) return; + const lang = language && language !== "und" ? language : null; + if (isDefault === 0 && lang) return; + if (string === null) walk.remove(name); + else await walk.value(name, string.dataStart, string.end); +} + +/** + * Reads the prompt carrier tags of a video File without touching its media data. + * + * @param {Blob} file - the chosen File (only `size` and `slice` are used) + * @param {Object} [options] + * @param {AbortSignal} [options.signal] - rejects the read with an AbortError + * @param {Uint8Array} [options.prefix] - the file's first bytes, already read for classification + * @param {Function} [options.yieldControl] - awaited about every 8 ms of work + * @param {string} [options.kind] - the classifier's kind, when the prefix was not kept + * @returns {Promise<{status:'complete', container:'iso-bmff'|'matroska', format:string|null, tags:Object, limited:string[]} + * | {status:'unreadable', reason:'invalid-container'|'read-failure'} + * | {status:'limited', reason:'metadata-limit'|'chunk-limit'} + * | {status:'unsupported', reason:'video-container-unsupported'}>} + */ +export async function readVideoTags(file, options = {}) { + const { signal, yieldControl = defaultYield } = options; + try { + throwIfAborted(signal); + if (!file || !Number.isSafeInteger(file.size) || typeof file.slice !== "function") throw readFailure("Not a readable file"); + let prefix = toBytes(options.prefix); + if (!prefix && options.kind === undefined) { + prefix = await createSliceReader(file, null, signal).read(0, Math.min(PREFIX_BYTES, file.size)); + } + const container = options.kind ?? detectKind(prefix); + if (container === "avi" || container === "asf") return { status: "unsupported", reason: "video-container-unsupported" }; + if (container !== "iso-bmff" && container !== "matroska") throw invalid("Not a supported video container"); + const walk = createWalk(createSliceReader(file, prefix, signal), signal, createPacer(yieldControl, signal)); + const format = container === "iso-bmff" ? await readIsoBmff(walk, file.size) : await readMatroska(walk, file.size); + walk.finish(); + return { status: "complete", container, format, tags: walk.tags, limited: walk.limited }; + } catch (error) { + if (error && error.name === "AbortError") throw error; + if (error instanceof VideoReadError) return { status: error.status, reason: error.reason }; + return { status: "unreadable", reason: "read-failure" }; + } +} + +// ------------------------------------------------- Python json port (loads/dumps) + +export class PythonJsonError extends Error { + constructor(message, recursion = false) { + super(message); + this.name = "PythonJsonError"; + this.recursion = recursion; + } +} + +/** A JSON number as Python keeps it: the lexeme, and whether it is a float. */ +class PyNumber { + constructor(text, isFloat) { + this.text = text; + this.isFloat = isFloat; + } +} + +const NUMBER = /-?(?:0|[1-9]\d*)(\.\d+)?([eE][-+]?\d+)?/y; +const STRING_STOP = /["\\\u0000-\u001f]/g; +const WHITESPACE = /[ \t\n\r]*/y; +const ESCAPES = { '"': '"', "\\": "\\", "/": "/", b: "\b", f: "\f", n: "\n", r: "\r", t: "\t" }; +const LITERALS = [["true", true], ["false", false], ["null", null], ["NaN", new PyNumber("NaN", true)], ["Infinity", new PyNumber("Infinity", true)], ["-Infinity", new PyNumber("-Infinity", true)]]; + +/** + * Python's ``json.loads``: objects become Maps (insertion order, last duplicate wins in + * the first position), numbers keep their lexeme, NaN/Infinity are accepted, raw + * control characters and trailing data are rejected. + */ +export function pythonJsonLoads(text) { + if (typeof text !== "string") throw new PythonJsonError("Not a string"); + let pos = 0; + const fail = (message) => new PythonJsonError(`${message} at ${pos}`); + const skip = () => { + WHITESPACE.lastIndex = pos; + pos = WHITESPACE.exec(text)[0].length + pos; + }; + function string() { + pos += 1; + let out = ""; + for (;;) { + STRING_STOP.lastIndex = pos; + const stop = STRING_STOP.exec(text); + if (!stop) throw fail("Unterminated string"); + out += text.slice(pos, stop.index); + pos = stop.index + 1; + const char = stop[0]; + if (char === '"') return out; + if (char !== "\\") throw fail("Invalid control character"); + const escape = text[pos]; + if (escape === "u") { + const hex = text.slice(pos + 1, pos + 5); + if (!/^[0-9a-fA-F]{4}$/.test(hex)) throw fail("Invalid \\uXXXX escape"); + pos += 5; + let code = parseInt(hex, 16); + if (code >= 0xd800 && code <= 0xdbff && text[pos] === "\\" && text[pos + 1] === "u") { + const low = text.slice(pos + 2, pos + 6); + if (/^[0-9a-fA-F]{4}$/.test(low)) { + const lowCode = parseInt(low, 16); + if (lowCode >= 0xdc00 && lowCode <= 0xdfff) { + out += String.fromCharCode(code, lowCode); + pos += 6; + continue; + } + } + } + out += String.fromCharCode(code); + } else if (escape in ESCAPES) { + out += ESCAPES[escape]; + pos += 1; + } else throw fail("Invalid \\escape"); + } + } + function value(depth) { + if (depth > JSON_MAX_DEPTH) throw new PythonJsonError("Maximum recursion depth exceeded", true); + const char = text[pos]; + if (char === '"') return string(); + if (char === "{") { + pos += 1; + const map = new Map(); + skip(); + if (text[pos] === "}") { + pos += 1; + return map; + } + for (;;) { + if (text[pos] !== '"') throw fail("Expecting property name enclosed in double quotes"); + const key = string(); + skip(); + if (text[pos] !== ":") throw fail("Expecting ':' delimiter"); + pos += 1; + skip(); + map.set(key, value(depth + 1)); + skip(); + if (text[pos] === "}") { + pos += 1; + return map; + } + if (text[pos] !== ",") throw fail("Expecting ',' delimiter"); + pos += 1; + skip(); + } + } + if (char === "[") { + pos += 1; + const list = []; + skip(); + if (text[pos] === "]") { + pos += 1; + return list; + } + for (;;) { + list.push(value(depth + 1)); + skip(); + if (text[pos] === "]") { + pos += 1; + return list; + } + if (text[pos] !== ",") throw fail("Expecting ',' delimiter"); + pos += 1; + skip(); + } + } + for (const [literal, result] of LITERALS) { + if (text.startsWith(literal, pos)) { + pos += literal.length; + return result; + } + } + NUMBER.lastIndex = pos; + const number = NUMBER.exec(text); + if (!number || number.index !== pos || !number[0]) throw fail("Expecting value"); + pos += number[0].length; + const isFloat = Boolean(number[1] || number[2]); + // CPython's int() refuses more digits than sys.get_int_max_str_digits(). + if (!isFloat && number[0].replace("-", "").length > INT_MAX_STR_DIGITS) throw fail("Exceeds the limit for integer string conversion"); + return new PyNumber(number[0], isFloat); + } + skip(); + const result = value(0); + skip(); + if (pos !== text.length) throw fail("Extra data"); + return result; +} + +/** Python's ``repr(float)``: shortest round-trip digits, exponent when decpt <= -4 or > 16. */ +function pythonFloatRepr(number) { + if (Number.isNaN(number)) return "NaN"; + if (number === Infinity) return "Infinity"; + if (number === -Infinity) return "-Infinity"; + if (number === 0) return Object.is(number, -0) ? "-0.0" : "0.0"; + const sign = number < 0 ? "-" : ""; + const [mantissa, exponent] = Math.abs(number).toExponential().split("e"); + const digits = mantissa.replace(".", ""); + const decpt = Number(exponent) + 1; + if (decpt <= -4 || decpt > 16) { + const shown = digits.length > 1 ? `${digits[0]}.${digits.slice(1)}` : digits; + const power = decpt - 1; + return `${sign}${shown}e${power < 0 ? "-" : "+"}${String(Math.abs(power)).padStart(2, "0")}`; + } + if (decpt <= 0) return `${sign}0.${"0".repeat(-decpt)}${digits}`; + if (decpt >= digits.length) return `${sign}${digits}${"0".repeat(decpt - digits.length)}.0`; + return `${sign}${digits.slice(0, decpt)}.${digits.slice(decpt)}`; +} + +/** Python's ``json.dumps`` default ``ensure_ascii`` string escaping. */ +function pythonEscape(text) { + let out = '"'; + for (let index = 0; index < text.length; index += 1) { + const char = text[index]; + const code = text.charCodeAt(index); + if (char === '"') out += '\\"'; + else if (char === "\\") out += "\\\\"; + else if (char === "\n") out += "\\n"; + else if (char === "\r") out += "\\r"; + else if (char === "\t") out += "\\t"; + else if (char === "\b") out += "\\b"; + else if (char === "\f") out += "\\f"; + else if (code < 0x20 || code > 0x7e) out += `\\u${code.toString(16).padStart(4, "0")}`; + else out += char; + } + return `${out}"`; +} + +/** Python's ``json.dumps(value)`` with default separators and ``ensure_ascii``, on a loads() value. */ +export function pythonJsonDumps(value) { + if (value === null) return "null"; + if (value === true) return "true"; + if (value === false) return "false"; + if (typeof value === "string") return pythonEscape(value); + if (value instanceof PyNumber) { + if (!value.isFloat) return value.text === "-0" ? "0" : value.text; + return pythonFloatRepr(Number(value.text)); + } + if (Array.isArray(value)) return `[${value.map(pythonJsonDumps).join(", ")}]`; + if (value instanceof Map) return `{${[...value].map(([key, item]) => `${pythonEscape(key)}: ${pythonJsonDumps(item)}`).join(", ")}}`; + throw new PythonJsonError("Not a JSON value"); +} + +// ------------------------------------------------------------ carrier selection + +/** utils/video_metadata._record: (text or null, record format or null). */ +function record(value, key) { + if (key === "workflow" && !(typeof value === "string" || value instanceof Map)) return [null, "invalid"]; + if (typeof value === "string") return [value, "original-text"]; + if (value instanceof Map) return [pythonJsonDumps(value), "reserialized-json"]; + return [null, null]; +} + +/** + * The one container carrier of prompt metadata, with the server's record budgets + * applied (utils/video_metadata.select_video_carrier, then the metadata loop of + * utils/prompt_inspector.read_inspection_file). + * + * @param {Object} tags - container tags as ffprobe would report them + * @param {Object} [options] + * @param {string[]} [options.limited] - records the reader left out (`['workflow']`) + * @returns {{status:'complete', records:Object, record_formats:Object, carrierText:string|null, limited:string[]} + * | {status:'unreadable', reason:'duplicate-metadata'|'invalid-metadata'} + * | {status:'limited', reason:'metadata-limit'}} + */ +export function selectVideoCarrier(tags, options = {}) { + let limitedRecords = new Set(options.limited || []); + const folded = new Map(); + const entries = tags && typeof tags === "object" && !Array.isArray(tags) ? Object.entries(tags) : []; + for (const [key, value] of entries) { + if (ENVELOPE_NAMES.has(key)) { + folded.set(key, value); + continue; + } + const name = key.toLowerCase(); + if (!PER_KEY_NAMES.has(name)) continue; + if (folded.has(name) && folded.get(name) !== value) return { status: "unreadable", reason: "duplicate-metadata" }; + folded.set(name, value); + } + const candidates = []; + const perKey = new Map(METADATA_KEYS.filter((key) => folded.has(key)).map((key) => [key, folded.get(key)])); + if (perKey.size) candidates.push([perKey, null]); + for (const tag of ENVELOPE_KEYS) { + const raw = folded.get(tag); + if (typeof raw !== "string") continue; + let envelope; + try { + envelope = pythonJsonLoads(raw); + } catch (error) { + if (error instanceof PythonJsonError && error.recursion) return { status: "unreadable", reason: "invalid-metadata" }; + continue; + } + if (envelope instanceof Map) candidates.push([envelope, raw]); + } + const selected = + candidates.find(([fields]) => ROLE_KEYS.some((key) => fields.has(key))) ?? candidates.find(([fields]) => fields.has("workflow")) ?? null; + const formats = {}; + if (selected === null) { + for (const key of limitedRecords) formats[key] = "limited"; + return { status: "complete", records: {}, record_formats: formats, carrierText: null, limited: [...limitedRecords] }; + } + const [fields, carrierText] = selected; + // An envelope carrier is whole; a left-out per-key workflow belonged to the carrier that lost. + if (carrierText !== null) limitedRecords = new Set(); + const metadata = {}; + for (const key of METADATA_KEYS) { + if (!fields.has(key)) continue; + if (fields.get(key) === OVERSIZED_WORKFLOW) { + // The reader left it unread: the server reads it, then finds it over budget. + limitedRecords.add(key); + continue; + } + const [text, format] = record(fields.get(key), key); + if (format === "invalid") { + formats[key] = format; + continue; + } + metadata[key] = text; + if (format !== null) formats[key] = format; + } + const records = {}; + let roleSize = 0; + for (const key of METADATA_KEYS) { + if (!(key in metadata)) continue; + const value = metadata[key]; + if (typeof value !== "string") return { status: "unreadable", reason: "invalid-metadata" }; + const size = utf8Length(value); + if (key === "workflow") { + if (size > WORKFLOW_METADATA_BYTES) { + limitedRecords.add(key); + continue; + } + } else { + roleSize += size; + if (roleSize > ROLE_METADATA_BYTES) return { status: "limited", reason: "metadata-limit" }; + } + records[key] = value; + if (!(key in formats)) formats[key] = "original-text"; + } + for (const key of limitedRecords) formats[key] = "limited"; + return { status: "complete", records, record_formats: formats, carrierText, limited: [...limitedRecords] }; +}