nodes/ComfyAssets-ComfyUI_PromptManager
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(inspect): video metadata carrier parity between Images and Inspect (#275)

Browse Source 
* fix(video): select one prompt carrier from per-key or comment tags

Current ComfyUI savers write one container tag per key (prompt, workflow,
parameters, comfyassets_prompt_pairs); only the old VideoHelperSuite comment
envelope was read. select_video_carrier picks exactly one carrier in the
order per-key tags, comment envelope, description: the first with a role
key wins, else the first with a workflow, and carriers never merge.
Matroska upper-cased tag names fold on our side; differing fold-duplicates
fail the read like duplicate PNG keys, identical ones are one record.
Per-key values stay original-text; envelope objects are reserialized.

* fix(inspect): read per-key video prompt tags

The inspector asked ffprobe only for comment and description, so videos
written by core SaveVideo/SaveWEBM and current VideoHelperSuite (one tag per
key) were inspected as having no metadata. It now asks for
prompt,workflow,parameters,comfyassets_prompt_pairs,comment,description and
hands the tags to the shared carrier selector. ffprobe stdout is bounded at
the role and workflow budgets plus 64 KiB of envelope; past that it probes
once more without the workflow tag within the role budget and marks the
per-key workflow limited, while an over-budget envelope still fails the
read. Differing fold-duplicate tags fail like duplicate PNG keys.

* fix(video): scan videos written by current ComfyUI savers

The output scan and live monitor read container tags through
metadata_from_tags, which now delegates to the shared carrier selector, so
per-key prompt, workflow, parameters and comfyassets_prompt_pairs tags
(including Matroska upper-cased names) are ingested with the default
resolver profile exactly like the same graph in a PNG. Fold-duplicate tags
and null markers are skipped quietly. Pins the behavior with a mocked
ffprobe scan test and real MP4/MKV/WebM round trips when ffmpeg is present.

* docs(testing): document video carrier selection

Record the shared video carrier order (per-key tags, comment envelope,
description), the fold-duplicate rule, the 6 MiB + 64 KiB probe bound with
the role-only re-probe and limited workflow, and that scans read the same
carrier as the inspector.

* fix(inspect): size video probe bounds for JSON escaping, exact-case envelopes

ffprobe JSON-escapes tag text, so bounds measured on its stdout could mark a
video workflow limited, or fail a role record, that the identical PNG reads
within its decoded budget. The first probe now allows twice the role and
workflow budgets plus 64 KiB (12 MiB + 64 KiB) and the role-only re-probe
twice the role budget plus 64 KiB, leaving the decoded per-record budgets as
the only gate; a reader-level test reads near-budget quote-dense records as
PNG and as video and asserts identical metadata and formats.

Envelope keys (comment, description) are matched exactly as the legacy
readers did; only the four per-key carriers fold Matroska case. Fixtures can
write the legacy ilst comment atom (metadata_tags=False), and the docs now
say the admin scan reads the carrier while the live monitor still processes
images only, with a follow-up recorded in TESTING.md.
This commit is contained in:
Vito
2026-10-22 05:04:15 -07:00
committed by GitHub
parent f95f66d76a
commit bc14f29db7
8 changed files with 836 additions and 125 deletions
Download Patch File Download Diff File Expand all files Collapse all files
utils/prompt_inspector.py
+65 -48
View File
@@ -23,7 +23,15 @@ from .prompt_pair_resolver import (
ResolverProfile,
resolve_prompt_pair,
)
from .video_metadata import ffprobe_path, is_video_path
from .video_metadata import (
ENVELOPE_TAG_KEYS,
ROLE_TAG_KEYS,
VIDEO_TAG_KEYS,
DuplicateVideoTagError,
ffprobe_path,
is_video_path,
select_video_carrier,
)
# Role records (prompt, parameters, the pair execution record) share one
# budget, encoded and decoded: roles cannot be trusted from a partial record,
@@ -36,7 +44,18 @@ WORKFLOW_METADATA_BYTES = 4 * 1024 * 1024
# repeated chunks cost bounded I/O instead of file size.
DUPLICATE_WORKFLOW_BYTES = 2 * WORKFLOW_METADATA_BYTES
DUPLICATE_ROLE_BYTES = 2 * ROLE_METADATA_BYTES
MAX_METADATA_BYTES = ROLE_METADATA_BYTES # video stdout bound, unchanged here
# ffprobe stdout bounds. ffprobe JSON-escapes tag text (at most doubling it),
# and the decoded per-record budgets below are the only gate that may make a
# record fail or ``limited``: a video must read exactly as the identical PNG.
# So the first probe allows twice the role and workflow budgets plus 64 KiB of
# JSON envelope; past that only the per-key ``workflow`` tag can be dropped,
# and the re-probe asks for role and envelope tags within twice the role
# budget (an envelope cannot be split, so an over-budget one fails the read).
MAX_METADATA_BYTES = ROLE_METADATA_BYTES
VIDEO_PROBE_BYTES = 2 * (ROLE_METADATA_BYTES + WORKFLOW_METADATA_BYTES) + 64 * 1024
VIDEO_REPROBE_BYTES = 2 * ROLE_METADATA_BYTES + 64 * 1024
VIDEO_PROBE_SECONDS = 3
_VIDEO_ROLE_PROBE_KEYS = ROLE_TAG_KEYS + ENVELOPE_TAG_KEYS
MAX_KEYWORD_BYTES = 80 # a 1-79 byte PNG keyword plus its NUL separator
MAX_HEADER_BYTES = 8 * 1024 * 1024
MAX_PNG_CHUNKS = 10_000
@@ -86,6 +105,7 @@ class _VideoInspection:
metadata: dict[str, str | None]
record_formats: dict[str, str]
carrier_text: str | None
limited: frozenset[str] = dataclass_field(default_factory=frozenset)
def _inflate(data: bytes, limit: int) -> bytes | None:
@@ -274,16 +294,16 @@ def _png_metadata(stream: Any, size: int) -> tuple[dict, frozenset, int, int]:
raise MetadataReadError("PNG chunk count exceeds inspection limits")
def _video_metadata(path: Path) -> _VideoInspection:
probe = ffprobe_path()
if not probe:
raise MetadataReadError("Video metadata reader is unavailable")
def _probe_video_tags(
probe: str, path: Path, keys: tuple[str, ...], limit: int
) -> bytes | None:
"""ffprobe's JSON for the container tags ``keys``; ``None`` past ``limit``."""
command = [
probe,
"-v",
"quiet",
"-show_entries",
"format_tags=comment,description",
"format_tags=" + ",".join(keys),
"-of",
"json",
str(path),
@@ -303,13 +323,13 @@ def _video_metadata(path: Path) -> _VideoInspection:
# Timeout and final cleanup can race with the process exiting.
pass
timer = threading.Timer(3, stop)
timer = threading.Timer(VIDEO_PROBE_SECONDS, stop)
timer.daemon = True
timer.start()
try:
raw = process.stdout.read(MAX_METADATA_BYTES + 1)
if len(raw) > MAX_METADATA_BYTES:
raise MetadataReadError("Video metadata exceeds inspection limits")
raw = process.stdout.read(limit + 1)
if len(raw) > limit:
return None
process.wait()
if process.returncode:
raise MetadataReadError("Video metadata could not be read")
@@ -318,49 +338,45 @@ def _video_metadata(path: Path) -> _VideoInspection:
stop()
process.wait()
process.stdout.close()
return raw
def _video_metadata(path: Path) -> _VideoInspection:
probe = ffprobe_path()
if not probe:
raise MetadataReadError("Video metadata reader is unavailable")
limited = frozenset()
raw = _probe_video_tags(probe, path, VIDEO_TAG_KEYS, VIDEO_PROBE_BYTES)
if raw is None:
raw = _probe_video_tags(
probe, path, _VIDEO_ROLE_PROBE_KEYS, VIDEO_REPROBE_BYTES
)
if raw is None:
raise MetadataReadError("Video metadata exceeds inspection limits")
# Only the per-key workflow tag was left out, so it is the record over
# budget: limited when the per-key carrier (or no carrier) is selected.
limited = frozenset({"workflow"})
probed = json.loads(raw)
if not isinstance(probed, dict) or not isinstance(probed.get("format"), dict):
raise MetadataReadError("Malformed video metadata")
tags = probed["format"].get("tags", {})
if not isinstance(tags, dict):
raise MetadataReadError("Malformed video metadata")
# The scan normalizer drops null fields and unknown keys. Inspection must
# retain explicit prompt/execution markers so malformed evidence cannot
# silently fall back to an unrelated saved recipe or another comment.
candidates = []
for key in ("comment", "description"):
raw = tags.get(key)
if not isinstance(raw, str):
continue
try:
envelope = json.loads(raw)
except ValueError:
continue
if not isinstance(envelope, dict):
continue
candidates.append((raw, envelope))
selected = next(
(item for item in candidates if _ROLE_METADATA_KEYS.intersection(item[1])),
None,
)
# The shared selector keeps explicit null role markers so malformed
# evidence cannot silently fall back to an unrelated saved recipe or
# another carrier; ``read_inspection_file`` refuses them.
try:
selected = select_video_carrier(tags)
except DuplicateVideoTagError:
raise MetadataReadError("Duplicate prompt metadata") from None
if selected is None:
selected = next((item for item in candidates if "workflow" in item[1]), None)
if selected is not None:
raw, envelope = selected
metadata, formats = {}, {}
for field in _METADATA_KEYS.intersection(envelope):
value = envelope[field]
if field == "workflow" and not isinstance(value, (str, dict)):
formats[field] = "invalid"
continue
metadata[field] = (
value if value is None or isinstance(value, str) else json.dumps(value)
)
formats[field] = (
"original-text" if isinstance(value, str) else "reserialized-json"
)
return _VideoInspection(metadata, formats, raw)
return _VideoInspection({}, {}, None)
return _VideoInspection({}, {}, None, limited)
metadata, formats, carrier = selected
if carrier is not None:
# An envelope carrier is whole; the dropped per-key workflow belonged
# to the carrier that lost.
limited = frozenset()
return _VideoInspection(metadata, formats, carrier, limited)
def read_inspection_file(path: Path) -> InspectionRead:
@@ -370,10 +386,11 @@ def read_inspection_file(path: Path) -> InspectionRead:
format_name = width = height = None
if is_video_path(path):
video = _video_metadata(path)
metadata, formats, carrier = (
metadata, formats, carrier, limited = (
video.metadata,
video.record_formats,
video.carrier_text,
video.limited,
)
else:
with path.open("rb") as stream:
utils/video_metadata.py
+126 -27
View File
@@ -1,14 +1,18 @@
"""Prompt metadata from video containers.
ComfyUI's video savers (core SaveVideo, VideoHelperSuite, ...) write the
queued graph into the container's ``comment`` tag as JSON:
``{"prompt": "<graph json>", "workflow": {...}}``. ffprobe reads it on every
platform; when ffprobe is not installed, videos are skipped with a single
notice instead of one error per file.
Current ComfyUI video savers (core SaveVideo/SaveWEBM, VideoHelperSuite since
``c67ff3e``) write one container tag per key, each a JSON string:
``prompt``, ``workflow``, ``parameters``, ``comfyassets_prompt_pairs``. Older
VideoHelperSuite wrote one ``comment`` tag holding a JSON envelope
``{"prompt": "<graph json>", "workflow": {...}}``; ``description`` is read as a
fallback envelope. ffprobe reads the tags on every platform; when ffprobe is
not installed, videos are skipped with a single notice instead of one error
per file.
``select_video_carrier`` picks exactly one carrier for both the inspector and
the scan, so Images and ingestion read the same prompts from the same file.
The result uses the same shape as PNG text chunks (string values keyed
``prompt`` / ``workflow`` / ``parameters``) so the existing prompt parser
handles both.
``prompt`` / ``workflow`` / ...) so the existing prompt parser handles both.
"""
import functools
@@ -20,10 +24,15 @@ import shutil
# Fixed argv, absolute binary, no shell.
import subprocess # nosec B404
from .pair_execution import EXECUTION_KEY
VIDEO_EXTENSIONS = (".mp4", ".mov", ".m4v", ".webm", ".mkv", ".avi", ".wmv")
FFPROBE_TIMEOUT = 20 # seconds; a stuck probe must not hang a worker thread
_COMMENT_TAGS = ("comment", "description")
_METADATA_KEYS = ("prompt", "workflow", "parameters")
# Carrier order: per-key tags, then the ``comment`` envelope, then ``description``.
ROLE_TAG_KEYS = ("prompt", "parameters", EXECUTION_KEY)
METADATA_TAG_KEYS = ("prompt", "workflow", "parameters", EXECUTION_KEY)
ENVELOPE_TAG_KEYS = ("comment", "description")
VIDEO_TAG_KEYS = METADATA_TAG_KEYS + ENVELOPE_TAG_KEYS
_NO_WINDOW = getattr(subprocess, "CREATE_NO_WINDOW", 0) # no console flash on Windows
logger = logging.getLogger("prompt_manager.video")
@@ -46,32 +55,122 @@ def reset_missing_notice():
_missing_notice_shown = False
def metadata_from_tags(tags) -> dict:
"""Normalise container tags into PNG-text-chunk shaped metadata.
class DuplicateVideoTagError(ValueError):
"""Two container tags fold to the same metadata key with different text."""
Returns ``{}`` when no tag carries a JSON object with a known key.
def _folded_tags(tags: dict) -> dict:
"""Known tags keyed by their carrier name.
Per-key prompt carriers fold ASCII case: Matroska muxers upper-case tag
names and ffprobe reports them as written. Two tags folding to one key are
a duplicate record: byte-identical duplicates are one record (as for PNG
text chunks), differing ones fail the read. Envelope keys (``comment``,
``description``) are read exactly as the legacy readers did, never folded.
"""
if not isinstance(tags, dict):
return {}
for tag in _COMMENT_TAGS:
raw = tags.get(tag)
folded = {}
for key, value in tags.items():
if not isinstance(key, str):
continue
if key in ENVELOPE_TAG_KEYS:
folded[key] = value
continue
name = key.lower()
if name not in METADATA_TAG_KEYS:
continue
if name in folded and folded[name] != value:
raise DuplicateVideoTagError(f"Duplicate video metadata tag: {name}")
folded[name] = value
return folded
def _record(value, key):
"""One carrier field as (metadata text or None, record format or None).
A null marker (or a non-text role value) has no record format: it is kept
in the metadata as ``None`` so the inspector refuses it instead of letting
fallback evidence win, while scans drop it.
"""
if key == "workflow" and not isinstance(value, (str, dict)):
return None, "invalid"
if isinstance(value, str):
return value, "original-text"
if isinstance(value, dict):
return json.dumps(value), "reserialized-json"
return None, None
def _carriers(folded: dict):
"""Candidate carriers in order: (fields, raw envelope text or None)."""
per_key = {key: folded[key] for key in METADATA_TAG_KEYS if key in folded}
if per_key:
yield per_key, None
for tag in ENVELOPE_TAG_KEYS:
raw = folded.get(tag)
if not isinstance(raw, str):
continue
try:
data = json.loads(raw)
envelope = json.loads(raw)
except ValueError:
continue
if not isinstance(data, dict):
if isinstance(envelope, dict):
yield envelope, raw
def select_video_carrier(tags):
"""Choose the one container carrier of prompt metadata.
Returns ``(metadata, record_formats, carrier_text)`` or ``None`` when no
carrier holds a known key. ``metadata`` maps known keys to their text, or
to ``None`` for an explicit null marker that the inspector must refuse;
``record_formats`` is ``original-text``, ``reserialized-json`` (an envelope
object) or ``invalid`` (a workflow that is neither text nor an object), and
``carrier_text`` is the exact envelope for reserialized values (``None``
for per-key tags). The first carrier holding a role key wins; otherwise the
first holding ``workflow``. Carriers are never merged.
Raises :class:`DuplicateVideoTagError` for differing fold-duplicate tags.
"""
if not isinstance(tags, dict):
return None
candidates = list(_carriers(_folded_tags(tags)))
selected = next(
(item for item in candidates if any(key in item[0] for key in ROLE_TAG_KEYS)),
None,
)
if selected is None:
selected = next((item for item in candidates if "workflow" in item[0]), None)
if selected is None:
return None
fields, carrier_text = selected
metadata, formats = {}, {}
for key in METADATA_TAG_KEYS:
if key not in fields:
continue
found = {}
for key in _METADATA_KEYS:
value = data.get(key)
if value is None:
continue
found[key] = value if isinstance(value, str) else json.dumps(value)
if found:
return found
return {}
text, record_format = _record(fields[key], key)
if record_format == "invalid":
formats[key] = record_format
continue
metadata[key] = text
if record_format is not None:
formats[key] = record_format
return metadata, formats, carrier_text
def metadata_from_tags(tags) -> dict:
"""Normalise container tags into PNG-text-chunk shaped metadata for scans.
Returns ``{}`` when no carrier holds a known key, and when fold-duplicate
tags make the file's evidence untrustworthy.
"""
try:
selected = select_video_carrier(tags)
except DuplicateVideoTagError as e:
logger.debug(f"Video tags skipped: {e}")
return {}
if selected is None:
return {}
metadata, _, _ = selected
return {key: value for key, value in metadata.items() if isinstance(value, str)}
def read_video_metadata(path, timeout=FFPROBE_TIMEOUT) -> dict: