Files
ComfyAssets-ComfyUI_PromptM…/utils/inspection_semantics.py
T
Vito a0fc5171e3 fix(inspector): pass text through unknown nodes instead of refusing them (#381) (#383)
* fix(inspector): pass text through unknown nodes instead of refusing them (#381)

The inspector refused any node class missing from the semantics table, so a
PromptManagerPair behind an rgthree Any Switch showed "Prompt uses a node the
inspector does not support". The inspection resolver (Python and browser) now
reads through unknown nodes: a text input is the text, positive/negative route
by output slot, a conditioning input is followed alone, otherwise every linked
input except core type inputs (pass_through.skip_inputs). PromptManager nodes
are authoritative; several different texts become one candidate each
("Several prompts reach the sampler"). A seeded text node with no conditioning
input is generated at run time, and a recorded PromptManager execution is not
checked against text that passed through an unknown node.

INSPECTION_RESOLVER_VERSION goes to 2; ingestion keeps the default profile.
A legacy parity gate records every committed graph whose inspection positive
matches the legacy parser's; no graph may leave it.

* fix(inspector): keep recorded pair text, string-only seed rule, role inputs and unique candidates (#381)

Review of the pass-through: a walk through an unknown node never overrides a
PromptManagerPair's execution record (any outcome but work-limit is
unverifiable while the record is checked); the run-time seed rule reads only
string outputs; a node with positive/negative inputs follows only the role of
the requested slot, nothing when it is unlinked; one text offered twice is one
candidate.

* fix(inspector): anchor on the saver that wrote the file and resolve stages on shared links (#381)

An inspected file's name now anchors the inspection on the one saver whose
filename_prefix (literal or traced) makes the file's stem <prefix>_<digits>[_];
the samplers in that saver's image ancestry are the candidates. No match,
several matches or an unreadable walk anchor nothing, so the file stays
"several possible prompts". In a latent walk, an earlier sampler reading the
same conditioning links as the later one is the same prompt, not an
unattributed stage. Python and the browser resolver behave the same; the
legacy parity gate states each graph that differs from legacy on purpose.

* fix(inspector): skip only plain first stages on shared links; an unfinished prefix scan anchors nothing (#381)

Review of the saver anchor: a same-links earlier stage is skipped only when it
is not a detailer, sets no prompt override and continues no other sampler, so
an earlier prompt or override behind it is never hidden. A work limit while
tracing a saver prefix ends the match scan without an anchor instead of
counting that saver as no match.

* test(inspector): cover every plain-first-stage outcome on both resolvers (#381)

The frontend coverage gate found local-resolver-stages.mjs branches at 92.44%
(< 93). Each way an earlier stage stops being a plain first stage (no latent,
literal, missing or cyclic latent, a sampler-shaped node, a detailer) is now a
resolver test and an anchor-first-stage-* parity vector.
2026-10-27 16:56:37 +00:00

633 lines
26 KiB
Python

"""Load and validate the declarative inspection node semantics table (#240).
The table lives once in ``web/js/inspection-semantics.json`` so a browser-local
resolver can load the same data. It maps node classes (or ``(class, output
slot)`` pairs) to a named strategy from the closed ``STRATEGIES`` set plus the
exact input names that strategy reads. Code implements each strategy; adding a
node that reuses one is a data-only change.
Only the inspection resolver profile consumes this table, through
``get_semantics()``, which reads and validates it on first use and caches only a
valid result. Importing this module reads nothing, and the default (ingestion)
profile never imports it, so a missing or bad table can never break ingestion.
An invalid table raises ``SemanticsError`` so a bad edit fails loudly instead of
silently changing what the inspector resolves. The file is bounded in size and
nesting before it is validated. Every class appears in exactly one prompt
category; an unlisted output of a listed class stays ``unsupported-node``. ``lineage``
(image and latent pass-through outputs for multi-stage chains) describes no
prompt, so its classes are not claimed by a category.
``Semantics.version`` is the SHA-256 of the table's canonical JSON (sorted
keys, no whitespace): any change of content, never of layout, moves it.
``class_version(doc, classes)`` is the narrower fingerprint Needs review
folds into the version of the items it records from the inspection profile
(``utils.review_capture``): the SHA-256 of ``class_projection``, the table
restricted to what it says about ``classes`` (the node classes of one graph)
plus the class-independent ``schema``, ``stage_inputs`` and ``pass_through``.
The resolver looks the table up only by the class of a graph node, so an edit
about other classes cannot change how that graph resolves and leaves the
version alone. A class the table does not list at all passes text through
(#381); that rule lives in code, so ``INSPECTION_RESOLVER_VERSION`` moves for it.
``pass_through.skip_inputs`` names the core type inputs (model, clip, image,
...) such a node's text is never read through.
"""
import hashlib
import json
from dataclasses import dataclass, replace
from functools import lru_cache
from pathlib import Path
from types import MappingProxyType
from typing import Any, Callable, Dict, FrozenSet, Mapping, Optional, Tuple, Union
SEMANTICS_PATH = (
Path(__file__).resolve().parents[1] / "web" / "js" / "inspection-semantics.json"
)
SCHEMA_VERSION = 1
# The real table is a few KiB and four levels deep; these only reject nonsense.
MAX_TABLE_BYTES = 1024 * 1024
MAX_NESTING = 8
_NAME_LENGTH = 128
# Comfy output slots are small; this only rejects nonsense, never a real node.
_MAX_SLOT = 255
STRATEGIES: Mapping[str, FrozenSet[str]] = MappingProxyType(
{
"samplers": frozenset({"ksampler", "custom-advanced", "detailer-stage"}),
"guiders": frozenset({"cfg", "basic"}),
"conditioning": frozenset({"passthrough", "zeroed"}),
"selectors": frozenset({"boolean-select"}),
"composites": frozenset({"combine"}),
"encoders": frozenset({"single", "parts-equal", "pm-compose", "flag-parts"}),
"strings": frozenset({"literal", "strip-newlines", "join-multi", "concat"}),
"savers": frozenset({"comfy-counter", "vhs-counter"}),
"lineage": frozenset({"continue"}),
}
)
# Shapes of the facts each sampler class provides (consumed by comfy-metadata.js).
FACT_SHAPES = frozenset({"ksampler", "custom", "custom-advanced"})
SAVER_PATTERNS = MappingProxyType(
{
"comfy-counter": "{name}_{counter:05}_.{ext}",
"vhs-counter": "{name}_{counter:05}.{ext}",
}
)
_SAMPLER_OPTIONS = frozenset({"prompt_overrides", "fact_overrides"})
_CONFIGURATION_KINDS = frozenset({"noise", "sampler", "sigmas", "guider"})
_SCALAR_KINDS = frozenset({"number", "string"})
_TYPE_FIELDS = frozenset({"type_fields"})
_CLASS_CATEGORIES = ("samplers", "guiders", "selectors", "composites", "savers")
_SLOT_CATEGORIES = ("conditioning", "encoders", "strings")
_SET_CATEGORIES = ("runtime", "non_file_outputs")
_FACTS_TABLES = (
"configuration_outputs",
"model_outputs",
"model_passthroughs",
"scalar_outputs",
)
_CATEGORIES = frozenset(
("schema",)
+ _CLASS_CATEGORIES
+ _SLOT_CATEGORIES
+ _SET_CATEGORIES
+ ("lineage", "stage_inputs", "pass_through", "facts")
)
# Input names that make an unlisted node a possible sampler stage in an ancestry
# walk (``shaped``: pipes, video embeds and custom-sampler parts carry a prompt
# without a ``positive`` link), and those that do so beside a linked ``model``
# (``with_model``: XlabsSampler's ``conditioning``, video samplers' ``samples``).
# ``model`` alone (ModelSampling*) or with ``clip`` (LoraLoader) is a model patch.
# The resolver and comfy-metadata.js both read these lists.
_STAGE_INPUT_LISTS = ("shaped", "with_model")
# Inputs of core ComfyUI types (model, clip, image, ...) that an unknown node's
# output is never read through when the resolver passes text through it (#381):
# a loader's lora tag reached through ``clip`` is not the prompt.
_PASS_THROUGH_KEYS = ("skip_inputs",)
_STRING_PARAMS = MappingProxyType(
{
"literal": frozenset(),
"concat": frozenset(),
"strip-newlines": frozenset({"flag"}),
"join-multi": frozenset({"count", "delimiter", "list_flag", "input_prefix"}),
}
)
Frozen = Union[Mapping[str, Any], Tuple[Any, ...], str, int, None]
class SemanticsError(ValueError):
"""The semantics table is unreadable or violates its schema."""
@dataclass(frozen=True)
class Semantics:
"""Validated, deeply frozen semantics. Slot categories key by ``(class, slot)``."""
samplers: Mapping[str, Mapping[str, Any]]
guiders: Mapping[str, Mapping[str, Any]]
conditioning: Mapping[Tuple[str, int], Mapping[str, Any]]
selectors: Mapping[str, Mapping[str, Any]]
composites: Mapping[str, Mapping[str, Any]]
encoders: Mapping[Tuple[str, int], Mapping[str, Any]]
strings: Mapping[Tuple[str, int], Mapping[str, Any]]
runtime: FrozenSet[str]
savers: Mapping[str, Mapping[str, Any]]
non_file_outputs: FrozenSet[str]
lineage: Mapping[Tuple[str, int], Mapping[str, Any]]
stage_inputs: Mapping[str, Tuple[str, ...]]
pass_through: Mapping[str, Tuple[str, ...]]
facts: Mapping[str, Any]
all_classes: FrozenSet[str]
# SHA-256 of the canonical table JSON (set by ``load_semantics``).
version: str = ""
# The canonical table JSON itself, for ``class_version``.
source: str = ""
def _freeze(value: Any) -> Frozen:
if isinstance(value, dict):
return MappingProxyType({key: _freeze(item) for key, item in value.items()})
if isinstance(value, list):
return tuple(_freeze(item) for item in value)
return value
def _reject_duplicate_keys(pairs: list) -> dict:
result: Dict[str, Any] = {}
for key, value in pairs:
if key in result:
raise SemanticsError(f"duplicate key {key!r}")
result[key] = value
return result
def _reject_constant(name: str) -> None:
raise SemanticsError(f"non-finite number {name} is not allowed")
def _name(value: Any, where: str, what: str = "input") -> str:
if not isinstance(value, str) or not value or len(value) > _NAME_LENGTH:
raise SemanticsError(f"{where}: missing or invalid {what} name {value!r}")
return value
def _names(value: Any, where: str, minimum: int = 1) -> None:
if not isinstance(value, list) or len(value) < minimum:
raise SemanticsError(f"{where}: needs at least {minimum} input name(s)")
for item in value:
_name(item, where)
if len(set(value)) != len(value):
raise SemanticsError(f"{where}: repeated input name")
def _fields(
entry: Any, where: str, required: FrozenSet[str], optional: FrozenSet[str]
) -> None:
if not isinstance(entry, dict):
raise SemanticsError(f"{where}: entry must be an object")
missing = sorted(required - entry.keys())
if missing:
raise SemanticsError(f"{where}: missing input field(s) {missing}")
unknown = sorted(entry.keys() - required - optional)
if unknown:
raise SemanticsError(f"{where}: unknown field(s) {unknown}")
def _strategy(entry: Any, category: str, where: str) -> str:
strategy = entry.get("strategy") if isinstance(entry, dict) else None
if not isinstance(strategy, str) or strategy not in STRATEGIES[category]:
raise SemanticsError(f"{where}: unknown strategy {strategy!r}")
return strategy
def _check_sampler(entry: dict, strategy: str, where: str) -> None:
stage = "image_stage" if strategy == "detailer-stage" else "latent"
_fields(
entry,
where,
frozenset({"strategy", "conditioning", stage, "facts"}),
_SAMPLER_OPTIONS,
)
_name(entry[stage], where)
# Inputs that, when set, replace the prompt the sampler's conditioning
# inputs carry (a detailer's wildcard); the prompt is then not read.
# ``fact_overrides`` (read by comfy-metadata.js): inputs that, when set,
# replace the settings the facts would read (a script or an XY plot).
for option in sorted(_SAMPLER_OPTIONS & entry.keys()):
_names(entry[option], f"{where}.{option}")
keys = {"guider"} if strategy == "custom-advanced" else {"positive", "negative"}
conditioning = entry["conditioning"]
_fields(conditioning, where + ".conditioning", frozenset(keys), frozenset())
for key in keys:
_name(conditioning[key], where + ".conditioning")
if entry["facts"] not in FACT_SHAPES:
raise SemanticsError(f"{where}: unknown facts shape {entry['facts']!r}")
def _check_guider(entry: dict, strategy: str, where: str) -> None:
if strategy == "cfg":
_fields(
entry, where, frozenset({"strategy", "positive", "negative"}), frozenset()
)
_name(entry["negative"], where)
else:
# Negative absent by design; its reason is fixed text, not an input.
_fields(
entry,
where,
frozenset({"strategy", "positive"}),
frozenset({"absent_reason"}),
)
if "absent_reason" in entry:
_name(entry["absent_reason"], where, "absent reason")
_name(entry["positive"], where)
def _check_selector(entry: dict, strategy: str, where: str) -> None:
_fields(entry, where, frozenset({"strategy", "condition", "branches"}), frozenset())
_name(entry["condition"], where)
branches = entry["branches"]
_fields(branches, where + ".branches", frozenset({"true", "false"}), frozenset())
_name(branches["true"], where)
_name(branches["false"], where)
def _check_composite(entry: dict, strategy: str, where: str) -> None:
# ``additive``: every part stays in effect (combine, concat), so a later
# stage that adds parts to an earlier stage's prompt keeps that prompt.
_fields(entry, where, frozenset({"strategy", "inputs"}), frozenset({"additive"}))
_names(entry["inputs"], where, minimum=2)
if "additive" in entry and not isinstance(entry["additive"], bool):
raise SemanticsError(f"{where}: additive must be a boolean")
def _check_saver(entry: dict, strategy: str, where: str) -> None:
_fields(
entry,
where,
frozenset({"strategy", "prefix_input", "image_input", "pattern"}),
frozenset(),
)
_name(entry["prefix_input"], where)
_name(entry["image_input"], where)
if entry["pattern"] != SAVER_PATTERNS[strategy]:
raise SemanticsError(f"{where}: pattern does not match {strategy}")
def _check_conditioning(entry: dict, strategy: str, where: str) -> None:
_fields(entry, where, frozenset({"strategy", "input"}), frozenset())
_name(entry["input"], where)
def _check_encoder(entry: dict, strategy: str, where: str) -> None:
if strategy == "flag-parts":
# A literal boolean ``flag`` false reads the first input; true needs
# every input equal (smZ CLIPTextEncode's with_SDXL).
_fields(entry, where, frozenset({"strategy", "inputs", "params"}), frozenset())
params = entry["params"]
if not isinstance(params, dict) or set(params) != {"flag"}:
raise SemanticsError(f"{where}: params must be exactly ['flag']")
_name(params["flag"], where + ".params")
_names(entry["inputs"], where, minimum=2)
return
_fields(entry, where, frozenset({"strategy", "inputs"}), frozenset({"compose"}))
_names(entry["inputs"], where, minimum=2 if strategy == "parts-equal" else 1)
if "compose" in entry and not isinstance(entry["compose"], bool):
raise SemanticsError(f"{where}: compose must be a boolean")
def _check_string(entry: dict, strategy: str, where: str) -> None:
required = _STRING_PARAMS[strategy]
if required:
_fields(entry, where, frozenset({"strategy", "inputs", "params"}), frozenset())
params = entry["params"]
if not isinstance(params, dict) or set(params) != required:
raise SemanticsError(f"{where}: params must be exactly {sorted(required)}")
for value in params.values():
_name(value, where + ".params")
else:
_fields(entry, where, frozenset({"strategy", "inputs"}), frozenset())
_names(entry["inputs"], where)
_CHECKS: Mapping[str, Callable[[dict, str, str], None]] = MappingProxyType(
{
"samplers": _check_sampler,
"guiders": _check_guider,
"selectors": _check_selector,
"composites": _check_composite,
"savers": _check_saver,
"conditioning": _check_conditioning,
"encoders": _check_encoder,
"strings": _check_string,
}
)
class _Owners:
"""Each class belongs to one category; each (class, slot) is listed once."""
def __init__(self) -> None:
self._category: Dict[str, str] = {}
self._outputs: set = set()
def claim(self, name: str, category: str, slot: Optional[int] = None) -> None:
owner = self._category.setdefault(name, category)
if owner != category:
raise SemanticsError(f"duplicate class {name!r} in {owner} and {category}")
key = (name, slot)
if slot is not None and key in self._outputs:
raise SemanticsError(f"{category}: duplicate output key {key!r}")
self._outputs.add(key)
def classes(self) -> FrozenSet[str]:
return frozenset(self._category)
def _class_category(doc: dict, category: str, owners: _Owners) -> Mapping:
table = doc[category]
if not isinstance(table, dict):
raise SemanticsError(f"category {category} must be an object")
result = {}
for name, entry in table.items():
where = f"{category}[{name!r}]"
_name(name, where, "class")
strategy = _strategy(entry, category, where)
_CHECKS[category](entry, strategy, where)
owners.claim(name, category)
result[name] = _freeze(entry)
return MappingProxyType(result)
def _slot(entry: dict, where: str) -> int:
slot = entry.get("slot")
# bool is an int subclass; a JSON true is not an output slot.
if type(slot) is not int or not 0 <= slot <= _MAX_SLOT:
raise SemanticsError(f"{where}: slot must be a non-negative int, not {slot!r}")
return slot
def _slot_category(doc: dict, category: str, owners: _Owners) -> Mapping:
rows = doc[category]
if not isinstance(rows, list):
raise SemanticsError(f"category {category} must be a list")
result = {}
for index, row in enumerate(rows):
where = f"{category}[{index}]"
if not isinstance(row, dict):
raise SemanticsError(f"{where}: entry must be an object")
name = _name(row.get("class"), where, "class")
slot = _slot(row, where)
entry = {
key: value for key, value in row.items() if key not in ("class", "slot")
}
strategy = _strategy(entry, category, where)
_CHECKS[category](entry, strategy, where)
owners.claim(name, category, slot)
result[(name, slot)] = _freeze(entry)
return MappingProxyType(result)
def _lineage(doc: dict) -> Mapping:
"""``(class, slot)`` outputs whose image or latent continues one input's.
These describe an output's image or latent, never its prompt, so they are
not prompt categories: a class listed here is not claimed, and stays
unsupported (or listed elsewhere) for text. Each output is listed once.
"""
rows = doc["lineage"]
if not isinstance(rows, list):
raise SemanticsError("category lineage must be a list")
result = {}
for index, row in enumerate(rows):
where = f"lineage[{index}]"
if not isinstance(row, dict):
raise SemanticsError(f"{where}: entry must be an object")
name = _name(row.get("class"), where, "class")
slot = _slot(row, where)
entry = {
key: value for key, value in row.items() if key not in ("class", "slot")
}
_strategy(entry, "lineage", where)
_fields(entry, where, frozenset({"strategy", "input"}), frozenset())
_name(entry["input"], where)
if (name, slot) in result:
raise SemanticsError(f"lineage: duplicate output key {(name, slot)!r}")
result[(name, slot)] = _freeze(entry)
return MappingProxyType(result)
def _set_category(doc: dict, category: str, owners: _Owners) -> FrozenSet[str]:
rows = doc[category]
if not isinstance(rows, list):
raise SemanticsError(f"category {category} must be a list")
seen: set = set()
for name in rows:
_name(name, category, "class")
if name in seen:
raise SemanticsError(f"{category}: duplicate class {name!r}")
seen.add(name)
owners.claim(name, category)
return frozenset(seen)
def _string_map(table: Any, where: str, allowed: Optional[FrozenSet[str]]) -> None:
if not isinstance(table, dict):
raise SemanticsError(f"{where} must be an object")
for name, value in table.items():
_name(name, where, "class")
_name(value, where)
if allowed is not None and value not in allowed:
raise SemanticsError(f"{where}[{name!r}]: unknown kind {value!r}")
def _stage_inputs(doc: dict) -> Mapping[str, Tuple[str, ...]]:
table = doc["stage_inputs"]
if not isinstance(table, dict) or set(table) != set(_STAGE_INPUT_LISTS):
raise SemanticsError(
f"stage_inputs must have exactly {list(_STAGE_INPUT_LISTS)}"
)
for key in _STAGE_INPUT_LISTS:
_names(table[key], f"stage_inputs.{key}")
if "model" in table[key]:
raise SemanticsError(f"stage_inputs.{key}: model is a patch input")
if set(table["shaped"]) & set(table["with_model"]):
raise SemanticsError("stage_inputs: an input is listed in both lists")
return _freeze(table)
def _pass_through(doc: dict) -> Mapping[str, Tuple[str, ...]]:
table = doc["pass_through"]
if not isinstance(table, dict) or set(table) != set(_PASS_THROUGH_KEYS):
raise SemanticsError(
f"pass_through must have exactly {list(_PASS_THROUGH_KEYS)}"
)
_names(table["skip_inputs"], "pass_through.skip_inputs")
for name in ("text", "positive", "negative", "conditioning"):
if name in table["skip_inputs"]:
raise SemanticsError(f"pass_through.skip_inputs: {name} carries text")
return _freeze(table)
def _facts(doc: dict) -> Mapping[str, Any]:
facts = doc["facts"]
if not isinstance(facts, dict) or set(facts) != set(_FACTS_TABLES):
raise SemanticsError(f"facts must have exactly {list(_FACTS_TABLES)}")
configuration = facts["configuration_outputs"]
if not isinstance(configuration, dict):
raise SemanticsError("facts.configuration_outputs must be an object")
for name, adapter in configuration.items():
where = f"facts.configuration_outputs[{name!r}]"
_name(name, where, "class")
_fields(adapter, where, frozenset({"kind", "fields"}), _TYPE_FIELDS)
if adapter["kind"] not in _CONFIGURATION_KINDS:
raise SemanticsError(f"{where}: unknown kind {adapter['kind']!r}")
_names(adapter["fields"], where)
if "type_fields" in adapter:
# A fact whose value is the provider's node type (D8: Flux2Scheduler).
_names(adapter["type_fields"], where)
if set(adapter["type_fields"]) & set(adapter["fields"]):
raise SemanticsError(f"{where}: type_fields repeat an input field")
_string_map(facts["model_outputs"], "facts.model_outputs", None)
_string_map(facts["scalar_outputs"], "facts.scalar_outputs", _SCALAR_KINDS)
passthroughs = facts["model_passthroughs"]
_names(passthroughs, "facts.model_passthroughs", minimum=0)
return _freeze(facts)
def _check_nesting(doc: Any) -> None:
"""Reject containers nested deeper than ``MAX_NESTING`` (iterative walk)."""
stack = [(doc, 1)]
while stack:
value, level = stack.pop()
if not isinstance(value, (dict, list)):
continue
if level > MAX_NESTING:
raise SemanticsError(f"semantics table nests deeper than {MAX_NESTING}")
items = value.values() if isinstance(value, dict) else value
stack.extend((item, level + 1) for item in items)
def _validate(doc: Any) -> Semantics:
if not isinstance(doc, dict):
raise SemanticsError("semantics table must be an object")
if set(doc) != _CATEGORIES:
missing = sorted(_CATEGORIES - doc.keys())
unknown = sorted(doc.keys() - _CATEGORIES)
raise SemanticsError(f"category mismatch: missing {missing}, unknown {unknown}")
if type(doc["schema"]) is not int or doc["schema"] != SCHEMA_VERSION:
raise SemanticsError(f"unsupported schema {doc['schema']!r}")
owners = _Owners()
tables: Dict[str, Any] = {}
for category in _CLASS_CATEGORIES:
tables[category] = _class_category(doc, category, owners)
for category in _SLOT_CATEGORIES:
tables[category] = _slot_category(doc, category, owners)
for category in _SET_CATEGORIES:
tables[category] = _set_category(doc, category, owners)
return Semantics(
lineage=_lineage(doc),
stage_inputs=_stage_inputs(doc),
pass_through=_pass_through(doc),
facts=_facts(doc),
all_classes=owners.classes(),
**tables,
)
def load_semantics(path: Optional[Union[str, Path]] = None) -> Semantics:
"""Read and validate a semantics table; ``SemanticsError`` on any problem."""
source = Path(path) if path is not None else SEMANTICS_PATH
try:
with source.open("rb") as stream:
data = stream.read(MAX_TABLE_BYTES + 1)
if len(data) > MAX_TABLE_BYTES:
raise SemanticsError(
f"semantics table {source.name} exceeds {MAX_TABLE_BYTES} bytes"
)
doc = json.loads(
data.decode("utf-8"),
object_pairs_hook=_reject_duplicate_keys,
parse_constant=_reject_constant,
)
except SemanticsError:
raise
except RecursionError:
raise SemanticsError(
f"semantics table {source.name} nests deeper than {MAX_NESTING}"
) from None
except (OSError, UnicodeDecodeError, ValueError) as error:
raise SemanticsError(f"cannot read semantics table {source.name}: {error}")
_check_nesting(doc)
return replace(
_validate(doc), version=table_version(doc), source=_canonical_json(doc)
)
# Table parts that hold no class names: they apply to every graph.
_CLASS_INDEPENDENT = frozenset({"schema", "stage_inputs", "pass_through"})
def _canonical_json(doc: Any) -> str:
return json.dumps(doc, sort_keys=True, separators=(",", ":"), ensure_ascii=True)
def table_version(doc: Any) -> str:
"""SHA-256 of ``doc`` as canonical JSON: content, not layout."""
return hashlib.sha256(_canonical_json(doc).encode("utf-8")).hexdigest()
def _about(value: Any, classes: FrozenSet[str]) -> Any:
"""The part of one category that names a class in ``classes``.
Categories are keyed by class (objects), list classes (strings) or list
entries that carry a ``class``.
"""
if isinstance(value, dict):
return {key: item for key, item in value.items() if key in classes}
if isinstance(value, list):
return [
entry
for entry in value
if (isinstance(entry, str) and entry in classes)
or (isinstance(entry, dict) and entry.get("class") in classes)
]
return value
def class_projection(doc: Mapping[str, Any], classes) -> dict:
"""What ``doc`` says about ``classes``, plus its class-independent parts."""
classes = frozenset(classes)
projected = {}
for category, value in doc.items():
if category in _CLASS_INDEPENDENT:
projected[category] = value
elif category == "facts" and isinstance(value, dict):
projected[category] = {
table: _about(entries, classes) for table, entries in value.items()
}
else:
projected[category] = _about(value, classes)
return projected
def class_version(doc: Mapping[str, Any], classes) -> str:
"""SHA-256 of ``class_projection(doc, classes)`` as canonical JSON."""
return table_version(class_projection(doc, classes))
@lru_cache(maxsize=256)
def _cached_class_version(source: str, classes: FrozenSet[str]) -> str:
return class_version(json.loads(source), classes)
def semantics_class_version(semantics: Semantics, classes) -> str:
"""``class_version`` of a loaded table (cached per table and class set)."""
return _cached_class_version(semantics.source, frozenset(classes))
@lru_cache(maxsize=1)
def get_semantics() -> Semantics:
"""The repository table, loaded on first use; failures are not cached."""
return load_semantics()