Add performance benchmarks for the non-Analyzer pipeline (MASTER_SPEC §26.6,
NFR-004) plus tests.
- scripts/benchmark.py: measures Knowledge Base load time, single-scene
resolution time, and full pipeline time (median over iterations) and records a
baseline to docs/benchmarks.md.
- docs/benchmarks.md: recorded baseline numbers (indicative, machine-dependent).
- tests/test_performance.py: a benchmark smoke test (functions run, all metrics
positive, a generous non-flaky bound), a check that the recorded baseline
exists, and two tests confirming the Knowledge Base is loaded once and reused
(the loader caches; the pipeline never reloads per compilation).
Co-authored-by: kunail0804 <kunail0804@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Add golden tests for the ten reference scenes (MASTER_SPEC §26.7, §26.8),
guarding determinism across changes.
- tests/regression/golden_scenes.py: the ten reference scenes (single character,
two characters, interaction, indoor, outdoor, fantasy, modern, unknown
concepts, ambiguous concepts, complex multi-character) plus the shared no-LLM
pipeline runner.
- tests/regression/golden/*.json: the golden prompt outputs for each scene.
- scripts/regenerate_goldens.py: the deliberate, documented way to regenerate the
goldens when a change is intended to alter output (never automatic).
- tests/regression/test_golden.py: asserts exact equality against the goldens and
determinism across repeated runs.
- tests/regression/golden/README.md: documents regeneration and the regression
convention (every fixed bug adds a permanent test that is never removed).
Co-authored-by: kunail0804 <kunail0804@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Implement whole-Knowledge-Base validation per MASTER_SPEC §15.10, §26.4:
compiler/common/kb_validation.py (importable by the loader) plus a CLI at
scripts/validate_knowledge_base.py.
validate_knowledge_base(entries) collects every problem (never stops at the
first); each Message carries a code, the offending id, and a description:
- duplicate canonical id -> SC0005
- invalid category -> SC0008 (drawn from the shared 19-category constant)
- circular expansion (incl. self-expansion) -> SC0003, via DFS cycle detection
- duplicate alias, alias colliding with a canonical id (no chains), missing
expansion target, and per-entry schema nonconformance (empty tags, unknown
fields, ...) -> umbrella SC0004
Findings are deterministically ordered.
The CLI loads every *.json (array of entries) from a directory, reports invalid
JSON / non-array files as SC0004, writes JSON findings to stdout, and exits
non-zero when invalid.
Tests: a passing fixture plus a fixture per failure mode, multi-problem
reporting, determinism, and the CLI on valid/invalid/broken-JSON inputs.
Co-authored-by: kunail0804 <kunail0804@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Scaffold the Phase 0 foundation per MASTER_SPEC §24:
- Directory layout: nodes/, compiler/{analyzer,validator,resolver,splitter,
builder,common}/, knowledge_base/, schemas/, prompts/, config/,
tests/{analyzer,resolver,validator,integration,regression}/, examples/,
scripts/, docs/.
- pyproject.toml targeting Python 3.11+, jsonschema as the single runtime dep,
ruff + pytest as dev deps, explicit package list, and ruff/pytest config.
- ruff for linting and formatting; pytest with repo root on pythonpath.
- GitHub Actions CI running install, ruff lint, ruff format check, and pytest
on Python 3.11 and 3.12 for pushes and pull requests.
- docs/ skeleton and CONTRIBUTING.md referencing the wiki.
- Layout test asserting the §24 structure and package importability.
Co-authored-by: kunail0804 <kunail0804@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
