DumiFlex
49f882f131
`drift-store.ts` polls /wp/api/modules/hashes and /wp/api/bundles/hashes every 5 seconds. Both the SPA and the in-graph WP_Context subscribe, so a ComfyUI tab with a Context node on the graph polls these whether or not the manager is ever opened. Measured against a live 2.13.1 install on 2026-08-06: 33,008 bytes per poll, no ETag, no Cache-Control, no Last-Modified, and 0 of 39 observed requests served from cache. That is ~24 MB/hour per open tab on an idle page. Invisible on localhost; over Tailscale it was the largest thing this extension does. Producing it was not cheap either, despite the docstring's "lightweight ... without measurable cost" -- that described the response, not the work. `ModuleRepository.list()` is `SELECT *` plus a json.loads of every payload in the library, and the handler kept three fields per row and discarded the rest. Two changes: - Conditional responses. `json_ok_revalidated` emits an ETag plus `Cache-Control: no-cache` -- store it, but revalidate -- and returns a bodied 304 when `If-None-Match` matches. Nothing changes for the client: the browser serves the cached body as a 200, so `fetch().then(r => r.json())` is untouched and the saving is entirely on the wire. - A cheap stamp behind the ETag. `library_stamp()` is COUNT/SUM(version)/MAX(updated_at) -- one aggregate row, no payloads. The expensive map is built only when the stamp moves, which is the whole point: computing an ETag from the response body would have done all the work anyway and then thrown it away. Also narrows the map builders. `list_hash_rows` selects the columns the drift map needs instead of every column via `_row_to_module`/`_row_to_bundle`, which parse `tags` (and `children`, for bundles) per row for a map that wants neither. Bundles get the bigger win -- `payload_hash` is a stored column there, so that path now parses nothing at all. Modules still parse `payload` because their hash is computed rather than stored; removing that needs a schema migration, and the stamp makes it moot in the steady state. Stamp precision, both directions verified: `set_community_origin` writes without a version bump and cannot change the map, so the ETag correctly holds. A favourite toggle goes through `update()`, whose single statement always does `version = version + 1`, so it does move the ETag and costs one refetch of an identical map. Accepted rather than fixed -- a rare user action against a second counter tracking payload-affecting columns only -- and pinned by a test that says so, and says a future failure would be an improvement.
128 lines
4.3 KiB
Python
128 lines
4.3 KiB
Python
"""Shared helpers for wp_api route handlers."""
|
|
from __future__ import annotations
|
|
|
|
import hashlib
|
|
from contextlib import contextmanager
|
|
from typing import Any
|
|
|
|
from aiohttp import web
|
|
|
|
from engine.db.connection import get_connection
|
|
from engine.syntax.tokenize import tokenize_text
|
|
from engine.syntax.types import TokenKind
|
|
|
|
|
|
@contextmanager
|
|
def db_session(request: web.Request):
|
|
"""Yield a SQLite connection that is auto-closed on exit.
|
|
|
|
If the aiohttp app has a ``wp_db`` key (injected by tests or the
|
|
application startup), that connection is yielded directly and is NOT
|
|
closed on exit (lifetime owned by the injector). Otherwise, a fresh
|
|
file-based connection is opened via ``get_connection()`` and closed
|
|
when the context manager exits.
|
|
|
|
SQLite WAL mode + per-request connections avoid cross-task locking.
|
|
"""
|
|
injected = request.app.get("wp_db") if hasattr(request, "app") else None
|
|
if injected is not None:
|
|
yield injected
|
|
return
|
|
conn = get_connection()
|
|
try:
|
|
yield conn
|
|
finally:
|
|
conn.close()
|
|
|
|
|
|
def json_ok(data: Any, *, status: int = 200) -> web.Response:
|
|
return web.json_response(data, status=status)
|
|
|
|
|
|
def json_error(message: str, *, status: int) -> web.Response:
|
|
return web.json_response({"error": message}, status=status)
|
|
|
|
|
|
def make_etag(*parts: object) -> str:
|
|
"""A strong ETag from any values that identify a response's content.
|
|
|
|
Callers pass a cheap fingerprint (row counts, version sums) rather than
|
|
the response body, so the body never has to be built to find out whether
|
|
it changed.
|
|
"""
|
|
raw = "\x1f".join(str(p) for p in parts)
|
|
return '"' + hashlib.sha256(raw.encode("utf-8")).hexdigest()[:32] + '"'
|
|
|
|
|
|
def matches_if_none_match(request: web.Request, etag: str) -> bool:
|
|
"""Whether the client already holds this exact representation.
|
|
|
|
`If-None-Match` is a comma-separated list and each entry may carry the
|
|
`W/` weak prefix, which is compared away — we only ever emit one tag, so
|
|
weak and strong comparison agree.
|
|
"""
|
|
header = request.headers.get("If-None-Match")
|
|
if not header:
|
|
return False
|
|
if header.strip() == "*":
|
|
return True
|
|
wanted = etag.strip()
|
|
for candidate in header.split(","):
|
|
value = candidate.strip()
|
|
if value.startswith("W/"):
|
|
value = value[2:]
|
|
if value == wanted:
|
|
return True
|
|
return False
|
|
|
|
|
|
def json_ok_revalidated(
|
|
request: web.Request, data: Any, *, etag: str,
|
|
) -> web.Response:
|
|
"""`json_ok` plus conditional-request support.
|
|
|
|
`Cache-Control: no-cache` means "store it, but revalidate before reuse" —
|
|
NOT "don't store it". That is exactly what a poll wants: the browser keeps
|
|
the body and sends `If-None-Match` on the next tick, and an unchanged
|
|
library answers 304 with no body at all.
|
|
|
|
Nothing changes for the caller in JavaScript. The browser serves the
|
|
cached body with a 200 on a 304, so `fetch(...).then(r => r.json())` is
|
|
untouched — the saving is entirely on the wire.
|
|
"""
|
|
headers = {"ETag": etag, "Cache-Control": "no-cache"}
|
|
if matches_if_none_match(request, etag):
|
|
# A 304 carries no body by definition; aiohttp enforces this.
|
|
return web.Response(status=304, headers=headers)
|
|
response = web.json_response(data)
|
|
response.headers.update(headers)
|
|
return response
|
|
|
|
|
|
def extract_referenced_uuids(payload: Any) -> set[str]:
|
|
"""Recursively scan strings inside `payload` for `@{8hex}` refs and
|
|
return the set of unique uuids referenced.
|
|
|
|
Used by the SPA Test Runner to determine which wildcards must be
|
|
loaded from the library before resolution can proceed (lazy catalog,
|
|
spec §2.10). Reuses the engine tokenizer so the regex shape stays
|
|
locked in one place."""
|
|
refs: set[str] = set()
|
|
|
|
def _scan(value: Any) -> None:
|
|
if isinstance(value, str):
|
|
for tok in tokenize_text(value):
|
|
if tok.kind == TokenKind.REF:
|
|
uuid = (tok.meta or {}).get("uuid")
|
|
if isinstance(uuid, str):
|
|
refs.add(uuid)
|
|
elif isinstance(value, dict):
|
|
for v in value.values():
|
|
_scan(v)
|
|
elif isinstance(value, list):
|
|
for item in value:
|
|
_scan(item)
|
|
|
|
_scan(payload)
|
|
return refs
|