SolitaryThinker
270fae959d
A scoped, CPU-testable realization of design_v3.md — the architecture where the atomic unit is a typed (recipe, runtime) ModelCard, the model owns loop semantics while the runtime owns loop lifecycle, one resident instance runs many loops, and training records behavior on the same loops it serves. Implements: - card/ loop/ runtime/ cache/ memory/ parallel/ parity/ extend/ program/ request/ training/ spanning design_v3 §4-§13: ModelCard + validate(); driven loops (init/next/advance/finalize); step-interleaving Engine with reservation-before-admission + per-class caches keyed by CacheKey; the C0-C4 consistency ladder + the non-negotiable batch-of-N interleave parity gate. - Inference: Wan2.1-1.3B (T2V), LTX2.3 (two-stage distilled, shared transformer), Wan-causal (chunk rollout + slab-KV streaming). - Training (Wan2.1-1.3B), each driving the SAME loops the engine serves: finetune (flow-match), DMD2 (teacher/critic distribution matching), DiffusionNFT (likelihood-free C2, samples from the decay-blended old policy, group-relative advantages, shared-prompt cache reuse), self-forcing (causal chunk loop). The engine never imports training (the §10 dependency rule, grep-verified). numpy-only core (no torch/GPU here); heavy Wan/LTX forwards are deterministic toy stand-ins with lazy torch-adapter seams (ComponentSpec.load_id/factory) for a GPU box. 49 tests pass via pytest and a zero-dependency runner; interleave parity verified (and a buggy module-global interceptor provably breaks it). Omni-ready spine (ar_decode/chunk_step loop kinds, multi-loop instances, LoopState.extension) for the phase-2 Cosmos3 + vllm-omni omni ports. Run: python3 -m pytest mini_fastvideo/tests/ -q ; python3 -m mini_fastvideo.examples
86 lines
3.0 KiB
Python
86 lines
3.0 KiB
Python
"""CacheKey — cache correctness is a contract, not hope (design_v3 §7.1).
|
||
|
||
> If a field can change output semantics, it is in the key. Incorrect reuse is worse
|
||
> than no reuse.
|
||
|
||
The serving hazard this kills: a workflow-cloud request that shares a prompt but
|
||
differs in te-LoRA stack must not serve stale embeddings — so the key is *partitioned*
|
||
by ``adapter_versions``, not flushed. An RL ``update_weights`` bumps ``weights_version``
|
||
and invalidates wholesale.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import hashlib
|
||
from dataclasses import dataclass, field
|
||
from typing import Any
|
||
|
||
|
||
def content_hash(obj: Any) -> str:
|
||
"""Stable content hash for feature-cache keys (text/vision embeddings, §7.2).
|
||
|
||
Lets K rollout samples of one prompt reuse a single text encode (design_v3 §10:
|
||
24× text-encode reduction).
|
||
"""
|
||
h = hashlib.sha256()
|
||
if isinstance(obj, str):
|
||
h.update(obj.encode("utf-8"))
|
||
elif isinstance(obj, (bytes, bytearray)):
|
||
h.update(obj)
|
||
elif hasattr(obj, "tobytes") and hasattr(obj, "shape"): # numpy / torch tensor
|
||
h.update(str(getattr(obj, "shape", "")).encode())
|
||
h.update(str(getattr(obj, "dtype", "")).encode())
|
||
try:
|
||
h.update(obj.tobytes())
|
||
except Exception:
|
||
h.update(repr(obj).encode())
|
||
else:
|
||
h.update(repr(obj).encode())
|
||
return h.hexdigest()[:32]
|
||
|
||
|
||
@dataclass(frozen=True)
|
||
class CacheKey:
|
||
model_id: str
|
||
component_id: str
|
||
loop_id: str | None = None
|
||
weights_version: str = "v0"
|
||
adapter_versions: tuple[tuple[str, str], ...] = () # sorted (adapter_id, version) pairs
|
||
precision: str = "float32"
|
||
parallel_plan_hash: str = ""
|
||
shape_sig: str = ""
|
||
layout_sig: str = ""
|
||
scheduler_sig: str | None = None
|
||
guidance_sig: str | None = None
|
||
seed: int | None = None
|
||
input_hashes: tuple[tuple[str, str], ...] = ()
|
||
step_index: int | None = None
|
||
contract_version: str = "v0"
|
||
|
||
@property
|
||
def hash(self) -> str:
|
||
return hashlib.sha256(repr(self).encode()).hexdigest()[:24]
|
||
|
||
def partition_field(self) -> tuple:
|
||
"""Fields that *partition* (not flush) a feature cache: adapters + weights."""
|
||
return (self.weights_version, self.adapter_versions)
|
||
|
||
@staticmethod
|
||
def adapters(d: dict[str, str] | None) -> tuple[tuple[str, str], ...]:
|
||
return tuple(sorted((d or {}).items()))
|
||
|
||
@staticmethod
|
||
def hashes(d: dict[str, str] | None) -> tuple[tuple[str, str], ...]:
|
||
return tuple(sorted((d or {}).items()))
|
||
|
||
|
||
@dataclass
|
||
class CachePolicy:
|
||
"""Runtime config for one cache class pool (design_v3 §7.2; designv2 cache plane)."""
|
||
class_name: str # "feature" | "residual" | "slab_kv" | "paged_kv"
|
||
max_bytes: int = 1 << 30
|
||
block_bytes: int = 1 << 16
|
||
eviction: str = "lru" # "lru" | "fifo" | "none"
|
||
reuse_across_requests: bool = True
|
||
per_component: dict[str, int] = field(default_factory=dict)
|
||
training_mode_disables_recycle: bool = False
|