SolitaryThinker
6220d02746
The last audit stress test. A cheap draft model proposes K tokens, the target verifies them in one batched step, and SpeculativeARLoop accepts the matching prefix + one target correction — a variable accepted-length per round (a ragged AR loop the model owns). - Exactness: the emitted sequence equals the target's OWN greedy decode for any draft quality (every accepted token is one the target would produce; the correction is the target's token) — the speedup is free. - Speedup scales with accept rate: draft-agree 0.3→1x, 0.7→3x, 1.0→4x=K tokens/round (fewer verify_rounds, the expensive model's latency steps, for the same output). - Two components (draft + target) co-scheduled on one resident instance; each round an AR_TOKEN WorkUnit. models/speculative/ (loop+card+program); backend ToyTargetModel/ToyDraftModel with a shared length-dependent target formula (no degenerate fixed point). 5 tests; full suite 153 passed. designv4 §9.16 + counts (153 tests, 26 files).
9 lines
359 B
Python
9 lines
359 B
Python
"""Speculative (draft-verify) decoding: a draft + target AR pair, exact + lower-latency (design_v3 §9.16)."""
|
|
from __future__ import annotations
|
|
|
|
from .card import build_speculative_card
|
|
from .loop import SpeculativeARLoop
|
|
from .program import build_speculative_program
|
|
|
|
__all__ = ["build_speculative_card", "build_speculative_program", "SpeculativeARLoop"]
|