SolitaryThinker
32a7a6b87b
Targets the three design_v3 claims that were most load-bearing AND least exercised (sessions/realtime, training-plane boundary, the WorkUnit-generality falsifier). All fit with no new runtime primitive (127 tests pass). 1. Interactive world-model session (runtime/session.py, §9.8) — the Session plane had ZERO coverage. WorldModelSession drives the causal chunk_rollout loop as a long-lived session: persistent cross-request world state on the Session.kv_handle, frame streaming, transactional step-boundary cancellation (a cancelled act leaves the world resumable), no cross-session smearing. Added only a continuation seam to the chunk loop (init seeds context from a world_context slot; default empty = unchanged one-shot path). 5 tests. 2. End-to-end RL over a cross-model workflow (training/methods/workflow_rl.py, §9.9) — trains BOTH flux-t2i and wan-i2v from ONE final-video reward. Rolls out the whole workflow with SDE capture in both instances; the same final advantage drives FlowGRPO PPO on each stage's transformer; two WeightSyncPlans on two instances. The earlier model (T2I) is trained by a reward on the final video — end-to-end credit across a model boundary — proven causal by a control (constant reward => zero advantage => nothing moves). 4 tests. 3. Heterogeneous WorkUnit co-scheduling (models/tiled/, §9.10) — the §17 falsifier. VAETileLoop makes VAE decode a loop of VAE_TILE units; tiling is exact (== one-shot, C0), and VAE_TILE + DIFFUSION_STEP pipelines interleave bit-identically and co-run in one batch. Validates the mechanism; the economic half (does it pay) stays a port-time measurement. 4 tests. designv4 §9.8–§9.10 + falsifier/layout/closing updates (127 tests, 20 files).
84 lines
4.3 KiB
Python
84 lines
4.3 KiB
Python
"""WorldModelSession — a long-lived interactive world-model rollout (design_v3 §16, §12, §15e).
|
|
|
|
The realtime/interactive plane, made concrete. A one-shot ``engine.run`` generates a clip and forgets;
|
|
a *session* continues the world: each ``.act(action)`` resumes the causal ``chunk_rollout`` loop from
|
|
the **persistent world state** carried on the ``Session`` (its ``kv_handle``), streams frames as chunks
|
|
complete, and honors **step-boundary cancellation**. This exercises the seams a one-shot path never
|
|
touches — cross-request persistent context, the ``CHUNK_ROLLOUT`` loop driven incrementally, the
|
|
``Stream`` frame channel, and ``CancelScope`` — using the exact runtime primitives the engine uses
|
|
(``RuntimeLoopContext`` + ``LoopRunner``), so interleaving two sessions is still parity-safe.
|
|
|
|
Cancellation is transactional: the persistent world state advances only on a *completed* act, so a
|
|
cancelled rollout leaves the session resumable from the last good frame.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
from typing import Any, Callable
|
|
|
|
from .._enums import ExecutionProfile
|
|
from ..loop.driver import LoopRunner
|
|
from ..models.common import text_encode_node_fn
|
|
from ..request import DiffusionParams, TaskType, make_request
|
|
from ..request.requests import Session
|
|
from ..request.streams import Stream
|
|
from .context import RuntimeLoopContext
|
|
|
|
|
|
class WorldModelSession:
|
|
def __init__(self, engine: Any, model_id: str, *, session_id: str = "world",
|
|
window: int = 2, loop_id: str = "chunk_rollout"):
|
|
self.engine = engine
|
|
self.model_id = model_id
|
|
self.window = window # how many prior chunks of world state to carry forward
|
|
self.loop_id = loop_id
|
|
self.session = Session(session_id=session_id)
|
|
self.stream = Stream(session_id)
|
|
self.session.streams["video"] = self.stream
|
|
self.world_context: list = [] # the persistent world chunks (the §16 cross-request KV)
|
|
self.session.kv_handle = self.world_context
|
|
self.step = 0
|
|
|
|
@property
|
|
def instance(self) -> Any:
|
|
if self.model_id not in self.engine._registry:
|
|
raise KeyError(f"no model {self.model_id!r} registered on engine")
|
|
return self.engine._registry[self.model_id][0]
|
|
|
|
def act(self, action: str, *, seed: int | None = None,
|
|
step_hook: Callable[["WorldModelSession", int], None] | None = None) -> Any:
|
|
"""Advance the world by one action, continuing from persistent context; stream frames.
|
|
|
|
Raises ``request.Cancelled`` if cancelled at a step boundary — and because the world state is
|
|
only committed after the loop completes, a cancelled act leaves the session resumable."""
|
|
inst = self.instance
|
|
req = make_request(TaskType.V2W, self.model_id, action,
|
|
diffusion=DiffusionParams(seed=self.step if seed is None else seed))
|
|
slots: dict[str, Any] = {}
|
|
text_encode_node_fn(inst, slots, req, None)
|
|
slots["world_context"] = list(self.world_context) # thread the persistent state in
|
|
ctx = RuntimeLoopContext(
|
|
inst, observers=self.engine.observers, interceptors=self.engine.interceptors,
|
|
slots=slots, stream=self.stream, cancel_scope=self.session.cancel_scope,
|
|
profile=ExecutionProfile.SERVE, metrics={},
|
|
request_id=f"{self.session.session_id}.{self.step}")
|
|
runner = LoopRunner(inst.loop(self.loop_id), ctx, req, inst)
|
|
n = 0
|
|
while not runner.done:
|
|
if step_hook is not None:
|
|
step_hook(self, n) # test seam: may call self.cancel() at a boundary
|
|
runner.step() # peek() raises request.Cancelled if cancelled
|
|
n += 1
|
|
result = runner.result
|
|
new_chunks = list(result.outputs.get("chunks") or [])
|
|
self.world_context = (self.world_context + new_chunks)[-self.window:] # slide window; commit
|
|
self.session.kv_handle = self.world_context
|
|
self.step += 1
|
|
return result
|
|
|
|
def frames(self) -> list:
|
|
"""All streamed media chunks so far across the session (the realtime frame channel)."""
|
|
return [e for e in self.stream.events if e.type == "media.chunk"]
|
|
|
|
def cancel(self) -> None:
|
|
self.session.cancel_scope.cancel()
|