# Dreamverse Architecture

## Overview

Dreamverse currently has two main runtime pieces:

- `apps/dreamverse/web/`: Next.js frontend
- `apps/dreamverse/dreamverse/`: Python FastAPI runtime

Today, the browser talks directly to the Dreamverse runtime over HTTP and a
single websocket on `/ws`. The frontend owns UI state and interaction flow. The
server owns generation state, prompt rewrite, prompt safety, websocket session
semantics, and GPU-backed execution.

Near-term OSS note:

- `apps/dreamverse/dreamverse/` is the current runtime implementation.
- A future `controller/` layer is planned for local-only compute management and
  provider orchestration, but it does not exist yet.

## Repo Map

### Frontend

- `apps/dreamverse/web/src/app/page.tsx`: main client orchestration,
  websocket connect, init payloads, send paths, and top-level app behavior
- `apps/dreamverse/web/src/lib/ws/reducer.ts`: reduces normalized websocket
  events into client stores
- `apps/dreamverse/web/src/stores/session.ts`: connection, mode, and top-level
  session UI
- `apps/dreamverse/web/src/stores/promptWindow.ts`: editable prompt window and
  seed prompt UI state
- `apps/dreamverse/web/src/stores/rewrite.ts`: rewrite activity timeline and
  inspection state
- `apps/dreamverse/web/src/stores/stream.ts`: playback and stream-related
  client state
- `apps/dreamverse/web/src/lib/prompts/promptWindowSnapshot.ts`: prompt-window snapshot
  building for rewrite requests

### Server

- `apps/dreamverse/dreamverse/main.py`: websocket endpoint, request handling,
  session state machine, rewrite orchestration, REST routes, and stream relay
- `apps/dreamverse/dreamverse/gpu_pool.py`: GPU worker processes, warmup, model
  loading, and `generate_video()` calls through FastVideo
- `apps/dreamverse/dreamverse/prompt_enhancer.py`: prompt enhancement, rollout
  rewrite execution, provider selection, and timeout/fallback behavior
- `apps/dreamverse/dreamverse/rewrite_prompt_payload.py`: canonical rewrite request payload
  building
- `apps/dreamverse/dreamverse/config.py`: runtime flags, prompt file paths, provider settings, and
  warmup config
- `apps/dreamverse/dreamverse/session_init_image.py`: validates and persists uploaded initial
  images for segment 1

## Current Split Of Responsibility

### Frontend owns

- local UI state and client-side stores
- prompt drafts and prompt window editing
- websocket connection management
- deciding which user action to send:
  - `session_init_v2`
  - `project_init_v1`
  - `append_prompt`
  - `rewrite_seed_prompts`
  - `simple_generate`
- showing rewrite progress, stream status, prompt history, and devtools views

### Server owns

- websocket session lifecycle and protocol
- GPU assignment and worker lifecycle
- the authoritative seed prompt memory used for generation
- prompt rewrite execution and prompt safety
- actual generation queue semantics
- stream chunk emission and segment lifecycle events
- prompt config and preset persistence routes
- health and readiness endpoints

Important rule:

- The frontend may propose prompt-window state for rewrite, but the server is
  the source of truth for the rewritten rollout and the active prompt memory
  used for generation.

## End-To-End Flow

1. The frontend opens `/ws`.
2. The frontend sends `session_init_v2` with the initial prompt-window state,
   preset metadata, and current toggles.
3. The server validates init data, persists an optional initial image, acquires
   a GPU slot, and emits session status such as `gpu_assigned`.
4. The server starts or resumes project generation and emits events like
   `ltx2_stream_start`, `ltx2_segment_start`, media init/chunks, and completion
   events.
5. The frontend reduces those websocket events into its stores and updates the
   UI.
6. User actions such as appending prompts, rewriting seed prompts, or starting
   a single custom clip go back to the server over the same websocket.

## Frontend Architecture

The frontend is store-driven.

- `page.tsx` wires together websocket setup, send helpers, reducer
  application, and top-level interaction flows.
- Store modules separate concerns like session state, rewrite state, prompt
  window state, and stream state.
- The websocket reducer is responsible for turning normalized runtime events
  into store updates. If the server event schema changes, the reducer must
  change with it.

The frontend is intentionally not responsible for:

- generating rewritten prompts locally
- deciding final prompt safety outcomes
- reconstructing server session state from scratch
- inventing its own generation semantics independent of the runtime

## Server Architecture

The current runtime is a FastAPI app with a single long-lived websocket per
session.

`apps/dreamverse/dreamverse/main.py` manages:

- websocket connect/init
- prompt queues
- project and segment state
- prompt enhancement/rewrite triggers
- stream relay from GPU workers to the browser
- session logging and REST endpoints

`apps/dreamverse/dreamverse/gpu_pool.py` manages:

- model loading through FastVideo
- one or more worker processes
- startup warmup
- user join/leave commands
- `USER_STEP` execution for each segment
- continuation state between segments

`apps/dreamverse/dreamverse/prompt_enhancer.py` manages:

- prompt enhancement for user-submitted prompts
- rollout rewrite requests for the prompt window
- provider selection and fallback across configured prompt providers
- response normalization and safety-aware failure handling

## FastAPI Surface

The server is a single FastAPI application created in
`apps/dreamverse/dreamverse/main.py`.

Current built-in FastAPI docs are enabled:

- `/docs`: Swagger UI
- `/redoc`: ReDoc
- `/openapi.json`: OpenAPI schema

The runtime also mounts the frontend static build at `/` when one of the
configured frontend static directories exists. It does not expose the backend
Python package as static content.

## HTTP API

The current HTTP API is small. Most realtime behavior still goes through the
websocket.

### Core health and status routes

- `GET /healthz`
  - process liveness probe
  - returns a small payload with `status`, `service`, and timestamp
- `GET /readyz`
  - readiness probe
  - returns `503` until prompt services are initialized and at least one GPU
    worker is ready
  - returns readiness and GPU pool summary fields such as ready workers, total
    GPUs, warmup counts, and queue size
- `GET /status`
  - returns the current GPU pool status payload from `gpu_pool`
- `GET /internal/monitor/sessions`
  - internal monitoring payload for session dashboards
  - includes pending session count, max available sessions, prompt provider
    success counts, and timestamp

### Prompt config routes

- `GET /prompt-system-config`
  - returns the editable prompt-system configuration currently loaded by
    `PromptEnhancer`
- `POST /prompt-system-config`
  - saves prompt-system configuration to disk and reloads prompt config in the
    runtime
  - current editable fields include:
    - next-segment system prompt
    - auto-extension system prompt
    - rewrite-window system prompt
    - rewrite-user system prompt
    - rewrite model
    - rewrite temperature

### Devtools-only preset routes

These exist only when `DEVTOOLS_ENABLED` is true in
`apps/dreamverse/dreamverse/config.py`.

- `GET /curated-presets`
  - returns merged curated presets, applying the local overlay file on top of
    the fallback file when both exist
- `POST /curated-presets/append`
  - appends a new curated preset to the overlay presets file
  - validates non-empty label, normalized id, and at least two non-empty
    segment prompts

## Websocket API

`WS /ws` is the main runtime API.

The websocket owns:

- session init
- project init and reset
- prompt append
- prompt rewrite
- generation toggles
- segment lifecycle events
- media stream delivery
- runtime error delivery

The websocket is the authoritative API for realtime Dreamverse behavior. The
HTTP routes mainly support health checks, devtools persistence, and monitoring.

## Prompt Rewrite Architecture

Prompt rewrite is a shared flow with strict ownership boundaries.

### Frontend responsibilities

- collect the rewrite instruction
- build the prompt-window snapshot from current client state
- send `rewrite_seed_prompts`
- show rewrite progress, raw output, fallback state, and resulting prompt list

### Server responsibilities

- validate and normalize the prompt-window payload
- choose rewrite model, system prompt, timeout, and temperature
- build the canonical prompt payload in
  `apps/dreamverse/dreamverse/rewrite_prompt_payload.py`
- execute rewrite through `PromptEnhancer`
- apply safety filtering to rewritten prompts
- replace the authoritative seed prompt memory when rewrite succeeds
- emit `seed_prompts_updated` and `rewrite_seed_prompts_complete`

Important rule:

- The frontend owns editable drafts.
- The server owns the accepted rollout.

After a successful rewrite, the frontend should replace its prompt-window view
from the server payload instead of preserving a locally-derived version.

## Prompt Modes

There are three related prompt paths in the current system:

### Initial rollout

- The frontend sends seed prompts during `session_init_v2`.
- The server uses those prompts as the initial seed prompt memory.
- If the rollout starts from an empty prompt window plus an initial rewrite
  instruction, the server can pause generation until rewrite completes.

### Live append

- The frontend sends `append_prompt`.
- The server may enhance that prompt, safety-check it, enqueue it, and use it
  as the next generated segment.

### Rewrite

- The frontend sends `rewrite_seed_prompts`.
- The server rewrites the entire seed prompt window or generates a new rollout,
  depending on the payload and current state.

## Initial Image And Segment Handling

The frontend currently sends `initial_image` as part of session init or
`simple_generate`.

The server:

- validates and persists the image
- uses it only for segment 1 when present
- keeps continuation state for later segments in the GPU worker

This means the runtime, not the frontend, decides how segment 1 image
conditioning and later continuation conditioning are applied.

## Websocket Contract

The websocket is the main integration surface between UI and runtime.

Typical incoming messages from the frontend:

- `session_init_v2`
- `project_init_v1`
- `append_prompt`
- `rewrite_seed_prompts`
- `simple_generate`
- `set_enhancement`
- `set_auto_extension`
- `set_loop_generation`

Typical outgoing messages from the server:

- `gpu_assigned`
- `ltx2_stream_start`
- `ltx2_segment_start`
- `segment_prompt_source`
- `prompt_received`
- `prompt_ready`
- `prompt_enhancing`
- `seed_prompts_updated`
- `rewrite_seed_prompts_complete`
- `media_init`
- `media_segment_complete`
- `project_idle`
- `error`

Binary websocket frames carry media chunks for playback.

## Current And Planned Architecture

Current architecture:

- browser -> `apps/dreamverse/web`
- `apps/dreamverse/web` -> `apps/dreamverse/dreamverse/main.py`
- `apps/dreamverse/dreamverse/main.py` ->
  `apps/dreamverse/dreamverse/gpu_pool.py`
- `gpu_pool.py` -> FastVideo runtime

Planned architecture:

- browser -> `apps/dreamverse/web`
- `apps/dreamverse/web` -> local `controller/`
- `controller/` -> local or remote Dreamverse runtime
- runtime -> FastVideo runtime

That future controller split should not move prompt rewrite, session state, or
generation semantics out of the runtime.