[docs] dreamverse-integration: D-18 — Option B+ monorepo plan

User decision after reviewing integration-review.md's Option D
recommendation. Combines Option D's "generic backend stays generic"
principle with Option B's monorepo subfolder layout for the
Dreamverse product. Result: single repo (FastVideo), Dreamverse
becomes apps/dreamverse/, generic backend stays at
fastvideo.entrypoints.streaming.*, Dreamverse repo gets archived
after migration.

Methodology: 3 parallel exploration agents (FastVideo build/CI
surface, Dreamverse product-vs-generic file split, Python+Next.js
monorepo precedents). Synthesis authored by ultrabrain category.
Oracle review applied as PASS-WITH-FIXES; 10 critical issues
addressed (import contract too tight, Phase 2 missing shim list,
prompt_enhancer misclassified, frontend CI Playwright/pnpm setup
broken, .gitignore product-asset issue, uv.lock missing, backend CI
path filters too narrow, cross-repo history claim wrong, Phase 6
too coarse, missing security/CORS risks).

* integration-plan.md (new, 1205 lines): executable 7-phase
  migration plan with file-by-file map, tooling stack (uv workspace
  + standalone pnpm), pyproject.toml/pre-commit/.gitignore/CI YAML
  diffs, risk register, verification gates, rollback. Phase 0 lands
  #1288; Phases 1-7 add skeleton, move backend, move FE, promote
  generic-pending, retire prompt enhancer fork (DR-1), CI/release
  cutover (split into 6a-6f), archive Dreamverse repo.

* integration-review.md: DEPRECATED banner added at top pointing to
  integration-plan.md. Body kept for drift-audit and OSS precedent
  reference (still authoritative). Reading-guide entry updated.

* decisions-log.md: D-18 entry capturing strategy decision,
  rationale (drops cross-repo coordination overhead from D-17 cycle;
  preserves architectural separation; OSS precedents support shape),
  why not A/B/C/D (concrete reasons for each), implications (apps/
  exclude in setuptools, no cross-repo history preservation, drift
  items fold into phases), and 4 deferred decisions (DR-2, VPO,
  history-import method, CORS/write-endpoint security).

* README.md: Last-reconciled bumped with D-18 reference. Reading-
  guide table updated: integration-plan.md is CURRENT;
  integration-review.md marked DEPRECATED.

Verified: pre-commit clean (memory dir excluded from yapf/ruff/mypy;
spaces-check passes). Oracle PASS-WITH-FIXES, all 10 critical fixes
applied (import contract relaxed to allow fastvideo.{api,configs,
entrypoints.streaming,entrypoints.video_generator}; Phase 2 lists 6
explicit shims for generic-pending; Phase 5 explicitly retires the
prompt fork; ci-dreamverse-frontend.yml uses pnpm/action-setup BEFORE
setup-node; Playwright deferred to Phase 4 until health routes land;
.gitignore unignores product assets; Phase 1 commits uv.lock with
--locked enforcement; backend CI watches fastvideo/api + streaming +
video_generator + configs; cross-repo history disposition documented;
Phase 6 split into 6a-6f substeps with independent verification
gates; CORS/write-endpoint risks added to register).

PR #1288: https://github.com/hao-ai-lab/FastVideo/pull/1288

Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
This commit is contained in:
SolitaryThinker
2026-05-05 10:37:11 -07:00
co-authored by Junda Su Matthew Noto XOR-op Zhang Peiyuan
parent 907620ff22
commit 1e66c74aec
4 changed files with 1300 additions and 10 deletions
@@ -5,16 +5,21 @@ Tracks the public API refactor (PRs 0-17), the LTX-2 streaming server
upstream, the Dreamverse switch from `FastVideo-internal` to public
`FastVideo`, and the NVFP4 quantization landing.
**Last reconciled:** 2026-05-05 (strategy reversal — single mega-PR).
FastVideo `will/ltx2_sr_port` @ `b36bdbc9` (36 commits ahead of main, post-D-17 STACK.md removal + integration-review.md addition).
Dreamverse `will/integrate-public-fastvideo` @ `ec8ef92`.
**Last reconciled:** 2026-05-05 (**D-18**: Option B+ chosen — Dreamverse
becomes `apps/dreamverse/` subfolder under FastVideo; generic backend stays
at `fastvideo.entrypoints.streaming.*`. See [integration-plan.md](integration-plan.md)
for the executable 7-phase migration plan;
[integration-review.md](integration-review.md) is **deprecated** but kept
for the drift audit and OSS precedent citations.).
FastVideo `will/ltx2_sr_port` @ HEAD (post-D-17 STACK.md removal +
integration-review.md addition + integration-plan.md addition + D-18
reconciliation). Dreamverse `will/integrate-public-fastvideo` @ `ec8ef92`.
PRs #1257 / #1258 / #1284 / #1286 MERGED to main. **PR #1287 CLOSED
(in favor of consolidation); PR #1288 OPEN as the single mega-PR
landing the entire `will/ltx2_sr_port` chain at once** (LTX-2 SR
runtime + NVFP4 + `generate_async`/Dynamo contract + agents memory dir,
71 files, +13,074/-583 LOC). Split branches kept as historical
bookmarks; STACK.md model is **abandoned** — see [decisions-log.md
D-17](decisions-log.md#d-17). Local backup
runtime + NVFP4 + `generate_async`/Dynamo contract + agents memory dir).
Split branches kept as historical bookmarks; STACK.md model **abandoned** —
see [decisions-log.md D-17](decisions-log.md#d-17). Local backup
`will/ltx2_sr_port-pre-1286-rebase` @ `1baa60bb` preserves the
pre-rebase chain.
@@ -68,7 +73,8 @@ file it in [open-threads.md](open-threads.md) before continuing.
| "Why was decision X made? What's resolved vs. open?" | [decisions-log.md](decisions-log.md) |
| "What should I work on next? Priority order?" | [open-threads.md](open-threads.md) |
| "Who should be co-authored on commits in this scope?" | [authors.md](authors.md) |
| "Have we drifted from the goal? Should Dreamverse merge into FastVideo?" | [integration-review.md](integration-review.md) |
| "How do we execute the Dreamverse → FastVideo monorepo merge?" | [integration-plan.md](integration-plan.md) ← **CURRENT** |
| "Historical drift audit + Option-D evaluation (deprecated by D-18)" | [integration-review.md](integration-review.md) (DEPRECATED) |
## Repo + worktree paths
@@ -6,7 +6,7 @@ question/decision, rationale, current status.
For implementation status see [pr-roadmap.md](pr-roadmap.md). For
follow-up actions see [open-threads.md](open-threads.md).
**Last updated:** 2026-05-05 (added D-12 — GpuPool layer separation, Oracle review post-#1257-merge; added D-13 — prompt enhancer / LLMProvider abstraction shape, Oracle review pre-#1258-merge; added D-14 — streaming auxiliaries cohesion, Oracle review during #1284 review cycle; added D-15 — streaming router placement + sticky/active-active deferral, Oracle review during #1286 review cycle; added D-16 — streaming router polish round 2, second-pass review on top of D-15 covering bridge cancellation hygiene, registry state machine, httpx hard-fail, replica YAML parsing, and `websockets` dep; added D-17 — strategy reversal: abandon 6-PR split in favor of single mega-PR #1288 on `will/ltx2_sr_port`).
**Last updated:** 2026-05-05 (added D-12 — GpuPool layer separation, Oracle review post-#1257-merge; added D-13 — prompt enhancer / LLMProvider abstraction shape, Oracle review pre-#1258-merge; added D-14 — streaming auxiliaries cohesion, Oracle review during #1284 review cycle; added D-15 — streaming router placement + sticky/active-active deferral, Oracle review during #1286 review cycle; added D-16 — streaming router polish round 2, second-pass review on top of D-15 covering bridge cancellation hygiene, registry state machine, httpx hard-fail, replica YAML parsing, and `websockets` dep; added D-17 — strategy reversal: abandon 6-PR split in favor of single mega-PR #1288 on `will/ltx2_sr_port`; added D-18 — Option B+ chosen: Dreamverse FE+product-server move into FastVideo as `apps/dreamverse/` subfolder while generic backend stays at `fastvideo.entrypoints.streaming.*`; integration-review.md deprecated, integration-plan.md is the executable migration plan).
## Status legend
@@ -16,6 +16,60 @@ follow-up actions see [open-threads.md](open-threads.md).
## Post-merge architecture decisions
### D-18: Option B+ — Dreamverse becomes `apps/dreamverse/` subfolder under FastVideo
**Status:** ✅ Resolved 2026-05-05. [integration-plan.md](integration-plan.md) is the executable migration plan; [integration-review.md](integration-review.md) is deprecated but kept for drift audit + OSS precedents.
**Source:** User decision after reviewing [integration-review.md](integration-review.md)'s Option D recommendation.
**Question:** [integration-review.md](integration-review.md) recommended **Option D** — Dreamverse stays a separate repo, generic backend (streaming runtime, GPU pool, prompt enhancer, router) merges into `fastvideo.entrypoints.streaming.*`. The user reviewed this and chose a different shape: keep the generic-backend principle from Option D but ALSO move the Dreamverse FE + product server into FastVideo as a subfolder (`apps/dreamverse/`). Combination is "Option B+" (Option B layout with Option D's backend principle).
**Decision:** Option B+. Concrete shape:
- **One repo**: `hao-ai-lab/FastVideo`. Dreamverse repo gets archived after migration completes.
- **Python ML library** stays at root: `fastvideo/`, `fastvideo-kernel/`.
- **Generic backend** stays at `fastvideo.entrypoints.streaming.*` (already there per #1257/#1258/#1284/#1286/#1288).
- **Dreamverse product** moves into `apps/dreamverse/{server,web,prompts,serve_configs,scripts}/`.
- **Tooling**: uv workspace for Python (`[tool.uv.workspace] members = ["apps/dreamverse/server"]`), standalone pnpm for the FE (no root `package.json`), split CI workflows with path-filter triggers.
**Rationale:**
- Drops the cross-repo coordination overhead identified in the post-#1286 rebase cycle (D-17 handled by consolidating into mega-PR; D-18 prevents the next round of cross-repo coordination from happening).
- Keeps the architectural separation Option D recommended (FastVideo owns reusable runtime; product owns product). The boundary is now `apps/dreamverse/` directory rather than two repos.
- Single repo means atomic cross-cutting refactors (e.g. GpuPool API change + Dreamverse adoption) ship as one PR.
- OSS precedents support the shape (chainlit uv-workspace + pnpm; open-webui Python + Svelte with paths-ignore CI). The librarian explicitly noted no precedent for "Python ML library + Next.js product merged into library namespace" — but this isn't that pattern. Dreamverse goes into a sibling directory, NOT into `fastvideo.entrypoints.dreamverse.*`. Library namespace stays clean.
**Why not Option D (separate repos):**
- Each upstream merge into FastVideo invalidates Dreamverse's lockfile/imports; the post-#1286 rebase showed this requires coordination overhead that scales with feature velocity.
- Cross-repo contract tests catch shape drift but not behavior drift.
- Two repos means two `AGENTS.md`, two CI configs, two release stories, two Dependabot dashboards.
**Why not Option C (full merge into `fastvideo.entrypoints.dreamverse.*`):**
- Forces FastVideo to ship Tailwind config + curated preset JSON + Next.js build artifacts.
- Locks Dreamverse product cadence to FastVideo PyPI releases.
- Librarian: "no 1:1 precedent for Python ML library + Next.js product merged into library namespace" — argues against this.
**Why not Option B (subfolder, but generic backend folded into `apps/dreamverse/server/`):**
- Other consumers (Dynamo, future streaming clients) need the backend without the Dreamverse product. Folding the backend under `apps/dreamverse/server/` would force Dynamo to either depend on `apps/` paths (ugly) or carry a fork.
**Implications:**
- [integration-review.md](integration-review.md) is **deprecated** (banner header + reading-guide demotion). Kept in tree for drift audit + OSS precedent reference.
- [integration-plan.md](integration-plan.md) is the **canonical executable plan** with 7 phases (Phase 0: land #1288; Phase 1: skeleton + tooling; Phase 2: backend move; Phase 3: FE move; Phase 4: promote generic-pending; Phase 5: prompt enhancer fork retirement; Phase 6: CI/release cutover; Phase 7: archive Dreamverse repo).
- Dreamverse repo will be **archived** at end of Phase 7 — not before.
- Dreamverse history does NOT migrate cross-repo via `git mv` (technical limitation); original history stays in archived Dreamverse repo, and Phase 2 PR body records the source SHA(s).
- New top-level `apps/` directory created — must be excluded from FastVideo PyPI wheel via `[tool.setuptools.packages.find] exclude = ["apps*", ...]`.
- Drift items from [integration-review.md](integration-review.md) get folded into specific phases of [integration-plan.md](integration-plan.md) (e.g. health routes → Phase 4, DR-1 → Phase 5).
**Open questions deferred to phase planning:**
- DR-2 (`cerebras_ifm`): public Literal vs Dreamverse-side custom provider — decide before Phase 5.
- VPO (`video_position_offset_sec` semantics): persistent vs per-segment — decide in Phase 4.
- Cross-repo history: fresh import vs `git subtree` import — decide before Phase 2.
- CORS / write-endpoint security policy: dev-only vs auth vs firewall — decide before Phase 6.
### D-17: Abandon 6-PR split — land everything as single mega-PR #1288
**Status:** ✅ Resolved 2026-05-05. PR #1287 closed; PR #1288 opened on `will/ltx2_sr_port` covering the full chain.
File diff suppressed because it is too large Load Diff
@@ -1,6 +1,31 @@
# Integration Review — Drift Audit + Path Forward
**Last updated:** 2026-05-05.
> # ⚠️ DEPRECATED — superseded by [integration-plan.md](integration-plan.md)
>
> This document recommended **Option D** (Dreamverse stays a separate repo;
> generic backend merges into FastVideo). On 2026-05-05 the team chose
> **Option B+** instead (Dreamverse FE + product server move into FastVideo
> as `apps/dreamverse/`; generic backend stays at
> `fastvideo.entrypoints.streaming.*` per Option D's principle).
> See [decisions-log.md D-18](decisions-log.md#d-18) for the strategy
> reversal rationale and [integration-plan.md](integration-plan.md) for the
> executable migration plan.
>
> **What's still authoritative in this file:**
> - **Part 1 — Drift audit** (the 17-row drift summary table). The drift
> findings remain valid; the migration plan in `integration-plan.md`
> folds them into specific phases.
> - **OSS precedent citations** (vLLM, BentoML, Ray Serve, TGI+ChatUI,
> Transformers.js, ComfyUI, AUTOMATIC1111). Reused in `integration-plan.md`.
>
> **What's superseded:**
> - **Part 2 — Recommendation (Option D)**. Replaced by Option B+ in the
> new plan. Read `integration-plan.md` for the current decision.
> - **Part 3 — Action items**. Replaced by the phased migration plan.
>
> Kept in tree for historical reference and audit trail. Do not delete.
**Last updated:** 2026-05-05 (deprecated header added).
**Scope:** FastVideo public `will/ltx2_sr_port` at the requested audit
anchor `b36bdbc9`; Dreamverse `will/integrate-public-fastvideo` at