[docs] dreamverse-integration: D-18 — Option B+ monorepo plan
User decision after reviewing integration-review.md's Option D recommendation. Combines Option D's "generic backend stays generic" principle with Option B's monorepo subfolder layout for the Dreamverse product. Result: single repo (FastVideo), Dreamverse becomes apps/dreamverse/, generic backend stays at fastvideo.entrypoints.streaming.*, Dreamverse repo gets archived after migration. Methodology: 3 parallel exploration agents (FastVideo build/CI surface, Dreamverse product-vs-generic file split, Python+Next.js monorepo precedents). Synthesis authored by ultrabrain category. Oracle review applied as PASS-WITH-FIXES; 10 critical issues addressed (import contract too tight, Phase 2 missing shim list, prompt_enhancer misclassified, frontend CI Playwright/pnpm setup broken, .gitignore product-asset issue, uv.lock missing, backend CI path filters too narrow, cross-repo history claim wrong, Phase 6 too coarse, missing security/CORS risks). * integration-plan.md (new, 1205 lines): executable 7-phase migration plan with file-by-file map, tooling stack (uv workspace + standalone pnpm), pyproject.toml/pre-commit/.gitignore/CI YAML diffs, risk register, verification gates, rollback. Phase 0 lands #1288; Phases 1-7 add skeleton, move backend, move FE, promote generic-pending, retire prompt enhancer fork (DR-1), CI/release cutover (split into 6a-6f), archive Dreamverse repo. * integration-review.md: DEPRECATED banner added at top pointing to integration-plan.md. Body kept for drift-audit and OSS precedent reference (still authoritative). Reading-guide entry updated. * decisions-log.md: D-18 entry capturing strategy decision, rationale (drops cross-repo coordination overhead from D-17 cycle; preserves architectural separation; OSS precedents support shape), why not A/B/C/D (concrete reasons for each), implications (apps/ exclude in setuptools, no cross-repo history preservation, drift items fold into phases), and 4 deferred decisions (DR-2, VPO, history-import method, CORS/write-endpoint security). * README.md: Last-reconciled bumped with D-18 reference. Reading- guide table updated: integration-plan.md is CURRENT; integration-review.md marked DEPRECATED. Verified: pre-commit clean (memory dir excluded from yapf/ruff/mypy; spaces-check passes). Oracle PASS-WITH-FIXES, all 10 critical fixes applied (import contract relaxed to allow fastvideo.{api,configs, entrypoints.streaming,entrypoints.video_generator}; Phase 2 lists 6 explicit shims for generic-pending; Phase 5 explicitly retires the prompt fork; ci-dreamverse-frontend.yml uses pnpm/action-setup BEFORE setup-node; Playwright deferred to Phase 4 until health routes land; .gitignore unignores product assets; Phase 1 commits uv.lock with --locked enforcement; backend CI watches fastvideo/api + streaming + video_generator + configs; cross-repo history disposition documented; Phase 6 split into 6a-6f substeps with independent verification gates; CORS/write-endpoint risks added to register). PR #1288: https://github.com/hao-ai-lab/FastVideo/pull/1288 Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
This commit is contained in:
co-authored by
Junda Su
Matthew Noto
XOR-op
Zhang Peiyuan
parent
907620ff22
commit
1e66c74aec
@@ -5,16 +5,21 @@ Tracks the public API refactor (PRs 0-17), the LTX-2 streaming server
|
||||
upstream, the Dreamverse switch from `FastVideo-internal` to public
|
||||
`FastVideo`, and the NVFP4 quantization landing.
|
||||
|
||||
**Last reconciled:** 2026-05-05 (strategy reversal — single mega-PR).
|
||||
FastVideo `will/ltx2_sr_port` @ `b36bdbc9` (36 commits ahead of main, post-D-17 STACK.md removal + integration-review.md addition).
|
||||
Dreamverse `will/integrate-public-fastvideo` @ `ec8ef92`.
|
||||
**Last reconciled:** 2026-05-05 (**D-18**: Option B+ chosen — Dreamverse
|
||||
becomes `apps/dreamverse/` subfolder under FastVideo; generic backend stays
|
||||
at `fastvideo.entrypoints.streaming.*`. See [integration-plan.md](integration-plan.md)
|
||||
for the executable 7-phase migration plan;
|
||||
[integration-review.md](integration-review.md) is **deprecated** but kept
|
||||
for the drift audit and OSS precedent citations.).
|
||||
FastVideo `will/ltx2_sr_port` @ HEAD (post-D-17 STACK.md removal +
|
||||
integration-review.md addition + integration-plan.md addition + D-18
|
||||
reconciliation). Dreamverse `will/integrate-public-fastvideo` @ `ec8ef92`.
|
||||
PRs #1257 / #1258 / #1284 / #1286 MERGED to main. **PR #1287 CLOSED
|
||||
(in favor of consolidation); PR #1288 OPEN as the single mega-PR
|
||||
landing the entire `will/ltx2_sr_port` chain at once** (LTX-2 SR
|
||||
runtime + NVFP4 + `generate_async`/Dynamo contract + agents memory dir,
|
||||
71 files, +13,074/-583 LOC). Split branches kept as historical
|
||||
bookmarks; STACK.md model is **abandoned** — see [decisions-log.md
|
||||
D-17](decisions-log.md#d-17). Local backup
|
||||
runtime + NVFP4 + `generate_async`/Dynamo contract + agents memory dir).
|
||||
Split branches kept as historical bookmarks; STACK.md model **abandoned** —
|
||||
see [decisions-log.md D-17](decisions-log.md#d-17). Local backup
|
||||
`will/ltx2_sr_port-pre-1286-rebase` @ `1baa60bb` preserves the
|
||||
pre-rebase chain.
|
||||
|
||||
@@ -68,7 +73,8 @@ file it in [open-threads.md](open-threads.md) before continuing.
|
||||
| "Why was decision X made? What's resolved vs. open?" | [decisions-log.md](decisions-log.md) |
|
||||
| "What should I work on next? Priority order?" | [open-threads.md](open-threads.md) |
|
||||
| "Who should be co-authored on commits in this scope?" | [authors.md](authors.md) |
|
||||
| "Have we drifted from the goal? Should Dreamverse merge into FastVideo?" | [integration-review.md](integration-review.md) |
|
||||
| "How do we execute the Dreamverse → FastVideo monorepo merge?" | [integration-plan.md](integration-plan.md) ← **CURRENT** |
|
||||
| "Historical drift audit + Option-D evaluation (deprecated by D-18)" | [integration-review.md](integration-review.md) (DEPRECATED) |
|
||||
|
||||
## Repo + worktree paths
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ question/decision, rationale, current status.
|
||||
For implementation status see [pr-roadmap.md](pr-roadmap.md). For
|
||||
follow-up actions see [open-threads.md](open-threads.md).
|
||||
|
||||
**Last updated:** 2026-05-05 (added D-12 — GpuPool layer separation, Oracle review post-#1257-merge; added D-13 — prompt enhancer / LLMProvider abstraction shape, Oracle review pre-#1258-merge; added D-14 — streaming auxiliaries cohesion, Oracle review during #1284 review cycle; added D-15 — streaming router placement + sticky/active-active deferral, Oracle review during #1286 review cycle; added D-16 — streaming router polish round 2, second-pass review on top of D-15 covering bridge cancellation hygiene, registry state machine, httpx hard-fail, replica YAML parsing, and `websockets` dep; added D-17 — strategy reversal: abandon 6-PR split in favor of single mega-PR #1288 on `will/ltx2_sr_port`).
|
||||
**Last updated:** 2026-05-05 (added D-12 — GpuPool layer separation, Oracle review post-#1257-merge; added D-13 — prompt enhancer / LLMProvider abstraction shape, Oracle review pre-#1258-merge; added D-14 — streaming auxiliaries cohesion, Oracle review during #1284 review cycle; added D-15 — streaming router placement + sticky/active-active deferral, Oracle review during #1286 review cycle; added D-16 — streaming router polish round 2, second-pass review on top of D-15 covering bridge cancellation hygiene, registry state machine, httpx hard-fail, replica YAML parsing, and `websockets` dep; added D-17 — strategy reversal: abandon 6-PR split in favor of single mega-PR #1288 on `will/ltx2_sr_port`; added D-18 — Option B+ chosen: Dreamverse FE+product-server move into FastVideo as `apps/dreamverse/` subfolder while generic backend stays at `fastvideo.entrypoints.streaming.*`; integration-review.md deprecated, integration-plan.md is the executable migration plan).
|
||||
|
||||
## Status legend
|
||||
|
||||
@@ -16,6 +16,60 @@ follow-up actions see [open-threads.md](open-threads.md).
|
||||
|
||||
## Post-merge architecture decisions
|
||||
|
||||
### D-18: Option B+ — Dreamverse becomes `apps/dreamverse/` subfolder under FastVideo
|
||||
|
||||
**Status:** ✅ Resolved 2026-05-05. [integration-plan.md](integration-plan.md) is the executable migration plan; [integration-review.md](integration-review.md) is deprecated but kept for drift audit + OSS precedents.
|
||||
**Source:** User decision after reviewing [integration-review.md](integration-review.md)'s Option D recommendation.
|
||||
|
||||
**Question:** [integration-review.md](integration-review.md) recommended **Option D** — Dreamverse stays a separate repo, generic backend (streaming runtime, GPU pool, prompt enhancer, router) merges into `fastvideo.entrypoints.streaming.*`. The user reviewed this and chose a different shape: keep the generic-backend principle from Option D but ALSO move the Dreamverse FE + product server into FastVideo as a subfolder (`apps/dreamverse/`). Combination is "Option B+" (Option B layout with Option D's backend principle).
|
||||
|
||||
**Decision:** Option B+. Concrete shape:
|
||||
|
||||
- **One repo**: `hao-ai-lab/FastVideo`. Dreamverse repo gets archived after migration completes.
|
||||
- **Python ML library** stays at root: `fastvideo/`, `fastvideo-kernel/`.
|
||||
- **Generic backend** stays at `fastvideo.entrypoints.streaming.*` (already there per #1257/#1258/#1284/#1286/#1288).
|
||||
- **Dreamverse product** moves into `apps/dreamverse/{server,web,prompts,serve_configs,scripts}/`.
|
||||
- **Tooling**: uv workspace for Python (`[tool.uv.workspace] members = ["apps/dreamverse/server"]`), standalone pnpm for the FE (no root `package.json`), split CI workflows with path-filter triggers.
|
||||
|
||||
**Rationale:**
|
||||
|
||||
- Drops the cross-repo coordination overhead identified in the post-#1286 rebase cycle (D-17 handled by consolidating into mega-PR; D-18 prevents the next round of cross-repo coordination from happening).
|
||||
- Keeps the architectural separation Option D recommended (FastVideo owns reusable runtime; product owns product). The boundary is now `apps/dreamverse/` directory rather than two repos.
|
||||
- Single repo means atomic cross-cutting refactors (e.g. GpuPool API change + Dreamverse adoption) ship as one PR.
|
||||
- OSS precedents support the shape (chainlit uv-workspace + pnpm; open-webui Python + Svelte with paths-ignore CI). The librarian explicitly noted no precedent for "Python ML library + Next.js product merged into library namespace" — but this isn't that pattern. Dreamverse goes into a sibling directory, NOT into `fastvideo.entrypoints.dreamverse.*`. Library namespace stays clean.
|
||||
|
||||
**Why not Option D (separate repos):**
|
||||
|
||||
- Each upstream merge into FastVideo invalidates Dreamverse's lockfile/imports; the post-#1286 rebase showed this requires coordination overhead that scales with feature velocity.
|
||||
- Cross-repo contract tests catch shape drift but not behavior drift.
|
||||
- Two repos means two `AGENTS.md`, two CI configs, two release stories, two Dependabot dashboards.
|
||||
|
||||
**Why not Option C (full merge into `fastvideo.entrypoints.dreamverse.*`):**
|
||||
|
||||
- Forces FastVideo to ship Tailwind config + curated preset JSON + Next.js build artifacts.
|
||||
- Locks Dreamverse product cadence to FastVideo PyPI releases.
|
||||
- Librarian: "no 1:1 precedent for Python ML library + Next.js product merged into library namespace" — argues against this.
|
||||
|
||||
**Why not Option B (subfolder, but generic backend folded into `apps/dreamverse/server/`):**
|
||||
|
||||
- Other consumers (Dynamo, future streaming clients) need the backend without the Dreamverse product. Folding the backend under `apps/dreamverse/server/` would force Dynamo to either depend on `apps/` paths (ugly) or carry a fork.
|
||||
|
||||
**Implications:**
|
||||
|
||||
- [integration-review.md](integration-review.md) is **deprecated** (banner header + reading-guide demotion). Kept in tree for drift audit + OSS precedent reference.
|
||||
- [integration-plan.md](integration-plan.md) is the **canonical executable plan** with 7 phases (Phase 0: land #1288; Phase 1: skeleton + tooling; Phase 2: backend move; Phase 3: FE move; Phase 4: promote generic-pending; Phase 5: prompt enhancer fork retirement; Phase 6: CI/release cutover; Phase 7: archive Dreamverse repo).
|
||||
- Dreamverse repo will be **archived** at end of Phase 7 — not before.
|
||||
- Dreamverse history does NOT migrate cross-repo via `git mv` (technical limitation); original history stays in archived Dreamverse repo, and Phase 2 PR body records the source SHA(s).
|
||||
- New top-level `apps/` directory created — must be excluded from FastVideo PyPI wheel via `[tool.setuptools.packages.find] exclude = ["apps*", ...]`.
|
||||
- Drift items from [integration-review.md](integration-review.md) get folded into specific phases of [integration-plan.md](integration-plan.md) (e.g. health routes → Phase 4, DR-1 → Phase 5).
|
||||
|
||||
**Open questions deferred to phase planning:**
|
||||
|
||||
- DR-2 (`cerebras_ifm`): public Literal vs Dreamverse-side custom provider — decide before Phase 5.
|
||||
- VPO (`video_position_offset_sec` semantics): persistent vs per-segment — decide in Phase 4.
|
||||
- Cross-repo history: fresh import vs `git subtree` import — decide before Phase 2.
|
||||
- CORS / write-endpoint security policy: dev-only vs auth vs firewall — decide before Phase 6.
|
||||
|
||||
### D-17: Abandon 6-PR split — land everything as single mega-PR #1288
|
||||
|
||||
**Status:** ✅ Resolved 2026-05-05. PR #1287 closed; PR #1288 opened on `will/ltx2_sr_port` covering the full chain.
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,6 +1,31 @@
|
||||
# Integration Review — Drift Audit + Path Forward
|
||||
|
||||
**Last updated:** 2026-05-05.
|
||||
> # ⚠️ DEPRECATED — superseded by [integration-plan.md](integration-plan.md)
|
||||
>
|
||||
> This document recommended **Option D** (Dreamverse stays a separate repo;
|
||||
> generic backend merges into FastVideo). On 2026-05-05 the team chose
|
||||
> **Option B+** instead (Dreamverse FE + product server move into FastVideo
|
||||
> as `apps/dreamverse/`; generic backend stays at
|
||||
> `fastvideo.entrypoints.streaming.*` per Option D's principle).
|
||||
> See [decisions-log.md D-18](decisions-log.md#d-18) for the strategy
|
||||
> reversal rationale and [integration-plan.md](integration-plan.md) for the
|
||||
> executable migration plan.
|
||||
>
|
||||
> **What's still authoritative in this file:**
|
||||
> - **Part 1 — Drift audit** (the 17-row drift summary table). The drift
|
||||
> findings remain valid; the migration plan in `integration-plan.md`
|
||||
> folds them into specific phases.
|
||||
> - **OSS precedent citations** (vLLM, BentoML, Ray Serve, TGI+ChatUI,
|
||||
> Transformers.js, ComfyUI, AUTOMATIC1111). Reused in `integration-plan.md`.
|
||||
>
|
||||
> **What's superseded:**
|
||||
> - **Part 2 — Recommendation (Option D)**. Replaced by Option B+ in the
|
||||
> new plan. Read `integration-plan.md` for the current decision.
|
||||
> - **Part 3 — Action items**. Replaced by the phased migration plan.
|
||||
>
|
||||
> Kept in tree for historical reference and audit trail. Do not delete.
|
||||
|
||||
**Last updated:** 2026-05-05 (deprecated header added).
|
||||
|
||||
**Scope:** FastVideo public `will/ltx2_sr_port` at the requested audit
|
||||
anchor `b36bdbc9`; Dreamverse `will/integrate-public-fastvideo` at
|
||||
|
||||
Reference in New Issue
Block a user