diff --git a/.agents/memory/dreamverse-integration/README.md b/.agents/memory/dreamverse-integration/README.md index c4160eb21..a085d9a00 100644 --- a/.agents/memory/dreamverse-integration/README.md +++ b/.agents/memory/dreamverse-integration/README.md @@ -5,8 +5,10 @@ Tracks the public API refactor (PRs 0-17), the LTX-2 streaming server upstream, the Dreamverse switch from `FastVideo-internal` to public `FastVideo`, and the NVFP4 quantization landing. -**Last reconciled:** 2026-05-03 (FastVideo `will/ltx2_sr_port` @ `156103b9`, -Dreamverse `will/integrate-public-fastvideo` @ `ec8ef92`). +**Last reconciled:** 2026-05-04 (FastVideo `will/ltx2_sr_port` @ `89a6484d` +post-#1258-merge rebase; Dreamverse `will/integrate-public-fastvideo` @ `ec8ef92`). +PRs #1257 (`will/api_7.6`) and #1258 (`will/api_7.7`) MERGED to main; PR #1284 +(`will/api_7.8`) OPEN, awaiting review. ## Reading guide — what to load when diff --git a/.agents/memory/dreamverse-integration/open-threads.md b/.agents/memory/dreamverse-integration/open-threads.md index 4659c2e7c..f8a327c23 100644 --- a/.agents/memory/dreamverse-integration/open-threads.md +++ b/.agents/memory/dreamverse-integration/open-threads.md @@ -6,11 +6,9 @@ recommended next action. For why each item is open see [decisions-log.md](decisions-log.md). For PR-level context see [pr-roadmap.md](pr-roadmap.md). -**Last updated:** 2026-05-04 (added DR-1/DR-2 — Dreamverse migration to -public prompt enhancer; added D-12-{A,B,C} — GpuPool follow-ups from -Oracle review; added D-13-{A,B} — prompt-enhancer follow-ups from Oracle -review; added 9-10 — minor commit-message cleanups; reorganized priority -table by category). +**Last updated:** 2026-05-04 (PRs 7.5 / 7.6 / 7.7 merged; PR 7.8 opened +as #1284; items #9 + #10 RESOLVED via the cleanup rebase that bundled +into 7.8 prep; DR-1 unblocked now that PR #1258 has merged). ## Priority overview @@ -21,7 +19,7 @@ table by category). | **2** | High | Fix pre-existing AbsMaxFP8 test failure | S | Self-contained quantization tech debt | | **VPO** | High | Decide `video_position_offset_sec` semantics (a vs b) | 30 min | Unblocks PR 7.6 state emission | | **D** | High | Implement `generate_async` (PR 7.10) | L | Closes Q-5/Q-9/PR-7.5 TODOs simultaneously; enables Dynamo backend; unblocks audio re-encode; enables `GpuPool.run_async()` migration (D-12-B) | -| **DR-1** | High | Dreamverse: create `prompting/_internal_compat.py` shim + replace local `prompt_enhancer.py` (1933 LOC) | M (~150-200 LOC shim, replace upstream wiring) | Lets Dreamverse stop carrying a 1933-LOC fork once PR #1258 lands | +| **DR-1** | High | Dreamverse: create `prompting/_internal_compat.py` shim + replace local `prompt_enhancer.py` (1933 LOC) — **PR #1258 has merged (`f673423b`); now actionable** | M (~150-200 LOC shim, replace upstream wiring) | Lets Dreamverse stop carrying a 1933-LOC fork | | **DR-2** | Med | Decide `cerebras_ifm` provider path: (a) public Literal + `CerebrasIFMProvider` shipped, OR (b) Dreamverse-side custom provider via `enhancer.register_provider(...)` | S (decision) + S-M (impl) | Resolves the cerebras_ifm gap left by PR #1258. Same item as legacy #3 below; DR-2 is the Dreamverse-side framing. | | **3** | Med | Add `cerebras_ifm` to `PromptEnhancerConfig.provider` Literal + provider | S-M | Public-side resolution if DR-2 picks (a) | | **4** | Med | Expose `layer_profile` on typed `engine.quantization` | M | Removes Dreamverse's `experimental["pipeline_config"]` dodge for stage profiles | @@ -35,10 +33,10 @@ table by category). | **6** | Low | Audio attention quantization profile + test update | S | Future audio quant exploration | | **7** | Low | Schema parity inventory cleanup (env-driven prompt fields) | S-M | Long-term consistency | | **8** | Low | Stale `apps/web/test-results/` dir cleanup | trivial | Cosmetic | -| **9** | Low | Commit-message cleanup: PR 8's 3 commits still have `[8/n] Improve API:` prefix (regex didn't match single-digit) | S | Cosmetic; will be visible when PR 8 is opened. Extend regex to `[\d+(\.\d+)?/n]`. | -| **10** | Low | Commit-message cleanup: PR 7.8/7.9 commits have `streaming: streaming X` duplication | S | Cosmetic; will be visible when PR 7.8/7.9 are opened. Drop redundant word. | | **11** | Low | Promote LTX-2 prompt orchestration (locked segments, segment_prompts JSON shape, rollout id/label) to `fastvideo.entrypoints.streaming.prompt.ltx2_orchestration` | M | Resolves Q-2 from decisions-log when a second LTX-2-style consumer appears | | **~~Source-doc disposition~~** | ~~Med~~ | ~~Disposition of 7 untracked source docs~~ | ~~trivial~~ | ✅ **Resolved 2026-05-03** — moved into [source-archive/](source-archive/) | +| **~~9~~** | ~~Low~~ | ~~Commit-message cleanup: PR 8's 3 commits still have `[8/n] Improve API:` prefix~~ | ~~S~~ | ✅ **Resolved 2026-05-04** — bundled into the will/api_7.8 prep rebase. PR 8's 3 commits now read `[type] streaming: ...` | +| **~~10~~** | ~~Low~~ | ~~Commit-message cleanup: PR 7.8/7.9 commits have `streaming: streaming X` duplication~~ | ~~S~~ | ✅ **Resolved 2026-05-04** — bundled into the will/api_7.8 prep rebase. 3 commits dedup'd. | --- @@ -417,51 +415,21 @@ run. **Effort:** Trivial. -### Item #9: PR 8 commits — `[8/n] Improve API:` prefix slipped through cleanup +### ~~Item #9~~ + ~~#10~~: Commit-message cleanups — ✅ Resolved 2026-05-04 -**Why:** During the PR-ref cleanup pass on `will/ltx2_sr_port`, the perl -substitution regex `^\[(feat|...)\] \[\d+\.\d+\/n\] Improve API: ` only -matched `[N.N/n]` (e.g. `[7.6/n]`). PR 8's three commits use `[8/n]` -(single-digit, no decimal) and weren't caught: +Both items resolved during the `will/api_7.8` prep rebase. A targeted +conditional script (`/tmp/opencode/cleanup_subjects_v2.sh` — only amends +when text actually changes) ran across 33 commits, modified 6: -``` -[docs] [8/n] Improve API: OpenAI HTTP contract reference -[docs] [8/n] Improve API: Dynamo native-backend integration reference -[test] [8/n] Improve API: contract tests for Dreamverse + Dynamo shapes -``` +- **#9 fix**: extended the regex from `\[\d+\.\d+/n\]` to + `\[\d+(\.\d+)?/n\]` so single-digit prefixes match. PR 8's 3 commits + now read `[type] streaming: ...` instead of `[type] [8/n] Improve API: ...`. +- **#10 fix**: added second substitution `streaming: streaming X` → + `streaming: X`. PR 7.8 / 7.9 commits no longer have the duplication. -These are on `will/api_8` (local-only), so it doesn't affect any open PR. - -**Action:** When PR 8 is opened, extend the regex to `\[\d+(\.\d+)?/n\]` -and re-run the cleanup rebase on the affected commits, OR simply edit -each by hand (only 3 commits). - -**Effort:** Small. - -**Dependencies:** Bundle with PR 8 prep. - -### Item #10: PR 7.8/7.9 commits — `streaming: streaming X` duplication - -**Why:** During the same PR-ref cleanup pass, the substitution turned -`[7.8/n] Improve API: streaming auxiliaries (...)` into `streaming: -streaming auxiliaries (...)` — the original subject already started with -"streaming" so the prefix substitution duplicated the word. Three -commits affected (one each on 7.8 + 7.9): - -``` -[feat] streaming: streaming auxiliaries (safety, rewrite, logger, mock) -[test] streaming: streaming auxiliaries coverage -[feat] streaming: streaming router (multi-replica load balancer) -``` - -These are on `will/api_7.8` / `will/api_7.9` (local-only). - -**Action:** When 7.8/7.9 are opened, drop the redundant "streaming" -word from each subject. Or edit each by hand (only 3 commits). - -**Effort:** Small. - -**Dependencies:** Bundle with PR 7.8/7.9 prep. +The conditional check skipped pre-commit-hook flakiness on no-op amends +(unlike the earlier first attempt). All affected commits verified clean +post-rebase. ### Item #11: Promote LTX-2 prompt orchestration to public (when 2nd consumer exists) @@ -489,13 +457,13 @@ If you have unbounded time and want to maximize forward progress: 1. **D-8 verify** (10 min) — eliminates uncertainty 2. **D-12-A docstring** (trivial) — caveat the GpuPool API publicly 3. **Item #2 AbsMaxFP8** (S) — clears tech debt -4. **Item VPO video_position_offset_sec** (30 min) — unblocks PR 7.6 -5. **DR-1 + DR-2 Dreamverse migration** (M) — after PR #1258 merges; replaces 1700+ LOC of local fork +4. **Item VPO video_position_offset_sec** (30 min) — unblocks PR 7.10 (since 7.6 has merged, this is now scoped to whatever consumer first reads the field) +5. **DR-1 + DR-2 Dreamverse migration** (M) — **now unblocked since PR #1258 merged**; replaces 1700+ LOC of local fork 6. **Item #4 layer_profile** (M) — closes Dreamverse quant escape hatch 7. **Item #1 build_app routes** (M-L) — closes FE-compat 8. **Item D generate_async** (L) — unlock PR; brings along D-12-B (run_async) + closes Q-5/Q-9/PR-7.5 TODOs 9. **Item #5 typed quant_config carrier** (L+L) — final form -10. **Items #6/#7/#8/#9/#10 + D-12-C/D-13-A/D-13-B** — cleanup polish +10. **Items #6/#7/#8 + D-12-C/D-13-A/D-13-B + #11** — cleanup polish (#9, #10 resolved 2026-05-04) If you have a specific user goal (e.g. "ship `BE_FLAVOR=fastvideo` flavor end-to-end"), that goal dictates the order — read this list as a diff --git a/.agents/memory/dreamverse-integration/pr-roadmap.md b/.agents/memory/dreamverse-integration/pr-roadmap.md index d39f4cda1..2f12505a3 100644 --- a/.agents/memory/dreamverse-integration/pr-roadmap.md +++ b/.agents/memory/dreamverse-integration/pr-roadmap.md @@ -7,7 +7,7 @@ For design rationale see [design.md](design.md). For streaming-specific PRs (7.5-7.10) see [streaming-server.md](streaming-server.md). For NVFP4 work that runs parallel to this sequence see [quantization.md](quantization.md). -**Last updated:** 2026-05-03. +**Last updated:** 2026-05-04 (PRs 7.5 / 7.6 / 7.7 merged; PR 7.8 opened as #1284). ## Status legend @@ -16,9 +16,9 @@ work that runs parallel to this sequence see [quantization.md](quantization.md). - 🟡 **Planned** — designed, not started - 🔵 **Future** — deferred to post-PR-13 cleanup -## Landed PRs (0-7) +## Landed PRs (0 → 7.7) -| # | PR | Status | Branch | Scope | +| # | PR | Status | Merge commit | Scope | |---|---|---|---|---| | 0 | #1218 [1/n] | ✅ | merged | Parity inventory + typed inference schema | | 1 | #1218 [1/n] | ✅ | merged | Strict parser/validation/overrides + API tests | @@ -26,26 +26,30 @@ work that runs parallel to this sequence see [quantization.md](quantization.md). | 3 | #1226 [3/n] | ✅ | merged | CLI/YAML-first typed config loading for `generate` and `serve` | | 4 | #1234 [4/n] | ✅ | merged | Preset registry + presets for all 13 model families; `SamplingParam` moved to `fastvideo/api/`; `configs/sample/` deleted entirely | | 5 | #1237 [5/n] | ✅ | merged | `ServeConfig.default_request` wired into stateless OpenAI server | -| 5.5 | (`5d1d71fc`) | ✅ | `will/api_5.5` | Streaming server package skeleton, typed `StreamingConfig`/`GpuPoolConfig`/`PromptEnhancerConfig`/`PromptSafetyConfig`/`WarmupConfig`, `streaming-serve` CLI stub | +| 5.5 | (`5d1d71fc`) | ✅ | merged | Streaming server package skeleton, typed `StreamingConfig`/`GpuPoolConfig`/`PromptEnhancerConfig`/`PromptSafetyConfig`/`WarmupConfig`, `streaming-serve` CLI stub | | 6 | #1239 [6/n] | ✅ | merged | LTX2 public preset + asset wiring + `gpu_pool.py` typed-kwarg translation | | 7 | #1250 [7/n] | ✅ | merged | Typed LTX2 continuation state + streaming session store + blob store | +| **7.5** | **#1251** | ✅ | `95fd29e0` (merged 2026-04-26) | Streaming server skeleton (WebSocket + fMP4 + single generator). 8 commits. Deferred TODOs (per-step progress, mid-segment cancellation) carried forward to PR 7.10. | +| **7.6** | **#1257** | ✅ | `eb0a4152` (merged 2026-05-04) | GPU pool upstream + worker subprocess + two-segment warmup. 7 commits squashed. APPROVED by Eigensystem. See [decisions-log.md D-12](decisions-log.md#d-12) for the architectural review. | +| **7.7** | **#1258** | ✅ | `f673423b` (merged 2026-05-04) | Prompt enhancer with `LLMProvider` abstraction. Built-in providers: cerebras, groq. 3 commits squashed. **Public Literal does NOT include `cerebras_ifm`** — open-threads.md item DR-2 covers the gap. See [decisions-log.md D-13](decisions-log.md#d-13) for the architectural review. | -## In flight (7.5 / 7.6) +## In flight (7.8) | # | PR | Status | Branch | Scope | |---|---|---|---|---| -| 7.5 | #1251 | 🟢 open for review | `will/api_7.5` | Streaming server skeleton (WebSocket + fMP4 + single generator). 8 commits shipped. Deferred TODOs: per-step progress events, mid-segment cancellation. | -| 7.6 | not yet PR'd | 🟢 branch ready (rebased on 7.5) | `will/api_7.6` | GPU pool upstream. 5 commits. 17/17 gpu_pool tests + 89/89 streaming tests green at HEAD. | +| **7.8** | **#1284** | 🟢 OPEN, MERGEABLE, BLOCKED on review | `will/api_7.8` (head `f95c03b6`) | Streaming auxiliaries — `prompt/safety.py` (optional fasttext, lazy import), `prompt/rewrite.py` (RewriteOptions/Result + build_rewrite splitting), `session_logger.py` (thread-safe JSONL append-only), `mock_server.py` (build_mock_app + MockGenerator for FE dev / protocol integration without GPU). 730 LOC across 7 files, 2 commits. | -## Planned (7.7 → 13) +## Planned (7.9 → 13) | # | Status | Branch | Scope | |---|---|---|---| -| 7.7 | 🟡 #1258 prepared, pending push | `will/api_7.7` | Prompt enhancer with `LLMProvider` abstraction. Built-in providers: cerebras, groq. **Public Literal does NOT include `cerebras_ifm`** — see [open-threads.md](open-threads.md). | -| 7.8 | 🟡 rebased on 7.7 | `will/api_7.8` | Streaming auxiliaries — `prompt/safety.py` (optional fasttext), `prompt/rewrite.py`, `session_logger.py`, `mock_server.py` | -| 7.9 | 🟡 rebased on 7.8 | `will/api_7.9` | Router upstream (multi-replica load balancer + WS proxy). Caveat: uses deprecated FastAPI `app.on_event("shutdown")` — migrate to lifespan handlers pre-merge. | -| 7.10 | 🟡 rebased on 7.9 | `will/api_7.10` | **Dynamo backend contract** — `VideoGenerator.generate_async`, `default_health_check_request()`, `VideoEvent` hierarchy, audio re-encode integration, mid-segment cancellation. | -| 8 | 🟡 rebased on 7.10 | `will/api_8` | Internal-UI ↔ public-server contract docs + Dynamo integration reference | +| 7.9 | 🟡 ready (rebased on 7.8) | `will/api_7.9` | Router upstream (multi-replica load balancer + WS proxy). 3 commits. Caveat: uses deprecated FastAPI `app.on_event("shutdown")` — migrate to lifespan handlers pre-merge. | +| 7.10 | 🟡 ready (rebased on 7.9) | `will/api_7.10` | **Dynamo backend contract** — `VideoGenerator.generate_async`, `default_health_check_request()`, `VideoEvent` hierarchy, audio re-encode integration, mid-segment cancellation. 3 commits. The "unlock PR" — closes Q-5/Q-9/PR-7.5 cancellation TODO + enables `GpuPool.run_async()` migration (D-12-B). | +| 8 | 🟡 ready (rebased on 7.10) | `will/api_8` | Internal-UI ↔ public-server contract docs + Dynamo integration reference. 3 commits. | +| LTX-2 SR | 🟡 ready (rebased on 8) | `will/ltx2_sr_runtime` | LTX-2 SR runtime port + i2v conditioning + alignment harness against FastVideo-internal. 9 commits. NOT in canonical PR plan — separate stream. | +| NVFP4 | 🟡 ready (rebased on LTX-2 SR) | `will/ltx2_nvfp4` | NVFP4 wire-up + per-component compile + typed `transformer_quant` flow. 6 commits. NOT in canonical PR plan. | +| LTX-2 fixes | 🟡 ready (rebased on NVFP4) | `will/ltx2_post_fixes` | Post-handoff parity fixes (Gemma `to()`, list-of-generators). 2 commits. | +| agents cleanup | 🟡 ready (top of stack) | `will/agents_cleanup` | `.agents/` Phase 1 cleanup + dreamverse-integration memory + STACK/CO-AUTHORS docs + D-12 + D-13 + open-threads tracking. 5 commits. **Independent of API stack** — could fast-track. | | 9 | 🟡 | — | LongCat preset migration + colocation (9 model-specific stage files) | | 10 | 🟡 | — | Hunyuan15 SR preset migration + colocation + SR field migration POC | | 11 | 🟡 | — | SSIM/performance test migration off legacy `generate_video(..., **kwargs)` |