From eae529a22ddce169d6567b89df43cc952f96897d Mon Sep 17 00:00:00 2001 From: SolitaryThinker Date: Tue, 5 May 2026 02:16:47 -0700 Subject: [PATCH] [docs] dreamverse-integration: add authors.md + track D-16 router polish MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * authors.md (new): dreamverse-integration-scoped mirror of the top-level CO-AUTHORS.md. Documents the 4 human co-authors credited on every commit in the integration scope, verified against PRs #1257/#1258/#1284/#1286. Self-contained so the memory dir is discoverable without traversing to the repo root. Cross-references CO-AUTHORS.md and STACK.md. Includes a 'Known gaps' section flagging that commits a152cb77 / 40e265b8 (the [fix] streaming: router polish pair from earlier in this session) are missing trailers and need an amend + force-push to fix. * README.md: register authors.md in the reading guide table. * decisions-log.md: add D-16 — Streaming router polish round 2 — capturing the 5 second-pass fixes applied on top of D-15's pre-merge polishes (bridge cancellation hygiene, registry UNKNOWN->HEALTHY immediate, httpx hard-fail, URL path/query/fragment/duplicate validation, per-index YAML parser errors, explicit websockets [streaming] dep, +7 test cases). Bumps Last updated header. * open-threads.md: bump Last updated header to mention the second-pass router commit and link to D-16. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> --- .../memory/dreamverse-integration/README.md | 1 + .../memory/dreamverse-integration/authors.md | 145 ++++++++++++++++++ .../dreamverse-integration/decisions-log.md | 59 ++++++- .../dreamverse-integration/open-threads.md | 8 +- 4 files changed, 211 insertions(+), 2 deletions(-) create mode 100644 .agents/memory/dreamverse-integration/authors.md diff --git a/.agents/memory/dreamverse-integration/README.md b/.agents/memory/dreamverse-integration/README.md index a085d9a00..12aa69a4e 100644 --- a/.agents/memory/dreamverse-integration/README.md +++ b/.agents/memory/dreamverse-integration/README.md @@ -22,6 +22,7 @@ PRs #1257 (`will/api_7.6`) and #1258 (`will/api_7.7`) MERGED to main; PR #1284 | "NVFP4? Layer profiles? `LinearBase` fallback? AbsMaxFP8?" | [quantization.md](quantization.md) | | "Why was decision X made? What's resolved vs. open?" | [decisions-log.md](decisions-log.md) | | "What should I work on next? Priority order?" | [open-threads.md](open-threads.md) | +| "Who should be co-authored on commits in this scope?" | [authors.md](authors.md) | ## Repo + worktree paths diff --git a/.agents/memory/dreamverse-integration/authors.md b/.agents/memory/dreamverse-integration/authors.md new file mode 100644 index 000000000..f6e13a4ab --- /dev/null +++ b/.agents/memory/dreamverse-integration/authors.md @@ -0,0 +1,145 @@ +# Authors — Dreamverse Integration + +**Status:** PERMANENT — keep around as the source of truth for who collaborated +on the dreamverse-integration work, even after every PR in the integration +scope has merged. +**Last updated:** 2026-05-05 + +This file documents the human co-authors credited on every commit in the +dreamverse-integration scope (FastVideo public-API refactor, streaming server +upstream, GPU pool, prompt enhancer, NVFP4 wire-up, LTX-2 SR port). The +4 collaborators below worked on the FastVideo-internal precursor of this code +and are credited as co-authors on every public-side upstream commit via Git's +standard +[`Co-authored-by`](https://docs.github.com/en/pull-requests/committing-changes-to-your-project/creating-and-editing-commits/creating-a-commit-with-multiple-authors) +trailer convention. + +Scope-wise this is the dreamverse-integration-flavored mirror of the +top-level [`CO-AUTHORS.md`](../../../CO-AUTHORS.md), which is scoped to the +broader `will/ltx2_sr_port` 10-PR stack. The roster is identical; this file +exists so the dreamverse-integration memory dir is self-contained and +discoverable without traversing to the repo root. + +## Co-author roster + +| GitHub user | Real name | GitHub ID | Trailer email | +|---|---|---|---| +| [`@Davids048`](https://github.com/Davids048) | Junda (David) Su | 90978028 | `90978028+Davids048@users.noreply.github.com` | +| [`@RandNMR73`](https://github.com/RandNMR73) | Matthew Noto | 99706358 | `99706358+RandNMR73@users.noreply.github.com` | +| [`@XOR-op`](https://github.com/XOR-op) | (unset) | 17672363 | `17672363+XOR-op@users.noreply.github.com` | +| [`@jzhang38`](https://github.com/jzhang38) | Zhang Peiyuan | 42993249 | `42993249+jzhang38@users.noreply.github.com` | + +## Verification — where these trailers appear + +Verified via `gh pr view --json commits --jq '.commits[].messageBody'` +across every PR in the integration scope: + +| PR | Branch | Status | Trailers present on every commit | +|---|---|---|---| +| #1257 | `will/api_7.6` (GPU pool upstream) | ✅ merged 2026-05-04 | yes (4/4) | +| #1258 | `will/api_7.7` (prompt enhancer + LLMProvider) | ✅ merged 2026-05-04 | yes (3/3) | +| #1284 | `will/api_7.8` (streaming auxiliaries) | ✅ merged 2026-05-04 | yes (2/2) | +| #1286 | `will/api_7.9` (streaming router) | 🟢 open | yes on commits 1-3; **commit `a152cb77` (`[fix] streaming: router polish`) is missing trailers — see "Known gaps" below** | + +Aggregate count across `will/ltx2_sr_port` (top of stack) at the time of +writing: 32-33 commits per co-author, matching the 32 commits in the stack +on top of base `cfccd292`. Numbers stay consistent because the rebase +command (see "How the trailers were applied" below) walks every commit. + +## Trailer block (copy-paste ready) + +The trailers added to every commit on `will/ltx2_sr_port` and every +dreamverse-integration PR: + +``` +Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> +Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> +Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> +Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> +``` + +For one-off `git commit -m` invocations, use `--trailer` flags: + +```bash +git commit -m "..." \ + --trailer "Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>" \ + --trailer "Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>" \ + --trailer "Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>" \ + --trailer "Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>" +``` + +`--trailer` is idempotent (dedupes by full `key: value`) so re-running is safe. + +## Why no-reply emails + +GitHub's `+@users.noreply.github.com` form is the most reliable +way to link a `Co-authored-by` trailer to a GitHub account. It: + +- Always works regardless of whether the user has a public verified email +- Survives the user changing their primary email +- Doesn't expose anyone's personal email to git history +- Is the format GitHub itself produces when you click "Add co-author" in the + web UI + +(All 4 collaborators have this email already used in `FastVideo-internal` +git history, verified via `git log --all` on that repo.) + +## How the trailers were applied (bulk rebase) + +```bash +git rebase --exec ' + git commit --amend --no-edit \ + --trailer "Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>" \ + --trailer "Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>" \ + --trailer "Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>" \ + --trailer "Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>" +' origin/main will/ltx2_sr_port +``` + +After running, re-slice all 10 split branches per [`STACK.md`](../../../STACK.md) +and force-push the published branches (`will/api_7.9`, `will/ltx2_sr_port`). + +## How to add a new co-author later + +1. Add the user to the roster table above (and the top-level + [`CO-AUTHORS.md`](../../../CO-AUTHORS.md) — keep them in sync). +2. Append their `Co-authored-by` line to the trailer block above. +3. Re-run the bulk rebase command on `will/ltx2_sr_port` — git's trailer + dedupe handles the existing 4; the new one gets appended. +4. Re-slice all split branches per [`STACK.md`](../../../STACK.md). +5. Force-push the published branches. + +## What we do NOT add + +Per the repo's top-level [`AGENTS.md`](../../../AGENTS.md): + +> Never add any coding agent or models such as Claude (or Claude Code), GPT, +> Codex or others as a co-author in commits or PRs. Do not include +> `Co-Authored-By: Claude ...` trailers or "Generated with Claude Code" and +> other such lines. + +So no `Co-authored-by: Claude `, no +`Generated with Claude Code` footer, no `Cursor ` +trailer (one such commit exists on `will/ltx2_sr_port` from a pre-policy +external contribution and stays grandfathered; new commits MUST NOT introduce +the pattern). Only human collaborators. + +## Known gaps + +| Gap | Where | How to fix | +|---|---|---| +| Commit `a152cb77` on `will/api_7.9` (PR #1286 head) is missing the 4 co-author trailers — `[fix] streaming: router polish — bridge cancel + state machine + deps` | `will/api_7.9`, pushed 2026-05-05 | Amend the commit with `--trailer` flags above + force-push `will/api_7.9` (requires explicit user confirmation per AGENTS.md force-push rule) | +| Commit `40e265b8` on `will/ltx2_sr_port` (the cherry-pick partner of `a152cb77`) is missing the same 4 trailers | `will/ltx2_sr_port`, pushed 2026-05-05 | Same — amend + force-push (or wait for the next bulk rebase across the stack) | + +These commits were authored by an automation flow that did not pass the +trailer block through to the `git commit` invocation. The fix is mechanical +once force-push is approved. + +## See also + +- [`../../../CO-AUTHORS.md`](../../../CO-AUTHORS.md) — top-level stack-scoped + co-authors file (same roster, broader scope) +- [`../../../STACK.md`](../../../STACK.md) — 10-PR split layout for + `will/ltx2_sr_port` (re-slice commands live here) +- [`pr-roadmap.md`](pr-roadmap.md) — per-PR status within the + dreamverse-integration scope diff --git a/.agents/memory/dreamverse-integration/decisions-log.md b/.agents/memory/dreamverse-integration/decisions-log.md index df6e393ac..d364ae15f 100644 --- a/.agents/memory/dreamverse-integration/decisions-log.md +++ b/.agents/memory/dreamverse-integration/decisions-log.md @@ -6,7 +6,7 @@ question/decision, rationale, current status. For implementation status see [pr-roadmap.md](pr-roadmap.md). For follow-up actions see [open-threads.md](open-threads.md). -**Last updated:** 2026-05-05 (added D-12 — GpuPool layer separation, Oracle review post-#1257-merge; added D-13 — prompt enhancer / LLMProvider abstraction shape, Oracle review pre-#1258-merge; added D-14 — streaming auxiliaries cohesion, Oracle review during #1284 review cycle; added D-15 — streaming router placement + sticky/active-active deferral, Oracle review during #1286 review cycle). +**Last updated:** 2026-05-05 (added D-12 — GpuPool layer separation, Oracle review post-#1257-merge; added D-13 — prompt enhancer / LLMProvider abstraction shape, Oracle review pre-#1258-merge; added D-14 — streaming auxiliaries cohesion, Oracle review during #1284 review cycle; added D-15 — streaming router placement + sticky/active-active deferral, Oracle review during #1286 review cycle; added D-16 — streaming router polish round 2, second-pass review on top of D-15 covering bridge cancellation hygiene, registry state machine, httpx hard-fail, replica YAML parsing, and `websockets` dep). ## Status legend @@ -172,6 +172,63 @@ All 4 review threads marked resolved on the GitHub PR. **Open thread it touches:** open-threads.md items #13 (sticky routing), #14 (bridge backpressure), #15 (multi-primary semantics). +### D-16: Streaming router polish round 2 — second-pass fixes on top of D-15 + +**Status:** ✅ Resolved. Applied as `[fix] streaming: router polish — bridge +cancel + state machine + deps` (`a152cb77` on `will/api_7.9`, `40e265b8` on +`will/ltx2_sr_port`). +**Source:** Second-pass review on PR #1286, 2026-05-05, after D-15's pre-merge +polishes landed. + +**Question:** D-15 closed the structural review (placement, sticky/active-active +deferral, basic `__post_init__` validation). On a second pass through the same +files, five latent issues surfaced that weren't covered by gemini's first pass +or Oracle's structural review. Apply them on top of the merged D-15 polishes, +or queue for a follow-up PR? + +**Decision:** Apply on top of `will/api_7.9` directly. All five are bug-class +or DX-class — none are scope-expanding architecture changes — so folding them +into PR #1286 keeps the router landing in one reviewable unit instead of +shipping a router PR plus an immediate follow-up fix PR. + +**Fixes applied:** + +| # | File | What | Why | +|---|---|---|---| +| 1 | `router/main.py::_bridge_session` | Replaced `asyncio.gather()` with `wait(FIRST_COMPLETED)` + explicit `cancel()`/drain + `_is_normal_disconnect()` classifier | `gather` waited for both directions; on client disconnect, the backend-reader task leaked and stayed pending. Backend `ConnectionClosed` also surfaced as an unhandled exception in server logs. New shape: first task to finish triggers explicit cancel of the other, both are drained, and only non-routine exceptions re-raise. | +| 2 | `router/registry.py::record_success` | Split state transitions: `UNKNOWN -> HEALTHY` is now immediate on first successful probe; only `UNHEALTHY -> HEALTHY` remains gated by `recovery_threshold` | Previously a fresh registry needed `recovery_threshold` consecutive successes before any replica was selectable. With default `recovery_threshold=2` and `health_check_interval=1s`, that meant 2-3s of `gpu_unavailable` rejections at startup. Now the first probe promotes immediately; recovery gating still protects against flapping replicas. | +| 3 | `router/registry.py::_build_default_probe` | Missing `httpx` now raises `RuntimeError` with install hint instead of yielding a "disabled" probe stub | Previous behavior: silently returned `(0.0, "httpx not installed; ...")` for every probe, which `record_failure` then folded into `UNHEALTHY` after `failure_threshold` cycles. Operators saw replicas drop UNHEALTHY with a confusing reason and no clear remediation. Hard-fail at startup is the right surface. | +| 4 | `router/config.py::__post_init__` | Extended D-15 polish #4 with: rejects `urlparse(url).path not in ("", "/")`, rejects `query`/`fragment`, rejects duplicate URLs across replicas | D-15's validation rejected non-`http(s)://` URLs and >1 primary; it didn't catch `http://host/api` (the router appends `/health` and `/v1/stream` itself, so a base-URL with path yields malformed routes) or `[{url: x}, {url: x}]` (replica registry keys by URL — duplicates would silently collapse to one entry, masking the misconfiguration). | +| 5 | `cli/router_serve.py::_load_router_config` | Replaced silent list-comprehension filter (`for r in replicas_raw if isinstance(r, dict) and r.get("url")`) with per-index `raise ValueError` | Original parser silently dropped malformed YAML entries. A single typo in `replicas[2].url` would yield 2 replicas instead of 3 with no log line. New shape: explicit per-index error message ("missing required key 'url'", "must be a mapping"). | +| 6 | `pyproject.toml::[streaming]` extra | Added `websockets` as explicit dep | `router/main.py::_bridge_session` does `import websockets` lazily and raises `RuntimeError` if missing. The `[streaming]` extra was an implicit transitive — anyone installing only `[streaming]` (and not the broader requirements) hit the runtime error. Now explicit. | + +**Tests added (7 cases in `fastvideo/tests/entrypoints/streaming/test_router.py`):** + +- `TestUnknownToHealthyImmediate.test_first_success_promotes_unknown` — first probe success transitions `UNKNOWN -> HEALTHY` regardless of `recovery_threshold` +- `TestUnknownToHealthyImmediate.test_unhealthy_recovery_still_gated_by_threshold` — `UNHEALTHY -> HEALTHY` still requires `recovery_threshold` successes +- `TestConfigValidation.test_rejects_path_in_url` / `test_rejects_query_in_url` / `test_rejects_fragment_in_url` / `test_rejects_duplicate_urls` / `test_accepts_trailing_slash` — `__post_init__` URL validation matrix + +**Verification:** 17/17 router tests pass on both branches. `pre-commit run` +clean (yapf / ruff / codespell / mypy). `lsp_diagnostics` clean on changed +regions; the one pre-existing `Task` generic-type warning at `main.py:37` is +unrelated and predates this commit. + +**In-flight pre-commit corrections (not part of the 6 fixes themselves):** + +- yapf auto-reformatted 4 files (kept verbatim). +- ruff `UP038`: rewrote `isinstance(exc, (CancelledError, WebSocketDisconnect))` + to `isinstance(exc, CancelledError | WebSocketDisconnect)`. +- mypy `[misc]`: renamed loop var `exc` (inside `for task in done`) to + `task_exc` to avoid name collision with the outer + `except ImportError as exc` binding. + +**Open thread it touches:** None new. Item #14 (bridge backpressure) and +item #13 (sticky routing) from D-15 remain deferred — this round addressed +**cancellation/disconnect** semantics on the bridge, which is distinct from +**throughput backpressure**. Item #14 still applies: at higher load, add +`_bridge_session()` max-size + timeout limits or recommend Envoy/HAProxy +in front. + ### D-14: Streaming auxiliaries (PR #1284) — cohesion + concrete-vs-Protocol scoping **Status:** ✅ Resolved (interim). Two polish items applied during review; one diff --git a/.agents/memory/dreamverse-integration/open-threads.md b/.agents/memory/dreamverse-integration/open-threads.md index 2bddd1d82..fd3d001b0 100644 --- a/.agents/memory/dreamverse-integration/open-threads.md +++ b/.agents/memory/dreamverse-integration/open-threads.md @@ -11,7 +11,13 @@ streaming router with 4 gemini review fixes + 4 Oracle polish items applied — `select()` docstring rewrite, parallel pooled health probes, `RouterConfig.__post_init__` validation, FastAPI on_event→lifespan migration; D-15 added to decisions-log; #13/#14/#15 added — sticky -routing extensibility, bridge backpressure, multi-primary semantics). +routing extensibility, bridge backpressure, multi-primary semantics. +Second-pass router polish committed as `a152cb77` on `will/api_7.9` / +`40e265b8` on `will/ltx2_sr_port` — bridge cancellation hygiene, +registry `UNKNOWN -> HEALTHY` immediate transition, httpx hard-fail, +URL path/query/fragment/duplicate rejection, per-index YAML parser +errors, explicit `websockets` `[streaming]` dep + 7 new test cases; +D-16 added to decisions-log capturing these.). ## Priority overview