cb082d8bbb2ca03902f660d6495ef52c026efb1b
728
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
cb082d8bbb |
[feat] streaming: promote /healthz /readyz /status to public build_app
Promote the Dreamverse-compatible health, readiness, and status probes into the public streaming package so integrators can compose them through build_app without product-local shims. Co-authored-by: Junda Su <junda@berkeley.edu> Co-authored-by: Matthew Noto <matthewnoto@gmail.com> Co-authored-by: XOR-op <xor-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <peiyuan_zhang@berkeley.edu> |
||
|
|
02ab3e45cd |
[feat] dreamverse-deploy skill + docs prereq additions
Adds an executable skill that takes a GPU index and (re)deploys the
migrated apps/dreamverse backend + Next.js frontend cleanly. Also extends
the contributing doc with the additional Phase 0 prereqs Oracle flagged
during the post-migration verification pass.
* .agents/skills/dreamverse-deploy/SKILL.md (new): documents the skill's
contract — args, env overrides, prereqs, what it does and does NOT do,
teardown, and the e2e wrapper command for follow-up Playwright runs.
* .agents/skills/dreamverse-deploy/scripts/dreamverse-deploy.sh (new):
- Validates prereqs upfront (apps/dreamverse, wrapper script, .venv,
flashinfer-python, pnpm, gcc-13/g++-13).
- Kills processes by port pattern AND by GPU UUID via
`nvidia-smi --query-compute-apps`. The earlier naive port-only kill
missed orphan multiproc_executor worker subprocesses, which then OOM'd
the next deploy on the same GPU.
- Sources ~/.env for provider API keys, exports the full env recipe
(CUDA_VISIBLE_DEVICES, FASTVIDEO_*, gcc-13 + NVCC_PREPEND_FLAGS for
flashinfer JIT against gcc-15 conda toolchain on B200/sm_100a).
- Launches backend via apps/dreamverse/scripts/dreamverse-server in a
detached setsid session, polls /readyz up to 5 min.
- Launches frontend via pnpm run dev:devtools (or `next dev --port` for
non-default port) with NEXT_PUBLIC_INCLUDE_DEVTOOLS=1, polls / up to
60s.
- Prints URLs, PIDs, log paths, stop command, e2e command at end.
- Supports `--stop` (port-pattern teardown) and `--stop <GPU>` (also
nukes any process holding that GPU).
* docs/contributing/dreamverse-development.md: Phase 0 prereqs section
extended per Oracle review (D-19 follow-up):
- Adds cerebras-cloud-sdk + openai to the install command (required by
the migrated prompt enhancer; without them, server fails to import
provider clients).
- Documents the gcc-13 / NVCC_PREPEND_FLAGS workaround for B200 hosts
with gcc-15 conda toolchain. Without these, flashinfer JIT compilation
fails at first NVFP4 use.
- Cross-links to the new dreamverse-deploy skill as the recommended
local-deploy path.
Verified: skill ran end-to-end on this dev node (GPU 4, ports 8009/5274),
backend boots /readyz=200 with /proc/$PID/cwd = apps/dreamverse/server,
frontend serves 200 on :5274, all 8 Playwright e2e specs pass against the
migrated stack in 5.1s.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
dd9a8f6aae |
[docs] dreamverse-integration: D-19 — migration executed on will/dreamverse-monorepo
Records the actual landing of D-18's Option B+ plan as 5 commits on will/dreamverse-monorepo (08828d96, f3a863ba, 876f7eb3, 1d47ede6, c1fe5d4c). 164 files, +53,294 LOC, 31,725 files under apps/dreamverse/. Significant deviation from integration-plan.md surfaced by Oracle review: the plan's Phase 2 prescription "DELETE generic-merged files (gpu_pool.py, av_streaming.py, worker_ipc.py, mock_server.py, session_init_image.py, session_logger.py) and rewire imports to public fastvideo.entrypoints. streaming.*" was INVALID — public APIs are not drop-ins for the Dreamverse product modules. Public `GpuPool` is an abstract base class with acquire/run/release/shutdown/health semantics; Dreamverse's product GPUPool has totally different shape (per-GPU subprocess workers with slot.user_step / register_stream_queue / acquire(client_id, websocket)). Substituting one for the other made apps/dreamverse/server/main.py:63 fail at boot with `TypeError: GpuPool() takes no arguments`. Fix-up commits (1d47ede6, c1fe5d4c) carried all 7 generic-merged files into apps/dreamverse/server/ as PRODUCT-LOCAL, reverted import rewrites to flat `from gpu_pool import GPUPool` style (matching Dreamverse's existing sys-path-injection convention), and reverted the public-surface deviations introduced in Phase 2. Verification: `git diff fbd823df.. c1fe5d4c -- fastvideo/entrypoints/streaming/` returns empty. Also documented: Phase 0 prerequisites for B200 / sm_100a + gcc-15 conda toolchain. flashinfer JIT requires CUDAHOSTCXX=/usr/bin/g++-13 and NVCC_PREPEND_FLAGS="-ccbin /usr/bin/gcc-13 -allow-unsupported-compiler" to bypass the nvcc gcc-version check. Without these, NVFP4 path fails to compile. E2E evidence (post-fix-up): 8/8 Playwright tests pass in 5.1s against the migrated backend with /proc/$PID/cwd = apps/dreamverse/server confirmed. GPU4 holds 50.9 GiB (NVFP4 model loaded). All routes (/healthz, /readyz, /status, /prompt-system-config, /curated-presets) return 200. * state.md: branch tip table adds will/dreamverse-monorepo @ c1fe5d4c with 5-commit migration summary + e2e evidence reference. * decisions-log.md: full D-19 entry with commit table, Oracle false- positive narrative (initial e2e run hit canonical Dreamverse install via legacy console_script, NOT migrated code; fix-up wrapper scripts resolve this), Phase 0 environment prereqs, e2e evidence dump, implications for the integration-plan.md (Phase 2 prescription invalidated), open follow-ups. * README.md: Last-reconciled bumped with D-19 reference. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
SolitaryThinker
|
0e0af68f01 |
[fix] dreamverse-monorepo: entrypoint + audio re-encode + verification
Make the repo-local wrapper scripts the canonical Dreamverse launch path while package=false defers proper console-script packaging. Restore the bounded audio re-encode path from the product backend and document the verification prerequisites used for the migrated server boot probe.\n\nCo-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>\nCo-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>\nCo-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>\nCo-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
SolitaryThinker
|
aef113a861 |
[fix] dreamverse-monorepo: carry generic-merged files product-local
Source: Dreamverse repo at will/integrate-public-fastvideo @ ec8ef92.\n\nCarry the generic-merged backend modules as Dreamverse product-local code because the public streaming APIs are not drop-in compatible with the product runtime yet. Revert the Phase 2 public compatibility shims so FastVideo's public streaming surface returns to the pre-migration state.\n\nCo-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>\nCo-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>\nCo-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>\nCo-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
9c2da7c2b0 |
[feat] dreamverse-monorepo: Phase 3 — frontend + public assets
Source: Dreamverse repo at will/integrate-public-fastvideo @ ec8ef92. Cross-repo move; original history retained in archived Dreamverse repo. Per .agents/memory/dreamverse-integration/integration-plan.md Phase 3. * Frontend moved to apps/dreamverse/web with Next/Vitest/Playwright config rewired for the monorepo location. * Product docs, scripts, launch helpers, prompt assets, and Dreamverse diagrams/assets moved under apps/dreamverse/. * Public streaming demo config added at examples/serving/streaming_demo.yaml with non-private placeholders. * Launch/smoke scripts now resolve DREAMVERSE_ROOT from apps/dreamverse and run frontend commands from apps/dreamverse/web. * Dreamverse asset allow-list narrowed in .gitignore so copied image/video assets can be staged without broadly tracking generated build output. * Vitest excludes Playwright e2e specs; obsolete page.integration assertions remain skipped until updated for the current migrated shell. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
86c8389cc9 |
[feat] dreamverse-monorepo: Phase 2 — backend move + import rewires
Source: Dreamverse repo at will/integrate-public-fastvideo @ ec8ef92.
Cross-repo move; original history retained in archived Dreamverse repo.
Per .agents/memory/dreamverse-integration/integration-plan.md Phase 2.
* PRODUCT files moved to apps/dreamverse/server/: main.py, config.py,
video_generation.py, session/controller.py, session/messages.py,
routes/presets.py + 5 product test files + product prompts.
* GENERIC-PENDING shims carried (replaced in Phase 4/5): runtime.py,
utils.py, prompt_safety.py, rewrite_prompt_payload.py,
routes/health.py, prompt_enhancer.py.
* Imports rewritten: gpu_pool/worker_ipc/session_init_image/session_logger
now import from fastvideo.entrypoints.streaming.* public modules.
* Public streaming compatibility aliases added for the migrated product
controller: GPUSlot/get_available_gpus, MediaChunk/MediaComplete,
cleanup_session_init_image, and SessionEventLogger.
* Path rewrites: _APP_ROOT/"apps"/"web" -> _APP_ROOT/"web" in config.py.
* GPU marker: test_realtime_stress.py and test_session_logging.py
marked @pytest.mark.gpu (skip-by-default in CI).
* New: apps/dreamverse/server/tests/test_import_contract.py — AST gate
forbidding fastvideo.{pipelines,models,layers,worker,fastvideo_args}.
Generic-merged files NOT copied (already on public FastVideo):
gpu_pool.py, av_streaming.py, worker_ipc.py, mock_server.py,
session_init_image.py, session_logger.py.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
6d925d5333 |
[feat] dreamverse-monorepo: Phase 1 — skeleton + tooling
Establish the apps/dreamverse/ subfolder + monorepo tooling per
.agents/memory/dreamverse-integration/integration-plan.md Phase 1.
No Dreamverse content moved yet — that's Phase 2/3.
* Add apps/dreamverse/{README.md, pyproject.toml, server/__init__.py,
server/tests/__init__.py}.
* Add docs/contributing/dreamverse-development.md with local dev commands.
* Root pyproject.toml: add [tool.uv.workspace] members = ["apps/dreamverse"];
add apps* to setuptools exclude; add gpu pytest marker.
* .pre-commit-config.yaml: exclude ^apps/dreamverse/web/.
* .gitignore: scope Node artifacts + unignore product assets.
* uv.lock: regenerated by `uv lock` to include the new workspace member.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
cbac854582 |
[docs] dreamverse-integration: GPU4 smoke validated; flashinfer prereq
Live smoke test on GPU4 confirms public fastvideo serve from will/ltx2_sr_port HEAD
|
||
|
|
996df583b9 |
[docs] dreamverse-integration: align test strategy with B+ + GPU4 hook
Audit found the test strategy in integration-plan.md was partially stale from the prior Option D / split-PR drafting and missing an explicit local-GPU verification path. This commit reconciles testing with the single-mega-PR (D-17) + Option B+ monorepo (D-18) reality and makes the GPU4-on-this-node hook explicit. * New "Test strategy" subsection (added before "File-by-file migration map"): test taxonomy table (unit / integration-with-fakes / live-GPU / FE build / FE Playwright / contract / SSIM), pytest marker scheme (`@pytest.mark.gpu`), pyproject.toml addopts to default-skip GPU tests in ubuntu-latest CI, and a per-phase responsibility matrix (which class runs in which gate per Phase 0..7). * GPU4 local verification hook documented: this dev node has 8x B200; GPU4 currently held by Dreamverse-side server PID 2453227 / port 8009; concrete kill / redeploy / smoke-test / cleanup commands so any phase touching the live-service path (Phase 2/3/4/5) can be validated locally without waiting for Buildkite-Modal. * Phase 2 verification gate: `uv run` command upgraded to `--locked --extra test pytest ... -m 'not gpu'` (matches Phase 1 `--locked` enforcement). Added explicit MANUAL GPU4 QA gate required, not optional: kill PID 2453227, deploy `fastvideo serve` on GPU4 from this branch, hit `/health`, run `pytest -m gpu` against the live deploy, capture output in PR. Rollback expanded to handle GPU4-discovered bugs. * Phase 3 verification gate: Playwright in CI is now explicitly DEFERRED to Phase 4 (matches `ci-dreamverse-frontend.yml` scaffold which already comments out Playwright steps until health routes land). Removed the contradictory "or PR note explains" clause. Added recommended manual GPU4 Playwright `frontend-shell.spec.ts` against the Phase 2 GPU4 deploy. Same `--locked --extra test` upgrade for the backend regression check. * Phase 4 verification gate: Frontend CI Playwright RE-ENABLED at end of this phase (the natural re-baseline once health routes land). Manual GPU4 full E2E for all 3 Playwright specs (backend-health + frontend-shell + preset-prompt-generation) is required to merge. * OSS precedent table: clarified that chainlit's "split CI" is a per-language CI split (frontend vs backend workflows), not a PR-level split — independent of D-17's single-mega-PR decision. Removes any reading-confusion that the testing strategy carries forward from the abandoned split. No claims, references, or commands tied to the abandoned split-PR branches (`will/api_7.10`, `will/api_8`, `will/ltx2_sr_runtime`, `will/ltx2_nvfp4`, `will/ltx2_post_fixes`, `will/agents_cleanup`) remain in the plan doc. All test commands route through `apps/dreamverse/server/` and `apps/dreamverse/web/` per the B+ layout. PR #1288: https://github.com/hao-ai-lab/FastVideo/pull/1288 Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
8b198bbcca |
[docs] dreamverse-integration: D-18 — Option B+ monorepo plan
User decision after reviewing integration-review.md's Option D recommendation. Combines Option D's "generic backend stays generic" principle with Option B's monorepo subfolder layout for the Dreamverse product. Result: single repo (FastVideo), Dreamverse becomes apps/dreamverse/, generic backend stays at fastvideo.entrypoints.streaming.*, Dreamverse repo gets archived after migration. Methodology: 3 parallel exploration agents (FastVideo build/CI surface, Dreamverse product-vs-generic file split, Python+Next.js monorepo precedents). Synthesis authored by ultrabrain category. Oracle review applied as PASS-WITH-FIXES; 10 critical issues addressed (import contract too tight, Phase 2 missing shim list, prompt_enhancer misclassified, frontend CI Playwright/pnpm setup broken, .gitignore product-asset issue, uv.lock missing, backend CI path filters too narrow, cross-repo history claim wrong, Phase 6 too coarse, missing security/CORS risks). * integration-plan.md (new, 1205 lines): executable 7-phase migration plan with file-by-file map, tooling stack (uv workspace + standalone pnpm), pyproject.toml/pre-commit/.gitignore/CI YAML diffs, risk register, verification gates, rollback. Phase 0 lands #1288; Phases 1-7 add skeleton, move backend, move FE, promote generic-pending, retire prompt enhancer fork (DR-1), CI/release cutover (split into 6a-6f), archive Dreamverse repo. * integration-review.md: DEPRECATED banner added at top pointing to integration-plan.md. Body kept for drift-audit and OSS precedent reference (still authoritative). Reading-guide entry updated. * decisions-log.md: D-18 entry capturing strategy decision, rationale (drops cross-repo coordination overhead from D-17 cycle; preserves architectural separation; OSS precedents support shape), why not A/B/C/D (concrete reasons for each), implications (apps/ exclude in setuptools, no cross-repo history preservation, drift items fold into phases), and 4 deferred decisions (DR-2, VPO, history-import method, CORS/write-endpoint security). * README.md: Last-reconciled bumped with D-18 reference. Reading- guide table updated: integration-plan.md is CURRENT; integration-review.md marked DEPRECATED. Verified: pre-commit clean (memory dir excluded from yapf/ruff/mypy; spaces-check passes). Oracle PASS-WITH-FIXES, all 10 critical fixes applied (import contract relaxed to allow fastvideo.{api,configs, entrypoints.streaming,entrypoints.video_generator}; Phase 2 lists 6 explicit shims for generic-pending; Phase 5 explicitly retires the prompt fork; ci-dreamverse-frontend.yml uses pnpm/action-setup BEFORE setup-node; Playwright deferred to Phase 4 until health routes land; .gitignore unignores product assets; Phase 1 commits uv.lock with --locked enforcement; backend CI watches fastvideo/api + streaming + video_generator + configs; cross-repo history disposition documented; Phase 6 split into 6a-6f substeps with independent verification gates; CORS/write-endpoint risks added to register). PR #1288: https://github.com/hao-ai-lab/FastVideo/pull/1288 Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
2f7f0cce5a |
[docs] dreamverse-integration: drift audit + integration tradeoff doc
Comprehensive review against ../Dreamverse and ../FastVideo-internal
plus 4-option integration tradeoff analysis with recommendation.
Driven by user request to (1) verify will/ltx2_sr_port has not drifted
from the integration goal, and (2) analyze whether Dreamverse should
remain a separate repo, become a subfolder, or merge into FastVideo's
namespace entirely.
Methodology: 4 parallel exploration agents mapped Dreamverse repo
structure, FastVideo-internal residual, FastVideo public cross-repo
surfaces, and OSS service-in-monorepo precedents (vLLM, BentoML, Ray
Serve, TGI+ChatUI, Transformers.js, ComfyUI, AUTOMATIC1111). Synthesis
authored by ultrabrain category. Oracle review applied as
PASS-WITH-FIXES; critical issues (broad zero-drift wording, stale Dynamo
mapping citation, broken Ray Serve URL, missing D-8/VPO/D-12-A/D-12-B/
SBS coverage) addressed in this commit.
* integration-review.md (new, 863 lines): Part 1 drift audit, Part 2
integration tradeoffs, Part 3 action items.
Part 1 — Drift audit
- Methodology: typed-public-boundary criterion, intentional refactor
vs drift, evidence sourced from worktree files + memory dir.
- Zero core typed API drift on the integration surface — but the
realtime-runtime contract surface (health routes) IS real drift.
- 8 numbered drift items (Dreamverse README/bootstrap, prompt
enhancer fork, cerebras_ifm gap, health routes, layerwise offload,
upsampler CLI, missing example config, LTX-2 stage equivalence
verification gap).
- 6 deferred/accepted residual items.
- 17-row drift summary table with priority / effort / status /
tracked-where / next-action columns. Includes D-8 (ltx2_image_crf
flow), VPO (video_position_offset_sec semantics — overdue),
D-12-A (GpuPool docstring), D-12-B (run_async migration), SBS
(session/blob lifecycle).
Part 2 — Integration path tradeoffs
- Option A: status quo (Dreamverse separate, depends on fastvideo).
- Option B: Dreamverse as subfolder under FastVideo.
- Option C: full merge into fastvideo.entrypoints.dreamverse.*.
- Option D: hybrid — backend merges, frontend stays separate.
- Comparison matrix on 13 axes.
- 7 OSS precedent rows with citations (vLLM, BentoML, Ray Serve,
TGI, Transformers.js, ComfyUI, AUTOMATIC1111).
- Recommendation: Option D (constrained) — backend merges as
generic FastVideo streaming, frontend stays separate. Mirrors
ComfyUI's late-stage frontend split. Strong precedent: TGI +
ChatUI. Critical librarian finding: NO 1:1 precedent exists for
"Python ML library + Next.js product merged into library
namespace" — argues against Option C.
- Conditions that would change the recommendation enumerated.
Part 3 — Action items: phased migration sketch (phases 0-5) with
concrete file paths, effort estimates, and cross-references to
open-threads.md / decisions-log.md.
* README.md: register integration-review.md in deep-dive reading guide.
Bump Last reconciled to
|
||
|
|
ec3fe68b5d |
[chore] remove STACK.md — split-PR plan abandoned per D-17
The 10-PR / 6-remaining-slice split tracked by STACK.md is no longer the landing strategy. Per [decisions-log.md D-17](.agents/memory/dreamverse-integration/decisions-log.md#d-17), the remaining `will/ltx2_sr_port` content ships as a single mega-PR (#1288) instead of 6 stacked PRs. The bulk-rebase formula and split bookmark table in STACK.md no longer reflect reality, so keeping the file in the repo would mislead future agents/contributors. Already deleted: * PR #1287 (was first slice of split) — closed * origin/will/api_7.10 — remote branch deleted * Local split bookmarks (will/api_7.10, will/api_8, will/ltx2_sr_runtime, will/ltx2_nvfp4, will/ltx2_post_fixes, will/agents_cleanup) — deleted Kept: * CO-AUTHORS.md — still the canonical roster reference (mirrored in .agents/memory/dreamverse-integration/authors.md but the top-level file is the source of truth for the broader stack history). * will/ltx2_sr_port-pre-1286-rebase — local safety backup of the pre-rebase chain, scheduled for deletion after #1288 merges. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
4077a1649e |
[docs] dreamverse-integration: D-17 — abandon split, single mega-PR #1288
User decision after the post-#1286 rebase + re-slice cycle. Closing #1287 (slice 1-3); opening #1288 on will/ltx2_sr_port covering the full 34-commit chain (71 files, +13,074/-583 LOC) at once. STACK.md model deprecated; split bookmarks no longer maintained. * decisions-log.md: add D-17 with rationale (review-coordination overhead vs structural benefit, layers not actually independent, CI/merge-queue simplicity), implications (STACK.md deprecated, bookmarks stale, runbook protocol replaced), and watch-outs (PR size, fallback to re-split if main moves significantly). Bumps Last updated header. * README.md: switch narrative — single mega-PR #1288 on will/ltx2_sr_port @ |
||
|
|
86a094a418 |
[docs] dreamverse-integration: reconcile post-#1286 merge + 7.10 split
PR #1286 merged at |
||
|
|
2ebb1b28f3 |
[docs] dreamverse-integration: add runbook + fresh-context onboarding
Make the memory dir self-sufficient as an onboarding package — a fresh agent should be able to resume work end-to-end (verify, commit, push, propagate to PR #1286) using only files in this directory. * runbook.md (new): operational how-to. Covers worktree contract, branch topology, verification (pre-commit binary path, pytest, lsp_diagnostics, gh CLI), commit workflow (subject/body conventions, required co-author trailers, the multi-`-m` trailer-parsing trap and its `-F` workaround), push + PR propagation (cherry-pick from ltx2_sr_port to api_7.9 to avoid force-push), memory-dir maintenance table, common pitfalls (pre-commit binary location, stash@{0} not ours, AbsMaxFP8 pre-existing failure, untracked nested clones, live ports, branch-switch by other agents, force-push policy, two known trailerless commits in PR #1286), 8-question self-test, and a "first 60 seconds" copy-paste orientation block. * README.md: replace single "Reading guide" with a two-tier structure. New "Fresh-context onboarding (read in order)" section gives 5 ordered steps for an agent picking up the work for the first time — worktree confirmation via runbook, then state.md, pr-roadmap.md, open-threads.md, runbook.md end-to-end, finishing with the runbook self-test as a context-loaded check. Original table preserved as "Deep-dive reading guide" with a new entry for runbook.md. Bumps "Last reconciled" header to 2026-05-05 / `09647a30` and updates the PR status line (PR #1284 merged; PR #1286 open at `a152cb77`, MERGEABLE). * state.md: bump "Current State" header from 2026-05-03 to 2026-05-05. Branch-tips table refreshed — `will/ltx2_sr_port` now at `09647a30` (was `156103b9`), 32 commits ahead of i2v base; new row for `will/api_7.9` (PR #1286 head `a152cb77`) noting it's an ancestor of the working branch. Added cross-links to runbook.md and authors.md in the front matter, plus a note about other agents sharing the worktree (covers the branch-switch scenario without inventing reconciliation logic). Verified: pre-commit clean (memory dir is yapf/ruff/mypy excluded; only spaces-check runs). All cross-links resolve to siblings in this directory or to top-level CO-AUTHORS.md / STACK.md / AGENTS.md as appropriate. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
eae529a22d |
[docs] dreamverse-integration: add authors.md + track D-16 router polish
* authors.md (new): dreamverse-integration-scoped mirror of the top-level CO-AUTHORS.md. Documents the 4 human co-authors credited on every commit in the integration scope, verified against PRs #1257/#1258/#1284/#1286. Self-contained so the memory dir is discoverable without traversing to the repo root. Cross-references CO-AUTHORS.md and STACK.md. Includes a 'Known gaps' section flagging that commits a152cb77 / 40e265b8 (the [fix] streaming: router polish pair from earlier in this session) are missing trailers and need an amend + force-push to fix. * README.md: register authors.md in the reading guide table. * decisions-log.md: add D-16 — Streaming router polish round 2 — capturing the 5 second-pass fixes applied on top of D-15's pre-merge polishes (bridge cancellation hygiene, registry UNKNOWN->HEALTHY immediate, httpx hard-fail, URL path/query/fragment/duplicate validation, per-index YAML parser errors, explicit websockets [streaming] dep, +7 test cases). Bumps Last updated header. * open-threads.md: bump Last updated header to mention the second-pass router commit and link to D-16. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
SolitaryThinker
|
8596c072ab |
[docs] dreamverse-integration: track D-15 + #13/#14/#15 — PR #1286 arch review
Adds D-15 to decisions-log.md: Oracle review of PR #1286 streaming router. Verdict — keep current shape (Alt A): in-repo at fastvideo/entrypoints/streaming/router/, FastAPI-based, single-primary failover, lazy httpx/websockets imports. Reject Alt B (separate package), Alt C (fold into server), Alt D (delegate to nginx/envoy/HAProxy as the sole answer), Alt E (sticky now), Alt F (round-robin now). Captures the 4 gemini review fixes: * High: select() docstring claimed round-robin but always returns [0] — rewrote with explicit MVP semantics * Medium: per-probe httpx.AsyncClient prevented connection reuse — refactored run_health_check_loop to share one client via _build_default_probe() async context manager * Medium: sequential probes could fall behind interval — now use asyncio.gather for parallel per-cycle probes * Medium: parser duplication — kept manual mapping intentionally because YAML has nested health_check: block while RouterConfig is flat; restructuring is beyond this PR's scope Plus 1 Oracle pre-merge polish: * RouterConfig.__post_init__ validation — empty replicas, non-positive intervals/timeouts, thresholds < 1, non-http(s) URLs, and >1 primary all raise ValueError so misconfigurations surface at config-load instead of confusing runtime failures Plus 1 prep-rebase item carried in: * Migrated FastAPI @app.on_event(startup/shutdown) to a single @contextlib.asynccontextmanager-based _lifespan() handler (deprecated API in newer FastAPI; was the 7.9 caveat in pr-roadmap.md) Adds open-threads.md items #13/#14/#15: * #13: sticky session routing extensibility (forward-compat from D-15) * #14: bridge backpressure note for high-scale deployments * #15: multi-primary semantics if active-active becomes required |
||
|
SolitaryThinker
|
952e0da232 |
[docs] dreamverse-integration: track D-14 + #12 — PR #1284 arch review
Adds D-14 to decisions-log.md: Oracle review of PR #1284 streaming auxiliaries. Verdict — keep current shape (Alt A): single PR, 4 modules under streaming/, mock_server in production module path, concrete PromptSafetyFilter. Reject Alt B (split into 4 PRs), Alt C (move mock to tests/), Alt D (premature observability extraction), Alts E/F (premature Protocol-ization). Captures the 4 gemini review fixes (high: session_logger race vs close; medium: rewrite regex in hot path, safety _ensure_loaded race, streaming extra missing prompt-safety) and 2 pre-merge polishes from Oracle (remove inert RewriteOptions.user_system_prompt_override field, sanitize session_id for filename safety as defense-in-depth). Adds open-threads.md item #12 (Low): when streaming server starts using PromptSafetyFilter, ensure operator-visible logging on SafetyDecision.UNAVAILABLE results — surfaces degraded-safety state per D-14's Watch-Out note. |
||
|
SolitaryThinker
|
0bb7bfae3e |
[docs] dreamverse-integration: reflect 7.5/7.6/7.7 merged + 7.8 opened
Updates following the PR landings during this session: * pr-roadmap.md: move PR 7.5 (#1251 merged 2026-04-26 as |
||
|
SolitaryThinker
|
75b613574e |
[docs] dreamverse-integration: track D-13 + 9 new follow-ups from this thread
Adds D-13 to decisions-log.md mirroring D-12's format: PR #1258 prompt enhancer / LLMProvider abstraction shape review by Oracle. Verdict — keep current shape (Alt A); promote to top-level fastvideo.prompt.* only when a second non-streaming consumer exists. Don't convert Protocol → ABC. Three deferred polishes captured. Adds 9 new items to open-threads.md priority table: * DR-1 (High): Dreamverse must create prompting/_internal_compat.py shim and delete most of the 1933-LOC local prompt_enhancer.py once PR #1258 merges. * DR-2 (Med): Decide cerebras_ifm provider path — public Literal vs. Dreamverse-side custom provider via register_provider(). * D-12-A (Med): GpuPool ABC docstring — mark experimental / server-internal until PR 7.10 cycle. * D-12-B (Med): Replace GpuPool.run() -> Any with run_async() -> AsyncIterator[VideoEvent] in PR 7.10 cycle. * D-13-A (Med): Document streaming/prompt/* as streaming-scoped in user-facing docs; avoid framework-level framing. * D-12-C (Low): Avoid locking PoolAssignment.gpu_id: int as public; prefer worker_id (already exists) or device_ids: list[int] for future multi-GPU-per-worker. * D-13-B (Low): Optional client_factory parameter for httpx pooling if metrics justify. * #9 (Low): PR 8's three commits still have [8/n] Improve API: prefix — regex didn't match single-digit version. Cleanup when PR 8 is opened. * #10 (Low): PR 7.8/7.9 commits have streaming: streaming X duplication. Cleanup when 7.8/7.9 are opened. * #11 (Low): Promote LTX-2 prompt orchestration (locked segments, segment_prompts JSON shape) to public when a second consumer appears. Updates recommended pull order to include the new items. |
||
|
SolitaryThinker
|
8039c86244 |
[docs] dreamverse-integration: track GpuPool architecture decision (D-12)
Adds D-12 to decisions-log.md, documenting the Oracle review of whether GpuPool (PR #1257, just merged) should be folded into VideoGenerator. Decision summary: keep GpuPool separate from VideoGenerator (Alt A) as interim, evolve to Alt C (thin async executor over generate_async) once PR 7.10 lands. Do not pursue Alt B (folding into VideoGenerator). Captures: * Three alternatives evaluated (A: status quo, B: VideoGenerator absorbs pool role, C: async executor over generate_async) * Key finding: MultiprocExecutor and SubprocessGpuPool are orthogonal, not redundant — both spawn subprocesses because CUDA contexts demand process boundaries, but they solve different problems (one-call-fast vs. many-sessions-throughput) * Sticky binding stays in pool, NOT in VideoGenerator (different consumers want different policies — sticky for streaming, lease for HTTP, queue for per-frame real-time) * Risks flagged: GpuPool.run() sync shape, PoolAssignment.gpu_id: int freeze, and the 'is this canonical serving API?' framing question Action items deferred to PR 7.10 cycle: replace run() with run_async() returning AsyncIterator[VideoEvent], clarify worker_id vs gpu_id field naming, mark GpuPool docstring as experimental until 7.10. |
||
|
|
af92021aa3 |
[docs] add STACK.md + CO-AUTHORS.md trackers for the will/ltx2_sr_port stack
* STACK.md tracks the 10-PR split of will/ltx2_sr_port: per-PR slice ranges, tip SHAs (live, not hardcoded), independence map, landing strategy, fast-track candidates, and re-slice commands to run after any rebase or trailer injection. * CO-AUTHORS.md documents the 4 GitHub users credited as co-authors via Co-authored-by trailers on every commit in the stack: - @Davids048 (Junda Su) - @RandNMR73 (Matthew Noto) - @XOR-op - @jzhang38 (Zhang Peiyuan) All 4 verified active in FastVideo-internal git history. Trailer emails use GitHub's <id>+<username>@users.noreply.github.com form for reliable account linkage. Both files are temporary trackers (delete after the full stack merges or as documented at the bottom of each file). Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
4b154835cd |
[docs] .agents/memory: add dreamverse-integration knowledge base
Consolidates the public-API refactor + LTX-2 streaming-server upstream
+ Dreamverse migration + NVFP4 quantization landing into a single agent
memory module. Future agents loading the dreamverse-integration story
can pull a single targeted slice (state, design, PR roadmap, streaming,
cross-repo, quantization, decisions, open threads) rather than wading
through ~200 KB of source documents.
Structure:
.agents/memory/dreamverse-integration/
├── README.md ← index + glossary + reading guide
├── state.md ← branches, commits, live services
├── design.md ← typed schema design
├── pr-roadmap.md ← PRs 0-17 status
├── streaming-server.md ← PRs 7.5-7.10 + Dynamo + build_app
├── cross-repo-surfaces.md ← Dreamverse 3-surface + Dynamo contract
├── quantization.md ← NVFP4 + LinearBase fallback + AbsMaxFP8
├── decisions-log.md ← D-1..D-11 + Q-1..Q-9 status
├── open-threads.md ← 12 active items prioritized
└── source-archive/ ← 7 archived source docs (pre-synthesis)
Source-archive contents (all previously untracked at repo root or in
.agents/exploration/): apirefactor.md (838 lines), PR-plan.md (1145
lines, was 'PR plan.md'), dreamverse_review.md (390 lines),
handoff-nvfp4-launch-demo.md, streaming-server-upstream-plan.md,
dreamverse_integration.md, video-generator-config-api-design.md.
Registered in .agents/memory/index.jsonl as
{name: dreamverse-integration, status: ready, trust: high}.
Also adds CLEANUP.md (temporary tracker) for the multi-phase .agents/
cleanup. Phase 1 (deletes) shipped in the prior commit; Phase 2+
(rewrites, additions, registry consolidation) are tracked there for a
follow-up.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
4a2a11f6b0 |
[chore] .agents: drop stale STATUS.md + sync-dashboard SOP + vapor skills
Phase 1 of .agents/ cleanup. All deletions are stale, redundant, or
unused; .agents/ memory + skills + workflows remain functional.
Removed:
* STATUS.md — hand-maintained dashboard, last synced 2026-03-02 with
wrong counts (claimed 8 skills/4 workflows/4 memory; actual 9/5/5)
and references to non-existent snake_case filenames. Strictly
redundant with .agents/{memory,skills}/index.jsonl.
* workflows/sync-dashboard.md — SOP that maintained the deleted
STATUS.md, with obsolete pre-PR-4 file paths.
* skills/index-related-work/, skills/search-related-work/ — vapor
skills operating on the empty .agents/memory/related-work/ registry.
The search skill literally requires 'index has entries' as a
prerequisite but none have ever been added. Re-introduce when the
registry is actually populated.
Updated skills/index.jsonl to drop the two removed entries (was 9
entries; now 7).
.agents/scripts/sync-skills.sh ran post-deletion and pruned 2 stale
.claude/skills/ symlinks pointing at the removed skill directories.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
c094262751 |
[fix]: unwrap list-of-generator before torch.randn in LTX-2 latent prep
InputValidationStage normalizes generator to a one-element list when num_videos_per_prompt == 1, but torch.randn only accepts a single torch.Generator. Unwrap the list here to keep the single-sample path working. Batched sampling (>1) currently collapses to the first generator — flagged in a comment for the future batched-inference path. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
a750c25bdd |
[fix]: avoid model.to() round-trip in Gemma encoder forward
Wrap the device move in an equality guard so Dynamo can DCE it under fullgraph=True (model.to() goes through _parse_to, which returns a non-Tensor torch.device that Dynamo can't trace). The model is already placed on the target device by prepare_for_compile, so the guard is a runtime no-op. Also drop the orig_device save/restore — the encoder should stay resident on the compute device between calls. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
aca68d9452 |
test(nvfp4): lock LTX-2 wiring + typed transformer_quant flow
Two new contract test files plus a few stale-name doc fixups so the
LTX-2 NVFP4 path can't silently regress.
* ``test_nvfp4_ltx2_wiring.py`` (6 tests): asserts that
- ``LTXSelfAttention``'s ``to_q/to_k/to_v/to_out`` are
``ReplicatedLinear`` (not plain ``nn.Linear``);
- ``NVFP4Config()`` attaches ``NVFP4QuantizeMethod`` to the
quantized subset and the ``layer_prefix`` matches the
``ltx2.blocks.<i>.<sub>.<proj>`` paths in
``NVFP4Config.fp4_layers``;
- non-tagged projections (cross-attn K/V, audio attn, audio FFN)
fall back to ``UnquantizedLinearMethod`` instead of crashing on
the quant_method assert;
- ``BasicAVTransformerBlock`` propagates ``quant_config`` and
``prefix`` correctly to all 4 attention modules + FFN at once.
These are CPU-only and don't need flashinfer.
* ``test_typed_quant_flow.py`` (4 tests): asserts that
- typed ``engine.quantization.transformer_quant: "NVFP4"`` resolves
through ``compat.py`` to a concrete ``NVFP4Config()`` instance;
- omitting the typed surface leaves the ``transformer_quant``
carrier ``None`` so legacy callers that mutate
``pipeline_config.dit_config.quant_config`` directly keep working;
- ``__post_init__._apply_transformer_quant`` pins the carrier onto
``dit_config.quant_config``;
- explicit ``dit_config.quant_config = …`` is preserved (the
explicit setter wins over the typed carrier).
Plus stale ``FP4Config`` → ``NVFP4Config`` doc updates in
``api/compat.py``, ``fastvideo_args.py``, and ``layers/linear.py``
that the previous rename commit missed (comments and docstrings only,
no behavior change).
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
b24c4f0d61 |
refactor(quant): rename FP4 → NVFP4 to disambiguate from other FP4 variants
The FP4 implementation is specifically NVIDIA's block-scaled FP4 (e2m1 mantissa, fp32 alpha, ``layout_128x4`` scale layout, group size 16) backed by FlashInfer's ``nvfp4_quantize`` and ``mm_fp4`` kernels. Naming the public surface ``FP4`` would collide with other FP4 variants we may want to support later (OCP-FP4 / MX-FP4 / e3m0). Mechanical rename, no behavior change: * ``fp4_config.py`` → ``nvfp4_config.py`` * ``FP4Config`` → ``NVFP4Config``; ``get_name()`` returns ``"nvfp4"`` * ``FP4QuantizeMethod`` → ``NVFP4QuantizeMethod`` * ``convert_model_to_fp4`` → ``convert_model_to_nvfp4`` * ``QuantizationMethods`` literal: ``"FP4"`` → ``"NVFP4"`` * registered buffer names: ``_fp4_weight``/``_fp4_alpha`` → ``_nvfp4_weight``/``_nvfp4_alpha`` * loader helper ``_maybe_convert_model_to_fp4`` → ``_maybe_convert_model_to_nvfp4`` * test file rename + symbol updates * Dreamverse worker updates its single import + call site Internal-scope torch op namespace ``fastvideo_fp4::*`` and ``_get_ltx2_fp4_stage_profile`` left as-is (purely internal naming that mirrors FastVideo-internal). Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
3a0f39d626 |
feat(ltx2): wire FP4 inference through fastvideo.layers.quantization
The public LTX-2 path was previously running full bf16 even when callers set ``pipeline_config.dit_config.quant_config = FP4Config()``, because: * the DiT used plain ``torch.nn.Linear`` for FP4-eligible projections (``attn1/attn2.to_q/k/v/out``, ``audio_to_video_attn``, ``video_to_audio_attn``, FFN ``fc_in``/``fc_out``), so ``LinearBase.quant_method.apply`` was never reached; * the loader did not call ``convert_model_to_fp4`` after weights loaded, so even matched layers had no ``_fp4_weight*`` buffers; * ``QuantizationMethods`` did not register ``"FP4"``, so the typed ``engine.quantization.transformer_quant`` surface had no way to select it. This commit completes the inference wire-up through the existing ``fastvideo.layers.quantization`` registry — no parallel pathway: * swap ``nn.Linear`` → ``ReplicatedLinear`` for the LTX-2 attention ``to_q``/``to_k``/``to_v``/``to_out``/``to_gate_compress`` and the FFN ``fc_in``/``fc_out`` (``GELUApprox.proj`` and ``FeedForward`` ``project_out``). Other linears (timestep MLP, caption proj, patchify, AdaLN scale/shift) stay ``nn.Linear`` to match internal. * port ``_supports_prequantized_input`` and ``_linear_project_with_optional_prequant`` so the attention forward quantizes input once and reuses the ``(x_fp4, x_scale, x_global_sf)`` tuple across q/k/v projections — matches internal exactly. * plumb ``quant_config`` and ``prefix`` through ``BasicAVTransformerBlock`` → ``_init_transformer_blocks`` → ``LTXModel`` → ``LTX2Transformer3DModel`` so each ``ReplicatedLinear`` gets the correct ``ltx2.blocks.<i>.<sub>.<proj>`` prefix that ``FP4Config.get_quant_method`` matches against. * register ``"FP4"`` in ``QuantizationMethods`` literal + ``get_quantization_config`` registry so typed ``engine.quantization.transformer_quant: "FP4"`` resolves to a concrete ``FP4Config()`` instance. * add ``transformer_quant`` carrier on ``FastVideoArgs`` and ``__post_init__._apply_transformer_quant`` to pin the resolved quant config onto ``dit_config.quant_config`` (without overwriting an explicit setter on the dit_config). * add ``_maybe_convert_model_to_fp4`` in ``fsdp_load.py`` that walks the loaded model once and registers ``_fp4_weight*``/``_fp4_alpha``/``_weight_global_sf`` buffers on layers whose ``quant_method`` is ``FP4QuantizeMethod``. flashinfer is imported lazily inside ``convert_model_to_fp4`` so this is a no-op on hosts without the FP4 kernels. * restore ``LinearBase`` fallback to ``UnquantizedLinearMethod`` when ``quant_config.get_quant_method`` returns ``None`` — ``FP4Config`` only tags a curated subset of layers, and the previous ``assert quant_method is not None`` would crash any non-tagged layer that received a quant_config. Non-FP4 callers are unaffected: ``UnquantizedLinearMethod.apply`` runs the standard ``F.linear`` path, and parameter names (``to_q.weight``, ``to_out.0.weight`` from ``nn.ModuleList``) are identical to the previous ``nn.Sequential`` layout. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
d21676efa7 |
feat(compile): per-component compile + transformer_refine + prepare hook
Bring the public ``ComposedPipelineBase.post_init`` compile dispatch to feature parity with FastVideo-internal: * Compile ``transformer_refine`` alongside ``transformer`` and ``transformer_2`` whenever the DiT compile flag is on. Without this the LTX-2 stage-2 refine pass silently runs eager while the main transformer is compiled — different inductor fusion choices vs. eager execution can produce last-bit divergences in bf16. * Drive the new per-component compile flags from ``FastVideoArgs``: text encoder (``enable_torch_compile_text_encoder``), VAE (``enable_torch_compile_vae``), audio VAE (``enable_torch_compile_audio_vae``). Each picks its own kwargs dict (``torch_compile_kwargs_*``) and falls back to the master ``torch_compile_kwargs`` when empty. * Call ``module.prepare_for_compile()`` on each compiled submodule before invoking ``torch.compile`` so model-specific external state (e.g. lazy HF loads) lands outside Dynamo's tracer. ``Gemma3`` implements this hook to materialize the Gemma weights up front. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
82e532727f |
feat(api): typed per-component CompileConfig + FastVideoArgs carriers
Extend the typed inference API so callers can drive torch.compile per component without falling back to the legacy flat-kwargs pathway. * ``CompileConfig`` now exposes ``vae_enabled``, ``audio_vae_enabled``, ``dit_kwargs``, ``text_encoder_kwargs``, ``vae_kwargs``, ``audio_vae_kwargs`` alongside the existing ``enabled`` / ``text_encoder_enabled`` switches. The master ``backend``/ ``fullgraph``/``mode``/``dynamic``/``extras`` continue to apply to every compiled submodule unless a per-component kwargs dict is non-empty (in which case it overrides entirely — matches the FastVideo-internal precedent). * Add the matching runtime carrier fields on ``FastVideoArgs``: ``enable_torch_compile_text_encoder/vae/audio_vae`` plus ``torch_compile_kwargs_dit/text_encoder/vae/audio_vae``. These let the legacy flat-kwargs path round-trip through the typed config. * ``api/compat.legacy_from_pretrained_to_config`` now lifts the new flat kwargs into ``CompileConfig``, and ``generator_config_to_fastvideo_args`` writes them back out so ``VideoGenerator.from_pretrained(model_path, ...)`` callers keep working unchanged. No behavior change yet — these surfaces are ports of carriers only. The composed pipeline base will start consuming them in the next commit. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
3ef2466810 |
fix(api): propagate generic refine_* args + match internal randn
Three small parity fixes uncovered while comparing the LTX-2 distilled path against FastVideo-internal: * `FastVideoArgs.__post_init__` now calls `_resolve_refine_args()` to copy the public-facing generic `refine_*` knobs onto their `ltx2_refine_*` runtime carriers (mirrors internal lines 303-322). Without this, callers that set `refine_lora_path=...` via the typed CLI/config surface would silently drop the value, surfacing later as the "applied to 0 layers" warning. * Revert the patch-noise sampler in `_randn_ltx2_video_latents` from `randn_tensor` back to `torch.randn` to bit-match internal under single-generator inference. `randn_tensor` is identical for a single `torch.Generator` but diverges for `list[Generator]` (per-sample seeds), which is the only place the two paths could disagree. * Classify the 19 previously-unclassified `refine_*` / `ltx2_refine_*` / `ltx2_audio_latent_path` / `ltx2_images` / `ltx2_image_crf` / `ltx2_conditioning_latent_*` / `ltx2_video_conditions` fields in the schema-parity inventory yaml so the parity test passes. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
a62b8e1e8e |
feat(ltx2): full i2v conditioning + continuation latent port
Replaces the T2V-only stub with the full internal port: * resolve_ltx2_images / _resize_and_center_crop / CRF re-encode * load_ltx2_conditioning_image + load_ltx2_conditioning_video_clip * _extract_video_latent / _insert_conditioning_latent * build_ltx2_image_conditioning composes (clean_latent, denoise_mask) from images + video clips + continuation latents (stage1 vs stage2) — returns None for plain T2V ForwardBatch + SamplingParam gain the missing fields the builder reads (ltx2_images, ltx2_image_crf, ltx2_conditioning_latent_stage1/2, ltx2_video_conditions). generate_video kwargs flow through sampling_param.update -> ForwardBatch(**shallow_asdict(...)) so Dreamverse's ltx2_image_crf=0.0 and continuation latents land on the batch correctly. 68/68 LTX-2 tests still green. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
040b7735c6 |
fix(registry): order LTX-2 detectors so distilled wins for distilled paths
The model-name detector loop in get_model_name_for_path tests both ``model_path.lower()`` AND ``pipeline_name`` against each detector, OR-ing the result. The pipeline_name for the distilled checkpoint is ``ltx2pipeline`` — a string that contains no "distilled" marker — so the *base* detector's ``and "distilled" not in path`` predicate returns True against it, even when the absolute model path clearly contains "distilled". Result: the resolver matches both LTX-2 base and LTX-2 distilled, warns "Multiple models matched … Using the first matched: '0'", and falls back to whichever was registered first. That used to be base (ltx2_base preset → cfg=3.0 / mod=3.0 / rescale=0.7 / stg=1.0), so SamplingParam.from_pretrained on a distilled-checkpoint path silently loaded the *full* LTX-2 sampling defaults — exactly the divergence the public-vs-internal Dreamverse alignment surfaced. Reorder so the distilled entry registers first. The registry's "first matched wins" tiebreak now lands on the more specific preset when both detectors fire, and SamplingParam.from_pretrained reaches the internal-aligned distilled defaults (mod=1.0 / rescale=0.0 / stg=0.0 / cfg_video=1.0 / cfg_audio=1.0). Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
f7103015a1 |
fix(api): align public SamplingParam ltx2 defaults with distilled
Internal FastVideo declares two LTX-2 SamplingParam variants:
LTX2SamplingParam (mod=3.0/rescale=0.7/stg=1.0 — full LTX-2) and
LTX2DistilledSamplingParam (mod=1.0/rescale=0.0/stg=0.0). Dreamverse
production runs against LTX2-Distilled-Diffusers, so the distilled
defaults are what the runtime should land on.
The public package only has the single SamplingParam class, and its
class-level defaults for these knobs were the *full* LTX-2 values.
Result: when Dreamverse (or any other consumer) called
``SamplingParam.from_pretrained("FastVideo/LTX2-Distilled-Diffusers")``,
the LTX2_DISTILLED preset's defaults dict didn't override these
fields (it didn't list them) so they fell through to the class
defaults — silently picking up *full*-model CFG / modality / rescale /
STG instead of the distilled ones. Streamed video output diverged
from the internal-ui reference.
Switch the class defaults to the distilled values (1.0 / 0.0 / 0.0).
The LTX2_BASE preset already overrides them explicitly to 3.0/0.7/1.0
in its ``defaults`` dict, so users selecting the full-LTX-2 preset
keep the full-model behavior.
The numerical alignment harness still reproduces the same 42.72 dB
PSNR / 52.87% exact pixel match because that test pinned the values
explicitly via legacy kwargs; this fix moves the *default* path on
the public side onto the same numerical track.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
50a792283c |
test(ltx2-sr): pin ltx2 sampling knobs in harness for parity diff
Initial alignment showed PSNR ~12 dB because public's LTX2_BASE preset sets ``ltx2_modality_scale_video=3.0 / rescale=0.7 / stg_video=1.0`` while internal's default ForwardBatch lands at ``1.0 / 0.0 / 0.0``. Pinning these explicitly on both runs aligns the denoising mechanics so the diff measures pipeline parity, not preset divergence. Result with both sides pinned to mod=1.0/rescale=0.0/stg=0.0: max_abs_diff = 203 mean_abs_diff = 0.788 rms_diff = 1.864 psnr_db = 42.72 dB exact pixels = 52.87% within 1 lvl = 86.94% within 4 lvls = 97.93% The ~47% near-but-not-exact bucket is consistent with bf16 reduction non-determinism on B200 plus the fact that the public LoRA matcher emitted "applied to 0 layers" for the LTX2-Distilled refine LoRA (internal side did not). Both runs effectively skip the refine LoRA, so they share the same effective stage-2 model — the residual diff is the right ballpark for "same algorithm, different bf16 reduction schedule". The typed-API path also sets the same overrides via setattr on the GenerationRequest so the typed path stays comparable. Once the typed SamplingConfig grows ltx2-specific fields the setattr workaround can go. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
df6fda5bb0 |
fix(ltx2-sr): close port gaps surfaced by alignment harness retries
LTX-2 SR pipeline now passes module load + stage graph build on the public side. Each retry of scripts/ltx2_sr_alignment.py surfaced one more gap; this commit closes them so the pipeline progresses to the upsample → stage-2 → decode tail. * harness: PipelineSelection (real class name; PipelineConfig was a typo) * FastVideoArgs: generic refine_* fields (preferred user-facing API) * LTX2Pipeline.initialize_pipeline: getattr-with-default for debug_model_sums / debug_model_detail (public args lack the debug fields; LTX-2 still flips the env-var toggles when set) * VAEConfig: use_temporal_scaling_frames default True (LTX-2 latent prep reads this in the schedule-shift gate) * LTX2LatentPreparationStage._randn_ltx2_video_latents: switch to randn_tensor so list-of-generators batches work (matches internal sampling order) * ModelRegistry: register LTX2LatentUpsampler in _UPSAMPLERS * component_loader: register spatial_upsampler / temporal_upsampler with UpsamplerLoader (was hitting the generic stub loader); UpsamplerLoader gains LTX-2 fallback when pipeline_config.upsampler_config isn't a multi-config tuple, plus the 'model.'-prefix-stripping weight load path the internal version uses 7 retries -> public side now reaches the upsample stage. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
870ec52127 |
test(ltx2-sr): add numerical alignment harness — public vs internal
Single script with three modes: * --label internal --output /tmp/ltx2_sr_internal.pt : runs the legacy from_pretrained kwargs path and saves frames + metadata. Used against PYTHONPATH=../FastVideo-internal as the reference run. * --label public --use-typed-api --output /tmp/ltx2_sr_public.pt : runs through the new GeneratorConfig + GenerationRequest typed API with pipeline.preset_overrides["refine"] driving the SR path. Used against PYTHONPATH=../FastVideo (the public package) for the candidate run. * --diff --reference R --candidate C : loads both .pt files and reports shape parity + max_abs / mean_abs / rms / psnr / pixel-bucket histograms over the uint8 frames. Pinned alignment fixture: PROMPT + seed + 1088x1920x121x24fps + 8 base steps + 3 refine steps mirrors basic_ltx2_upscale.py upstream so we exercise the same code path the user has been generating with. Deliberately skips FP4 (no quant config) and Dreamverse runtime knobs to keep the bf16 path clean per request. The two runs are intentionally invoked in separate processes via PYTHONPATH overrides because both repos register as the `fastvideo` package and can't be imported together. The diff script then runs in either env to load both .pt files and compute the alignment metrics. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
55b7c95998 |
feat(ltx2): wire SR pipeline graph + port denoising/latent-prep stages
Brings the public LTX2Pipeline into parity with the internal SR-capable
version end-to-end:
* LTX2DenoisingStage gains the stage-2 control surface
(sigmas_override, num_inference_steps_override, force_guidance_scale,
initial_audio_latents_key) plus the i2v conditioning round-trip
(clean_latent + denoise_mask via LTX2_VIDEO_*_KEY) and the audio
refine path. The file is now ~1:1 with the internal stage; only the
i2v_conditioning import path differs (rerouted to the public
ltx2_image_conditioning module).
* LTX2LatentPreparationStage takes vae=, exposes the official LTX-2
noise sampling order via _randn_ltx2_video_latents (patchify -> noise
in token order -> unpatchify), and applies image conditioning when
build_ltx2_image_conditioning returns a state. Matches internal.
* LTX2Pipeline switches its base from ComposedPipelineBase to
LoRAPipeline (refine-LoRA support); create_pipeline_stages adds the
refine_init / upsample / refine_lora / refine_denoising chain when
ltx2_refine_enabled is True; load_modules pulls
spatial_upsampler (and optionally transformer_refine) from the
resolved upsampler path, with model_index.json defaults
(fastvideo_refine_*) feeding the FastVideoArgs.ltx2_refine_*
carriers.
stages/__init__.py re-exports the new refine stages so importers
outside the basic/ltx2 package keep working. 68/68 LTX-2-related tests
still pass after the rewrite (api translation, gpu_pool, contract,
streaming).
Reachability: setting pipeline.preset_overrides["refine"] = {"enabled":
True, ...} and components.upsampler_weights flows through compat.py
into the new ltx2_refine_* args, the pipeline branches, and load_modules
brings in the upsampler. T2V SR is now end-to-end runnable; numerical
alignment harness comes next.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
f173bf57c5 |
feat(ltx2): port LTX-2 SR runtime — upsampler, refine stages, refine args
Lays the runtime-side pieces the LTX2_TWO_STAGE preset will exercise.
Behaviour matches FastVideo-internal 1:1 for the text-to-video SR path;
i2v / continuation conditioning is stubbed with a NotImplementedError
that fires only on those code paths so the public T2V SR run is clean.
* fastvideo/models/upsamplers/ltx2_upsampler.py — verbatim copy of the
internal LTX-2 latent upsampler (PixelShuffleND, BlurDownsample,
SpatialRationalResampler, ResBlock, LatentUpsampler, LTX2LatentUpsampler,
upsample_video). Re-exported via models/upsamplers/__init__.py.
* fastvideo/pipelines/basic/ltx2/stages/ltx2_image_conditioning.py —
minimal i2v-conditioning helpers: the four ForwardBatch.extra keys,
apply_ltx2_gaussian_noiser, post_process_ltx2_denoised, and a T2V-only
build_ltx2_image_conditioning that returns None when the request has
no image inputs / no continuation latents (the SR T2V path).
* fastvideo/pipelines/basic/ltx2/stages/ltx2_refine.py — the three
refine stages adapted to public imports: LTX2RefineInitStage halves
the request resolution and stashes the original target;
LTX2UpsampleStage runs the latent upsampler, optionally re-applies
conditioning, and mixes refine-noise scaled by the stage-2 sigma;
LTX2RefineLoRAStage swaps in a refine-specific LoRA (no-op when path
is unset).
* fastvideo/fastvideo_args.py — add the runtime carrier fields
ltx2_refine_{enabled,upsampler_path,transformer_path,lora_path,
num_inference_steps,guidance_scale,add_noise,noise_path,
audio_noise_path}. The typed public surface (ComponentConfig
upsampler_weights/lora_path + pipeline.preset_overrides.refine) keeps
flowing through compat.py:273-279, which expands the dict into these
ltx2_refine_* kwargs at construction time.
Not yet wired: the LTX2 pipeline's create_pipeline_stages still goes
denoise -> audio_decode -> decode without the refine_init / upsample
/ refine_denoise injection. The remaining piece is teaching
LTX2DenoisingStage to accept sigmas_override / num_inference_steps_override
/ force_guidance_scale / initial_audio_latents_key (the internal
version is 725 lines vs public 395; ~330 lines of stage-2 logic still
need bringing across) and to flip the pipeline graph based on
ltx2_refine_enabled. That follow-up unlocks the numerical alignment run.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
|
||
|
|
5c9a884c3f |
feat(quantization): upstream LTX-2 FP4Config with lazy flashinfer
Ports the FP4 quantization config + ops from FastVideo-internal (fastvideo.layers.quantization.fp4_config). flashinfer is imported lazily inside _require_flashinfer() rather than at module top so ``import fastvideo`` stays cheap on hosts that don't ship flashinfer — the FP4 kernels themselves still raise a clear ImportError pointing at ``pip install flashinfer-python`` when invoked. Behavior is identical to the internal version: * FP4Config: LTX-2 layer set (48 transformer blocks across attn1/attn2 /audio_to_video_attn/video_to_audio_attn/ffn plus adaln_single.linear), layer_profile selects between "base" and "refine" stage subsets. * FP4QuantizeMethod: per-layer quantize_input + apply with stage-aware routing for refine-only layers (audio_to_video_attn.to_q, video_to_audio_attn.to_k, video_to_audio_attn.to_v fall through to dense F.linear during the base stage). * convert_model_to_fp4(): pre-quantize weight buffers + alpha onto module attrs at load time so the forward path doesn't pay the per-step weight-side quantize cost. Required for Dreamverse integration: dreamverse-server's GPU worker runs ``pipeline_config.dit_config.quant_config = FP4Config()`` during LTX-2 model load, which is the dominant inference path. test_fp4_config covers the lazy-flashinfer contract: import succeeds without flashinfer; layer_profile round-trips via from_config; calling into _require_flashinfer raises ImportError with an actionable message when flashinfer is missing. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
9704a8e39e |
[test] streaming: contract tests for Dreamverse + Dynamo shapes
fastvideo/tests/contract/ guards against drift in the two public-facing adapter shapes that PR 8 locks down: * test_dynamo_shape.py: mock Dynamo-style handler implementing the adapter template from docs/design/server_contracts/dynamo.md. It imports only fastvideo + fastvideo.api, translates a Dynamo-shape NvCreateVideoRequest dict into a typed GenerationRequest, runs it through an async generator that matches endpoint.serve_endpoint(handler.generate, ...), and round-trips the continuation-state envelope. A source scan asserts the adapter function body touches no private FastVideo module. * test_dreamverse_shape.py: the flat init-time kwarg bag the internal gpu_pool.py passes today lands entirely on typed GeneratorConfig fields (no leakage into pipeline.experimental) after PR 6's typed replacement work. Per-request Dreamverse shape round-trips through legacy_generate_call_to_request -> normalize_generation_request; output.return_state (PR 7) stays wired through. Failures in this suite appear at FastVideo CI — before the Dynamo-side integration or private Dreamverse adapter see the drift. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
65cb91ed70 |
[docs] streaming: Dynamo native-backend integration reference
docs/design/server_contracts/dynamo.md documents the integration shape the ai-dynamo/dynamo native backend package consumes. Per the refactor decision, no Dynamo code lives in FastVideo — the backend package ships in the Dynamo repo at components/src/dynamo/fastvideo/ with the layout mirroring components/src/dynamo/sglang/. Captured here: * the 8-symbol public surface Dynamo imports (VideoGenerator plus ContinuationState, GenerationRequest, SamplingConfig, InputConfig, OutputConfig, VideoEvent, VideoResult) and what is available today vs. what lands in PR 7.10 * target backend package layout matching the sglang template * full NvCreateVideoRequest <-> GenerationRequest mapping table and VideoFinalEvent <-> NvVideosResponse reverse mapping * worked examples: aggregated handler (works today with generate_video + asyncio.to_thread), streaming handler (post-PR 7.10 via generate_async), health check payload (swapping to default_health_check_request() after PR 7.10) * init function sketch, registration (ModelType::Videos fast path), args adapter * contract guarantees that let the adapter be written once and not chase FastVideo drift * explicit no-import list for the adapter Also wires the server_contracts/ section into mkdocs.yml under Design. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
11bdafa812 |
[docs] streaming: OpenAI HTTP contract reference
docs/design/server_contracts/ lands the contract reference set that PR 8 locks down ahead of the streaming + Dynamo upstreams. This commit adds the index page plus the OpenAI HTTP reference: * endpoint catalogue (/v1/videos/generations, list, status, content, images, models, health) * VideoGenerationsRequest shape with SGLang-compatible extensions * merge precedence (body > default_request > hardcoded fallback) with a pointer to the explicit-path tracking mechanism that makes it work * continuation-state roundtrip on the stateless surface * HTTP error code table * explicit non-goals: flat legacy kwargs, private Dreamverse fields, raw tensor payloads — all excluded from this boundary by design Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
d81117fae1 |
[test] streaming: generate_async coverage + refreshed streaming test
* tests/contract/test_generate_async.py: event ordering (Progress -> Final); exactly-one-final per request; batch expansion emits one Final per sub-result; VideoFinalEvent carries frames / full GenerationResult / continuation_state as expected; health-check request is minimal and round-trips through normalize_generation_ request; public fastvideo.api exports the new symbols; a mock Dynamo-style async handler wraps generate_async using only the public import surface. * tests/api/test_cli_translation.py: the streaming-dispatch test no longer expects NotImplementedError (PR 7.5 landed the live server); it now injects a fake run_server and asserts the CLI hands the typed streaming ServeConfig through unchanged. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
f24c2ede40 |
[feat] streaming: VideoGenerator.generate_async + health helper
* VideoGenerator.generate_async(request) yields VideoEvent objects: one VideoProgressEvent at start, one VideoFinalEvent per resulting GenerationResult (prompt-batch expansion emits one Final per sub-result). The aggregated code path runs the existing sync generate() inside asyncio.to_thread; future work threads per-step progress events from inside the pipeline's denoise loop without changing the public contract. * VideoGenerator.default_health_check_request() returns a 256x256 / 8-frame / 1-step typed GenerationRequest. Dynamo uses it to derive its health_check_payload without knowing FastVideo internals. * _final_event_from_result packages GenerationResult into a VideoFinalEvent, reading the encoded MP4 off disk when the pipeline wrote one so streaming consumers don't re-encode. Also: streaming_server.run_server validates streaming config before loading the generator so the existing ValueError fires before the expensive model-load path. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
|
873dc3f225 |
[feat] streaming: VideoEvent hierarchy + VideoResult alias
Add the typed event dataclasses VideoGenerator.generate_async will emit, plus a VideoResult alias for the public docs: * VideoProgressEvent / VideoPartialEvent / VideoFinalEvent * VideoEvent = union of the three * VideoResult = GenerationResult Re-exported from fastvideo.api so the Dynamo backend package, the streaming server, and the stateless OpenAI server share one set of names. Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |
||
|
MookandSolitaryThinker
|
c17d33bf33 |
[ci] Replace flaky LTX-2 pixel SSIM with latent-slice cosine regression (#1253)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com> |
||
|
|
2aaeee2ab8 |
[feat] Improve API: streaming router (multi-replica load balancer + ws proxy) (#1286)
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com> Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com> Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com> Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com> |