Compare commits
90
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9f94673c34 | ||
|
|
c150889330 | ||
|
|
7f783e30b7 | ||
|
|
ec9db97431 | ||
|
|
fd236a06a7 | ||
|
|
54fe0be488 | ||
|
|
3a398a2bbd | ||
|
|
4e76897f1f | ||
|
|
95d8e74d65 | ||
|
|
bcd0ad7dee | ||
|
|
9acbf92455 | ||
|
|
ce4fa8ef2d | ||
|
|
41dcd065ae | ||
|
|
c0cb2228ac | ||
|
|
2fa0f70ee8 | ||
|
|
e172215880 | ||
|
|
23e3b325a4 | ||
|
|
44d0990199 | ||
|
|
59b6578906 | ||
|
|
fd8372d172 | ||
|
|
33eb7ea13c | ||
|
|
051d21958d | ||
|
|
bf29e20bf5 | ||
|
|
291fa5d9a6 | ||
|
|
954daf7fb3 | ||
|
|
e05c04a2f6 | ||
|
|
9fb74b9732 | ||
|
|
521dee0e82 | ||
|
|
229419208e | ||
|
|
65f3b946b9 | ||
|
|
191fcbf46c | ||
|
|
755a4e4470 | ||
|
|
9709b7513b | ||
|
|
32cd603515 | ||
|
|
d4bdd3621a | ||
|
|
e2f8322842 | ||
|
|
6966f9e0bc | ||
|
|
743f4ed5f9 | ||
|
|
1c04ace573 | ||
|
|
6cbff73687 | ||
|
|
dec8b10939 | ||
|
|
133a5278af | ||
|
|
da856274cc | ||
|
|
a253856147 | ||
|
|
6e25d94ebc | ||
|
|
cae8fa18dc | ||
|
|
821e5a0832 | ||
|
|
c1abc42782 | ||
|
|
ef15ea2391 | ||
|
|
1ea2517e22 | ||
|
|
0c63528c59 | ||
|
|
b063f8ca41 | ||
|
|
e7fff0173a | ||
|
|
d82abc271e | ||
|
|
970409962f | ||
|
|
055586703d | ||
|
|
5d89f86675 | ||
|
|
19a51a1fe6 | ||
|
|
d3232cea5a | ||
|
|
0c90c8c24d | ||
|
|
4c08ffce49 | ||
|
|
af4a77553c | ||
|
|
c096fda1eb | ||
|
|
8f47e85be0 | ||
|
|
90d3bd19eb | ||
|
|
afb4f7d3c5 | ||
|
|
02e1143f22 | ||
|
|
e2f4d1a7b5 | ||
|
|
f037351146 | ||
|
|
1ee11e08dc | ||
|
|
595f0ea60e | ||
|
|
d921832cd2 | ||
|
|
629697629a | ||
|
|
a25313beec | ||
|
|
dbde64385b | ||
|
|
9d909f5f04 | ||
|
|
76b0550c15 | ||
|
|
384c1e9493 | ||
|
|
b1dbcc93f6 | ||
|
|
b93833772e | ||
|
|
30b523edd6 | ||
|
|
6aab7f3832 | ||
|
|
9cd53fe5f8 | ||
|
|
6a32cf3a5e | ||
|
|
98be9b3da2 | ||
|
|
c53e85b767 | ||
|
|
40a8bd2d3b | ||
|
|
31aa115611 | ||
|
|
98ac10a528 | ||
|
|
a5a6d171e5 |
@@ -1,207 +0,0 @@
|
||||
# v2 ← M\*: Architecture Gap-Analysis & Improvement Roadmap
|
||||
|
||||
**Status:** exploration, flagged for review. **Date:** 2026-06-19.
|
||||
**Source paper:** *M\*: A Modular, Extensible, Serving System for Multimodal Models* (arXiv 2606.12688,
|
||||
Stanford/UW/CMU; Jha, Sagan, Kamahori, …, Kasikci, S. Wang). It is a universal serving runtime for composite
|
||||
multimodal models built on the **Walk Graph** abstraction (a model is a dataflow graph `G`; a request is a
|
||||
*Walk* — a labeled subgraph — and the runtime executes walks). It beats vLLM-Omni (~20% lower T2I latency on
|
||||
**BAGEL**, up to 2.64× on I2I), SGLang-Omni (2.7× TTS throughput on **Qwen3-Omni**), and native V-JEPA2
|
||||
rollout (12.5×). It explicitly names **FastVideo's own** sparse/sliding-tile attention, xDiT/PipeFusion/USP,
|
||||
Inferix, and FlashDrive as techniques integratable into the graph runtime.
|
||||
|
||||
**Method:** a 28-agent workflow — 6 parallel v2-subsystem maps → 10 M\*-dimension analyses, each
|
||||
*adversarially verified against the actual v2 code* → synthesis + a completeness critic. The critic's
|
||||
corrections and three P0 claims were then **spot-verified by hand** (file:line below). This doc folds those
|
||||
corrections in; it is the corrected, authoritative synthesis.
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive summary
|
||||
|
||||
v2 already implements the **harder half** of M\*'s thesis and in several axes **exceeds** it:
|
||||
|
||||
- v2's `Program` *is* M\*'s graph `G` (typed `ComponentNode`/`ModelLoopNode` + edges).
|
||||
- v2's `shared_weight_components` *is* M\*'s cross-Walk node sharing — BAGEL/Cosmos3/LTX2 each bind two
|
||||
`ModelLoopNode`s to **one resident transformer** (`instance.component()` returns the same live object). This
|
||||
is the exact MoT serving property the omni cards in this repo already express.
|
||||
- v2 adds three things M\* (serving-only) has **no equivalent for**: a required+validated per-loop **cost
|
||||
model**, a non-negotiable **interleave bit-parity gate**, and an **integrated training plane** (RL→distill
|
||||
flywheel driving the *same* serving Loop).
|
||||
- The `extend/` plugin seam (interceptors/observers/registry with capability negotiation) is precisely the
|
||||
hook M\*'s "extensible / integrate FastVideo-STA, xDiT, Inferix, FlashDrive" call-out asks for — **v2
|
||||
already has the seam M\* only gestures at.**
|
||||
|
||||
What v2 lacks is M\*'s **declarative authoring layer above the substrate**, and — the key insight — *much of
|
||||
that substrate is already authored but inert*: v2 has declared the metadata for "minimum components per
|
||||
request" (`required_for`/`optional_for` on every omni card) and "branch as a cache axis" (`guidance_sig`,
|
||||
`CacheKey`) but **never wired it to an executor**. The substrate is ~80% built and switched off.
|
||||
|
||||
**Highest-leverage cluster:** three small, parity-safe wires that turn on inert substrate and unblock the
|
||||
BAGEL/Qwen-Omni/Cosmos3 latency wins M\* measured **on the exact models this repo already runs** — plus one
|
||||
P1 that aligns v2 with the paper's headline "extensible" claim using a seam v2 already has.
|
||||
|
||||
### Verified P0 correctness findings (spot-checked by hand)
|
||||
1. **Runner divergence (real bug).** `v2/runtime/engine.py:88` → `nodes = self.program.nodes`;
|
||||
`v2/runtime/disaggregated.py:96` → `nodes = self.program.active_nodes(self.request)`. The inline and
|
||||
disaggregated runners execute *different node sets*. ✅ confirmed.
|
||||
2. **EOS is faked.** `v2/recipes/omni/ar_loop.py` docstring says "done on EOS/max_tokens"; `next()` (`:46-48`)
|
||||
checks **only** `max_tokens`. M\*'s marquee `DynamicLoop` use case (EOS) is unimplemented in the loop that
|
||||
serves the Qwen-Omni Thinker/Talker and Cosmos3 reasoner. ✅ confirmed.
|
||||
3. **`required_for`/`optional_for` have zero runtime consumers** (grep outside `specs.py`/recipes/tests is
|
||||
empty). The min-components metadata is declared on every card and never read. ✅ confirmed.
|
||||
|
||||
---
|
||||
|
||||
## 2. Dimension table (corrected)
|
||||
|
||||
| # | Dimension | v2 status | Gap | Priority | Effort | Payoff | Action |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 1 | Min-components per request (`required_for` + `when_task`) | substrate built, **inert** | real, cheap | **P0** | S | Consume `required_for` in `active_nodes`; unify `engine.py:88` onto `active_nodes`; deliver via registry/card builder so all ~40 cards inherit it |
|
||||
| 2 | Real EOS + declarative `DynamicLoop` | early-exit emergent; **EOS faked** | real | **P0** | S | `ARDecodeLoop` honors `eos_id` + `req.sampling.stop`; add `LoopSpec.dynamic_stop` + `register_loop_stop`. **Training-enabling** (world-model rollout horizon) |
|
||||
| 3 | CFG/branch as label over one paged KV pool | absent (`PagedKVCache` is a counter) | real | **P1** | L | `(namespace,label)` paged store w/ one budget; reuse `guidance_sig` for hash (NOT `partition_field`); by-ref via existing `InProcKVConnector`. AR path only (diffusion has no KV) |
|
||||
| 4 | `extend/` plugin seam → integrate FastVideo-STA / Inferix | **seam exists, unused for attn** | real (paper headline) | **P1** | M | Expose FastVideo sparse/sliding-tile attention + Inferix block-diffusion as `Interceptor`/`EngineKind` plugins — the paper's named integration targets, on this repo's own code |
|
||||
| 5 | `ParitySpec.output_determinism` (C3 distributional) | C3 rung defined, **0 users** | real, dormant | **P1** | S | Add field; `compare_outputs` consults it. **Training-enabling** (SDE/FlowGRPO stochastic rollouts) |
|
||||
| 6 | Registry-driven delivery of #1 | present, not leveraged | integration | **P1** | S | Express `when_task`/min-components through `WorkflowRegistry`/card builders, not 3 bespoke recipe patches |
|
||||
| 7 | Serving conductor + pluggable data plane | conductor exists (`serving/http.py`); **single-process transport** | real | **P2** | L | v2 already has the step-scheduled worker surface; gap is ZeroMQ/Mooncake + direct worker→worker tensor routing (today `InProcKVConnector` only) |
|
||||
| 8 | Fleet/Dynamo placement + replicas | **live** (`deploy/fleet.py`,`dynamo.py`) | partial | **P2** | M | Fleet-level placement/affinity/replica is real & ≥M\*; missing piece is only the intra-engine `(node,Walk)→rank` map decoupled from model code |
|
||||
| 9 | Per-node TP / SP + cross-rank transport | axis vocab **exists** (`sp` incl.); not wired to runtime | partial | **P2** | XL | Wire declarative degrees into runtime; Wan/LTX are **SP-native** (TP is a no-op there); populate `parallel_plan_hash` on the serving cache path |
|
||||
| 10 | Named Walks + per-model state machine | `Program`=G, sharing real; no Walk/SM | real | **P2** | M | Defer until a *re-entrant* phase graph (Thinker↔Talker, rollout) needs it; #1 captures the min-components win without it |
|
||||
| 11 | Declarative `Parallel/Sequential/Loop` IR | imperative loop classes | real (authoring) | **P2** | M | Thin Section IR lowering to flat `Program`; scope to one AR recipe |
|
||||
| 12 | Streaming `ChunkPolicy` + `StreamBuffer` | causal-chunk emit **already ships** (`wan_causal`); `EdgeKind.STREAM` inert | real | **P2** | L | Declarative `ChunkPolicy` vocab over the existing chunk mechanism; needs concurrent producer/consumer runner (= pipelined scheduling). Inferix integration point |
|
||||
| 13 | Speculative deferred-termination; loop-spanning CUDA graphs; N+1 prefetch; attn double-buffer | absent / per-step capture (14 cards) | real | **P3** | L | Gate behind a real GPU executor; unobservable on CPU-toy CI; loop-span needs an `allows_interleaving=False` carve-out |
|
||||
| — | Cost model + interleave/consistency parity | **exceeds M\*** | none | **guard** | — | Do not regress; keep `step_cost_model` mandatory + `bit_identical` default |
|
||||
| — | Integrated training plane (flywheel, weight-sync) | **exceeds M\*** | none | **guard** | — | Protect train==serve loop identity with a toy fixture |
|
||||
|
||||
---
|
||||
|
||||
## 3. P0/P1 deep-dives (sequenced)
|
||||
|
||||
```
|
||||
PR-1 (P0) min-components ──┐
|
||||
PR-2 (P0) real EOS ─┼─► prereqs for honest "DynamicLoop" + min-component claims; both training-enabling
|
||||
PR-3 (P1) output_determinism (independent)
|
||||
PR-5 (P1) extend/ plugin: FastVideo-STA / Inferix as Interceptors (independent; highest paper-alignment)
|
||||
PR-4 (P1) CFG-as-label paged pool ──► depends on PR-2 (AR loop is the only KV consumer)
|
||||
```
|
||||
PR-1, PR-2, PR-3, PR-5 are mutually independent; PR-4 depends on PR-2.
|
||||
|
||||
### PR-1 (P0) — Turn on the inert min-components substrate + fix runner divergence
|
||||
- **Change.** Extend `Program.active_nodes(request)` (`v2/program/specs.py`) to also drop any node whose bound
|
||||
`ComponentSpec.required_for` (`v2/card/specs.py:144`) excludes `request.task` (and isn't in `optional_for`).
|
||||
**Fix the bug:** change `v2/runtime/engine.py:88` to `nodes = self.program.active_nodes(self.request)` so the
|
||||
inline `ProgramRunner` matches `DisaggregatedRunner` (`disaggregated.py:96`). Deliver the `when_task` gating
|
||||
through the **registry/card builder** (`recipes/__init__.py`, `program/workflow.py:WorkflowRegistry`) so all
|
||||
~40 cards inherit it uniformly — not three bespoke `program.py` patches.
|
||||
- **Why (this repo's models).** BAGEL T2I currently steps the AR-text loop and Cosmos3 t2v materializes the
|
||||
reasoner even though the cards declare `transformer required_for={'reason','t2i'}`, `vae required_for={'t2i'}`.
|
||||
On the GPU backend that is wasted resident-weight load + wasted steps on every single-modality request —
|
||||
exactly M\*'s "execute the MINIMUM components per request," delivered by consuming existing metadata.
|
||||
- **Risk/invariant.** Validate in `ModelCard.validate()` that every active node's `reads` are produced by an
|
||||
active node for each declared `TaskType` (avoid dropping a producer). Pure node-id filtering ⇒ serial and
|
||||
interleaved still walk the same filtered list ⇒ §9.3 interleave bit-parity holds by construction. CPU-toy clean.
|
||||
|
||||
### PR-2 (P0) — Real EOS + declarative `dynamic_stop` *(also training-enabling)*
|
||||
- **Change.** In `v2/recipes/omni/ar_loop.py`, `advance()` reads the emitted token; if it equals the model
|
||||
`eos_id` (toy backend exposes `EOS=0`) or matches `req.sampling.stop` (`params.py:21`, currently dead),
|
||||
register termination; `next()` returns `Done()` on stop OR `max_tokens`. Add `StopRegistry` to `LoopState` +
|
||||
`register_loop_stop(name)` to the `LoopContext` protocol (`contracts.py:204`) and to
|
||||
`DisaggregatedRunner`'s `RuntimeLoopContext`. Add `LoopSpec.dynamic_stop: bool=False`, opt the AR cards in.
|
||||
- **Why.** The docstring-vs-code lie sits in the loop serving Qwen-Omni Thinker/Talker and the Cosmos3 reasoner;
|
||||
M\*'s second named `DynamicLoop` use case (world-model **rollout horizon**) is exactly what `self_forcing` RL
|
||||
needs — so this is both a serving-credibility fix and a training enabler (raise its payoff accordingly).
|
||||
- **Risk/invariant.** `dynamic_stop=False` is byte-identical back-compat. Must pass **all three** parity gates:
|
||||
serial==interleaved AND disaggregated==inline. **Not** in this PR: speculative deferred-termination (unobservable
|
||||
on CPU-toy, fights the interleave invariant — P3, gated on GPU executor).
|
||||
|
||||
### PR-3 (P1) — `ParitySpec.output_determinism` (close the dormant C3 hole) *(training-enabling)*
|
||||
- **Change.** Add `output_determinism: str = "bit_identical"` to `ParitySpec` (`card/specs.py:88`); make
|
||||
`compare_outputs` (`parity/interleave_gate.py:54`) consult it (`bit_identical` → today's exact check;
|
||||
`distributional` → a moment/tolerance check — land a simple moment match first; a real KS test is new code).
|
||||
- **Why.** `ConsistencyLevel.C3` is defined and used by zero recipes; an SDE/FlowGRPO stochastic rollout cannot
|
||||
honestly declare its parity contract and would falsely fail the bit-identical gate. Additive; default unchanged.
|
||||
|
||||
### PR-5 (P1) — Expose FastVideo's own attention + Inferix as `extend/` plugins *(highest paper-alignment)*
|
||||
- **Change.** Use the existing `extend/{interceptors,observers,registry}.py` seam (capability-negotiated, with
|
||||
per-(request,branch) `plugin_state` that already passes the interleave gate) to register FastVideo's
|
||||
sparse/sliding-tile attention and Inferix-style block-diffusion as `Interceptor`s / an `EngineKind` plugin.
|
||||
- **Why.** M\*'s title is "Modular, **Extensible**" and it explicitly lists FastVideo-STA, xDiT/PipeFusion/USP,
|
||||
Inferix, FlashDrive as integratable. v2 already has the seam M\* only describes — this is where v2 most
|
||||
directly answers the paper, using this repo's own attention code. Low risk (the seam + capability negotiation
|
||||
already exist and are tested).
|
||||
|
||||
### PR-4 (P1) — CFG/branch as a LABEL over one paged KV pool
|
||||
- **Change.** Rewrite `PagedKVCache` (`cache/classes.py:155-172`) from a block *counter* into a real
|
||||
`(namespace,label)->[block-handle]` store with **one shared `total_blocks` budget** (M\*'s single-pool
|
||||
property). Reuse the existing-but-unpopulated `CacheKey.guidance_sig` (`keys.py:53`) for the hash. Thread the
|
||||
label through `ar_loop.py` (alloc/append/get per `(request_id, branch)`; prefill once per shared-prefix label;
|
||||
combine via `CFGPolicy.combine`). Wire `ResourceRequest.cache_blocks` (`contracts.py:64`, zero consumers) into
|
||||
admission per (class,label).
|
||||
- **Why.** The dossier-identified driver of M\*'s BAGEL win (3 CFG contexts as 3 labels over ONE pool vs dense
|
||||
per-context). Targets AR_DECODE (BAGEL `generate_text`, omni Thinker); **correctly excludes diffusion**
|
||||
(Wan/LTX are bidirectional, no KV — their CFG stays dense-but-batched).
|
||||
- **Corrections to bake in.** Do **NOT** add `branch_label` to `CacheKey.partition_field()` (CFG branches share
|
||||
embeddings; partitioning by branch is a semantic bug). Do **NOT** add a new by-ref type — reuse
|
||||
`InProcKVConnector` + `TransferManifest.cache_key`. Wiring `cache_blocks` admission is greenfield ⇒ effort **L**.
|
||||
CPU version proves label/sharing semantics; the real latency win needs a FlashInfer paged kernel (out of scope)
|
||||
— **merge** with a future "real KVCacheEngine" effort rather than landing isolated.
|
||||
|
||||
---
|
||||
|
||||
## 4. What v2 already does ≥ M\* — do NOT regress
|
||||
1. **Required+validated cost model** on every `LoopSpec` (13-kind `WorkUnitKind`) — typed, pre-GPU-validated.
|
||||
2. **Interleave bit-parity as a hard gate** (`parity.interleave_required=True` on 40+ cards). M\* has no such
|
||||
gate (its speculative scheduling deliberately wastes steps). Load-bearing invariant; every new primitive
|
||||
must pass it.
|
||||
3. **C0–C4 consistency ladder** wired into RL methods, with first-divergence tap reporting. No M\* equivalent.
|
||||
4. **Integrated training plane** — DiffusionNFT/DMD2/self_forcing, RL→distill flywheel, `WeightSyncController`
|
||||
hot weight-sync with drain-to-boundary + scoped cache invalidation, driving the **same** serving Loop.
|
||||
M\* is serving-only. Protect with a toy fixture asserting `rollout_loop` drives the served Loop object.
|
||||
5. **CPU-toy parity for the whole stack** — loops/CFG/caches/parity/RL run in CI without a GPU. Every new
|
||||
primitive must ship a toy exercise (this is what makes all PRs above testable without H100s).
|
||||
6. **Partition-not-flush cache invalidation** + four independent per-class pools.
|
||||
7. **`extend/` plugin seam** with capability negotiation (a 4-step distilled card *rejects* a residual-skip
|
||||
interceptor) — M\* describes extensibility; v2 has the mechanism.
|
||||
8. **Dynamo citizenship** (`deploy/dynamo.py`: one `DeploymentCard`+cost model, two consumers) — beyond M\*'s
|
||||
self-contained runtime.
|
||||
|
||||
---
|
||||
|
||||
## 5. Dropped / merged / deferred (and why)
|
||||
- **DROP declarative `Parallel` as a CFG-execution win.** The runner walks nodes linearly (ignores
|
||||
`Program.edges`), so `Parallel` lowers to sequential sugar and the CFG 3-pass braid is already one
|
||||
co-scheduled `WorkPlan.run`; splitting it risks the interleave gate. Salvage only the no-op refactor
|
||||
extracting `branch_forward` from `WanDenoiseLoop._velocity`. Reassign `Parallel` to the placement workstream.
|
||||
- **MERGE the full Walk/state-machine layer** into "defer until a re-entrant phase graph needs it" (PR-1 gets the
|
||||
min-components win with ~20 lines, no new abstraction). If built: the validator must check a walk's node-id
|
||||
order is a *subsequence* of `program.nodes` (not just membership) or the runner can reorder and break parity.
|
||||
- **MERGE `StreamBuffer`/`ChunkPolicy` into pipelined-scheduling.** Causal-chunk emit *already ships*
|
||||
(`wan_causal/loop.py` per-chunk `StepResult.emit` + slab-KV); the gap is the declarative `ChunkPolicy` vocab
|
||||
+ a concurrent producer/consumer runner. If built: keep all policies pure (per-request `StreamBuffer` history,
|
||||
not shared edge state) and restrict the bit-identical claim to the token-only handoff.
|
||||
- **MERGE CFG-fan-out exec + cross-rank transport + PD loop-splitting into a multi-GPU-runtime program.** These
|
||||
need real collectives (`v2/distributed/` is a stub) and KV-by-reference (KV lives in `CacheManager`, not the
|
||||
transferable `slots`). **Keep cheaply now:** the *declarative* halves — per-component degree, `(node,Walk)`
|
||||
placement key with node-only fallback, `ReplicaSet` under `LocalFleet`, and populate `parallel_plan_hash` on
|
||||
the **serving** cache path (it is already populated in `training/behavior.py:40` — the gap is serving-only).
|
||||
- **DEFER** speculative deferred-termination, loop-spanning CUDA graphs, N+1 prefetch, attention-plan
|
||||
double-buffer — all gated on a real GPU executor; benefit unobservable on CPU-toy CI. Keep the cheap
|
||||
`EngineKind` tag (`STATELESS|KV_CACHE|DIFFUSION`) now. Correct the stale `cudagraph.py:51-52` docstring
|
||||
(per-step capture ships in 14 cards, not just wan21).
|
||||
- **RESCOPE per-node TP.** Wan/LTX use `ReplicatedLinear` + **sequence parallelism** (`sp`), not TP; the `sp`
|
||||
axis already exists in `parallel/plan.py:AXIS_NAMES`. The work is wiring degrees into the runtime, not
|
||||
inventing vocabulary; a `tp_size=2` "one-line activation" is a no-op for the shipped models.
|
||||
|
||||
---
|
||||
|
||||
## 6. The first integration test, if/when multi-GPU placement work starts
|
||||
The **live Qwen-Omni 2-GPU bring-up** (Thinker on rank 0, Talker+Code2Wav on rank 1; see
|
||||
`v2_debug_videos/vlm.md` Session 4) is the natural first validation target for any `(node,Walk)→rank`
|
||||
placement work — it is the one place this repo already has real multi-rank composite-model execution.
|
||||
|
||||
---
|
||||
|
||||
## Anchor files for P0/P1
|
||||
`v2/program/specs.py`, `v2/runtime/engine.py` (**line 88 fix**), `v2/runtime/disaggregated.py`,
|
||||
`v2/recipes/omni/ar_loop.py`, `v2/loop/contracts.py`, `v2/card/specs.py`, `v2/cache/{classes.py,keys.py}`,
|
||||
`v2/parity/interleave_gate.py`, `v2/extend/{interceptors,registry}.py`, `recipes/__init__.py` +
|
||||
`v2/program/workflow.py` (registry-driven delivery).
|
||||
@@ -1,20 +1,23 @@
|
||||
---
|
||||
name: reseed-performance-baseline
|
||||
description: Re-seed the HF performance-tracking baseline for an intentional runtime, dependency, or environment-caused benchmark shift using one or more reviewed normalized performance JSONs. Use when performance CI fails because metrics such as latency, throughput, component time, or peak memory changed for an accepted reason and the rolling median baseline in FastVideo/performance-tracking must be advanced from a consistent batch of reviewed source results. The workflow backs up existing history under /tmp, validates all source JSONs for the same (model_id, gpu_type), rejects internally inconsistent source batches, uploads one success=true reseed record per accepted source JSON, and offers to clean local temp state after a successful upload.
|
||||
description: Re-seed the HF performance-tracking baseline for an intentional runtime, dependency, environment-caused benchmark shift, or reviewed v2 calibration using one or more reviewed normalized performance JSONs. Use when performance CI fails because metrics such as latency, throughput, component time, or peak memory changed for an accepted reason and the rolling median baseline in FastVideo/performance-tracking must be advanced, or when a new v2 exact comparable identity needs its first approved baseline. The workflow backs up existing history under /tmp, validates all source JSONs for the same legacy (model_id, gpu_type) target or the same v2 exact identity, rejects internally inconsistent source batches, uploads one success=true baseline record per accepted source JSON, and offers to clean local temp state after a successful upload.
|
||||
---
|
||||
|
||||
# Re-seed Performance Baseline
|
||||
|
||||
## Purpose
|
||||
|
||||
Replace or advance the rolling performance baseline for a single
|
||||
`(model_id, gpu_type)` pair in the HF dataset
|
||||
`FastVideo/performance-tracking`.
|
||||
Replace or advance the rolling performance baseline in the HF dataset
|
||||
`FastVideo/performance-tracking`. Legacy targets are scoped by
|
||||
`(model_id, gpu_type)`. V2 targets are scoped by exact comparable identity:
|
||||
`workload_id`, `variant_id`, `benchmark_version`, `hardware_profile_id`,
|
||||
`software_profile_id`, and `recipe_fingerprint`.
|
||||
|
||||
Performance comparison uses the median of up to the last 5 successful records
|
||||
for the same model and GPU. Failed records are useful audit history, but they
|
||||
do not move the future baseline because `compare_baseline.py` loads records
|
||||
with `successful_only=True`.
|
||||
Performance comparison uses the median of up to the last 5 successful,
|
||||
baseline-eligible records for the same target. Failed or calibration-only
|
||||
records are useful audit history, but they do not move the future baseline
|
||||
because `compare_baseline.py` loads records with `successful_only=True` and
|
||||
`baseline_eligible_only=True`.
|
||||
|
||||
This skill now reseeds from a reviewed batch of one or more source performance
|
||||
JSONs. It uploads one new `success=true` record per accepted source JSON; it
|
||||
@@ -22,11 +25,13 @@ does not blindly replicate one measurement into 3 or 5 records. The effective
|
||||
reseed size is therefore dynamic and equals the number of provided, validated,
|
||||
internally consistent source JSONs.
|
||||
|
||||
If the operator provides fewer than 3 records, call out that the last-5 rolling
|
||||
median may not move immediately. If the operator provides 3 consistent shifted
|
||||
records, the rolling median usually moves immediately. If the operator provides
|
||||
5 consistent shifted records, the last-5 window is effectively reset to the new
|
||||
runtime profile.
|
||||
For baseline shifts with existing history, if the operator provides fewer than
|
||||
3 records, call out that the last-5 rolling median may not move immediately. If
|
||||
the operator provides 3 consistent shifted records, the rolling median usually
|
||||
moves immediately. If the operator provides 5 consistent shifted records, the
|
||||
last-5 window is effectively reset to the new runtime profile. For the first
|
||||
approved v2 baseline of a new exact identity, one reviewed calibration seed is
|
||||
enough for the next comparable run to leave `CALIBRATION_NEEDED`.
|
||||
|
||||
These records are intentional operator-approved baseline resets, not ordinary
|
||||
independent main-branch persistence. Mark them clearly with provenance fields
|
||||
@@ -66,10 +71,10 @@ approval, then upload reviewed accepted baseline records.
|
||||
|
||||
| Parameter | Required | Description |
|
||||
|-----------|----------|-------------|
|
||||
| `model_id` | Yes | Benchmark id, e.g. `wan-t2v-1.3b-2gpu`. This maps to the HF subdirectory after `sanitize(model_id)`. |
|
||||
| `gpu_type` | Yes | Exact GPU device string from the performance record, e.g. the L40S device name emitted by CI. Baselines are GPU-specific. |
|
||||
| `model_id` | Legacy required; v2 inferred | Benchmark id, e.g. `wan-t2v-1.3b-2gpu`. This maps to the HF subdirectory after `sanitize(model_id)`. For v2 records, use the `model_id` from each source artifact only as the upload directory; comparison is by exact identity. |
|
||||
| `gpu_type` | Legacy required; v2 inferred | Exact GPU device string from the performance record, e.g. the L40S device name emitted by CI. V2 hardware matching uses `hardware_profile_id`; preserve `gpu_type` as display metadata. |
|
||||
| `source_results` | Yes | One or more local paths or Buildkite artifact URLs for accepted shifted performance JSONs. Prefer normalized `normalized_perf_*.json` artifacts emitted by `compare_baseline.py`. Accept `source_result` as an alias only for a single JSON. |
|
||||
| `max_intra_batch_regression` | No | Maximum allowed regression of any source JSON against the source batch median. Default: `PERF_MAX_REGRESSION` if set, otherwise `0.05` (5%). |
|
||||
| `max_intra_batch_regression` | No | Maximum allowed regression of any source JSON against the source batch median. Default: `0.05` (5%). |
|
||||
| `intent_rationale` | Yes | One-line explanation for why the baseline shift is legitimate. This is written into provenance and should be reused in the PR. |
|
||||
|
||||
Hardcoded defaults:
|
||||
@@ -78,10 +83,14 @@ Hardcoded defaults:
|
||||
supported by the code, but use the default unless the user explicitly asks).
|
||||
- Local sync root: `/tmp/perf-tracking` (`PERFORMANCE_TRACKING_ROOT` override
|
||||
is supported).
|
||||
- Prepared-record staging root: `/tmp/performance_reseed_prepared`
|
||||
(`PERFORMANCE_RESEED_STAGING_ROOT` override is supported). Keep it separate
|
||||
and non-nested from the sync root.
|
||||
- Backup root: `/tmp/performance_reseed_backup`.
|
||||
- Download scratch root for source artifact URLs: `/tmp/performance_reseed_source`.
|
||||
- Baseline window: last 5 `success=true` records for the same
|
||||
`(model_id, gpu_type)`.
|
||||
- Baseline window: last 5 `success=true`, `baseline_eligible=true` records
|
||||
for the same legacy `(model_id, gpu_type)` target or the same v2 exact
|
||||
comparable identity.
|
||||
- Reseed count: dynamic. Upload exactly one accepted seed record per validated
|
||||
source JSON.
|
||||
|
||||
@@ -115,12 +124,24 @@ with open(source_result, encoding="utf-8") as f:
|
||||
record = json.load(f)
|
||||
```
|
||||
|
||||
Stop if any normalized record's `model_id` or `gpu_type` does not match the
|
||||
requested `model_id` and `gpu_type`.
|
||||
Classify the source batch before continuing:
|
||||
|
||||
- **Legacy source records** have no v2 exact identity fields. Stop if any
|
||||
normalized record's `model_id` or `gpu_type` does not match the requested
|
||||
`model_id` and `gpu_type`.
|
||||
- **V2 source records** have exact identity fields. Stop unless every source
|
||||
record has all six comparable identity fields and they are identical across
|
||||
the batch: `workload_id`, `variant_id`, `benchmark_version`,
|
||||
`hardware_profile_id`, `software_profile_id`, and `recipe_fingerprint`.
|
||||
Do not fall back to legacy `(model_id, gpu_type)` matching for v2 records.
|
||||
|
||||
The source records may have `success: false` when they came from failed
|
||||
rolling baseline comparisons. That is expected; only the reviewed reseed
|
||||
records become new `success: true` baseline records after explicit approval.
|
||||
For a first v2 baseline seed, the source records must instead be successful
|
||||
scheduled-main full-suite `CALIBRATION_NEEDED` normalized artifacts. Reject PR,
|
||||
local, direct-run, non-main-branch, or non-full-suite calibration artifacts as
|
||||
seed sources.
|
||||
|
||||
Sort validated source records by their original `timestamp` ascending before
|
||||
preparing the seed records. If a source timestamp is missing or unparsable,
|
||||
@@ -148,8 +169,7 @@ For each metric with at least two non-null source values:
|
||||
4. Stop if any source record regresses against the batch median by more than
|
||||
`max_intra_batch_regression`.
|
||||
|
||||
Default `max_intra_batch_regression` to `PERF_MAX_REGRESSION` when set,
|
||||
otherwise `0.05`. Print a table with per-source values, batch median, and
|
||||
Default `max_intra_batch_regression` to `0.05`. Print a table with per-source values, batch median, and
|
||||
worst intra-batch regression.
|
||||
|
||||
This check prevents uploading a mixed batch where one JSON is materially
|
||||
@@ -183,7 +203,7 @@ present, that run is not a valid source for baseline reseeding.
|
||||
|
||||
### 2. Sync and back up existing HF records under /tmp
|
||||
|
||||
Use `fastvideo/tests/performance/hf_store.py` helpers directly. Do **not** use
|
||||
Use `fastvideo/performance/hf_store.py` helpers directly. Do **not** use
|
||||
`compare_baseline.py` as a sync shortcut; on full main runs it can persist
|
||||
records, while this step must only fetch and back up existing history.
|
||||
|
||||
@@ -192,16 +212,16 @@ The sync command pattern is:
|
||||
```bash
|
||||
export PERFORMANCE_TRACKING_ROOT="${PERFORMANCE_TRACKING_ROOT:-/tmp/perf-tracking}"
|
||||
export HF_REPO_ID="${HF_REPO_ID:-FastVideo/performance-tracking}"
|
||||
PYTHONPATH=fastvideo/tests/performance python -c 'from hf_store import sync_from_hf; import os; sync_from_hf(os.environ["PERFORMANCE_TRACKING_ROOT"], strict=True)'
|
||||
python -c 'from fastvideo.performance.hf_store import sync_from_hf; import os; sync_from_hf(os.environ["PERFORMANCE_TRACKING_ROOT"], strict=True)'
|
||||
```
|
||||
|
||||
Then back up only the sanitized model directory under `/tmp`:
|
||||
For legacy records, back up the sanitized model directory under `/tmp`:
|
||||
|
||||
```bash
|
||||
SHORT_COMMIT=$(git rev-parse --short=12 HEAD)
|
||||
TIMESTAMP=$(date -u +%Y%m%d_%H%M%S)
|
||||
MODEL_SAFE=$(PYTHONPATH=fastvideo/tests/performance python - <<'PY'
|
||||
from hf_store import sanitize
|
||||
MODEL_SAFE=$(python - <<'PY'
|
||||
from fastvideo.performance.hf_store import sanitize
|
||||
print(sanitize("<model_id>"))
|
||||
PY
|
||||
)
|
||||
@@ -210,6 +230,16 @@ mkdir -p "$BACKUP_DIR"
|
||||
cp -R "${PERFORMANCE_TRACKING_ROOT}/${MODEL_SAFE}" "$BACKUP_DIR/" 2>/dev/null || true
|
||||
```
|
||||
|
||||
For v2 records, back up the full local tracking root after sync. Exact identity
|
||||
lookup scans across model directories, so a benchmark rename may have relevant
|
||||
history outside the current source artifact's `model_id` directory:
|
||||
|
||||
```bash
|
||||
BACKUP_DIR="/tmp/performance_reseed_backup/${TIMESTAMP}_${SHORT_COMMIT}_v2_exact_identity"
|
||||
mkdir -p "$BACKUP_DIR"
|
||||
cp -R "${PERFORMANCE_TRACKING_ROOT}" "$BACKUP_DIR/tracking-root"
|
||||
```
|
||||
|
||||
Write provenance next to the backup:
|
||||
|
||||
```bash
|
||||
@@ -232,10 +262,12 @@ first baseline seed. Continue, but report that baseline history was empty.
|
||||
|
||||
### 3. Compute old baseline and candidate shift
|
||||
|
||||
Load the last 5 successful records for the target:
|
||||
Load the last 5 successful baseline records for the target.
|
||||
|
||||
For legacy targets:
|
||||
|
||||
```python
|
||||
from hf_store import load_records_for_model
|
||||
from fastvideo.performance.hf_store import load_records_for_model
|
||||
|
||||
records = load_records_for_model(
|
||||
"/tmp/perf-tracking",
|
||||
@@ -243,6 +275,28 @@ records = load_records_for_model(
|
||||
"<gpu_type>",
|
||||
last_n=5,
|
||||
successful_only=True,
|
||||
baseline_eligible_only=True,
|
||||
)
|
||||
```
|
||||
|
||||
For v2 exact-identity targets:
|
||||
|
||||
```python
|
||||
from fastvideo.performance.hf_store import load_records_for_identity
|
||||
|
||||
records = load_records_for_identity(
|
||||
"/tmp/perf-tracking",
|
||||
{
|
||||
"workload_id": "<workload_id>",
|
||||
"variant_id": "<variant_id>",
|
||||
"benchmark_version": "<benchmark_version>",
|
||||
"hardware_profile_id": "<hardware_profile_id>",
|
||||
"software_profile_id": "<software_profile_id>",
|
||||
"recipe_fingerprint": "<recipe_fingerprint>",
|
||||
},
|
||||
last_n=5,
|
||||
successful_only=True,
|
||||
baseline_eligible_only=True,
|
||||
)
|
||||
```
|
||||
|
||||
@@ -258,7 +312,8 @@ medians after appending the proposed seed records, and source batch spread for:
|
||||
|
||||
Also print how many successful old records exist. Make clear:
|
||||
|
||||
- 1 seed record usually does not move a last-5 median by itself.
|
||||
- 1 seed record usually does not move an existing last-5 median by itself, but
|
||||
it is enough to establish the first v2 baseline for a new exact identity.
|
||||
- 3 consistent seed records usually move the last-5 median immediately.
|
||||
- 5 consistent seed records effectively reset the last-5 window.
|
||||
- The records are intentional approved baseline resets and must be labeled
|
||||
@@ -268,10 +323,10 @@ Also print how many successful old records exist. Make clear:
|
||||
|
||||
Require an explicit confirmation phrase before preparing the upload:
|
||||
|
||||
> About to RE-SEED performance baseline for `<model_id>` on `<gpu_type>`.
|
||||
> About to RE-SEED performance baseline for `<target description>`.
|
||||
> This will upload `<N>` new `success=true` records to
|
||||
> `FastVideo/performance-tracking/<sanitize(model_id)>/`, one per accepted
|
||||
> source JSON.
|
||||
> `FastVideo/performance-tracking/<sanitize(model_id)>/` or the source
|
||||
> artifact's v2 model directory, one per accepted source JSON.
|
||||
>
|
||||
> Reason: `<intent_rationale>`
|
||||
> Source results: `<source_results>`
|
||||
@@ -289,25 +344,63 @@ Do not continue unless the user types exactly `confirm performance reseed`.
|
||||
|
||||
### 5. Create the accepted seed records
|
||||
|
||||
Create one seed record from each normalized source result. Do not copy the
|
||||
Create one seed record from each normalized source result.
|
||||
|
||||
For first v2 baseline seeds, use the scoped utility. It validates exact
|
||||
identity, requires successful scheduled-main full-suite `CALIBRATION_NEEDED`
|
||||
source artifacts, preserves the normalized v2 identity and metadata fields,
|
||||
and writes seed records with `success=true`, `baseline_eligible=true`, and
|
||||
`comparison_status=PASS`:
|
||||
|
||||
```bash
|
||||
python fastvideo/tests/performance/seed_baseline.py \
|
||||
--source-result <normalized_perf_1.json> \
|
||||
--source-result <normalized_perf_2.json> \
|
||||
--intent-rationale "<intent_rationale>" \
|
||||
--max-intra-batch-regression 0.05 \
|
||||
--tracking-root "${PERFORMANCE_TRACKING_ROOT}" \
|
||||
--staging-root "${PERFORMANCE_RESEED_STAGING_ROOT:-/tmp/performance_reseed_prepared}"
|
||||
```
|
||||
|
||||
The utility is prepare-only and intentionally has no upload option. Upload the
|
||||
scoped records only after the separate confirmation in step 6.
|
||||
|
||||
The utility validates against an isolated fresh HF snapshot and leaves
|
||||
`PERFORMANCE_TRACKING_ROOT` untouched; that argument only proves the staging
|
||||
root is separate from the operator's tracking mirror. Before writing, it stops
|
||||
if the exact identity already has a successful baseline-eligible record or if
|
||||
the workload/variant/version already trusts another recipe. It atomically
|
||||
reserves the exact identity and writes a digest-protected upload manifest bound
|
||||
to the current HF endpoint, repository id, and repository type. Keep the
|
||||
prepared records, manifest, source files, and reservation unchanged until the
|
||||
operation is uploaded or explicitly cleaned up.
|
||||
|
||||
If the prepared seed records look correct, upload only those scoped records in
|
||||
step 7. Do not rerun the utility with a different source list after approval.
|
||||
|
||||
For legacy reseeds or accepted v2 baseline shifts from regression artifacts,
|
||||
create one seed record from each normalized source result. Do not copy the
|
||||
source JSON wholesale.
|
||||
|
||||
Infer the baseline field allowlist from all existing HF records for the target
|
||||
`(model_id, gpu_type)` after syncing, including both `success=true` and
|
||||
`success=false` records. Use the union of non-provenance keys present in those
|
||||
target records, preserving only fields that also exist in the normalized
|
||||
source record or are explicitly set by the reseed workflow. Always include
|
||||
`model_id`, `timestamp`, and `success` because the upload path and baseline
|
||||
loader depend on them. Always set `timestamp` to a fresh reseed timestamp and
|
||||
`success` to `true`. Do not include unrelated source-only fields that are
|
||||
absent from existing HF records.
|
||||
after syncing, including both `success=true` and `success=false` records. For
|
||||
legacy targets the target is `(model_id, gpu_type)`. For v2 baseline-shift
|
||||
reseeds the target is the exact comparable identity. Use the union of
|
||||
non-provenance keys present in those target records, preserving only fields
|
||||
that also exist in the normalized source record or are explicitly set by the
|
||||
reseed workflow. Always include `model_id`, `timestamp`, `success`,
|
||||
`baseline_eligible`, and `comparison_status` because the upload path and
|
||||
baseline loader depend on them. Always set `timestamp` to a fresh reseed
|
||||
timestamp, `success` to `true`, `baseline_eligible` to `true`, and
|
||||
`comparison_status` to `PASS`. Do not include unrelated source-only fields
|
||||
that are absent from existing HF records.
|
||||
|
||||
Exclude existing provenance or operator metadata from the inferred baseline
|
||||
field allowlist. At minimum, exclude keys prefixed with `baseline_reseed` and
|
||||
any fields known to be local-only audit metadata.
|
||||
|
||||
If there are no previous HF records for the target model/GPU, fall back to this
|
||||
default baseline field list:
|
||||
If there are no previous HF records for the target, fall back to this default
|
||||
baseline field list:
|
||||
|
||||
- `model_id`
|
||||
- `timestamp`
|
||||
@@ -320,6 +413,22 @@ default baseline field list:
|
||||
- `dit_time_s`
|
||||
- `vae_decode_time_s`
|
||||
- `success`
|
||||
- `baseline_eligible`
|
||||
- `comparison_status`
|
||||
|
||||
For v2 baseline-shift reseeds with no previous HF records for the exact
|
||||
identity, also preserve:
|
||||
|
||||
- `workload_id`
|
||||
- `variant_id`
|
||||
- `benchmark_version`
|
||||
- `recipe_fingerprint`
|
||||
- `hardware_profile_id`
|
||||
- `software_profile_id`
|
||||
- `recipe`
|
||||
- `hardware_profile`
|
||||
- `software_profile`
|
||||
- `software_comparison_profile`
|
||||
|
||||
Do not upload extra fields from the source artifact.
|
||||
|
||||
@@ -335,6 +444,22 @@ Optional provenance fields are allowed and useful:
|
||||
- `baseline_reseed_operator`
|
||||
- `baseline_reseed_max_intra_batch_regression`
|
||||
|
||||
The v2 calibration seed utility writes analogous first-seed provenance:
|
||||
|
||||
- `baseline_seed: true`
|
||||
- `baseline_seed_reason`
|
||||
- `baseline_seed_source_result`
|
||||
- `baseline_seed_source_status`
|
||||
- `baseline_seed_source_timestamp`
|
||||
- `baseline_seed_source_success`
|
||||
- `baseline_seed_source_run_source`
|
||||
- `baseline_seed_source_branch`
|
||||
- `baseline_seed_source_test_scope`
|
||||
- `baseline_seed_source_pr_number`
|
||||
- `baseline_seed_batch_size`
|
||||
- `baseline_seed_batch_index`
|
||||
- `baseline_seed_operator`
|
||||
|
||||
Use a fresh reseed timestamp for each seed record, not the original source
|
||||
result timestamp. This is required because
|
||||
`load_records_for_model(..., last_n=5)` keeps the last records after loading
|
||||
@@ -357,7 +482,8 @@ Prefer uploading new accepted seed records so failed history remains visible.
|
||||
Print:
|
||||
|
||||
- Backup directory path under `/tmp`.
|
||||
- Prepared local record paths under `PERFORMANCE_TRACKING_ROOT`.
|
||||
- Prepared local record paths under `PERFORMANCE_RESEED_STAGING_ROOT`.
|
||||
- Prepared upload-manifest path under the identity reservation.
|
||||
- HF paths that will receive the new records.
|
||||
- Old rolling medians.
|
||||
- Source batch medians, source batch spread, reseed count, and candidate
|
||||
@@ -369,22 +495,36 @@ prepared records plus backup on disk.
|
||||
|
||||
### 7. Upload only the scoped records
|
||||
|
||||
Use the shared storage helper so the path and repo type match CI:
|
||||
For a first v2 calibration seed, use the manifest uploader after the user
|
||||
replies exactly `upload`:
|
||||
|
||||
```python
|
||||
from hf_store import upload_record
|
||||
|
||||
upload_record("<local_record_path>", record, strict=True)
|
||||
```bash
|
||||
python -c 'from fastvideo.tests.performance.seed_baseline import upload_prepared_seed_manifest; print(upload_prepared_seed_manifest("<prepared_manifest>"))'
|
||||
```
|
||||
|
||||
Run it once per prepared record. Each upload goes to:
|
||||
The uploader verifies the source and prepared-record digests, pins and scans
|
||||
the current HF revision, rechecks exact-identity and recipe-cohort conflicts,
|
||||
and writes the entire batch in one commit whose `parent_commit` must still be
|
||||
current. A concurrent Hub update makes the commit fail. Do not retry
|
||||
automatically: preserve staging, refresh/review remote state, and request a new
|
||||
explicit `upload` after the conflict is understood. Each record goes to:
|
||||
|
||||
```text
|
||||
FastVideo/performance-tracking/<sanitize(model_id)>/<record_filename>.json
|
||||
```
|
||||
|
||||
Never bulk upload the whole tracking root. Never modify another model's
|
||||
directory in the same operation.
|
||||
Never call `upload_record()` once per first-seed record: that can partially
|
||||
land the batch and has no compare-and-swap guard.
|
||||
|
||||
For a legacy reseed or an accepted v2 baseline shift, the first-seed manifest
|
||||
validator does not apply because an eligible baseline already exists. Upload
|
||||
only the individually reviewed records prepared in step 5 with the shared
|
||||
`upload_record(local_path, record, strict=True)` helper. Stop on the first
|
||||
failure and report exactly which records reached HF; do not silently rerun or
|
||||
replicate the remainder.
|
||||
|
||||
Never bulk upload the tracking or staging root, and never modify another
|
||||
model's directory in the same operation.
|
||||
|
||||
### 8. Report outcome and offer cleanup
|
||||
|
||||
@@ -406,9 +546,14 @@ distinguish an accepted baseline shift from a hidden regression.
|
||||
After the upload is verified, ask whether the user wants to clear temporary
|
||||
local state. Explain what each directory is for:
|
||||
|
||||
- `PERFORMANCE_TRACKING_ROOT`, usually `/tmp/perf-tracking`: local synced
|
||||
mirror of `FastVideo/performance-tracking` plus the prepared local seed
|
||||
records used for scoped upload.
|
||||
- `PERFORMANCE_TRACKING_ROOT`, usually `/tmp/perf-tracking`: read-only local
|
||||
synced mirror used for operator review and reporting. First-v2 preparation
|
||||
independently proves remote state from a fresh temporary HF snapshot.
|
||||
- `PERFORMANCE_RESEED_STAGING_ROOT`, usually
|
||||
`/tmp/performance_reseed_prepared`: prepared local seed records used for the
|
||||
scoped upload, plus the identity reservation and digest manifest. Keeping
|
||||
this separate prevents aborted preparations from appearing in later
|
||||
baseline reads.
|
||||
- `/tmp/performance_reseed_backup/<...>`: local backup of the target model's
|
||||
pre-reseed HF history plus `PROVENANCE.txt`, kept so a bad reseed can be
|
||||
audited or corrected.
|
||||
@@ -418,14 +563,18 @@ local state. Explain what each directory is for:
|
||||
Ask:
|
||||
|
||||
> Reseed succeeded. Do you want me to delete the local temp tracking mirror,
|
||||
> source downloads, and reseed backup under `/tmp`? These files are local
|
||||
> safety/audit artifacts only; HF already has the uploaded records.
|
||||
> this reseed's prepared staging records, source downloads, and reseed backup
|
||||
> under `/tmp`? These files are local safety/audit artifacts only; HF already
|
||||
> has the uploaded records.
|
||||
>
|
||||
> Reply `cleanup reseed temp` to delete them, anything else to keep them.
|
||||
|
||||
Do not delete anything unless the user replies exactly
|
||||
`cleanup reseed temp`. If cleanup is requested, remove only the specific
|
||||
directories created for this reseed. Never remove unrelated `/tmp` contents.
|
||||
directories and prepared record paths created for this reseed. Do not remove
|
||||
the shared staging root when it contains other records. Remove this operation's
|
||||
identity reservation only with its prepared records and manifest, and never
|
||||
remove unrelated `/tmp` contents.
|
||||
|
||||
## Failure modes and handling
|
||||
|
||||
@@ -437,19 +586,34 @@ directories created for this reseed. Never remove unrelated `/tmp` contents.
|
||||
against the source batch median by more than `max_intra_batch_regression`.
|
||||
Ask for cleaner sources or a reviewed explanation before continuing.
|
||||
- **Too few source records to move the median.** Continue only after making
|
||||
clear that one or two records may not immediately move the last-5 median.
|
||||
clear that one or two records may not immediately move an existing last-5
|
||||
median. This warning does not block a first v2 calibration seed for an exact
|
||||
identity with no eligible baseline yet.
|
||||
- **The source results are noisy or suspicious.** Stop. Reseeding amplifies
|
||||
those measurements into the baseline, so they must be reviewed first.
|
||||
- **HF sync fails.** Stop for destructive reseeds. A stale or empty sync can
|
||||
make the old baseline look missing.
|
||||
- **The exact v2 identity already has an eligible baseline.** Stop. The
|
||||
`CALIBRATION_NEEDED` artifact is stale; use the reviewed baseline-shift path
|
||||
instead of the first-seed utility.
|
||||
- **The workload/variant/version trusts another recipe.** Stop. The source is
|
||||
stale relative to the current recipe cohort and must not bypass
|
||||
`RECIPE_MISMATCH` by creating a second trusted recipe.
|
||||
- **The staging root already has a prepared seed for the exact identity.**
|
||||
Stop and reuse, upload, or explicitly clean that preparation. Do not prepare
|
||||
another copy of the same measurement.
|
||||
- **The conditional Hub commit loses its parent race.** Stop without retrying.
|
||||
Keep the preparation, refresh and review the new remote state, then request
|
||||
a new explicit `upload` only if the seed is still valid.
|
||||
- **Candidate still violates fixed thresholds.** Report that this skill only
|
||||
handles the rolling HF baseline; update benchmark JSON thresholds in code
|
||||
review if maintainers accept the new absolute limit.
|
||||
- **The user aborts at either confirmation.** Leave the backup and prepared
|
||||
records on disk. Nothing should be uploaded.
|
||||
- **The user declines cleanup.** Keep `/tmp/perf-tracking`, the source
|
||||
download directory if any, and `/tmp/performance_reseed_backup/<...>` in
|
||||
place for audit/debugging.
|
||||
- **The user declines cleanup.** Keep `/tmp/perf-tracking`, the prepared seed
|
||||
records under `/tmp/performance_reseed_prepared`, the source download
|
||||
directory if any, and `/tmp/performance_reseed_backup/<...>` in place for
|
||||
audit/debugging.
|
||||
- **A bad seed was uploaded.** Use the backup and HF history to identify the
|
||||
uploaded file, then remove or supersede it with an explicitly reviewed
|
||||
corrective record. Do not silently rewrite unrelated history.
|
||||
@@ -460,8 +624,9 @@ directories created for this reseed. Never remove unrelated `/tmp` contents.
|
||||
intentional baseline replacement.
|
||||
- `fastvideo/tests/performance/compare_baseline.py` — normalization, rolling
|
||||
median comparison, and persistence rules.
|
||||
- `fastvideo/tests/performance/hf_store.py` — HF sync, record loading,
|
||||
`sanitize()`, and `upload_record()`.
|
||||
- `fastvideo/performance/hf_store.py` — HF sync and record loading helpers.
|
||||
- `fastvideo/tests/performance/seed_baseline.py` — first-seed preparation,
|
||||
staging reservation, manifest validation, and conditional batch upload.
|
||||
- `fastvideo/tests/performance/test_inference_performance.py` — source result
|
||||
JSON schema.
|
||||
- `.buildkite/performance-benchmarks/tests/*.json` — fixed absolute benchmark
|
||||
@@ -474,3 +639,4 @@ directories created for this reseed. Never remove unrelated `/tmp` contents.
|
||||
| 2026-05-03 | Initial version. Sister workflow to `reseed-ssim-references`, scoped to one performance `(model_id, gpu_type)` baseline seed with backup, confirmation, provenance, and `success=true` upload. |
|
||||
| 2026-05-03 | Previous policy: replicate one approved shifted source result into 3 success records by default, or 5 only when explicitly requested. Add provenance marker for replicated-source reseeds. Superseded by the 2026-05-08 dynamic multi-source policy. |
|
||||
| 2026-05-08 | Replace fixed 3/5 replication with dynamic multi-source reseeding: upload one seed record per reviewed source JSON, validate intra-batch consistency, move backup/source scratch under `/tmp`, and ask whether to clean temp state after successful upload. |
|
||||
| 2026-07-13 | Keep first-v2-seed preparation outside the canonical mirror, reserve staging identities atomically, reject stale or replayed calibration seeds, and upload reviewed manifests with a single parent-guarded Hub commit. |
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
{
|
||||
"benchmark_id": "wan-t2v-1.3b-2gpu",
|
||||
"config_schema_version": 2,
|
||||
"workload_id": "wan-t2v",
|
||||
"variant_id": "1.3b-sp2",
|
||||
"benchmark_version": 3,
|
||||
"description": "Wan2.1 T2V 1.3B inference performance",
|
||||
"model": {
|
||||
"model_path": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
|
||||
+30
-1
@@ -1,6 +1,8 @@
|
||||
env:
|
||||
IMAGE_VERSION: "py3.12-latest"
|
||||
BUILDKITE_CLEAN_CHECKOUT: true
|
||||
# Buildkite only launches Modal; remote jobs initialize their own submodules.
|
||||
BUILDKITE_GIT_SUBMODULES: false
|
||||
|
||||
notify:
|
||||
- github_commit_status:
|
||||
@@ -114,6 +116,17 @@ steps:
|
||||
limit: 2
|
||||
agents:
|
||||
queue: "default"
|
||||
- label: ":test_tube: LoRA Extraction Tests"
|
||||
if: build.env("TEST_SCOPE") == "direct" && build.env("TEST_TYPE") == "lora_extraction"
|
||||
command: "timeout 90m .buildkite/scripts/pr_test.sh"
|
||||
retry:
|
||||
automatic:
|
||||
- exit_status: 128
|
||||
limit: 3
|
||||
- exit_status: -1
|
||||
limit: 2
|
||||
agents:
|
||||
queue: "default"
|
||||
- label: ":test_tube: Training Tests"
|
||||
if: build.env("TEST_SCOPE") == "direct" && build.env("TEST_TYPE") == "training"
|
||||
command: "timeout 90m .buildkite/scripts/pr_test.sh"
|
||||
@@ -371,6 +384,21 @@ steps:
|
||||
- TEST_TYPE=inference_lora
|
||||
agents:
|
||||
queue: "default"
|
||||
- path:
|
||||
- "scripts/lora_extraction/**"
|
||||
- "fastvideo/tests/lora_extraction/**"
|
||||
- "fastvideo/models/loader/**"
|
||||
- "fastvideo/training/training_utils.py"
|
||||
- "fastvideo/layers/lora/**"
|
||||
- "pyproject.toml"
|
||||
- "docker/Dockerfile"
|
||||
config:
|
||||
command: "timeout 90m .buildkite/scripts/pr_test.sh"
|
||||
label: ":test_tube: LoRA Extraction Tests"
|
||||
env:
|
||||
- TEST_TYPE=lora_extraction
|
||||
agents:
|
||||
queue: "default"
|
||||
- path:
|
||||
- "fastvideo/**"
|
||||
- "pyproject.toml"
|
||||
@@ -410,7 +438,7 @@ steps:
|
||||
- "pyproject.toml"
|
||||
- "docker/Dockerfile"
|
||||
config:
|
||||
command: "timeout 15m .buildkite/scripts/pr_test.sh"
|
||||
command: "timeout 25m .buildkite/scripts/pr_test.sh"
|
||||
label: ":test_tube: LoRA Training Tests"
|
||||
env:
|
||||
- TEST_TYPE=training_lora
|
||||
@@ -455,6 +483,7 @@ steps:
|
||||
- "fastvideo/layers/**"
|
||||
- "fastvideo/worker/**"
|
||||
- "fastvideo/entrypoints/**"
|
||||
- "fastvideo/performance/**"
|
||||
- "fastvideo/tests/performance/**"
|
||||
- ".buildkite/performance-benchmarks/**"
|
||||
- "pyproject.toml"
|
||||
|
||||
@@ -76,10 +76,27 @@ EFFECTIVE_PR=${BUILDKITE_PULL_REQUEST:-false}
|
||||
if [ "$EFFECTIVE_PR" = "false" ] && [ -n "${PR_NUMBER:-}" ]; then
|
||||
EFFECTIVE_PR=$PR_NUMBER
|
||||
fi
|
||||
MODAL_ENV="BUILDKITE_REPO=$BUILDKITE_REPO BUILDKITE_COMMIT=$BUILDKITE_COMMIT BUILDKITE_PULL_REQUEST=$EFFECTIVE_PR BUILDKITE_BRANCH=${BUILDKITE_BRANCH:-} TEST_SCOPE=${TEST_SCOPE:-} BUILDKITE_BUILD_URL=${BUILDKITE_BUILD_URL:-} BUILDKITE_BUILD_ID=${BUILDKITE_BUILD_ID:-} BUILDKITE_JOB_ID=${BUILDKITE_JOB_ID:-} IMAGE_VERSION=$IMAGE_VERSION"
|
||||
MODAL_ENV="BUILDKITE_REPO=$BUILDKITE_REPO BUILDKITE_COMMIT=$BUILDKITE_COMMIT BUILDKITE_PULL_REQUEST=$EFFECTIVE_PR BUILDKITE_BRANCH=${BUILDKITE_BRANCH:-} BUILDKITE_SOURCE=${BUILDKITE_SOURCE:-} TEST_SCOPE=${TEST_SCOPE:-} BUILDKITE_BUILD_URL=${BUILDKITE_BUILD_URL:-} BUILDKITE_BUILD_ID=${BUILDKITE_BUILD_ID:-} BUILDKITE_JOB_ID=${BUILDKITE_JOB_ID:-} IMAGE_VERSION=$IMAGE_VERSION"
|
||||
|
||||
POST_RUN_HOOK=""
|
||||
|
||||
is_truthy() {
|
||||
case "${1:-}" in
|
||||
1|true|TRUE|yes|YES|on|ON) return 0 ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
ssim_bootstrap_args() {
|
||||
local title="${PR_TITLE:-}"
|
||||
local message="${BUILDKITE_MESSAGE:-}"
|
||||
if is_truthy "${FASTVIDEO_SSIM_BOOTSTRAP_MODE:-}" \
|
||||
|| [[ "$title" == *"[new-model]"* ]] \
|
||||
|| [[ "$message" == *"[new-model]"* ]]; then
|
||||
printf ' --bootstrap-mode'
|
||||
fi
|
||||
}
|
||||
|
||||
upload_performance_artifacts() {
|
||||
SHORT_SHA=${BUILDKITE_COMMIT:0:7}
|
||||
LOCAL_DIR="downloaded_reports"
|
||||
@@ -172,7 +189,12 @@ case "$TEST_TYPE" in
|
||||
;;
|
||||
"ssim")
|
||||
log "Running SSIM tests..."
|
||||
MODAL_COMMAND="$MODAL_ENV HF_API_KEY=$HF_API_KEY python3 -m modal run $MODAL_SSIM_TEST_FILE::run_ssim_tests"
|
||||
SSIM_BOOTSTRAP_ARGS=$(ssim_bootstrap_args)
|
||||
if [ -n "$SSIM_BOOTSTRAP_ARGS" ]; then
|
||||
log "SSIM bootstrap mode enabled for new-model reference draft generation"
|
||||
fi
|
||||
MODAL_COMMAND="$MODAL_ENV HF_API_KEY=$HF_API_KEY python3 -m modal run "
|
||||
MODAL_COMMAND+="$MODAL_SSIM_TEST_FILE::run_ssim_tests$SSIM_BOOTSTRAP_ARGS"
|
||||
;;
|
||||
"training")
|
||||
log "Running training tests..."
|
||||
|
||||
Executable
+133
@@ -0,0 +1,133 @@
|
||||
#!/usr/bin/env bash
|
||||
# Gate the expensive Buildkite full suite on the cheap GitHub checks.
|
||||
#
|
||||
# Polls the workflow runs for the PR head commit and only exits 0 once the
|
||||
# watched cheap workflows (pre-commit, docs build) have succeeded, so the
|
||||
# 'ready' label cannot burn ~20 GPU lanes on a head that a cheap check has
|
||||
# already doomed.
|
||||
#
|
||||
# Semantics:
|
||||
# - watched run completed with a bad conclusion -> exit 1 (fail CLOSED:
|
||||
# no full suite; the next push re-arms via the 'synchronize' trigger)
|
||||
# - watched run cancelled -> still pending: the docs
|
||||
# workflow's repo-global 'pages' concurrency group cancels runs superseded
|
||||
# by unrelated pushes, so 'cancelled' is not a verdict on this PR
|
||||
# - watched runs pending -> poll until done
|
||||
# - docs run absent -> not applicable after a
|
||||
# short grace period ('Deploy Documentation' is path-filtered on PRs)
|
||||
# - pre-commit run absent -> keep polling: pre-commit
|
||||
# is never path-filtered, so its absence is always anomalous
|
||||
# - 'ready' label removed while waiting -> exit 1 (fail CLOSED:
|
||||
# un-labeling is a deliberate maintainer action)
|
||||
# - GitHub API unreachable or timeout -> exit 0 (fail OPEN,
|
||||
# loud warning: never brick CI on a GitHub outage)
|
||||
#
|
||||
# Required env: PR_SHA (PR head commit), PR_NUMBER, GITHUB_REPOSITORY, GH_TOKEN.
|
||||
set -euo pipefail
|
||||
|
||||
: "${PR_SHA:?PR_SHA (PR head commit) is required}"
|
||||
: "${PR_NUMBER:?PR_NUMBER (pull request number) is required}"
|
||||
: "${GITHUB_REPOSITORY:?GITHUB_REPOSITORY is required}"
|
||||
|
||||
# Workflow-level `name:` values that must be green before the full suite
|
||||
# may start. "Deploy Documentation" is path-filtered on PRs, so its run may
|
||||
# legitimately never exist; pre-commit always runs, so it must appear.
|
||||
WATCHED_NAMES='["pre-commit", "Deploy Documentation"]'
|
||||
WATCHED_REGEX='^(pre-commit|Deploy Documentation)$'
|
||||
POLL_SECS="${POLL_SECS:-20}"
|
||||
GRACE_SECS="${GRACE_SECS:-60}"
|
||||
MAX_WAIT_SECS="${MAX_WAIT_SECS:-1500}"
|
||||
|
||||
# Bound each API call so a hung connection hits the 3-strike fail-open path
|
||||
# instead of pinning the loop until the job timeout (which would fail closed
|
||||
# on exactly the GitHub-outage case this script is meant to survive).
|
||||
if command -v timeout >/dev/null 2>&1; then
|
||||
gh_api() { timeout 30 gh api "$@"; }
|
||||
else
|
||||
gh_api() { gh api "$@"; } # macOS dev boxes; CI always has coreutils timeout
|
||||
fi
|
||||
|
||||
# The workflow checked the label before starting the gate, but the wait can
|
||||
# last ~25 min: re-check once before any exit 0 and fail closed if 'ready'
|
||||
# was removed in the meantime. An API error here proceeds (the label was
|
||||
# present when the gate started; never brick CI on an outage).
|
||||
recheck_ready_label() {
|
||||
local pr_json
|
||||
if pr_json=$(gh_api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" 2>/dev/null); then
|
||||
if ! jq -e '[.labels[]?.name] | index("ready")' <<<"$pr_json" >/dev/null 2>&1; then
|
||||
echo "::error::PR #${PR_NUMBER} no longer has the 'ready' label —" \
|
||||
"NOT triggering the Buildkite full suite. Re-add the label to re-arm."
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
echo "::warning::Could not re-check the 'ready' label on PR #${PR_NUMBER}; proceeding (it was present when the gate started)."
|
||||
fi
|
||||
}
|
||||
|
||||
start=$(date +%s)
|
||||
api_fails=0
|
||||
missing=""
|
||||
|
||||
while true; do
|
||||
elapsed=$(( $(date +%s) - start ))
|
||||
|
||||
if runs_json=$(gh_api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${PR_SHA}&per_page=100" 2>/dev/null) \
|
||||
&& state=$(jq --arg re "$WATCHED_REGEX" '
|
||||
[.workflow_runs[]? | select(.name // "" | test($re))]
|
||||
| group_by(.name) | map(max_by(.id))
|
||||
| map({name, status, conclusion})' <<<"$runs_json" 2>/dev/null); then
|
||||
api_fails=0
|
||||
echo "t+${elapsed}s watched checks: $(jq -c . <<<"$state")"
|
||||
|
||||
failed=$(jq -r '[.[] | select(.status == "completed"
|
||||
and (.conclusion | IN("success", "skipped", "neutral", "cancelled") | not))]
|
||||
| map(.name) | join(", ")' <<<"$state")
|
||||
if [ -n "$failed" ]; then
|
||||
echo "::error::Cheap check(s) failed on ${PR_SHA}: ${failed}." \
|
||||
"NOT triggering the Buildkite full suite. Push a fix (the 'ready'" \
|
||||
"label re-arms on every push), or re-run the failed check and then" \
|
||||
"re-run this workflow."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 'cancelled' counts as pending: wait for a re-run to reach a real verdict
|
||||
# (bounded by MAX_WAIT, then the fail-open below).
|
||||
pending=$(jq '[.[] | select(.status != "completed" or .conclusion == "cancelled")] | length' <<<"$state")
|
||||
missing=$(jq -r --argjson watched "$WATCHED_NAMES" '($watched - map(.name)) | join(", ")' <<<"$state")
|
||||
if [ "$pending" -eq 0 ]; then
|
||||
if [ -z "$missing" ]; then
|
||||
recheck_ready_label
|
||||
echo "All watched cheap checks are green — full suite may proceed."
|
||||
exit 0
|
||||
fi
|
||||
case "$missing" in
|
||||
*pre-commit*)
|
||||
echo "pre-commit run not found for ${PR_SHA} yet; waiting (pre-commit is never path-filtered, so its absence is anomalous)."
|
||||
;;
|
||||
*)
|
||||
if [ "$elapsed" -ge "$GRACE_SECS" ]; then
|
||||
recheck_ready_label
|
||||
echo "::warning::Watched run(s) never appeared for ${PR_SHA}: ${missing} (path-filtered, likely not applicable). Proceeding on the checks that did run."
|
||||
exit 0
|
||||
fi
|
||||
echo "Waiting up to ${GRACE_SECS}s grace for path-filtered run(s) to appear: ${missing}."
|
||||
;;
|
||||
esac
|
||||
fi
|
||||
else
|
||||
api_fails=$(( api_fails + 1 ))
|
||||
echo "::warning::GitHub API error querying workflow runs for ${PR_SHA} (attempt ${api_fails}/3)."
|
||||
if [ "$api_fails" -ge 3 ]; then
|
||||
recheck_ready_label
|
||||
echo "::warning::FAILING OPEN: cannot query GitHub check status — triggering the full suite WITHOUT the cheap-check gate."
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$elapsed" -ge "$MAX_WAIT_SECS" ]; then
|
||||
recheck_ready_label
|
||||
echo "::warning::FAILING OPEN: watched checks still pending after $(( MAX_WAIT_SECS / 60 )) min${missing:+ (never appeared: ${missing})} — triggering the full suite anyway."
|
||||
exit 0
|
||||
fi
|
||||
sleep "$POLL_SECS"
|
||||
done
|
||||
Executable
+122
@@ -0,0 +1,122 @@
|
||||
#!/usr/bin/env bash
|
||||
# Self-test for gate_full_suite.sh using a mocked `gh`. No network, runs on
|
||||
# any dev box: bash .github/scripts/test_gate_full_suite.sh
|
||||
set -u
|
||||
here=$(cd "$(dirname "$0")" && pwd)
|
||||
tmp=$(mktemp -d)
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
|
||||
# Mock gh. Asserts the exact endpoint (including head_sha) it is called
|
||||
# with — an endpoint typo in the gate script fails the test rather than
|
||||
# silently serving canned data. On the runs endpoint it serves
|
||||
# $MOCK_DIR/response_<call#>.json, sticking on the highest existing file,
|
||||
# and exits 1 if none exist (simulates a GitHub API outage). On the pulls
|
||||
# endpoint it serves $MOCK_DIR/pr.json, defaulting to a 'ready'-labeled PR.
|
||||
cat > "$tmp/gh" <<'EOF'
|
||||
#!/usr/bin/env bash
|
||||
if [ "${1:-}" != "api" ]; then
|
||||
echo "unexpected gh invocation: $*" >> "$MOCK_DIR/endpoint_error"
|
||||
exit 2
|
||||
fi
|
||||
case "${2:-}" in
|
||||
"repos/o/r/actions/runs?head_sha=deadbeef&per_page=100")
|
||||
n=$(( $(cat "$MOCK_DIR/count" 2>/dev/null || echo 0) + 1 ))
|
||||
echo "$n" > "$MOCK_DIR/count"
|
||||
while [ "$n" -gt 0 ]; do
|
||||
if [ -f "$MOCK_DIR/response_$n.json" ]; then
|
||||
cat "$MOCK_DIR/response_$n.json"
|
||||
exit 0
|
||||
fi
|
||||
n=$(( n - 1 ))
|
||||
done
|
||||
echo "api outage" >&2
|
||||
exit 1
|
||||
;;
|
||||
"repos/o/r/pulls/42")
|
||||
if [ -f "$MOCK_DIR/pr.json" ]; then
|
||||
cat "$MOCK_DIR/pr.json"
|
||||
else
|
||||
echo '{"labels": [{"name": "ready"}]}'
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
echo "unexpected gh endpoint: $2" >> "$MOCK_DIR/endpoint_error"
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
EOF
|
||||
chmod +x "$tmp/gh"
|
||||
|
||||
PC_OK='{"name": "pre-commit", "id": 1, "status": "completed", "conclusion": "success"}'
|
||||
PC_BAD='{"name": "pre-commit", "id": 1, "status": "completed", "conclusion": "failure"}'
|
||||
PC_PENDING='{"name": "pre-commit", "id": 1, "status": "in_progress", "conclusion": null}'
|
||||
DOCS_OK='{"name": "Deploy Documentation", "id": 2, "status": "completed", "conclusion": "success"}'
|
||||
DOCS_BAD='{"name": "Deploy Documentation", "id": 2, "status": "completed", "conclusion": "failure"}'
|
||||
DOCS_CANCELLED='{"name": "Deploy Documentation", "id": 2, "status": "completed", "conclusion": "cancelled"}'
|
||||
OTHER='{"name": "Trigger Full Suite", "id": 3, "status": "in_progress", "conclusion": null}'
|
||||
NULL_NAME='{"name": null, "id": 4, "status": "completed", "conclusion": "failure"}'
|
||||
PC_OK_RERUN='{"name": "pre-commit", "id": 5, "status": "completed", "conclusion": "success"}'
|
||||
|
||||
fails=0
|
||||
want_log="" # optional: expect() also greps out.log for this regex, then resets
|
||||
pr_json="" # optional: served for the pulls (label re-check) endpoint, then resets
|
||||
raw_body="" # optional: serve responses verbatim instead of wrapping in workflow_runs
|
||||
expect() { # <name> <expected-exit> <response json>...
|
||||
local name=$1 want=$2 dir i=1
|
||||
shift 2
|
||||
dir=$(mktemp -d "$tmp/test_XXXXXX")
|
||||
for body in "$@"; do
|
||||
if [ -n "$raw_body" ]; then
|
||||
printf '%s' "$body" > "$dir/response_$i.json"
|
||||
else
|
||||
printf '{"workflow_runs": [%s]}' "$body" > "$dir/response_$i.json"
|
||||
fi
|
||||
i=$(( i + 1 ))
|
||||
done
|
||||
[ -n "$pr_json" ] && printf '%s' "$pr_json" > "$dir/pr.json"
|
||||
( export PATH="$tmp:$PATH" MOCK_DIR="$dir" PR_SHA=deadbeef PR_NUMBER=42 \
|
||||
GITHUB_REPOSITORY=o/r POLL_SECS=0 GRACE_SECS=1 MAX_WAIT_SECS=3
|
||||
bash "$here/gate_full_suite.sh" > "$dir/out.log" 2>&1 )
|
||||
local rc=$?
|
||||
if [ "$rc" -ne "$want" ]; then
|
||||
echo "FAIL: $name (exit $rc, want $want)"
|
||||
cat "$dir/out.log"
|
||||
fails=1
|
||||
elif [ -f "$dir/endpoint_error" ]; then
|
||||
echo "FAIL: $name (mock gh got an unexpected call)"
|
||||
cat "$dir/endpoint_error"
|
||||
fails=1
|
||||
elif [ -n "$want_log" ] && ! grep -Eq "$want_log" "$dir/out.log"; then
|
||||
echo "FAIL: $name (log does not match: $want_log)"
|
||||
cat "$dir/out.log"
|
||||
fails=1
|
||||
else
|
||||
echo "ok: $name"
|
||||
fi
|
||||
want_log="" pr_json="" raw_body=""
|
||||
}
|
||||
|
||||
expect "both green -> proceed" 0 "$PC_OK, $DOCS_OK, $OTHER, $NULL_NAME"
|
||||
expect "docs build failed -> blocked" 1 "$PC_OK, $DOCS_BAD"
|
||||
expect "pre-commit failed -> blocked" 1 "$PC_BAD"
|
||||
expect "pending then green -> proceed" 0 "$PC_PENDING" "$PC_OK, $DOCS_OK"
|
||||
want_log="never appeared.*Deploy Documentation"
|
||||
expect "docs run absent (path-filtered) -> proceed after grace" 0 "$PC_OK"
|
||||
expect "API outage -> fail open" 0
|
||||
want_log="FAILING OPEN"
|
||||
expect "pending past MAX_WAIT -> fail open" 0 "$PC_PENDING"
|
||||
want_log="FAILING OPEN"
|
||||
expect "unrelated runs only -> no grace, fail open at MAX_WAIT" 0 "$OTHER"
|
||||
expect "cancelled docs then green -> proceed" 0 \
|
||||
"$PC_OK, $DOCS_CANCELLED" "$PC_OK, $DOCS_OK"
|
||||
want_log="FAILING OPEN"
|
||||
expect "cancelled docs forever -> fail open at MAX_WAIT" 0 "$PC_OK, $DOCS_CANCELLED"
|
||||
want_log="FAILING OPEN"
|
||||
expect "pre-commit absent -> no grace, fail open at MAX_WAIT" 0 "$DOCS_OK"
|
||||
expect "duplicate run names -> latest wins" 0 "$PC_BAD, $PC_OK_RERUN, $DOCS_OK"
|
||||
raw_body=1
|
||||
expect "garbage response body -> fail open" 0 "this is not json"
|
||||
pr_json='{"labels": [{"name": "other"}]}'
|
||||
expect "ready label removed mid-gate -> blocked" 1 "$PC_OK, $DOCS_OK"
|
||||
|
||||
exit "$fails"
|
||||
@@ -1,7 +1,11 @@
|
||||
name: pre-commit
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
# pull_request_target instead of pull_request: the workflow definition and
|
||||
# the hook config are always taken from the BASE branch, so fork /
|
||||
# first-time-contributor PRs run immediately without a maintainer clicking
|
||||
# "Approve and run". The PR head is checked out as data only.
|
||||
pull_request_target:
|
||||
branches: [main]
|
||||
workflow_call:
|
||||
inputs:
|
||||
@@ -15,12 +19,36 @@ permissions:
|
||||
|
||||
jobs:
|
||||
pre-commit:
|
||||
if: github.event_name == 'workflow_call' || github.event.pull_request.draft != true
|
||||
if: github.event.pull_request.draft != true
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
ref: ${{ inputs.ref || '' }}
|
||||
# For PR events, lint the PR head — but keep the hook definitions from
|
||||
# the base branch so an untrusted PR cannot alter what gets executed.
|
||||
# The gate scripts are saved too: the self-test step below executes them,
|
||||
# so it must run the base-branch copies, not the PR head's.
|
||||
- name: Save trusted hook config and gate scripts
|
||||
if: github.event_name == 'pull_request_target'
|
||||
run: |
|
||||
cp .pre-commit-config.yaml "$RUNNER_TEMP/trusted-pre-commit-config.yaml"
|
||||
cp -a .github/scripts "$RUNNER_TEMP/trusted-scripts"
|
||||
echo "GATE_SCRIPTS_DIR=$RUNNER_TEMP/trusted-scripts" >> "$GITHUB_ENV"
|
||||
# allow-unsafe-pr-checkout acknowledges checkout's pull_request_target
|
||||
# guard: the head is data for the trusted hooks to lint; nothing from it
|
||||
# is executed (config and gate scripts are pinned to the base branch
|
||||
# above) and credentials are not persisted. SHA-pinned to v4.4.0 because
|
||||
# actionlint's action schema does not know the new input yet.
|
||||
- uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
|
||||
if: github.event_name == 'pull_request_target'
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.head.sha }}
|
||||
persist-credentials: false
|
||||
allow-unsafe-pr-checkout: true
|
||||
- name: Restore trusted hook config
|
||||
if: github.event_name == 'pull_request_target'
|
||||
run: cp "$RUNNER_TEMP/trusted-pre-commit-config.yaml" .pre-commit-config.yaml
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
@@ -30,3 +58,8 @@ jobs:
|
||||
- uses: pre-commit/action@v3.0.1
|
||||
with:
|
||||
extra_args: --all-files --hook-stage manual
|
||||
# After pre-commit so a self-test failure cannot mask lint failures.
|
||||
# GATE_SCRIPTS_DIR points at the base-branch copy on fork PRs (set above);
|
||||
# push / workflow_call runs use the checked-out tree directly.
|
||||
- name: Full-suite gate self-test
|
||||
run: bash "${GATE_SCRIPTS_DIR:-.github/scripts}/test_gate_full_suite.sh"
|
||||
|
||||
@@ -52,6 +52,7 @@ jobs:
|
||||
core.setOutput('pr_sha', pr.head.sha);
|
||||
core.setOutput('pr_branch', pr.head.ref);
|
||||
core.setOutput('pr_number', String(prNumber));
|
||||
core.setOutput('pr_title', pr.title);
|
||||
|
||||
- name: Trigger Full Suite
|
||||
if: steps.perm.outputs.has_write == 'true'
|
||||
@@ -60,6 +61,7 @@ jobs:
|
||||
PR_SHA: ${{ steps.label.outputs.pr_sha }}
|
||||
PR_BRANCH: ${{ steps.label.outputs.pr_branch }}
|
||||
PR_NUMBER: ${{ steps.label.outputs.pr_number }}
|
||||
PR_TITLE: ${{ steps.label.outputs.pr_title }}
|
||||
BK_ORG: ${{ vars.BUILDKITE_ORG_SLUG }}
|
||||
BK_PIPELINE: ${{ vars.BUILDKITE_PIPELINE_SLUG }}
|
||||
run: |
|
||||
@@ -71,6 +73,7 @@ jobs:
|
||||
--arg commit "$PR_SHA" \
|
||||
--arg branch "$PR_BRANCH" \
|
||||
--arg message "Full Suite for PR #${PR_NUMBER} (via /merge)" \
|
||||
--arg pr_title "$PR_TITLE" \
|
||||
--argjson pr_id "$PR_NUMBER" \
|
||||
'{
|
||||
commit: $commit,
|
||||
@@ -80,11 +83,12 @@ jobs:
|
||||
pull_request_id: $pr_id,
|
||||
pull_request_base_branch: "main",
|
||||
env: {
|
||||
TEST_SCOPE: "full",
|
||||
FULL_SUITE: "true",
|
||||
PR_NUMBER: ($pr_id | tostring)
|
||||
}
|
||||
}')"
|
||||
TEST_SCOPE: "full",
|
||||
FULL_SUITE: "true",
|
||||
PR_NUMBER: ($pr_id | tostring),
|
||||
PR_TITLE: $pr_title
|
||||
}
|
||||
}')"
|
||||
|
||||
parse-command:
|
||||
if: >-
|
||||
@@ -125,7 +129,7 @@ jobs:
|
||||
set -euo pipefail
|
||||
TEST_NAME=$(echo "$COMMENT" | grep -oP '(?<=/test\s)\S+' | head -1 || true)
|
||||
|
||||
VALID="encoder vae transformer kernel unit dreamverse ssim training lora-inference lora-training distillation self-forcing vsa vmoba performance api train-framework eval full fastcheck pre-commit"
|
||||
VALID="encoder vae transformer kernel unit dreamverse ssim training lora-inference lora-training lora-extraction distillation self-forcing vsa vmoba performance api train-framework eval full fastcheck pre-commit"
|
||||
if [ -z "$TEST_NAME" ] || ! echo "$VALID" | grep -qw "$TEST_NAME"; then
|
||||
echo "Unknown test: '$TEST_NAME'. Valid: $VALID"
|
||||
exit 1
|
||||
@@ -136,6 +140,7 @@ jobs:
|
||||
[kernel]=kernel_tests [unit]=unit_test [dreamverse]=dreamverse_app
|
||||
[ssim]=ssim [training]=training
|
||||
[lora-inference]=inference_lora [lora-training]=training_lora
|
||||
[lora-extraction]=lora_extraction
|
||||
[distillation]=distillation_dmd [self-forcing]=self_forcing
|
||||
[vsa]=training_vsa [vmoba]=inference_vmoba
|
||||
[performance]=performance [api]=api_server
|
||||
@@ -240,6 +245,7 @@ jobs:
|
||||
TEST_SCOPE: ${{ needs.parse-command.outputs.test_scope }}
|
||||
FULL_SUITE: ${{ needs.parse-command.outputs.full_suite }}
|
||||
TEST_TYPE: ${{ needs.parse-command.outputs.test_type }}
|
||||
PR_TITLE: ${{ github.event.issue.title }}
|
||||
BK_ORG: ${{ vars.BUILDKITE_ORG_SLUG }}
|
||||
BK_PIPELINE: ${{ vars.BUILDKITE_PIPELINE_SLUG }}
|
||||
run: |
|
||||
@@ -256,6 +262,7 @@ jobs:
|
||||
--arg full_suite "$FULL_SUITE" \
|
||||
--arg test_type "$TEST_TYPE" \
|
||||
--arg pr_number "$PR_NUMBER" \
|
||||
--arg pr_title "$PR_TITLE" \
|
||||
'{
|
||||
commit: $commit,
|
||||
branch: $branch,
|
||||
@@ -265,8 +272,9 @@ jobs:
|
||||
pull_request_base_branch: "main",
|
||||
env: {
|
||||
TEST_SCOPE: $test_scope,
|
||||
FULL_SUITE: $full_suite,
|
||||
TEST_TYPE: $test_type,
|
||||
PR_NUMBER: $pr_number
|
||||
}
|
||||
}')"
|
||||
FULL_SUITE: $full_suite,
|
||||
TEST_TYPE: $test_type,
|
||||
PR_NUMBER: $pr_number,
|
||||
PR_TITLE: $pr_title
|
||||
}
|
||||
}')"
|
||||
|
||||
@@ -7,6 +7,7 @@ on:
|
||||
permissions:
|
||||
contents: read
|
||||
pull-requests: read
|
||||
actions: read
|
||||
|
||||
concurrency:
|
||||
group: full-suite-${{ github.event.pull_request.number }}
|
||||
@@ -18,6 +19,8 @@ jobs:
|
||||
(github.event.action == 'labeled' && github.event.label.name == 'ready')
|
||||
|| github.event.action == 'synchronize'
|
||||
runs-on: ubuntu-latest
|
||||
# Gate below may wait for cheap checks (up to MAX_WAIT_SECS = 25 min).
|
||||
timeout-minutes: 35
|
||||
steps:
|
||||
- name: Check ready label
|
||||
id: check
|
||||
@@ -49,6 +52,20 @@ jobs:
|
||||
"https://api.buildkite.com/v2/organizations/${{ vars.BUILDKITE_ORG_SLUG }}/pipelines/${{ vars.BUILDKITE_PIPELINE_SLUG }}/builds/${build_num}/cancel"
|
||||
done
|
||||
|
||||
# Checks out the BASE branch (default for pull_request_target), so PR
|
||||
# authors cannot tamper with the gate script.
|
||||
- name: Checkout gate script
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||
|
||||
- name: Wait for pre-commit and docs build
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PR_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
run: bash .github/scripts/gate_full_suite.sh
|
||||
|
||||
- name: Trigger Buildkite Full Suite
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
env:
|
||||
@@ -56,6 +73,7 @@ jobs:
|
||||
PR_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
PR_BRANCH: ${{ github.event.pull_request.head.ref }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
PR_TITLE: ${{ github.event.pull_request.title }}
|
||||
BK_ORG: ${{ vars.BUILDKITE_ORG_SLUG }}
|
||||
BK_PIPELINE: ${{ vars.BUILDKITE_PIPELINE_SLUG }}
|
||||
run: |
|
||||
@@ -67,6 +85,7 @@ jobs:
|
||||
--arg commit "$PR_SHA" \
|
||||
--arg branch "$PR_BRANCH" \
|
||||
--arg message "Full Suite for PR #${PR_NUMBER}" \
|
||||
--arg pr_title "$PR_TITLE" \
|
||||
--argjson pr_id "$PR_NUMBER" \
|
||||
'{
|
||||
commit: $commit,
|
||||
@@ -78,6 +97,7 @@ jobs:
|
||||
env: {
|
||||
TEST_SCOPE: "full",
|
||||
FULL_SUITE: "true",
|
||||
PR_NUMBER: ($pr_id | tostring)
|
||||
PR_NUMBER: ($pr_id | tostring),
|
||||
PR_TITLE: $pr_title
|
||||
}
|
||||
}')"
|
||||
|
||||
@@ -13,12 +13,33 @@ on:
|
||||
required: false
|
||||
default: false
|
||||
type: boolean
|
||||
# Auto-rebuild the CUDA images when their Dockerfile changes on main. The CUDA
|
||||
# matrix is the only lane that builds from docker/Dockerfile, so a path-scoped
|
||||
# push trigger is a sufficient change detector on its own -- no separate
|
||||
# detect-changes/paths-filter job is needed now that there is a single
|
||||
# in-scope Dockerfile. Dreamverse (apps/dreamverse/docker/Dockerfile) and the
|
||||
# rocm Dockerfile stay manual-dispatch only.
|
||||
push:
|
||||
branches: [main]
|
||||
paths:
|
||||
- 'docker/Dockerfile'
|
||||
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
packages: write
|
||||
|
||||
# One static group, no cancellation: every run of this workflow writes the same
|
||||
# mutable registry tags (latest, py3.12-latest, ...), so runs must serialize —
|
||||
# concurrent push/dispatch runs would race on those tags, and cancelling a run
|
||||
# mid-publish can strand the cu126/cu130 tag families at different commits. An
|
||||
# in-flight superseded build wastes its runner time, but its tags are then
|
||||
# overwritten by the newer queued run. GitHub keeps a single pending run per
|
||||
# group: the newest queued run replaces any older queued one.
|
||||
concurrency:
|
||||
group: infra-build-image
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
# CUDA matrix: Python 3.12 x {12.6.3, 13.0.0} x {amd64, arm64}. Each architecture
|
||||
# builds natively and pushes only by digest; publish-cuda-manifests is the sole
|
||||
@@ -28,7 +49,11 @@ jobs:
|
||||
# aliases; 13.0.0/cu130 is published under explicit versioned tags. Flash-attn
|
||||
# 2.8.3 comes from the architecture-specific prebuilt releases.
|
||||
build-cuda-images:
|
||||
if: ${{ github.event.inputs.build_cuda_matrix == 'true' }}
|
||||
# Runs on a manual dispatch when build_cuda_matrix is set, or automatically
|
||||
# on a push that changed docker/Dockerfile (inputs are null on push). The
|
||||
# repository guard keeps fork syncs from auto-building; manual dispatch
|
||||
# still works in forks.
|
||||
if: ${{ (github.event_name == 'push' && github.repository == 'hao-ai-lab/FastVideo') || github.event.inputs.build_cuda_matrix == 'true' }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
@@ -75,10 +100,10 @@ jobs:
|
||||
secrets: inherit
|
||||
|
||||
publish-cuda-manifests:
|
||||
# !cancelled(): a failed sibling build leg must not skip the manifests for a
|
||||
# CUDA lane whose own digests all exist; the digest-count check below fails
|
||||
# the incomplete lane loudly instead.
|
||||
if: ${{ !cancelled() && github.event.inputs.build_cuda_matrix == 'true' }}
|
||||
# !cancelled(): publish lanes whose digests exist even if a sibling build
|
||||
# leg failed (the digest-count check fails incomplete lanes); it also
|
||||
# bypasses skipped-needs propagation, hence the explicit skipped check.
|
||||
if: ${{ !cancelled() && needs.build-cuda-images.result != 'skipped' }}
|
||||
needs: build-cuda-images
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
|
||||
+5
-2
@@ -6,6 +6,7 @@ results/
|
||||
wandb/
|
||||
*.ipynb
|
||||
*.jpg
|
||||
!examples/dataset/lingbotworld2/image.jpg
|
||||
*.safetensors
|
||||
*.mp4
|
||||
*.png
|
||||
@@ -34,6 +35,7 @@ env
|
||||
*.log
|
||||
weights/
|
||||
logs/
|
||||
/Z-Image/
|
||||
official_weights/
|
||||
converted_weights/
|
||||
|
||||
@@ -72,8 +74,7 @@ docs/distillation/examples/
|
||||
# Python pickle files
|
||||
*.pkl
|
||||
|
||||
# Reference videos
|
||||
!fastvideo/tests/ssim/reference_videos/**/*.mp4
|
||||
# Reference videos (negations must come after the catch-all on line below)
|
||||
|
||||
# Static images
|
||||
!docs/assets/images/**/*.png
|
||||
@@ -127,6 +128,8 @@ apps/dreamverse/web/.env.production.local
|
||||
.sisyphus/
|
||||
openspec/
|
||||
fastvideo/tests/ssim/reference_videos/**
|
||||
!fastvideo/tests/ssim/reference_videos/**/*.mp4
|
||||
!fastvideo/tests/ssim/reference_videos/**/*.png
|
||||
|
||||
# Editor logs and local Python version pins (accidentally committed)
|
||||
*.nvimlog
|
||||
|
||||
@@ -10,8 +10,6 @@ exclude: |
|
||||
scripts/.*|
|
||||
fastvideo/dataset/.*|
|
||||
fastvideo/models/.*|
|
||||
v2/(layers|attention|platforms|configs|distributed|models|logging_utils|third_party|hooks|api)/.*|
|
||||
v2/(envs|logger|utils|version|forward_context|fastvideo_args)\.py|
|
||||
^apps/dreamverse/web/.*|
|
||||
examples/.*|
|
||||
\.agents/.*|
|
||||
|
||||
@@ -9,7 +9,7 @@
|
||||
**FastVideo is a unified post-training and real-time inference framework for accelerated video generation.**
|
||||
|
||||
## NEWS
|
||||
- `2026/06/23`: Release FastWan-QAD: 5s of Video generated in 1.8s E2E. [FastWan-QAD models](https://huggingface.co/FastVideo/FastWan-QAD-FP8-1.3B), check out the [Blog](https://haoailab.com/blogs/fastwan-qad/).
|
||||
- `2026/06/23`: Release FastWan-QAD: 5s of Video generated in 1.8s E2E. See the [FastWan-QAD models](https://huggingface.co/FastVideo/FastWan-QAD-FP8-1.3B), [Attn-QAT training guide](https://haoailab.com/FastVideo/training/attn_qat/), and [blog](https://haoailab.com/blogs/fastwan-qad/).
|
||||
- `2026/03/17`: Release demo: Into the Dreamverse: Vibe Directing in FastVideo, check out the [Blog](https://haoailab.com/blogs/dreamverse/).
|
||||
- `2026/03/13`: Release demo: Create a 5s 1080p Video in 4.5s with FastVideo on a Single GPU, check out the [Blog](https://haoailab.com/blogs/fastvideo_realtime_1080p/).
|
||||
- `2025/11/19`: Release [CausalWan2.2 I2V A14B Preview](https://huggingface.co/FastVideo/CausalWan2.2-I2V-A14B-Preview-Diffusers) models, [Blog](https://hao-ai-lab.github.io/blogs/fastvideo_causalwan_preview/) and [Inference Code!](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_self_forcing_causal_wan2_2_i2v.py).
|
||||
|
||||
@@ -84,9 +84,12 @@ RUN source /opt/venv/bin/activate \
|
||||
FFMPEG_NATIVE_CXX=/usr/bin/g++ \
|
||||
bash /opt/FastVideo/apps/dreamverse/scripts/install_native_ffmpeg.sh
|
||||
|
||||
# FASTVIDEO_FA4: FA4 (flash_attn.cute) is opt-in; this image installs it via
|
||||
# the dreamverse extra and is validated with it, so enable it here.
|
||||
ENV FASTVIDEO_DREAMVERSE_HOME=/var/lib/dreamverse \
|
||||
STREAM_MODE=av_fmp4 \
|
||||
FASTVIDEO_ENABLE_PROMPT_SAFETY=0 \
|
||||
FASTVIDEO_FA4=1 \
|
||||
HF_HOME=/root/.cache/huggingface
|
||||
|
||||
RUN mkdir -p /var/lib/dreamverse
|
||||
|
||||
@@ -39,6 +39,17 @@ export FASTVIDEO_GENERATION_SEGMENT_CAP="${FASTVIDEO_GENERATION_SEGMENT_CAP:-6}"
|
||||
export FASTVIDEO_PROMPT_AUTO_SLEEP_MS="${FASTVIDEO_PROMPT_AUTO_SLEEP_MS:-120}"
|
||||
export FASTVIDEO_PROMPT_AUTO_TIMEOUT_MS="${FASTVIDEO_PROMPT_AUTO_TIMEOUT_MS:-1800}"
|
||||
|
||||
if [[ "${ENABLE_TORCH_COMPILE}" == "1" ]]; then
|
||||
# Persist Inductor, AOTAutograd, and Triton artifacts across launches.
|
||||
export DREAMVERSE_TORCH_COMPILE_CACHE_ROOT="${DREAMVERSE_TORCH_COMPILE_CACHE_ROOT:-${HOME}/.cache/dreamverse/torch_compile}"
|
||||
export TORCHINDUCTOR_CACHE_DIR="${TORCHINDUCTOR_CACHE_DIR:-${DREAMVERSE_TORCH_COMPILE_CACHE_ROOT}/inductor}"
|
||||
export TRITON_CACHE_DIR="${TRITON_CACHE_DIR:-${DREAMVERSE_TORCH_COMPILE_CACHE_ROOT}/triton}"
|
||||
export TORCHINDUCTOR_FX_GRAPH_CACHE="${TORCHINDUCTOR_FX_GRAPH_CACHE:-1}"
|
||||
export TORCHINDUCTOR_AUTOGRAD_CACHE="${TORCHINDUCTOR_AUTOGRAD_CACHE:-1}"
|
||||
mkdir -p "${TORCHINDUCTOR_CACHE_DIR}" "${TRITON_CACHE_DIR}"
|
||||
echo "[launch-demo] torch.compile cache: ${DREAMVERSE_TORCH_COMPILE_CACHE_ROOT}"
|
||||
fi
|
||||
|
||||
cd "${DREAMVERSE_ROOT}"
|
||||
|
||||
if ! command -v dreamverse-server >/dev/null 2>&1; then
|
||||
|
||||
@@ -1 +1 @@
|
||||
PUBLIC_API_BASE_URL=http://localhost:8189/api
|
||||
NEXT_PUBLIC_API_BASE_URL=http://localhost:8189/api
|
||||
@@ -13,6 +13,12 @@
|
||||
# testing
|
||||
/coverage
|
||||
|
||||
# playwright
|
||||
/test-results/
|
||||
/playwright-report/
|
||||
/blob-report/
|
||||
/.playwright/
|
||||
|
||||
# next.js
|
||||
/.next/
|
||||
/out/
|
||||
@@ -39,4 +45,3 @@ yarn-error.log*
|
||||
# typescript
|
||||
*.tsbuildinfo
|
||||
next-env.d.ts
|
||||
.svelte-kit
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
{
|
||||
"useTabs": true,
|
||||
"tabWidth": 4
|
||||
}
|
||||
@@ -11,19 +11,63 @@ The UI currently supports:
|
||||
- Datasets
|
||||
- Gallery View
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
apps/fastvideo_studio/
|
||||
├── server.py / job_runner.py / database.py # FastAPI backend + job lifecycle
|
||||
├── mock_server.py # In-memory API mock for e2e tests
|
||||
├── models/ # Pydantic request models (shared with the mock)
|
||||
├── training_config.py # Studio workloads → fastvideo/train YAML configs
|
||||
├── tests/ # Backend unit tests (pytest)
|
||||
├── e2e/ # Playwright specs (run against the mock)
|
||||
└── src/
|
||||
├── app/ # Next.js App Router pages (thin routes)
|
||||
├── components/
|
||||
│ ├── shell/ # App chrome: header, sidebars, layout
|
||||
│ ├── jobs/ # Job queue, cards, create-job modal, log sidebar
|
||||
│ ├── datasets/ # Dataset cards, upload, captions
|
||||
│ └── ui/ # shadcn-style primitives (shared theme)
|
||||
├── stores/ # Framework-agnostic state + React bridge (hooks/)
|
||||
├── lib/ # API client, types, option persistence
|
||||
└── test/ # Vitest setup + factories
|
||||
```
|
||||
|
||||
The visual theme (slate light/dark palettes, IBM Plex type, `#356cff` accent)
|
||||
is shared with `apps/dreamverse`; the toggle in the header persists the choice
|
||||
per browser.
|
||||
|
||||
## Testing
|
||||
|
||||
```bash
|
||||
npm run typecheck # tsc
|
||||
npm test # vitest unit tests
|
||||
npm run e2e # Playwright against the in-memory mock backend
|
||||
python -m pytest tests/ # backend unit tests (from apps/fastvideo_studio)
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
|
||||
The easiest way to run the application is:
|
||||
For local development, install dependencies and start the Next.js dev server:
|
||||
|
||||
```bash
|
||||
cd apps/fastvideo_studio
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
|
||||
You can then access the app at [http://localhost:3000](http://localhost:3000).
|
||||
Start the Python API server (default port 8189) in a separate terminal — see below.
|
||||
|
||||
For a production build of the web app together with the API server:
|
||||
|
||||
```bash
|
||||
cd apps/fastvideo_studio
|
||||
npm install
|
||||
npm run build
|
||||
npm run start
|
||||
npm run start:all
|
||||
```
|
||||
|
||||
You can then access the app at [http://localhost:3000](http://localhost:3000).
|
||||
|
||||
### Running API and Web Separately
|
||||
|
||||
The UI is composed of two separate components:
|
||||
@@ -31,7 +75,7 @@ The UI is composed of two separate components:
|
||||
- API Server
|
||||
- Web Server
|
||||
|
||||
To run each component separately, you can use the commands `npm run start:web` and `npm run start:api`.
|
||||
To run each component separately, you can use the commands `npm run start:web` (after `npm run build`) and `npm run start:api`.
|
||||
|
||||
You can also run the API server with this command from the `apps/` directory:
|
||||
|
||||
|
||||
@@ -251,11 +251,10 @@ class Database:
|
||||
sp_size, negative_prompt,
|
||||
data_path, max_train_steps, train_batch_size, learning_rate,
|
||||
num_latent_t, validation_dataset_file, lora_rank,
|
||||
ltx2_first_frame_conditioning_p,
|
||||
dmd_use_vsa, dmd_vsa_sparsity, dmd_denoising_steps,
|
||||
min_timestep_ratio, max_timestep_ratio, real_score_guidance_scale,
|
||||
real_score_guidance_scale,
|
||||
generator_update_interval, real_score_model_path, fake_score_model_path
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
job["id"],
|
||||
@@ -297,12 +296,9 @@ class Database:
|
||||
job.get("num_latent_t", 20),
|
||||
job.get("validation_dataset_file", ""),
|
||||
job.get("lora_rank", 32),
|
||||
job.get("ltx2_first_frame_conditioning_p"),
|
||||
1 if job.get("dmd_use_vsa") else 0,
|
||||
job.get("dmd_vsa_sparsity", 0.8),
|
||||
job.get("dmd_denoising_steps", "1000,757,522"),
|
||||
job.get("min_timestep_ratio", 0.02),
|
||||
job.get("max_timestep_ratio", 0.98),
|
||||
job.get("real_score_guidance_scale", 3.5),
|
||||
job.get("generator_update_interval", 5),
|
||||
job.get("real_score_model_path", ""),
|
||||
@@ -401,24 +397,6 @@ class Database:
|
||||
for file_name, caption in captions.items():
|
||||
self.upsert_dataset_caption(dataset_id, file_name, caption)
|
||||
|
||||
def update_dataset(self, dataset_id: str, updates: dict[str, Any]) -> None:
|
||||
"""Update dataset fields. Only provided keys are updated."""
|
||||
if not updates:
|
||||
return
|
||||
allowed: set[str] = set()
|
||||
cols = []
|
||||
vals = []
|
||||
for k, v in updates.items():
|
||||
if k in allowed:
|
||||
cols.append(f"{k} = ?")
|
||||
vals.append(v)
|
||||
if not cols:
|
||||
return
|
||||
vals.append(dataset_id)
|
||||
sql = f"UPDATE datasets SET {', '.join(cols)} WHERE id = ?"
|
||||
self._execute(sql, tuple(vals))
|
||||
self._commit()
|
||||
|
||||
def delete_dataset(self, dataset_id: str) -> bool:
|
||||
"""Delete a dataset. Returns True if a row was deleted."""
|
||||
cur = self._execute("DELETE FROM datasets WHERE id = ?", (dataset_id, ))
|
||||
@@ -443,7 +421,7 @@ class Database:
|
||||
cur = self._execute("SELECT * FROM settings WHERE id = 1")
|
||||
row = cur.fetchone()
|
||||
if not row:
|
||||
return _default_settings_dict()
|
||||
return default_settings_dict()
|
||||
t2v = ((row["default_model_id_t2v"] or row["default_model_id"] or "") if "default_model_id_t2v" in row else
|
||||
(row["default_model_id"] or ""))
|
||||
i2v = ((row["default_model_id_i2v"] or "") if "default_model_id_i2v" in row else "")
|
||||
@@ -557,8 +535,6 @@ def _row_to_job(row: sqlite3.Row) -> dict[str, Any]:
|
||||
}
|
||||
float_defaults = {
|
||||
"dmd_vsa_sparsity": 0.8,
|
||||
"min_timestep_ratio": 0.02,
|
||||
"max_timestep_ratio": 0.98,
|
||||
"real_score_guidance_scale": 3.5,
|
||||
}
|
||||
result = {
|
||||
@@ -625,7 +601,7 @@ def _row_to_job(row: sqlite3.Row) -> dict[str, Any]:
|
||||
return result
|
||||
|
||||
|
||||
def _default_settings_dict() -> dict[str, Any]:
|
||||
def default_settings_dict() -> dict[str, Any]:
|
||||
"""Return default settings as API-style dict."""
|
||||
return {
|
||||
"defaultModelId": DEFAULT_SETTINGS["default_model_id"],
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import { expect, test } from '@playwright/test';
|
||||
|
||||
import { skipWithoutMock } from './helpers';
|
||||
|
||||
/**
|
||||
* Create-job flow: open the Create Job modal on /inference, fill the prompt
|
||||
* (the model auto-selects once the mock's /api/models loads), submit, and
|
||||
* confirm the new job lands in the queue.
|
||||
*/
|
||||
test.describe('create inference job', () => {
|
||||
skipWithoutMock();
|
||||
|
||||
test('creates a T2V job and shows it in the queue', async ({ page }) => {
|
||||
await page.goto('/inference');
|
||||
|
||||
// The "Create Job" button reveals a workload menu on hover; wait for the
|
||||
// T2V item to become visible before clicking so the CSS hover transition
|
||||
// can't race the click.
|
||||
await page.getByRole('button', { name: /create job/i }).hover();
|
||||
const t2vItem = page.getByRole('menuitem', { name: /T2V/i });
|
||||
await expect(t2vItem).toBeVisible();
|
||||
await t2vItem.click();
|
||||
|
||||
const dialog = page.getByRole('dialog');
|
||||
await expect(dialog).toBeVisible();
|
||||
|
||||
// Wait for the mock's model catalogue to populate the dropdown (more than
|
||||
// just the disabled placeholder), then pick one explicitly — the app's
|
||||
// auto-selection is racy.
|
||||
const modelSelect = dialog.getByLabel('Model', { exact: true });
|
||||
await expect(modelSelect.locator('option')).not.toHaveCount(1);
|
||||
await modelSelect.selectOption({ index: 1 });
|
||||
|
||||
const prompt = `e2e raccoon in sunflowers ${Date.now()}`;
|
||||
await dialog.getByLabel('Prompt', { exact: true }).fill(prompt);
|
||||
|
||||
await dialog.getByRole('button', { name: 'Create Job' }).click();
|
||||
|
||||
// Modal closes and the queue refreshes with the newly created job.
|
||||
await expect(dialog).toBeHidden();
|
||||
await expect(page.getByText(prompt)).toBeVisible();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,37 @@
|
||||
import { expect, test } from '@playwright/test';
|
||||
|
||||
import { API_BASE, skipWithoutMock } from './helpers';
|
||||
|
||||
/**
|
||||
* Datasets page: the seeded datasets render as cards, and the Create Dataset
|
||||
* modal opens from the header action.
|
||||
*/
|
||||
test.describe('datasets', () => {
|
||||
skipWithoutMock();
|
||||
|
||||
test('lists the seeded datasets', async ({ page, request }) => {
|
||||
// Read the seeded names from the mock so the assertion tracks the fixture.
|
||||
const res = await request.get(`${API_BASE}/datasets`);
|
||||
const datasets = (await res.json()) as Array<{ name: string }>;
|
||||
expect(datasets.length).toBeGreaterThan(0);
|
||||
|
||||
await page.goto('/datasets');
|
||||
|
||||
for (const ds of datasets) {
|
||||
await expect(page.getByText(ds.name, { exact: true })).toBeVisible();
|
||||
}
|
||||
});
|
||||
|
||||
test('opens the Create Dataset modal', async ({ page }) => {
|
||||
await page.goto('/datasets');
|
||||
|
||||
await page.getByRole('button', { name: 'Add Dataset' }).click();
|
||||
|
||||
const dialog = page.getByRole('dialog');
|
||||
await expect(dialog).toBeVisible();
|
||||
await expect(
|
||||
dialog.getByRole('heading', { name: /Add Dataset/i }),
|
||||
).toBeVisible();
|
||||
await expect(dialog.getByLabel('Name', { exact: true })).toBeVisible();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,41 @@
|
||||
import { expect, test } from '@playwright/test';
|
||||
|
||||
import { API_BASE, skipWithoutMock } from './helpers';
|
||||
|
||||
/**
|
||||
* Gallery page: the seeded completed inference job surfaces as a media tile
|
||||
* (an <article> wrapping a <video>) captioned with its prompt.
|
||||
*/
|
||||
test.describe('gallery', () => {
|
||||
skipWithoutMock();
|
||||
|
||||
test('shows a media tile for the seeded completed job', async ({
|
||||
page,
|
||||
request,
|
||||
}) => {
|
||||
const res = await request.get(`${API_BASE}/jobs?job_type=inference`);
|
||||
const jobs = (await res.json()) as Array<{
|
||||
status: string;
|
||||
output_path: string | null;
|
||||
prompt: string;
|
||||
}>;
|
||||
const completed = jobs.find(
|
||||
(j) => j.status === 'completed' && j.output_path,
|
||||
);
|
||||
expect(completed, 'mock should seed a completed inference job').toBeTruthy();
|
||||
|
||||
await page.goto('/gallery');
|
||||
|
||||
await expect(
|
||||
page.getByRole('heading', { level: 1, name: 'Gallery' }),
|
||||
).toBeVisible();
|
||||
|
||||
// The completed job renders as an <article> containing a <video> tile.
|
||||
const tile = page
|
||||
.locator('article')
|
||||
.filter({ has: page.locator('video') });
|
||||
await expect(tile.first()).toBeVisible();
|
||||
|
||||
await expect(page.getByText(completed!.prompt)).toBeVisible();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,30 @@
|
||||
import { expect, test } from '@playwright/test';
|
||||
|
||||
import { skipWithoutMock } from './helpers';
|
||||
|
||||
/**
|
||||
* GPU status page: the mock's fake GPUs render as cards with meters, and the
|
||||
* page is reachable from the primary sidebar.
|
||||
*/
|
||||
test.describe('gpus', () => {
|
||||
skipWithoutMock();
|
||||
|
||||
test('lists the mock GPUs with utilization meters', async ({ page }) => {
|
||||
await page.goto('/gpus');
|
||||
|
||||
await expect(
|
||||
page.getByRole('heading', { level: 1, name: 'GPUs' }),
|
||||
).toBeVisible();
|
||||
|
||||
await expect(page.getByText('GPU 0')).toBeVisible();
|
||||
await expect(page.getByText('GPU 1')).toBeVisible();
|
||||
expect(await page.getByRole('meter').count()).toBe(4);
|
||||
});
|
||||
|
||||
test('is reachable from the sidebar', async ({ page }) => {
|
||||
await page.goto('/inference');
|
||||
await page.getByRole('link', { name: 'GPUs' }).click();
|
||||
await expect(page).toHaveURL(/\/gpus$/);
|
||||
await expect(page.getByText('GPU 0')).toBeVisible();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,44 @@
|
||||
import { test, type APIRequestContext } from '@playwright/test';
|
||||
|
||||
/**
|
||||
* Port and base URL of the mock API — the single source of truth shared by
|
||||
* playwright.config.ts (which boots the mock and points `npm run dev` at it
|
||||
* via NEXT_PUBLIC_API_BASE_URL) and the specs (which probe/read mock state
|
||||
* directly through the Playwright `request` fixture).
|
||||
*
|
||||
* The default port is deliberately NOT the real server's default (8189): the
|
||||
* e2e specs create and (if autostart is on) launch jobs, so running them
|
||||
* against a real backend would mutate the developer's database and fire real
|
||||
* GPU runs. API_BASE is derived solely from MOCK_API_PORT (it does not inherit
|
||||
* NEXT_PUBLIC_API_BASE_URL) so the specs always talk to the mock.
|
||||
*/
|
||||
export const MOCK_API_PORT = process.env.MOCK_API_PORT || '8190';
|
||||
|
||||
export const API_BASE = `http://127.0.0.1:${MOCK_API_PORT}/api`;
|
||||
|
||||
export const MOCK_SKIP_MESSAGE =
|
||||
`Mock backend not reachable at ${API_BASE}/__mock__ ` +
|
||||
`(start fastvideo_studio.mock_server on port ${MOCK_API_PORT}).`;
|
||||
|
||||
/**
|
||||
* True only when the mock server answers on the configured port. Probes the
|
||||
* mock-only /api/__mock__ sentinel (which the real server does not serve) so
|
||||
* the suite refuses to run its mutating specs against a real backend.
|
||||
*/
|
||||
export async function mockIsUp(request: APIRequestContext): Promise<boolean> {
|
||||
try {
|
||||
const res = await request.get(`${API_BASE}/__mock__`);
|
||||
if (!res.ok()) return false;
|
||||
const body = await res.json();
|
||||
return body?.mock === true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/** Self-skip every test in the enclosing describe when the mock is down. */
|
||||
export function skipWithoutMock(): void {
|
||||
test.beforeEach(async ({ request }) => {
|
||||
test.skip(!(await mockIsUp(request)), MOCK_SKIP_MESSAGE);
|
||||
});
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
import { expect, test } from '@playwright/test';
|
||||
|
||||
import { skipWithoutMock } from './helpers';
|
||||
|
||||
/**
|
||||
* App-shell smoke: the Next.js frontend hydrates, renders the FastVideo logo
|
||||
* and the primary-sidebar navigation, and routing between the top-level
|
||||
* sections works. Each spec self-skips when the mock backend isn't reachable.
|
||||
*/
|
||||
test.describe('app shell', () => {
|
||||
skipWithoutMock();
|
||||
|
||||
test('loads with the logo and primary-sidebar nav', async ({ page }) => {
|
||||
await page.goto('/');
|
||||
// The root route redirects to /inference.
|
||||
await expect(page).toHaveURL(/\/inference$/);
|
||||
|
||||
await expect(page.getByRole('img', { name: /fastvideo/i })).toBeVisible();
|
||||
|
||||
for (const label of ['Inference', 'Datasets', 'Gallery', 'Settings']) {
|
||||
await expect(page.getByRole('link', { name: label })).toBeVisible();
|
||||
}
|
||||
});
|
||||
|
||||
test('navigates between the primary sections', async ({ page }) => {
|
||||
await page.goto('/inference');
|
||||
await expect(
|
||||
page.getByRole('heading', { level: 1, name: 'Jobs' }),
|
||||
).toBeVisible();
|
||||
|
||||
const sections: Array<{ link: string; url: RegExp; title: string }> = [
|
||||
{ link: 'Datasets', url: /\/datasets$/, title: 'Datasets' },
|
||||
{ link: 'Gallery', url: /\/gallery$/, title: 'Gallery' },
|
||||
{ link: 'Settings', url: /\/settings$/, title: 'Settings' },
|
||||
{ link: 'Inference', url: /\/inference$/, title: 'Jobs' },
|
||||
];
|
||||
|
||||
for (const section of sections) {
|
||||
await page.getByRole('link', { name: section.link }).click();
|
||||
await expect(page).toHaveURL(section.url);
|
||||
// The header <h1> is the only level-1 heading and reflects the route.
|
||||
await expect(
|
||||
page.getByRole('heading', { level: 1, name: section.title }),
|
||||
).toBeVisible();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -1,2 +0,0 @@
|
||||
// Minimal ESLint config for SvelteKit (add typescript-eslint / eslint-plugin-svelte as needed)
|
||||
export default [];
|
||||
@@ -0,0 +1,74 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""GPU telemetry for the studio status page, via NVML (nvidia-ml-py)."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
logger = logging.getLogger("fastvideo.studio.gpu")
|
||||
|
||||
_nvml_initialized = False
|
||||
|
||||
|
||||
def _ensure_nvml() -> Any:
|
||||
"""Import and initialize NVML once; raises on machines without it."""
|
||||
global _nvml_initialized # noqa: PLW0603
|
||||
import pynvml
|
||||
if not _nvml_initialized:
|
||||
pynvml.nvmlInit()
|
||||
_nvml_initialized = True
|
||||
return pynvml
|
||||
|
||||
|
||||
def _device_snapshot(pynvml: Any, index: int) -> dict[str, Any]:
|
||||
handle = pynvml.nvmlDeviceGetHandleByIndex(index)
|
||||
name = pynvml.nvmlDeviceGetName(handle)
|
||||
if isinstance(name, bytes):
|
||||
name = name.decode()
|
||||
mem = pynvml.nvmlDeviceGetMemoryInfo(handle)
|
||||
util = pynvml.nvmlDeviceGetUtilizationRates(handle)
|
||||
|
||||
# Optional sensors: not every GPU/driver exposes them.
|
||||
temperature: int | None = None
|
||||
power_watts: float | None = None
|
||||
power_limit_watts: float | None = None
|
||||
with contextlib.suppress(pynvml.NVMLError):
|
||||
temperature = int(pynvml.nvmlDeviceGetTemperature(handle, pynvml.NVML_TEMPERATURE_GPU))
|
||||
with contextlib.suppress(pynvml.NVMLError):
|
||||
power_watts = pynvml.nvmlDeviceGetPowerUsage(handle) / 1000.0
|
||||
power_limit_watts = (pynvml.nvmlDeviceGetEnforcedPowerLimit(handle) / 1000.0)
|
||||
|
||||
return {
|
||||
"index": index,
|
||||
"name": name,
|
||||
"utilization": int(util.gpu),
|
||||
"memory_used_mib": int(mem.used / (1024 * 1024)),
|
||||
"memory_total_mib": int(mem.total / (1024 * 1024)),
|
||||
"temperature_c": temperature,
|
||||
"power_watts": power_watts,
|
||||
"power_limit_watts": power_limit_watts,
|
||||
}
|
||||
|
||||
|
||||
def get_gpu_snapshot() -> dict[str, Any]:
|
||||
"""Return {"available", "gpus", "error"} — never raises.
|
||||
|
||||
``available: False`` covers both "no NVIDIA driver/library on this host"
|
||||
and transient NVML failures; the frontend shows ``error`` as-is.
|
||||
"""
|
||||
try:
|
||||
pynvml = _ensure_nvml()
|
||||
count = pynvml.nvmlDeviceGetCount()
|
||||
gpus = [_device_snapshot(pynvml, i) for i in range(count)]
|
||||
return {"available": True, "gpus": gpus, "error": None}
|
||||
except ImportError:
|
||||
return {
|
||||
"available": False,
|
||||
"gpus": [],
|
||||
"error": "nvidia-ml-py is not installed on the API server host.",
|
||||
}
|
||||
except Exception as exc: # NVMLError, driver issues, …
|
||||
logger.warning("GPU snapshot failed: %s", exc)
|
||||
return {"available": False, "gpus": [], "error": str(exc)}
|
||||
@@ -23,12 +23,13 @@ import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
import yaml
|
||||
|
||||
from fastvideo.utils import get_mp_context
|
||||
from fastvideo_studio.database import Database
|
||||
from fastvideo_studio.training_config import (
|
||||
build_training_args,
|
||||
build_training_config,
|
||||
get_training_env,
|
||||
get_training_module_info,
|
||||
)
|
||||
|
||||
logger = logging.getLogger("fastvideo.studio.job_runner")
|
||||
@@ -160,13 +161,10 @@ class Job:
|
||||
num_latent_t: int = 20
|
||||
validation_dataset_file: str = ""
|
||||
lora_rank: int = 32
|
||||
ltx2_first_frame_conditioning_p: float | None = None
|
||||
# DMD options
|
||||
dmd_use_vsa: bool = False
|
||||
dmd_vsa_sparsity: float = 0.8
|
||||
dmd_denoising_steps: str = "1000,757,522"
|
||||
min_timestep_ratio: float = 0.02
|
||||
max_timestep_ratio: float = 0.98
|
||||
real_score_guidance_scale: float = 3.5
|
||||
generator_update_interval: int = 5
|
||||
real_score_model_path: str = ""
|
||||
@@ -221,12 +219,9 @@ class Job:
|
||||
"num_width": self.width,
|
||||
"validation_dataset_file": self.validation_dataset_file,
|
||||
"lora_rank": self.lora_rank,
|
||||
"ltx2_first_frame_conditioning_p": self.ltx2_first_frame_conditioning_p,
|
||||
"dmd_use_vsa": self.dmd_use_vsa,
|
||||
"dmd_vsa_sparsity": self.dmd_vsa_sparsity,
|
||||
"dmd_denoising_steps": self.dmd_denoising_steps,
|
||||
"min_timestep_ratio": self.min_timestep_ratio,
|
||||
"max_timestep_ratio": self.max_timestep_ratio,
|
||||
"real_score_guidance_scale": self.real_score_guidance_scale,
|
||||
"generator_update_interval": self.generator_update_interval,
|
||||
"real_score_model_path": self.real_score_model_path or "",
|
||||
@@ -330,12 +325,9 @@ class JobRunner:
|
||||
num_latent_t=row.get("num_latent_t", 20),
|
||||
validation_dataset_file=row.get("validation_dataset_file", "") or "",
|
||||
lora_rank=row.get("lora_rank", 32),
|
||||
ltx2_first_frame_conditioning_p=row.get("ltx2_first_frame_conditioning_p"),
|
||||
dmd_use_vsa=row.get("dmd_use_vsa", False),
|
||||
dmd_vsa_sparsity=float(row.get("dmd_vsa_sparsity", 0.8)),
|
||||
dmd_denoising_steps=row.get("dmd_denoising_steps", "1000,757,522") or "1000,757,522",
|
||||
min_timestep_ratio=float(row.get("min_timestep_ratio", 0.02)),
|
||||
max_timestep_ratio=float(row.get("max_timestep_ratio", 0.98)),
|
||||
real_score_guidance_scale=float(row.get("real_score_guidance_scale", 3.5)),
|
||||
generator_update_interval=int(row.get("generator_update_interval", 5)),
|
||||
real_score_model_path=row.get("real_score_model_path", "") or "",
|
||||
@@ -412,12 +404,9 @@ class JobRunner:
|
||||
num_latent_t: int = 20,
|
||||
validation_dataset_file: str = "",
|
||||
lora_rank: int = 32,
|
||||
ltx2_first_frame_conditioning_p: float | None = None,
|
||||
dmd_use_vsa: bool = False,
|
||||
dmd_vsa_sparsity: float = 0.8,
|
||||
dmd_denoising_steps: str = "1000,757,522",
|
||||
min_timestep_ratio: float = 0.02,
|
||||
max_timestep_ratio: float = 0.98,
|
||||
real_score_guidance_scale: float = 3.5,
|
||||
generator_update_interval: int = 5,
|
||||
real_score_model_path: str = "",
|
||||
@@ -457,12 +446,9 @@ class JobRunner:
|
||||
num_latent_t=num_latent_t,
|
||||
validation_dataset_file=validation_dataset_file or "",
|
||||
lora_rank=lora_rank,
|
||||
ltx2_first_frame_conditioning_p=ltx2_first_frame_conditioning_p,
|
||||
dmd_use_vsa=dmd_use_vsa,
|
||||
dmd_vsa_sparsity=dmd_vsa_sparsity,
|
||||
dmd_denoising_steps=dmd_denoising_steps,
|
||||
min_timestep_ratio=min_timestep_ratio,
|
||||
max_timestep_ratio=max_timestep_ratio,
|
||||
real_score_guidance_scale=real_score_guidance_scale,
|
||||
generator_update_interval=generator_update_interval,
|
||||
real_score_model_path=real_score_model_path or "",
|
||||
@@ -733,46 +719,23 @@ class JobRunner:
|
||||
self._save_job(job)
|
||||
return
|
||||
|
||||
module_info = get_training_module_info(job.workload_type, job.model_id)
|
||||
if not module_info:
|
||||
env = os.environ.copy()
|
||||
env.update(get_training_env())
|
||||
|
||||
try:
|
||||
# Job.to_dict() carries every key the config builder reads
|
||||
# (extra keys are ignored by its .get() lookups).
|
||||
train_config = build_training_config(job.to_dict(), job_output_dir)
|
||||
except ValueError as exc:
|
||||
job.status = JobStatus.FAILED
|
||||
job.error = f"Unknown workload type: {job.workload_type}"
|
||||
job.error = str(exc)
|
||||
job.finished_at = time.time()
|
||||
self._save_job(job)
|
||||
return
|
||||
|
||||
module_path, _pipeline_workload, use_vsa, _is_lora = module_info
|
||||
dmd_use_vsa = (job.workload_type.startswith("dmd_") and getattr(job, "dmd_use_vsa", False))
|
||||
env = os.environ.copy()
|
||||
env.update(get_training_env(use_vsa or dmd_use_vsa))
|
||||
|
||||
job_dict = {
|
||||
"model_id": job.model_id,
|
||||
"data_path": job.data_path,
|
||||
"workload_type": job.workload_type,
|
||||
"num_gpus": job.num_gpus,
|
||||
"max_train_steps": job.max_train_steps,
|
||||
"train_batch_size": job.train_batch_size,
|
||||
"learning_rate": job.learning_rate,
|
||||
"num_latent_t": job.num_latent_t,
|
||||
"num_height": job.height,
|
||||
"num_width": job.width,
|
||||
"num_frames": job.num_frames,
|
||||
"validation_dataset_file": job.validation_dataset_file,
|
||||
"lora_rank": job.lora_rank,
|
||||
"ltx2_first_frame_conditioning_p": job.ltx2_first_frame_conditioning_p,
|
||||
}
|
||||
if job.workload_type.startswith("dmd_") or job.workload_type.startswith("self_forcing_"):
|
||||
job_dict["dmd_use_vsa"] = getattr(job, "dmd_use_vsa", False)
|
||||
job_dict["dmd_vsa_sparsity"] = getattr(job, "dmd_vsa_sparsity", 0.8)
|
||||
job_dict["dmd_denoising_steps"] = getattr(job, "dmd_denoising_steps", "1000,757,522")
|
||||
job_dict["min_timestep_ratio"] = getattr(job, "min_timestep_ratio", 0.02)
|
||||
job_dict["max_timestep_ratio"] = getattr(job, "max_timestep_ratio", 0.98)
|
||||
job_dict["real_score_guidance_scale"] = getattr(job, "real_score_guidance_scale", 3.5)
|
||||
job_dict["generator_update_interval"] = getattr(job, "generator_update_interval", 5)
|
||||
job_dict["real_score_model_path"] = (getattr(job, "real_score_model_path", "") or job.model_id)
|
||||
job_dict["fake_score_model_path"] = (getattr(job, "fake_score_model_path", "") or job.model_id)
|
||||
train_args = build_training_args(job_dict, job_output_dir)
|
||||
config_path = os.path.join(job_output_dir, "train_config.yaml")
|
||||
with open(config_path, "w", encoding="utf-8") as f:
|
||||
yaml.safe_dump(train_config, f, sort_keys=False)
|
||||
|
||||
repo_root = Path(__file__).resolve().parent.parent
|
||||
torchrun_cmd = [
|
||||
@@ -783,9 +746,12 @@ class JobRunner:
|
||||
str(job.num_gpus),
|
||||
"--nnodes",
|
||||
"1",
|
||||
str(repo_root / module_path),
|
||||
] + train_args
|
||||
buf.write(f"Starting training: {' '.join(torchrun_cmd[:12])}...")
|
||||
"-m",
|
||||
"fastvideo.train.entrypoint.train",
|
||||
"--config",
|
||||
config_path,
|
||||
]
|
||||
buf.write(f"Starting training: {' '.join(torchrun_cmd)}")
|
||||
buf.phase = "starting"
|
||||
|
||||
try:
|
||||
@@ -825,12 +791,21 @@ class JobRunner:
|
||||
job._process.wait()
|
||||
exit_code = job._process.returncode or 0
|
||||
|
||||
if exit_code == 0:
|
||||
if job._stop_event.is_set():
|
||||
# Terminated by stop_job() without the reader loop observing the
|
||||
# flag (e.g. the process died between log lines).
|
||||
job.status = JobStatus.STOPPED
|
||||
buf.phase = "stopped"
|
||||
elif exit_code == 0:
|
||||
job.status = JobStatus.COMPLETED
|
||||
buf.progress = 100.0
|
||||
buf.phase = "done"
|
||||
# Training outputs checkpoints, not video
|
||||
ckpt_dirs = sorted(Path(job_output_dir).glob("checkpoint-*"))
|
||||
# Training outputs checkpoints, not video. Sort by step number,
|
||||
# not lexically ("checkpoint-1000" < "checkpoint-500" as strings).
|
||||
ckpt_dirs = sorted(
|
||||
Path(job_output_dir).glob("checkpoint-*"),
|
||||
key=lambda p: int(m.group(1)) if (m := re.fullmatch(r"checkpoint-(\d+)", p.name)) else -1,
|
||||
)
|
||||
if ckpt_dirs:
|
||||
job.output_path = str(ckpt_dirs[-1])
|
||||
else:
|
||||
|
||||
@@ -0,0 +1,660 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""
|
||||
In-memory mock of the FastVideo Studio API (``server.py``).
|
||||
|
||||
This mock implements the same ``/api`` routes and response shapes as the real
|
||||
FastAPI server but keeps everything in memory and never touches the FastVideo
|
||||
library, a GPU, or a database. It is meant to back the Playwright e2e suite so
|
||||
the Next.js frontend can be exercised end-to-end without a real backend.
|
||||
|
||||
Job lifecycle is simulated by recording a start timestamp and *computing* the
|
||||
status on read: a started job reports ``running`` for a few seconds and then
|
||||
flips to ``completed`` with an ``output_path``, so polling the job list / logs
|
||||
shows progression. Generated media is a tiny 1s ``testsrc`` MP4 built lazily
|
||||
with ffmpeg and cached.
|
||||
|
||||
Usage (from the ``apps/`` directory)::
|
||||
|
||||
PYTHONPATH=.. python -m fastvideo_studio.mock_server --port 8190
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import os
|
||||
import random
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from typing import Annotated, Any
|
||||
|
||||
from fastapi import FastAPI, File, HTTPException, UploadFile
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, PlainTextResponse
|
||||
|
||||
from fastvideo_studio.database import default_settings_dict
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, SettingsUpdate, UpdateCaptionRequest,
|
||||
model_label)
|
||||
|
||||
# --- Config -----------------------------------------------------------------
|
||||
|
||||
# How long a started job stays "running" before it flips to "completed".
|
||||
COMPLETE_AFTER_SECONDS = 3.0
|
||||
FFMPEG_BIN = shutil.which(os.getenv("FASTVIDEO_FFMPEG_BIN", "ffmpeg"))
|
||||
|
||||
# A small catalogue of fake models keyed by workload type. Mirrors the real
|
||||
# server's {id, label} shape (label derived from the HF-style path).
|
||||
_MODELS_BY_WORKLOAD: dict[str, list[str]] = {
|
||||
"t2v": [
|
||||
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
"FastVideo/FastHunyuan-diffusers",
|
||||
],
|
||||
"i2v": [
|
||||
"Wan-AI/Wan2.1-I2V-14B-480P-Diffusers",
|
||||
],
|
||||
"t2i": [
|
||||
"black-forest-labs/FLUX.1-schnell",
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _models_for(workload_type: str | None) -> list[dict[str, str]]:
|
||||
if workload_type:
|
||||
paths = _MODELS_BY_WORKLOAD.get(workload_type, [])
|
||||
else:
|
||||
seen: dict[str, None] = {}
|
||||
for paths_for_workload in _MODELS_BY_WORKLOAD.values():
|
||||
for path in paths_for_workload:
|
||||
seen.setdefault(path, None)
|
||||
paths = list(seen)
|
||||
return [{"id": path, "label": model_label(path)} for path in paths]
|
||||
|
||||
|
||||
# The real settings catalogue lives in database.py; the mock only pre-fills
|
||||
# the default model ids so the Create Job modal auto-selects one.
|
||||
_DEFAULT_SETTINGS: dict[str, Any] = {
|
||||
**default_settings_dict(),
|
||||
"defaultModelId": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
"defaultModelIdT2v": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
"defaultModelIdI2v": "Wan-AI/Wan2.1-I2V-14B-480P-Diffusers",
|
||||
"defaultModelIdT2i": "black-forest-labs/FLUX.1-schnell",
|
||||
}
|
||||
|
||||
# --- In-memory state --------------------------------------------------------
|
||||
|
||||
_state_lock = threading.Lock()
|
||||
_settings: dict[str, Any] = dict(_DEFAULT_SETTINGS)
|
||||
_jobs: dict[str, dict[str, Any]] = {}
|
||||
_datasets: dict[str, dict[str, Any]] = {}
|
||||
# dataset_id -> {"file_names": [...], "captions": {file_name: caption}}
|
||||
_dataset_files: dict[str, dict[str, Any]] = {}
|
||||
|
||||
# Lazily-built, cached media clips (shared by every job/dataset media response),
|
||||
# keyed by extension: "mp4" (1s testsrc video) or "png" (single testsrc frame).
|
||||
_mock_media_cache: dict[str, str] = {}
|
||||
_mock_media_lock = threading.Lock()
|
||||
|
||||
|
||||
def _build_mock_media(kind: str) -> str:
|
||||
"""Build (once) a tiny testsrc clip of the given ``kind`` with ffmpeg; cache the path."""
|
||||
with _mock_media_lock:
|
||||
cached = _mock_media_cache.get(kind)
|
||||
if cached and os.path.isfile(cached):
|
||||
return cached
|
||||
if not FFMPEG_BIN:
|
||||
raise HTTPException(status_code=500, detail="ffmpeg is required to build mock media")
|
||||
fd, path = tempfile.mkstemp(prefix="fvstudio_mock_", suffix=f".{kind}")
|
||||
os.close(fd)
|
||||
if kind == "png":
|
||||
command = [
|
||||
FFMPEG_BIN,
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
"testsrc=size=320x240:rate=1",
|
||||
"-frames:v",
|
||||
"1",
|
||||
"-f",
|
||||
"image2",
|
||||
path,
|
||||
]
|
||||
else:
|
||||
command = [
|
||||
FFMPEG_BIN,
|
||||
"-hide_banner",
|
||||
"-loglevel",
|
||||
"error",
|
||||
"-y",
|
||||
"-f",
|
||||
"lavfi",
|
||||
"-i",
|
||||
"testsrc=size=320x240:rate=24",
|
||||
"-t",
|
||||
"1",
|
||||
"-c:v",
|
||||
"libx264",
|
||||
"-preset",
|
||||
"ultrafast",
|
||||
"-pix_fmt",
|
||||
"yuv420p",
|
||||
"-movflags",
|
||||
"+faststart",
|
||||
"-f",
|
||||
"mp4",
|
||||
path,
|
||||
]
|
||||
try:
|
||||
subprocess.run(command, check=True, capture_output=True)
|
||||
except (subprocess.CalledProcessError, OSError) as exc:
|
||||
with contextlib.suppress(OSError):
|
||||
os.remove(path)
|
||||
raise HTTPException(status_code=500, detail=f"ffmpeg failed to build mock {kind}: {exc}") from exc
|
||||
if not os.path.isfile(path) or os.path.getsize(path) == 0:
|
||||
raise HTTPException(status_code=500, detail=f"ffmpeg produced no mock {kind} bytes")
|
||||
_mock_media_cache[kind] = path
|
||||
return path
|
||||
|
||||
|
||||
# --- Job helpers ------------------------------------------------------------
|
||||
|
||||
|
||||
def _new_job_dict(req: CreateJobRequest) -> dict[str, Any]:
|
||||
"""Build a job dict (mirrors job_runner.Job.to_dict()) in the pending state."""
|
||||
job_id = str(uuid.uuid4())
|
||||
return {
|
||||
"id": job_id,
|
||||
"model_id": req.model_id,
|
||||
"prompt": req.prompt,
|
||||
"workload_type": req.workload_type or "t2v",
|
||||
"job_type": req.job_type or "inference",
|
||||
"image_path": req.image_path or "",
|
||||
"status": "pending",
|
||||
"created_at": time.time(),
|
||||
"started_at": None,
|
||||
"finished_at": None,
|
||||
"error": None,
|
||||
"output_path": None,
|
||||
"log_file_path": None,
|
||||
"num_inference_steps": req.num_inference_steps,
|
||||
"num_frames": req.num_frames,
|
||||
"height": req.height,
|
||||
"width": req.width,
|
||||
"guidance_scale": req.guidance_scale,
|
||||
"guidance_rescale": req.guidance_rescale,
|
||||
"fps": req.fps,
|
||||
"seed": req.seed,
|
||||
"negative_prompt": req.negative_prompt or "",
|
||||
"num_gpus": req.num_gpus,
|
||||
"data_path": req.data_path or "",
|
||||
"progress": 0.0,
|
||||
"progress_msg": "",
|
||||
"phase": "pending",
|
||||
}
|
||||
|
||||
|
||||
def _advance_job(job: dict[str, Any]) -> None:
|
||||
"""Flip a running job to completed once enough wall-clock time has passed.
|
||||
|
||||
Status is *computed on read* from the recorded start timestamp, so polling
|
||||
the job list / logs naturally shows pending -> running -> completed.
|
||||
"""
|
||||
if job["status"] != "running" or not job.get("started_at"):
|
||||
return
|
||||
elapsed = time.time() - job["started_at"]
|
||||
if elapsed >= COMPLETE_AFTER_SECONDS:
|
||||
job["status"] = "completed"
|
||||
job["finished_at"] = job["started_at"] + COMPLETE_AFTER_SECONDS
|
||||
job["progress"] = 100.0
|
||||
job["progress_msg"] = "50/50 steps"
|
||||
job["phase"] = "done"
|
||||
ext = "png" if job.get("workload_type") == "t2i" else "mp4"
|
||||
job["output_path"] = f"/mock/outputs/{job['id']}/output.{ext}"
|
||||
|
||||
|
||||
def _public_job(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Advance + return a copy safe to serialize."""
|
||||
_advance_job(job)
|
||||
return dict(job)
|
||||
|
||||
|
||||
_LOG_TAIL = [
|
||||
"Loading model...",
|
||||
"Model loaded.",
|
||||
"Starting generation...",
|
||||
"Denoising step 10/50",
|
||||
"Denoising step 20/50",
|
||||
"Denoising step 30/50",
|
||||
"Denoising step 40/50",
|
||||
"Denoising step 50/50",
|
||||
"Generation complete. Saving output...",
|
||||
"Saved output file.",
|
||||
"Job completed successfully.",
|
||||
]
|
||||
|
||||
|
||||
def _log_sequence(job: dict[str, Any]) -> list[str]:
|
||||
return [
|
||||
f"Job {job['id']} started",
|
||||
f"Model: {job['model_id']}",
|
||||
f"Prompt: {job['prompt']}",
|
||||
*_LOG_TAIL,
|
||||
]
|
||||
|
||||
|
||||
def _compute_logs(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Return JobLogs-shaped data, growing the visible lines as time passes."""
|
||||
seq = _log_sequence(job)
|
||||
status = job["status"]
|
||||
if status == "pending":
|
||||
return {"lines": [], "progress": 0.0, "progress_msg": "", "phase": "pending"}
|
||||
if status == "completed":
|
||||
return {"lines": seq, "progress": 100.0, "progress_msg": "50/50 steps", "phase": "done"}
|
||||
if status in ("failed", "stopped"):
|
||||
# Keep the total line count monotonic across running -> terminal (the
|
||||
# `after` cursor relies on it): show every non-final line plus a terminal
|
||||
# notice, which is always >= any prefix a running job revealed.
|
||||
lines = seq[:-1] + [f"Job {status}."]
|
||||
return {"lines": lines, "progress": job.get("progress", 0.0), "progress_msg": "", "phase": status}
|
||||
# running: reveal a prefix proportional to elapsed time, but never the final
|
||||
# "completed" line — that appears only once the job is actually completed.
|
||||
elapsed = time.time() - (job.get("started_at") or time.time())
|
||||
frac = max(0.0, min(elapsed / COMPLETE_AFTER_SECONDS, 0.99))
|
||||
reveal = min(max(4, round(frac * len(seq))), len(seq) - 1)
|
||||
lines = seq[:reveal]
|
||||
if frac < 0.2:
|
||||
phase = "loading model"
|
||||
elif frac < 0.85:
|
||||
phase = "denoising"
|
||||
else:
|
||||
phase = "saving"
|
||||
return {
|
||||
"lines": lines,
|
||||
"progress": round(frac * 100.0, 1),
|
||||
"progress_msg": f"{int(frac * 50)}/50 steps",
|
||||
"phase": phase,
|
||||
}
|
||||
|
||||
|
||||
# --- Dataset helpers --------------------------------------------------------
|
||||
|
||||
|
||||
def _dataset_stats(dataset_id: str) -> tuple[int, int]:
|
||||
files = _dataset_files.get(dataset_id, {}).get("file_names", [])
|
||||
count = len(files)
|
||||
# Fake but stable per-file size so the UI shows a non-zero footprint.
|
||||
return count, count * 1_048_576
|
||||
|
||||
|
||||
def _public_dataset(dataset: dict[str, Any]) -> dict[str, Any]:
|
||||
count, size = _dataset_stats(dataset["id"])
|
||||
return {**dataset, "file_count": count, "size_bytes": size}
|
||||
|
||||
|
||||
def _seed() -> None:
|
||||
"""Seed a couple of datasets and one completed inference job."""
|
||||
now = time.time()
|
||||
seeds = [
|
||||
("Sunset Clips", ["sunset_01.mp4", "sunset_02.mp4"], {
|
||||
"sunset_01.mp4": "A sunset over the ocean",
|
||||
"sunset_02.mp4": "A sunset over the mountains"
|
||||
}),
|
||||
("City Timelapse", ["city_01.mp4"], {
|
||||
"city_01.mp4": "A busy city intersection at night"
|
||||
}),
|
||||
]
|
||||
for idx, (name, file_names, captions) in enumerate(seeds):
|
||||
dataset_id = str(uuid.uuid4())
|
||||
_datasets[dataset_id] = {"id": dataset_id, "name": name, "created_at": now - 600 + idx}
|
||||
_dataset_files[dataset_id] = {"file_names": list(file_names), "captions": dict(captions)}
|
||||
|
||||
job_id = str(uuid.uuid4())
|
||||
_jobs[job_id] = {
|
||||
"id": job_id,
|
||||
"model_id": "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
"prompt": "A curious raccoon peers through a field of yellow sunflowers",
|
||||
"workload_type": "t2v",
|
||||
"job_type": "inference",
|
||||
"image_path": "",
|
||||
"status": "completed",
|
||||
"created_at": now - 300,
|
||||
"started_at": now - 280,
|
||||
"finished_at": now - 250,
|
||||
"error": None,
|
||||
"output_path": f"/mock/outputs/{job_id}/output.mp4",
|
||||
"log_file_path": f"/mock/logs/{job_id}.log",
|
||||
"num_inference_steps": 50,
|
||||
"num_frames": 81,
|
||||
"height": 480,
|
||||
"width": 832,
|
||||
"guidance_scale": 5.0,
|
||||
"guidance_rescale": 0.0,
|
||||
"fps": 24,
|
||||
"seed": 1024,
|
||||
"negative_prompt": "",
|
||||
"num_gpus": 1,
|
||||
"data_path": "",
|
||||
"progress": 100.0,
|
||||
"progress_msg": "50/50 steps",
|
||||
"phase": "done",
|
||||
}
|
||||
|
||||
|
||||
# --- App --------------------------------------------------------------------
|
||||
|
||||
app = FastAPI(title="FastVideo Studio Mock API", version="0.1.0")
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
_seed()
|
||||
|
||||
|
||||
@app.get("/api/__mock__")
|
||||
def mock_sentinel() -> dict[str, bool]:
|
||||
"""Mock-only marker so the e2e suite can prove it isn't hitting a real
|
||||
backend before running its mutating specs (the real server has no such
|
||||
route)."""
|
||||
return {"mock": True}
|
||||
|
||||
|
||||
# --- Settings ---------------------------------------------------------------
|
||||
|
||||
|
||||
@app.get("/api/settings")
|
||||
def get_settings() -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
return dict(_settings)
|
||||
|
||||
|
||||
@app.put("/api/settings")
|
||||
def update_settings(settings: SettingsUpdate) -> dict[str, Any]:
|
||||
updates = settings.model_dump(exclude_unset=True)
|
||||
with _state_lock:
|
||||
_settings.update({k: v for k, v in updates.items() if v is not None})
|
||||
return dict(_settings)
|
||||
|
||||
|
||||
# --- Models -----------------------------------------------------------------
|
||||
|
||||
|
||||
@app.get("/api/models")
|
||||
def list_models(workload_type: str | None = None) -> list[dict[str, Any]]:
|
||||
return _models_for(workload_type)
|
||||
|
||||
|
||||
# --- GPUs -------------------------------------------------------------------
|
||||
|
||||
|
||||
@app.get("/api/gpus")
|
||||
def list_gpus() -> dict[str, Any]:
|
||||
"""Two fake GPUs with slight per-request jitter so the page looks live."""
|
||||
gpus = []
|
||||
for index, (base_util, used_mib) in enumerate([(62, 61_440), (7, 4_096)]):
|
||||
gpus.append({
|
||||
"index": index,
|
||||
"name": "NVIDIA Mock GPU 80GB",
|
||||
"utilization": max(0, min(100, base_util + random.randint(-5, 5))),
|
||||
"memory_used_mib": used_mib + random.randint(-256, 256),
|
||||
"memory_total_mib": 81_920,
|
||||
"temperature_c": 55 + random.randint(-3, 3),
|
||||
"power_watts": 310.0 + random.randint(-20, 20),
|
||||
"power_limit_watts": 700.0,
|
||||
})
|
||||
return {"available": True, "gpus": gpus, "error": None}
|
||||
|
||||
|
||||
# --- Uploads ----------------------------------------------------------------
|
||||
|
||||
|
||||
@app.post("/api/upload-image")
|
||||
async def upload_image(file: Annotated[UploadFile, File()]) -> dict[str, str]:
|
||||
name = file.filename or "image.png"
|
||||
return {"path": f"/mock/uploads/{uuid.uuid4().hex}_{os.path.basename(name)}"}
|
||||
|
||||
|
||||
@app.post("/api/upload-raw-dataset")
|
||||
async def upload_raw_dataset(files: Annotated[list[UploadFile], File()]) -> dict[str, Any]:
|
||||
video_exts = {".mp4", ".webm", ".avi", ".mov", ".mkv"}
|
||||
file_names: list[str] = []
|
||||
for uf in files:
|
||||
name = os.path.basename(uf.filename or f"{uuid.uuid4().hex}.mp4")
|
||||
if os.path.splitext(name)[1].lower() in video_exts:
|
||||
file_names.append(name)
|
||||
if not file_names:
|
||||
raise HTTPException(status_code=400, detail="No video files found.")
|
||||
upload_id = uuid.uuid4().hex
|
||||
path = f"/mock/uploads/{upload_id}"
|
||||
return {"path": path, "upload_id": upload_id, "file_names": file_names}
|
||||
|
||||
|
||||
# --- Jobs -------------------------------------------------------------------
|
||||
|
||||
|
||||
@app.get("/api/jobs")
|
||||
def list_jobs(job_type: str | None = None) -> list[dict[str, Any]]:
|
||||
# _public_job mutates the shared job dict via _advance_job, so it must run
|
||||
# under the lock (matching every other job route) to avoid racing start/stop.
|
||||
with _state_lock:
|
||||
jobs = list(_jobs.values())
|
||||
if job_type:
|
||||
jobs = [j for j in jobs if j.get("job_type") == job_type]
|
||||
jobs.sort(key=lambda j: j["created_at"], reverse=True)
|
||||
return [_public_job(j) for j in jobs]
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}")
|
||||
def get_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
@app.post("/api/jobs", status_code=201)
|
||||
def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
job = _new_job_dict(req)
|
||||
with _state_lock:
|
||||
_jobs[job["id"]] = job
|
||||
if _settings.get("autoStartJob"):
|
||||
_start(job)
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
def _start(job: dict[str, Any]) -> None:
|
||||
job["status"] = "running"
|
||||
job["started_at"] = time.time()
|
||||
job["finished_at"] = None
|
||||
job["error"] = None
|
||||
job["output_path"] = None
|
||||
# The real server assigns the log path once the job starts running; mirror
|
||||
# that so the Job Details "Download Log" button (gated on log_file_path) works.
|
||||
job["log_file_path"] = f"/mock/logs/{job['id']}.log"
|
||||
job["progress"] = 0.0
|
||||
job["progress_msg"] = ""
|
||||
job["phase"] = "starting"
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/start")
|
||||
def start_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
if job["status"] == "running":
|
||||
raise HTTPException(status_code=409, detail="Job is already running")
|
||||
if job["status"] == "completed":
|
||||
raise HTTPException(status_code=409, detail="Job already completed. Delete and re-create to run again.")
|
||||
_start(job)
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/stop")
|
||||
def stop_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
if job["status"] != "running":
|
||||
raise HTTPException(status_code=409, detail=f"Job is not running (status={job['status']})")
|
||||
job["status"] = "stopped"
|
||||
job["finished_at"] = time.time()
|
||||
job["phase"] = "stopped"
|
||||
return dict(job)
|
||||
|
||||
|
||||
@app.delete("/api/jobs/{job_id}")
|
||||
def delete_job(job_id: str) -> dict[str, str]:
|
||||
with _state_lock:
|
||||
if _jobs.pop(job_id, None) is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
return {"detail": f"Job {job_id} deleted"}
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/logs")
|
||||
def get_job_logs(job_id: str, after: int = 0) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
result = _compute_logs(job)
|
||||
all_lines = result["lines"]
|
||||
return {
|
||||
"lines": all_lines[after:],
|
||||
"total": len(all_lines),
|
||||
"progress": result["progress"],
|
||||
"progress_msg": result["progress_msg"],
|
||||
"phase": result["phase"],
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/video")
|
||||
def get_video(job_id: str) -> FileResponse:
|
||||
with _state_lock:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_advance_job(job)
|
||||
output_path = job.get("output_path")
|
||||
if job["status"] != "completed" or not output_path:
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
# Mirror the real server: image workloads (t2i) output a .png served as an
|
||||
# image; everything else is a video.
|
||||
if output_path.endswith(".png"):
|
||||
return FileResponse(_build_mock_media("png"), media_type="image/png", filename=f"job_{job_id}.png")
|
||||
return FileResponse(_build_mock_media("mp4"), media_type="video/mp4", filename=f"job_{job_id}.mp4")
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/download_log")
|
||||
def download_log(job_id: str) -> PlainTextResponse:
|
||||
with _state_lock:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_advance_job(job)
|
||||
lines = _compute_logs(job)["lines"]
|
||||
return PlainTextResponse("\n".join(lines) + "\n", media_type="text/plain")
|
||||
|
||||
|
||||
# --- Datasets ---------------------------------------------------------------
|
||||
|
||||
|
||||
@app.get("/api/datasets")
|
||||
def list_datasets() -> list[dict[str, Any]]:
|
||||
with _state_lock:
|
||||
datasets = sorted(_datasets.values(), key=lambda d: d["created_at"], reverse=True)
|
||||
return [_public_dataset(d) for d in datasets]
|
||||
|
||||
|
||||
@app.get("/api/datasets/{dataset_id}")
|
||||
def get_dataset(dataset_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
dataset = _datasets.get(dataset_id)
|
||||
if dataset is None:
|
||||
raise HTTPException(status_code=404, detail="Dataset not found")
|
||||
return _public_dataset(dataset)
|
||||
|
||||
|
||||
@app.post("/api/datasets", status_code=201)
|
||||
def create_dataset(req: CreateDatasetRequest) -> dict[str, Any]:
|
||||
if not req.upload_path:
|
||||
raise HTTPException(status_code=400, detail="upload_path is required. Upload media files first.")
|
||||
if not req.file_names:
|
||||
raise HTTPException(status_code=400, detail="No media files found.")
|
||||
dataset_id = str(uuid.uuid4())
|
||||
dataset = {"id": dataset_id, "name": req.name, "created_at": time.time()}
|
||||
captions = {fn: (req.captions.get(fn, "") if req.captions else "") for fn in req.file_names}
|
||||
with _state_lock:
|
||||
_datasets[dataset_id] = dataset
|
||||
_dataset_files[dataset_id] = {"file_names": list(req.file_names), "captions": captions}
|
||||
return _public_dataset(dataset)
|
||||
|
||||
|
||||
@app.get("/api/datasets/{dataset_id}/files")
|
||||
def get_dataset_files(dataset_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
if dataset_id not in _datasets:
|
||||
raise HTTPException(status_code=404, detail="Dataset not found")
|
||||
files = _dataset_files.get(dataset_id, {"file_names": [], "captions": {}})
|
||||
return {"file_names": list(files["file_names"]), "captions": dict(files["captions"])}
|
||||
|
||||
|
||||
@app.put("/api/datasets/{dataset_id}/captions")
|
||||
def update_dataset_caption(dataset_id: str, req: UpdateCaptionRequest) -> dict[str, str]:
|
||||
with _state_lock:
|
||||
if dataset_id not in _datasets:
|
||||
raise HTTPException(status_code=404, detail="Dataset not found")
|
||||
files = _dataset_files.setdefault(dataset_id, {"file_names": [], "captions": {}})
|
||||
files["captions"][req.file_name] = req.caption
|
||||
return {"detail": "Caption updated"}
|
||||
|
||||
|
||||
@app.get("/api/datasets/{dataset_id}/media/{file_name:path}")
|
||||
def serve_dataset_media(dataset_id: str, file_name: str) -> FileResponse:
|
||||
with _state_lock:
|
||||
if dataset_id not in _datasets:
|
||||
raise HTTPException(status_code=404, detail="Dataset not found")
|
||||
if file_name not in _dataset_files.get(dataset_id, {}).get("file_names", []):
|
||||
raise HTTPException(status_code=404, detail="File not found")
|
||||
return FileResponse(_build_mock_media("mp4"), media_type="video/mp4")
|
||||
|
||||
|
||||
@app.delete("/api/datasets/{dataset_id}")
|
||||
def delete_dataset(dataset_id: str) -> dict[str, str]:
|
||||
with _state_lock:
|
||||
if _datasets.pop(dataset_id, None) is None:
|
||||
raise HTTPException(status_code=404, detail="Dataset not found")
|
||||
_dataset_files.pop(dataset_id, None)
|
||||
return {"detail": f"Dataset {dataset_id} deleted"}
|
||||
|
||||
|
||||
def main() -> None:
|
||||
import argparse
|
||||
|
||||
import uvicorn
|
||||
|
||||
parser = argparse.ArgumentParser(description="FastVideo Studio mock API server")
|
||||
parser.add_argument("--host", default="127.0.0.1", help="Bind address (default: 127.0.0.1)")
|
||||
# Default off the real server's 8189 so the mock never shadows a real API.
|
||||
parser.add_argument("--port", type=int, default=8190, help="Port number (default: 8190)")
|
||||
args = parser.parse_args()
|
||||
|
||||
uvicorn.run(app, host=args.host, port=args.port, log_level="info")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -1,14 +1,22 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Pydantic request/response models for the API."""
|
||||
"""Pydantic request/response models for the API, plus tiny shared helpers
|
||||
usable by both the real server and the dependency-light mock server."""
|
||||
|
||||
from fastvideo_studio.models.create_job_request import CreateJobRequest
|
||||
from fastvideo_studio.models.settings_update import SettingsUpdate
|
||||
from fastvideo_studio.models.create_dataset_request import CreateDatasetRequest
|
||||
from fastvideo_studio.models.update_caption_request import UpdateCaptionRequest
|
||||
|
||||
|
||||
def model_label(model_path: str) -> str:
|
||||
"""Derive a readable label from an HF-style model path."""
|
||||
return model_path.split("/")[-1].replace("-", " ").replace("_", " ")
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CreateJobRequest",
|
||||
"SettingsUpdate",
|
||||
"CreateDatasetRequest",
|
||||
"UpdateCaptionRequest",
|
||||
"model_label",
|
||||
]
|
||||
|
||||
@@ -17,7 +17,6 @@ class CreateJobRequest(BaseModel):
|
||||
num_latent_t: int = 20
|
||||
validation_dataset_file: str = ""
|
||||
lora_rank: int = 32
|
||||
ltx2_first_frame_conditioning_p: float | None = None
|
||||
negative_prompt: str = ""
|
||||
num_inference_steps: int = 50
|
||||
num_frames: int = 81
|
||||
@@ -41,8 +40,6 @@ class CreateJobRequest(BaseModel):
|
||||
dmd_use_vsa: bool = False
|
||||
dmd_vsa_sparsity: float = 0.8
|
||||
dmd_denoising_steps: str = "1000,757,522"
|
||||
min_timestep_ratio: float = 0.02
|
||||
max_timestep_ratio: float = 0.98
|
||||
real_score_guidance_scale: float = 3.5
|
||||
generator_update_interval: int = 5
|
||||
real_score_model_path: str = ""
|
||||
|
||||
@@ -0,0 +1,13 @@
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import type { NextConfig } from 'next';
|
||||
|
||||
const configDir = path.dirname(fileURLToPath(import.meta.url));
|
||||
|
||||
const nextConfig: NextConfig = {
|
||||
// Point tracing at the monorepo root (apps/fastvideo_studio -> apps -> repo
|
||||
// root) so Next stops warning about multiple lockfiles in the workspace.
|
||||
outputFileTracingRoot: path.join(configDir, '..', '..'),
|
||||
};
|
||||
|
||||
export default nextConfig;
|
||||
Generated
+4618
-1038
File diff suppressed because it is too large
Load Diff
@@ -4,33 +4,54 @@
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite dev",
|
||||
"build": "vite build",
|
||||
"preview": "vite preview",
|
||||
"start": "concurrently --kill-others-on-fail \"npm:start:api\" \"npm:start:web\"",
|
||||
"dev": "next dev --port 3000",
|
||||
"build": "next build",
|
||||
"start": "next start --port 3000",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"test": "vitest run",
|
||||
"test:watch": "vitest",
|
||||
"test:coverage": "vitest run --coverage",
|
||||
"e2e": "playwright test",
|
||||
"start:api": "cd .. && python -m fastvideo_studio.server",
|
||||
"start:web": "node build",
|
||||
"lint": "eslint ."
|
||||
"start:web": "next start --port 3000",
|
||||
"start:all": "concurrently --kill-others-on-fail \"npm:start:api\" \"npm:start:web\""
|
||||
},
|
||||
"dependencies": {
|
||||
"@sveltejs/kit": "2.57.1",
|
||||
"svelte": "5.55.7"
|
||||
"@radix-ui/react-dialog": "^1.1.0",
|
||||
"@radix-ui/react-label": "^2.1.8",
|
||||
"@radix-ui/react-scroll-area": "^1.2.10",
|
||||
"@radix-ui/react-select": "^2.2.6",
|
||||
"@radix-ui/react-separator": "^1.1.8",
|
||||
"@radix-ui/react-slider": "^1.2.0",
|
||||
"@radix-ui/react-slot": "^1.2.4",
|
||||
"@radix-ui/react-switch": "^1.1.0",
|
||||
"@radix-ui/react-tabs": "^1.1.0",
|
||||
"class-variance-authority": "^0.7.1",
|
||||
"clsx": "^2.1.1",
|
||||
"lucide-react": "^0.577.0",
|
||||
"next": "15.5.18",
|
||||
"react": "^19.1.0",
|
||||
"react-dom": "^19.1.0",
|
||||
"sonner": "^2.0.7",
|
||||
"tailwind-merge": "^3.5.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@sveltejs/adapter-node": "^5.5.4",
|
||||
"@sveltejs/vite-plugin-svelte": "^5.0.0",
|
||||
"@types/node": "^20",
|
||||
"@playwright/test": "^1.59.1",
|
||||
"@tailwindcss/postcss": "^4.2.1",
|
||||
"@testing-library/dom": "^10.4.1",
|
||||
"@testing-library/jest-dom": "^6.9.1",
|
||||
"@testing-library/react": "^16.3.0",
|
||||
"@testing-library/user-event": "^14.6.1",
|
||||
"@types/node": "^22",
|
||||
"@types/react": "^19.1.0",
|
||||
"@types/react-dom": "^19.1.0",
|
||||
"@vitejs/plugin-react": "^4.5.2",
|
||||
"@vitest/coverage-v8": "^3.2.4",
|
||||
"concurrently": "^9",
|
||||
"eslint": "^9",
|
||||
"typescript": "^5",
|
||||
"vite": "6.4.2"
|
||||
},
|
||||
"overrides": {
|
||||
"cookie": "0.7.0",
|
||||
"devalue": "5.8.1",
|
||||
"flatted": "3.4.2",
|
||||
"minimatch": "3.1.4",
|
||||
"picomatch": "4.0.4",
|
||||
"postcss": "8.5.10"
|
||||
"jsdom": "^26.1.0",
|
||||
"postcss": "8.5.10",
|
||||
"tailwindcss": "^4.2.1",
|
||||
"typescript": "^5.8.3",
|
||||
"vitest": "^3.2.4"
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import { defineConfig, devices } from '@playwright/test';
|
||||
|
||||
import { API_BASE, MOCK_API_PORT } from './e2e/helpers';
|
||||
|
||||
/**
|
||||
* Playwright config for FastVideo Studio end-to-end tests.
|
||||
*
|
||||
* The Next.js frontend runs on port 3000 and talks to the in-memory mock API
|
||||
* (fastvideo_studio.mock_server) on port 8189. The webServer block boots both
|
||||
* the mock backend and `npm run dev` (pointed at the mock via
|
||||
* NEXT_PUBLIC_API_BASE_URL) if nothing is already listening, so the suite works
|
||||
* both locally and in CI. Set PLAYWRIGHT_SKIP_WEBSERVER to reuse externally
|
||||
* managed servers.
|
||||
*/
|
||||
export default defineConfig({
|
||||
testDir: './e2e',
|
||||
fullyParallel: false,
|
||||
workers: 1,
|
||||
forbidOnly: !!process.env.CI,
|
||||
retries: process.env.CI ? 2 : 0,
|
||||
reporter: process.env.CI ? 'github' : 'list',
|
||||
timeout: 120_000,
|
||||
expect: { timeout: 30_000 },
|
||||
use: {
|
||||
baseURL: process.env.PLAYWRIGHT_BASE_URL ?? 'http://127.0.0.1:3000',
|
||||
headless: true,
|
||||
viewport: { width: 1280, height: 720 },
|
||||
screenshot: 'only-on-failure',
|
||||
trace: 'retain-on-failure',
|
||||
},
|
||||
projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],
|
||||
webServer: process.env.PLAYWRIGHT_SKIP_WEBSERVER
|
||||
? undefined
|
||||
: [
|
||||
{
|
||||
command: `PYTHONPATH=.. python -m fastvideo_studio.mock_server --port ${MOCK_API_PORT}`,
|
||||
// Probe the mock-only sentinel, not /api/models (which a real server
|
||||
// also serves), so a real backend on this port is never mistaken for
|
||||
// the mock. Always boot a fresh mock in CI.
|
||||
url: `http://127.0.0.1:${MOCK_API_PORT}/api/__mock__`,
|
||||
reuseExistingServer: !process.env.CI,
|
||||
timeout: 120_000,
|
||||
},
|
||||
{
|
||||
command: 'npm run dev',
|
||||
url: 'http://127.0.0.1:3000',
|
||||
reuseExistingServer: true,
|
||||
timeout: 120_000,
|
||||
env: {
|
||||
NEXT_PUBLIC_API_BASE_URL: API_BASE,
|
||||
},
|
||||
},
|
||||
],
|
||||
});
|
||||
@@ -0,0 +1,5 @@
|
||||
export default {
|
||||
plugins: {
|
||||
'@tailwindcss/postcss': {},
|
||||
},
|
||||
};
|
||||
@@ -1,142 +0,0 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""
|
||||
Runs preprocessing subprocess for datasets.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from collections.abc import Callable
|
||||
|
||||
logger = logging.getLogger("fastvideo.studio.preprocess_runner")
|
||||
|
||||
|
||||
def build_preprocess_args(
|
||||
dataset_id: str,
|
||||
raw_path: str,
|
||||
output_dir: str,
|
||||
workload_type: str,
|
||||
model_path: str,
|
||||
dataset_type: str = "merged",
|
||||
num_gpus: int = 1,
|
||||
) -> list[str]:
|
||||
"""Build CLI args for v1_preprocessing_new."""
|
||||
return [
|
||||
"--model-path",
|
||||
model_path,
|
||||
"--mode",
|
||||
"preprocess",
|
||||
"--workload-type",
|
||||
workload_type,
|
||||
"--preprocess.video_loader_type",
|
||||
"torchvision",
|
||||
"--preprocess.dataset_type",
|
||||
dataset_type,
|
||||
"--preprocess.dataset_path",
|
||||
raw_path,
|
||||
"--preprocess.dataset_output_dir",
|
||||
output_dir,
|
||||
"--preprocess.preprocess_video_batch_size",
|
||||
"2",
|
||||
"--preprocess.dataloader_num_workers",
|
||||
"0",
|
||||
"--preprocess.max_height",
|
||||
"480",
|
||||
"--preprocess.max_width",
|
||||
"832",
|
||||
"--preprocess.num_frames",
|
||||
"77",
|
||||
"--preprocess.train_fps",
|
||||
"16",
|
||||
"--preprocess.samples_per_file",
|
||||
"8",
|
||||
"--preprocess.flush_frequency",
|
||||
"8",
|
||||
"--preprocess.video_length_tolerance_range",
|
||||
"5",
|
||||
]
|
||||
|
||||
|
||||
def run_preprocess(
|
||||
dataset_id: str,
|
||||
raw_path: str,
|
||||
output_dir: str,
|
||||
workload_type: str,
|
||||
model_path: str,
|
||||
dataset_type: str,
|
||||
num_gpus: int,
|
||||
log_file_path: str,
|
||||
on_status_change: Callable[[str, str | None], None],
|
||||
stop_event: threading.Event,
|
||||
) -> None:
|
||||
"""Run preprocessing subprocess. Calls on_status_change(status, error)."""
|
||||
repo_root = Path(__file__).resolve().parent.parent
|
||||
preprocess_module = "fastvideo.pipelines.preprocess.v1_preprocessing_new"
|
||||
|
||||
args = build_preprocess_args(
|
||||
dataset_id=dataset_id,
|
||||
raw_path=raw_path,
|
||||
output_dir=output_dir,
|
||||
workload_type=workload_type,
|
||||
model_path=model_path,
|
||||
dataset_type=dataset_type,
|
||||
num_gpus=num_gpus,
|
||||
)
|
||||
|
||||
cmd = [
|
||||
sys.executable,
|
||||
"-m",
|
||||
"torch.distributed.run",
|
||||
"--nproc_per_node",
|
||||
str(num_gpus),
|
||||
"--nnodes",
|
||||
"1",
|
||||
"-m",
|
||||
preprocess_module,
|
||||
] + args
|
||||
|
||||
env = os.environ.copy()
|
||||
env["TOKENIZERS_PARALLELISM"] = "false"
|
||||
|
||||
try:
|
||||
on_status_change("preprocessing", None)
|
||||
with open(log_file_path, "w", encoding="utf-8") as log_file:
|
||||
proc = subprocess.Popen(
|
||||
cmd,
|
||||
cwd=str(repo_root),
|
||||
env=env,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.STDOUT,
|
||||
text=True,
|
||||
bufsize=1,
|
||||
)
|
||||
assert proc.stdout is not None
|
||||
for line in iter(proc.stdout.readline, ""):
|
||||
if stop_event.is_set():
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=30)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
on_status_change("stopped", "Preprocessing stopped by user")
|
||||
return
|
||||
line = line.rstrip()
|
||||
if line:
|
||||
log_file.write(line + "\n")
|
||||
log_file.flush()
|
||||
|
||||
proc.wait()
|
||||
exit_code = proc.returncode or 0
|
||||
|
||||
if exit_code == 0:
|
||||
on_status_change("ready", None)
|
||||
else:
|
||||
on_status_change("failed", f"Preprocessing exited with code {exit_code}")
|
||||
except Exception as exc:
|
||||
on_status_change("failed", f"{type(exc).__name__}: {exc}")
|
||||
logger.exception("Preprocessing failed for dataset %s", dataset_id)
|
||||
@@ -32,8 +32,10 @@ from fastapi.responses import FileResponse
|
||||
|
||||
from fastvideo.registry import (get_registered_model_paths, get_registered_models_with_workloads)
|
||||
from fastvideo_studio.database import Database, _get_db_path
|
||||
from fastvideo_studio.gpu import get_gpu_snapshot
|
||||
from fastvideo_studio.job_runner import JobRunner, JobStatus
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, SettingsUpdate, UpdateCaptionRequest)
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, SettingsUpdate, UpdateCaptionRequest,
|
||||
model_label)
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
@@ -43,15 +45,9 @@ logger = logging.getLogger("fastvideo.studio.api")
|
||||
|
||||
DEFAULT_OUTPUT_DIR = os.path.join(os.path.dirname(__file__), "..", "outputs", "ui_jobs")
|
||||
|
||||
|
||||
def _get_model_label(model_path: str) -> str:
|
||||
"""Derive a readable label from a HF model path."""
|
||||
return model_path.split("/")[-1].replace("-", " ").replace("_", " ")
|
||||
|
||||
|
||||
_available_models: list[dict[str, str]] = [{
|
||||
"id": path,
|
||||
"label": _get_model_label(path)
|
||||
"label": model_label(path)
|
||||
} for path in get_registered_model_paths()]
|
||||
|
||||
job_runner: JobRunner
|
||||
@@ -100,6 +96,12 @@ def update_settings(settings: SettingsUpdate) -> dict[str, Any]:
|
||||
return database.get_settings()
|
||||
|
||||
|
||||
@app.get("/api/gpus")
|
||||
def list_gpus() -> dict[str, Any]:
|
||||
"""Return an NVML snapshot of every GPU on the API server host."""
|
||||
return get_gpu_snapshot()
|
||||
|
||||
|
||||
@app.get("/api/models")
|
||||
def list_models(workload_type: str | None = None) -> list[dict[str, Any]]:
|
||||
"""Return the catalogue of available video-generation models.
|
||||
@@ -148,7 +150,7 @@ async def upload_image(file: Annotated[UploadFile, File()], ) -> dict[str, str]:
|
||||
return {"path": os.path.abspath(dest_path)}
|
||||
|
||||
|
||||
ALLOWED_VIDEO_EXTENSIONS = {".mp4", ".webm", ".avi", ".mov"}
|
||||
ALLOWED_VIDEO_EXTENSIONS = {".mp4", ".webm", ".avi", ".mov", ".mkv"}
|
||||
|
||||
|
||||
def _filter_video_files(files: list[UploadFile]) -> list[UploadFile]:
|
||||
@@ -156,6 +158,23 @@ def _filter_video_files(files: list[UploadFile]) -> list[UploadFile]:
|
||||
return [f for f in files if Path(f.filename or "").suffix.lower() in ALLOWED_VIDEO_EXTENSIONS]
|
||||
|
||||
|
||||
def _path_is_within(child: str, parent: str) -> bool:
|
||||
"""True if ``child`` resolves to a location inside ``parent``."""
|
||||
try:
|
||||
parent_real = os.path.realpath(parent)
|
||||
return os.path.commonpath([os.path.realpath(child), parent_real]) == parent_real
|
||||
except (ValueError, OSError):
|
||||
return False
|
||||
|
||||
|
||||
def _staging_base_path() -> str:
|
||||
"""Root under which raw uploads are staged (settings override or default)."""
|
||||
settings = database.get_settings() if database is not None else {}
|
||||
raw_path = (settings.get("datasetUploadPath") or settings.get("dataset_upload_path") or "")
|
||||
base_path = (raw_path.strip() if raw_path and isinstance(raw_path, str) else "")
|
||||
return datasets_upload_dir if not base_path else os.path.abspath(base_path)
|
||||
|
||||
|
||||
@app.post("/api/upload-raw-dataset")
|
||||
async def upload_raw_dataset(files: Annotated[list[UploadFile], File()], ) -> dict[str, Any]:
|
||||
"""
|
||||
@@ -167,10 +186,7 @@ async def upload_raw_dataset(files: Annotated[list[UploadFile], File()], ) -> di
|
||||
status_code=503,
|
||||
detail="Database not initialized",
|
||||
)
|
||||
settings = database.get_settings()
|
||||
raw_path = (settings.get("datasetUploadPath") or settings.get("dataset_upload_path") or "")
|
||||
base_path = (raw_path.strip() if raw_path and isinstance(raw_path, str) else "")
|
||||
base_path = (datasets_upload_dir if not base_path else os.path.abspath(base_path))
|
||||
base_path = _staging_base_path()
|
||||
if not base_path:
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
@@ -250,6 +266,19 @@ def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
f"Valid options: {sorted(valid_ids)}"),
|
||||
)
|
||||
|
||||
# Training jobs reference a dataset by id; resolve it to the on-disk media
|
||||
# directory the trainer reads (the UI has no free-text path field). Falls
|
||||
# through unchanged for inference and for anything already a real path.
|
||||
def _resolve_dataset_path(value: str) -> str:
|
||||
media_dir = _dataset_media_dir(value) if value else ""
|
||||
return media_dir if media_dir and os.path.isdir(media_dir) else value
|
||||
|
||||
data_path = req.data_path or ""
|
||||
validation_dataset_file = req.validation_dataset_file or ""
|
||||
if job_type != "inference":
|
||||
data_path = _resolve_dataset_path(data_path)
|
||||
validation_dataset_file = _resolve_dataset_path(validation_dataset_file)
|
||||
|
||||
job = job_runner.create_job(
|
||||
job_id=str(uuid.uuid4()),
|
||||
model_id=req.model_id,
|
||||
@@ -257,14 +286,13 @@ def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
workload_type=req.workload_type or "t2v",
|
||||
job_type=job_type,
|
||||
image_path=req.image_path or "",
|
||||
data_path=req.data_path or "",
|
||||
data_path=data_path,
|
||||
max_train_steps=req.max_train_steps,
|
||||
train_batch_size=req.train_batch_size,
|
||||
learning_rate=req.learning_rate,
|
||||
num_latent_t=req.num_latent_t,
|
||||
validation_dataset_file=req.validation_dataset_file or "",
|
||||
validation_dataset_file=validation_dataset_file,
|
||||
lora_rank=req.lora_rank,
|
||||
ltx2_first_frame_conditioning_p=req.ltx2_first_frame_conditioning_p,
|
||||
negative_prompt=req.negative_prompt,
|
||||
num_inference_steps=req.num_inference_steps,
|
||||
num_frames=req.num_frames,
|
||||
@@ -287,8 +315,6 @@ def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
dmd_use_vsa=req.dmd_use_vsa,
|
||||
dmd_vsa_sparsity=req.dmd_vsa_sparsity,
|
||||
dmd_denoising_steps=req.dmd_denoising_steps or "1000,757,522",
|
||||
min_timestep_ratio=req.min_timestep_ratio,
|
||||
max_timestep_ratio=req.max_timestep_ratio,
|
||||
real_score_guidance_scale=req.real_score_guidance_scale,
|
||||
generator_update_interval=req.generator_update_interval,
|
||||
real_score_model_path=req.real_score_model_path or "",
|
||||
@@ -418,6 +444,13 @@ def create_dataset(req: CreateDatasetRequest) -> dict[str, Any]:
|
||||
status_code=400,
|
||||
detail="No media files found. Ensure at least one image or video.",
|
||||
)
|
||||
# upload_path must be a staging dir produced by /api/upload-raw-dataset,
|
||||
# not an arbitrary client path — the code below copies then rmtrees it.
|
||||
if not _path_is_within(req.upload_path, _staging_base_path()):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail="upload_path must be a staged upload directory.",
|
||||
)
|
||||
dataset_id = str(uuid.uuid4())
|
||||
created_at = time.time()
|
||||
dest_dir = os.path.join(datasets_upload_dir, dataset_id)
|
||||
@@ -491,8 +524,9 @@ def serve_dataset_media(dataset_id: str, file_name: str) -> FileResponse:
|
||||
ds = database.get_dataset(dataset_id)
|
||||
if ds is None:
|
||||
raise HTTPException(status_code=404, detail="Dataset not found")
|
||||
media_path = os.path.join(_dataset_media_dir(dataset_id), file_name)
|
||||
if not os.path.isfile(media_path):
|
||||
media_dir = _dataset_media_dir(dataset_id)
|
||||
media_path = os.path.realpath(os.path.join(media_dir, file_name))
|
||||
if not (_path_is_within(media_path, media_dir) and os.path.isfile(media_path)):
|
||||
raise HTTPException(status_code=404, detail="File not found")
|
||||
import mimetypes
|
||||
mime, _ = mimetypes.guess_type(media_path)
|
||||
|
||||
@@ -1,66 +0,0 @@
|
||||
/* CSS Variables */
|
||||
:root {
|
||||
--header-height: 74px;
|
||||
--bg: #0f1117;
|
||||
--surface: #1a1d27;
|
||||
--border: #2a2d3a;
|
||||
--text: #e4e4e7;
|
||||
--text-dim: #9ca3af;
|
||||
--accent: #356cff;
|
||||
--accent-h: #818cf8;
|
||||
--green: #22c55e;
|
||||
--red: #ef4444;
|
||||
--yellow: #eab308;
|
||||
--blue: #3b82f6;
|
||||
--radius: 8px;
|
||||
--shadow: 0 2px 8px rgba(0, 0, 0, 0.35);
|
||||
}
|
||||
|
||||
/* Reset */
|
||||
*,
|
||||
*::before,
|
||||
*::after {
|
||||
box-sizing: border-box;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
/* Base body styles */
|
||||
body {
|
||||
font-family:
|
||||
"Inter",
|
||||
system-ui,
|
||||
-apple-system,
|
||||
sans-serif;
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
line-height: 1.6;
|
||||
min-height: 100vh;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
/* Base form input styles (used globally) */
|
||||
select,
|
||||
input,
|
||||
textarea {
|
||||
background: var(--bg);
|
||||
color: var(--text);
|
||||
border: 1px solid var(--border);
|
||||
border-radius: var(--radius);
|
||||
padding: 0.55rem 0.75rem;
|
||||
font-size: 0.9rem;
|
||||
font-family: inherit;
|
||||
outline: none;
|
||||
transition: border-color 0.15s;
|
||||
}
|
||||
|
||||
select:focus,
|
||||
input:focus,
|
||||
textarea:focus {
|
||||
border-color: var(--accent);
|
||||
}
|
||||
|
||||
textarea {
|
||||
resize: vertical;
|
||||
}
|
||||
@@ -1,13 +0,0 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<link rel="icon" href="%sveltekit.assets%/fastvideo.ico" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>FastVideo</title>
|
||||
%sveltekit.head%
|
||||
</head>
|
||||
<body class="antialiased" data-sveltekit-preload-data="hover">
|
||||
<div style="display: contents">%sveltekit.body%</div>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,85 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import AddDatasetButton from '@/components/datasets/AddDatasetButton';
|
||||
import CreateDatasetModal from '@/components/datasets/CreateDatasetModal';
|
||||
import DatasetCard from '@/components/datasets/DatasetCard';
|
||||
import { HeaderActions } from '@/components/shell/HeaderActionsContext';
|
||||
import { Card } from '@/components/ui/card';
|
||||
import { useStore } from '@/hooks/useStore';
|
||||
import { getDatasets } from '@/lib/api';
|
||||
import type { Dataset } from '@/lib/api';
|
||||
import {
|
||||
setActiveDataset,
|
||||
setActiveDatasetId,
|
||||
} from '@/stores/activeDataset';
|
||||
import {
|
||||
createDatasetModalStore,
|
||||
setCreateDatasetModalOpen,
|
||||
} from '@/stores/createDatasetModalOpen';
|
||||
|
||||
export default function DatasetsPage() {
|
||||
const [datasets, setDatasets] = React.useState<Dataset[]>([]);
|
||||
const [error, setError] = React.useState<string | null>(null);
|
||||
const { open } = useStore(createDatasetModalStore);
|
||||
|
||||
const fetchDatasets = React.useCallback(async () => {
|
||||
try {
|
||||
setDatasets(await getDatasets());
|
||||
setError(null);
|
||||
} catch (err) {
|
||||
console.error('Failed to fetch datasets:', err);
|
||||
// Distinguish an API outage from a genuinely empty list, so the user
|
||||
// isn't told they have no datasets when the server is unreachable.
|
||||
setError(err instanceof Error ? err.message : 'Failed to load datasets');
|
||||
}
|
||||
}, []);
|
||||
|
||||
React.useEffect(() => {
|
||||
fetchDatasets();
|
||||
}, [fetchDatasets]);
|
||||
|
||||
function handleSelectDataset(ds: Dataset) {
|
||||
setActiveDataset(ds);
|
||||
setActiveDatasetId(ds.id);
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<HeaderActions>
|
||||
<AddDatasetButton />
|
||||
</HeaderActions>
|
||||
<main className="mx-auto flex w-full max-w-[850px] flex-col gap-6 px-4 pb-12">
|
||||
<Card className="p-6">
|
||||
<div>
|
||||
{error ? (
|
||||
<p className="py-8 text-center text-destructive">{error}</p>
|
||||
) : datasets.length === 0 ? (
|
||||
<p className="py-8 text-center text-muted-foreground">
|
||||
No datasets yet.
|
||||
</p>
|
||||
) : (
|
||||
datasets.map((ds) => (
|
||||
<DatasetCard
|
||||
key={ds.id}
|
||||
dataset={ds}
|
||||
onUpdated={fetchDatasets}
|
||||
onSelect={() => handleSelectDataset(ds)}
|
||||
/>
|
||||
))
|
||||
)}
|
||||
</div>
|
||||
</Card>
|
||||
</main>
|
||||
<CreateDatasetModal
|
||||
isOpen={open}
|
||||
onClose={() => setCreateDatasetModalOpen(false)}
|
||||
onSuccess={() => {
|
||||
fetchDatasets();
|
||||
setCreateDatasetModalOpen(false);
|
||||
}}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
'use client';
|
||||
|
||||
import CreateJobButton from '@/components/jobs/CreateJobButton';
|
||||
import { HeaderActions } from '@/components/shell/HeaderActionsContext';
|
||||
import JobQueue from '@/components/jobs/JobQueue';
|
||||
|
||||
export default function DistillationPage() {
|
||||
return (
|
||||
<>
|
||||
<HeaderActions>
|
||||
<CreateJobButton jobType="distillation" />
|
||||
</HeaderActions>
|
||||
<JobQueue jobType="distillation" />
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
'use client';
|
||||
|
||||
import CreateJobButton from '@/components/jobs/CreateJobButton';
|
||||
import { HeaderActions } from '@/components/shell/HeaderActionsContext';
|
||||
import JobQueue from '@/components/jobs/JobQueue';
|
||||
import type { JobType } from '@/lib/types';
|
||||
|
||||
// 'lora' is a backend job_type (LoRA finetunes) that isn't part of the
|
||||
// JobType union; the finetuning queue lists both alongside full finetunes.
|
||||
const FINETUNING_LIST = ['finetuning', 'lora'] as JobType[];
|
||||
|
||||
export default function FinetuningPage() {
|
||||
return (
|
||||
<>
|
||||
<HeaderActions>
|
||||
<CreateJobButton jobType="finetuning" />
|
||||
</HeaderActions>
|
||||
<JobQueue jobType="finetuning" jobTypesForList={FINETUNING_LIST} />
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import GalleryPage from './page';
|
||||
import { HeaderActionsProvider } from '@/components/shell/HeaderActionsContext';
|
||||
import { getJobsList } from '@/lib/api';
|
||||
import type { Job } from '@/lib/types';
|
||||
import { makeJob as makeBaseJob } from '@/test/factories';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
getJobsList: vi.fn(),
|
||||
getJobVideoUrl: (id: string) => `http://test.local/api/jobs/${id}/video`,
|
||||
}));
|
||||
|
||||
const makeJob = (overrides: Partial<Job> = {}): Job =>
|
||||
makeBaseJob({
|
||||
model_id: 'wan',
|
||||
prompt: 'a cat surfing a wave',
|
||||
status: 'completed',
|
||||
created_at: 1,
|
||||
finished_at: 2,
|
||||
output_path: '/out/clip.mp4',
|
||||
...overrides,
|
||||
});
|
||||
|
||||
function renderGallery() {
|
||||
return render(
|
||||
<HeaderActionsProvider>
|
||||
<GalleryPage />
|
||||
</HeaderActionsProvider>,
|
||||
);
|
||||
}
|
||||
|
||||
describe('GalleryPage', () => {
|
||||
it('renders a grid item for a completed inference job', async () => {
|
||||
vi.mocked(getJobsList).mockResolvedValue([
|
||||
makeJob({ prompt: 'a cat surfing a wave' }),
|
||||
]);
|
||||
|
||||
renderGallery();
|
||||
|
||||
expect(await screen.findByText('a cat surfing a wave')).toBeInTheDocument();
|
||||
expect(getJobsList).toHaveBeenCalledWith('inference');
|
||||
});
|
||||
|
||||
it('shows the empty state when no completed videos exist', async () => {
|
||||
vi.mocked(getJobsList).mockResolvedValue([
|
||||
makeJob({ status: 'running', output_path: null }),
|
||||
]);
|
||||
|
||||
renderGallery();
|
||||
|
||||
expect(
|
||||
await screen.findByText('No completed videos yet'),
|
||||
).toBeInTheDocument();
|
||||
expect(
|
||||
screen.queryByText('a cat surfing a wave'),
|
||||
).not.toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,112 @@
|
||||
'use client';
|
||||
|
||||
import { Loader2 } from 'lucide-react';
|
||||
import { useEffect, useState } from 'react';
|
||||
|
||||
import { Card } from '@/components/ui/card';
|
||||
import { getJobVideoUrl, getJobsList } from '@/lib/api';
|
||||
import type { Job } from '@/lib/types';
|
||||
|
||||
function isImage(job: Job): boolean {
|
||||
return job.output_path?.toLowerCase().endsWith('.png') ?? false;
|
||||
}
|
||||
|
||||
export default function GalleryPage() {
|
||||
const [jobs, setJobs] = useState<Job[]>([]);
|
||||
const [isLoading, setIsLoading] = useState(true);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false;
|
||||
async function load() {
|
||||
try {
|
||||
const list = await getJobsList('inference');
|
||||
if (cancelled) return;
|
||||
const sorted = [...list].sort(
|
||||
(a, b) =>
|
||||
(b.finished_at ?? b.created_at ?? 0) -
|
||||
(a.finished_at ?? a.created_at ?? 0),
|
||||
);
|
||||
setJobs(sorted);
|
||||
} catch (e) {
|
||||
if (!cancelled) {
|
||||
setError(e instanceof Error ? e.message : 'Failed to load jobs');
|
||||
}
|
||||
} finally {
|
||||
if (!cancelled) setIsLoading(false);
|
||||
}
|
||||
}
|
||||
void load();
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, []);
|
||||
|
||||
const galleryJobs = jobs.filter(
|
||||
(j) =>
|
||||
j.status === 'completed' &&
|
||||
j.output_path &&
|
||||
(j.job_type === 'inference' || !j.job_type),
|
||||
);
|
||||
|
||||
return (
|
||||
<div className="mx-auto w-full max-w-[1200px] px-4 pb-12 pt-4">
|
||||
<Card className="p-6">
|
||||
<h2 className="mb-1 text-2xl font-semibold text-foreground">Gallery</h2>
|
||||
<p className="mb-6 text-sm text-muted-foreground">
|
||||
Generated videos from completed inference jobs. Captions show the
|
||||
prompt used for each generation.
|
||||
</p>
|
||||
|
||||
{isLoading ? (
|
||||
<div className="flex items-center gap-3 p-8 text-muted-foreground">
|
||||
<Loader2 className="h-6 w-6 animate-spin text-primary" />
|
||||
<span>Loading gallery…</span>
|
||||
</div>
|
||||
) : error ? (
|
||||
<p className="py-8 text-destructive">{error}</p>
|
||||
) : galleryJobs.length === 0 ? (
|
||||
<p className="py-8 text-center text-muted-foreground">
|
||||
No completed videos yet
|
||||
</p>
|
||||
) : (
|
||||
<div className="grid grid-cols-[repeat(auto-fill,minmax(280px,1fr))] gap-5">
|
||||
{galleryJobs.map((job) => (
|
||||
<article
|
||||
key={job.id}
|
||||
className="flex flex-col overflow-hidden rounded-lg border border-border bg-background"
|
||||
>
|
||||
<div className="relative aspect-video overflow-hidden bg-muted">
|
||||
{isImage(job) ? (
|
||||
// eslint-disable-next-line @next/next/no-img-element
|
||||
<img
|
||||
src={getJobVideoUrl(job.id)}
|
||||
alt={job.prompt}
|
||||
className="block h-full w-full object-contain"
|
||||
loading="lazy"
|
||||
/>
|
||||
) : (
|
||||
<video
|
||||
src={getJobVideoUrl(job.id)}
|
||||
className="block h-full w-full object-contain"
|
||||
muted
|
||||
loop
|
||||
playsInline
|
||||
preload="metadata"
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
<p
|
||||
className="line-clamp-3 border-t border-border px-4 py-3 text-sm text-muted-foreground"
|
||||
title={job.prompt}
|
||||
>
|
||||
{job.prompt || '—'}
|
||||
</p>
|
||||
</article>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</Card>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,221 @@
|
||||
@import "tailwindcss";
|
||||
|
||||
@custom-variant dark (&:is(.dark *));
|
||||
|
||||
/*
|
||||
* FastVideo Studio theme — shared with the Dreamverse design system:
|
||||
* slate light/dark palettes, #356cff accent blue, IBM Plex type. The studio
|
||||
* defaults to dark (see the theme init script in layout.tsx); the toggle in
|
||||
* the header persists the choice to localStorage.
|
||||
*/
|
||||
:root {
|
||||
color-scheme: light;
|
||||
|
||||
--header-height: 74px;
|
||||
|
||||
--accent-blue: #356cff;
|
||||
|
||||
--background: #f5f4f4;
|
||||
--foreground: #0f172a;
|
||||
|
||||
--card: #ffffff;
|
||||
--card-foreground: #0f172a;
|
||||
|
||||
--popover: #ffffff;
|
||||
--popover-foreground: #0f172a;
|
||||
|
||||
--primary: #0f172a;
|
||||
--primary-foreground: #f8fafc;
|
||||
|
||||
--secondary: #f1f5f9;
|
||||
--secondary-foreground: #1e293b;
|
||||
|
||||
--muted: #f1f5f9;
|
||||
--muted-foreground: #64748b;
|
||||
|
||||
--accent: #e2e8f0;
|
||||
--accent-foreground: #0f172a;
|
||||
|
||||
--destructive: #ef4444;
|
||||
--destructive-foreground: #ffffff;
|
||||
|
||||
--border: #e2e8f0;
|
||||
--input: #cbd5e1;
|
||||
--ring: #94a3b8;
|
||||
|
||||
--radius: 0.5rem;
|
||||
}
|
||||
|
||||
.dark {
|
||||
color-scheme: dark;
|
||||
|
||||
--accent-blue: #356cff;
|
||||
|
||||
--background: #0f172a;
|
||||
--foreground: #e2e8f0;
|
||||
|
||||
--card: #0f172a;
|
||||
--card-foreground: #f1f5f9;
|
||||
|
||||
--popover: #0f172a;
|
||||
--popover-foreground: #f1f5f9;
|
||||
|
||||
--primary: #f1f5f9;
|
||||
--primary-foreground: #0f172a;
|
||||
|
||||
--secondary: #1e293b;
|
||||
--secondary-foreground: #f1f5f9;
|
||||
|
||||
--muted: #1e293b;
|
||||
--muted-foreground: #94a3b8;
|
||||
|
||||
--accent: #1e293b;
|
||||
--accent-foreground: #f1f5f9;
|
||||
|
||||
--destructive: #991b1b;
|
||||
--destructive-foreground: #fecaca;
|
||||
|
||||
--border: #334155;
|
||||
--input: #334155;
|
||||
--ring: #cbd5e1;
|
||||
}
|
||||
|
||||
@theme inline {
|
||||
--font-sans: var(--font-plex-sans), ui-sans-serif, system-ui, sans-serif;
|
||||
--font-mono: var(--font-plex-mono), ui-monospace, monospace;
|
||||
|
||||
--color-accent-blue: var(--accent-blue);
|
||||
|
||||
--color-background: var(--background);
|
||||
--color-foreground: var(--foreground);
|
||||
|
||||
--color-card: var(--card);
|
||||
--color-card-foreground: var(--card-foreground);
|
||||
|
||||
--color-popover: var(--popover);
|
||||
--color-popover-foreground: var(--popover-foreground);
|
||||
|
||||
--color-primary: var(--primary);
|
||||
--color-primary-foreground: var(--primary-foreground);
|
||||
|
||||
--color-secondary: var(--secondary);
|
||||
--color-secondary-foreground: var(--secondary-foreground);
|
||||
|
||||
--color-muted: var(--muted);
|
||||
--color-muted-foreground: var(--muted-foreground);
|
||||
|
||||
--color-accent: var(--accent);
|
||||
--color-accent-foreground: var(--accent-foreground);
|
||||
|
||||
--color-destructive: var(--destructive);
|
||||
--color-destructive-foreground: var(--destructive-foreground);
|
||||
|
||||
--color-border: var(--border);
|
||||
--color-input: var(--input);
|
||||
--color-ring: var(--ring);
|
||||
|
||||
--radius-sm: calc(var(--radius) - 4px);
|
||||
--radius-md: calc(var(--radius) - 2px);
|
||||
--radius-lg: var(--radius);
|
||||
--radius-xl: calc(var(--radius) + 4px);
|
||||
}
|
||||
|
||||
/* ——— Resets ——— */
|
||||
|
||||
html,
|
||||
body {
|
||||
min-height: 100%;
|
||||
}
|
||||
|
||||
html {
|
||||
background: var(--background);
|
||||
}
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
line-height: 1.6;
|
||||
color: var(--foreground);
|
||||
font-family: var(--font-plex-sans), ui-sans-serif, system-ui, sans-serif;
|
||||
background: var(--background);
|
||||
}
|
||||
|
||||
/* Dreamverse's dark-mode backdrop: subtle radial glows over a deep fade. */
|
||||
body::after {
|
||||
content: "";
|
||||
position: fixed;
|
||||
inset: 0;
|
||||
z-index: -1;
|
||||
opacity: 0;
|
||||
pointer-events: none;
|
||||
transition: opacity 300ms ease;
|
||||
background:
|
||||
radial-gradient(circle at top, rgba(56, 189, 248, 0.14), transparent 36%),
|
||||
radial-gradient(circle at right top, rgba(129, 140, 248, 0.12), transparent 28%),
|
||||
linear-gradient(180deg, #020617 0%, #000000 100%);
|
||||
background-attachment: fixed;
|
||||
}
|
||||
|
||||
.dark body::after {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
a {
|
||||
color: inherit;
|
||||
}
|
||||
|
||||
summary {
|
||||
list-style: none;
|
||||
}
|
||||
|
||||
summary::-webkit-details-marker {
|
||||
display: none;
|
||||
}
|
||||
|
||||
::selection {
|
||||
background: rgba(56, 189, 248, 0.25);
|
||||
}
|
||||
|
||||
.dark ::selection {
|
||||
background: rgba(56, 189, 248, 0.35);
|
||||
color: #ffffff;
|
||||
}
|
||||
|
||||
/* Smooth the light/dark switch (class applied briefly by the theme toggle). */
|
||||
html.theme-transition,
|
||||
html.theme-transition *,
|
||||
html.theme-transition *::before,
|
||||
html.theme-transition *::after {
|
||||
transition:
|
||||
background-color 300ms ease,
|
||||
color 300ms ease,
|
||||
border-color 300ms ease,
|
||||
box-shadow 300ms ease,
|
||||
fill 300ms ease,
|
||||
stroke 300ms ease !important;
|
||||
}
|
||||
|
||||
/* ——— Base layer ——— */
|
||||
|
||||
@layer base {
|
||||
button,
|
||||
input,
|
||||
textarea,
|
||||
select {
|
||||
font: inherit;
|
||||
}
|
||||
|
||||
img,
|
||||
svg,
|
||||
video,
|
||||
canvas {
|
||||
display: block;
|
||||
max-width: 100%;
|
||||
}
|
||||
|
||||
* {
|
||||
@apply border-border;
|
||||
}
|
||||
body {
|
||||
@apply bg-background text-foreground;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
'use client';
|
||||
|
||||
import GpuGrid from '@/components/system/GpuGrid';
|
||||
|
||||
export default function GpusPage() {
|
||||
return (
|
||||
<main className="mx-auto flex w-full max-w-[1100px] flex-col gap-6 px-4 pb-12 pt-6">
|
||||
<GpuGrid />
|
||||
</main>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
'use client';
|
||||
|
||||
import CreateJobButton from '@/components/jobs/CreateJobButton';
|
||||
import { HeaderActions } from '@/components/shell/HeaderActionsContext';
|
||||
import JobQueue from '@/components/jobs/JobQueue';
|
||||
|
||||
export default function InferencePage() {
|
||||
return (
|
||||
<>
|
||||
<HeaderActions>
|
||||
<CreateJobButton jobType="inference" />
|
||||
</HeaderActions>
|
||||
<JobQueue jobType="inference" />
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
import type { Metadata } from 'next';
|
||||
import { IBM_Plex_Mono, IBM_Plex_Sans } from 'next/font/google';
|
||||
|
||||
import { AppShell } from '@/components/shell/AppShell';
|
||||
import './globals.css';
|
||||
|
||||
const plexSans = IBM_Plex_Sans({
|
||||
subsets: ['latin'],
|
||||
weight: ['400', '500', '600', '700'],
|
||||
variable: '--font-plex-sans',
|
||||
display: 'swap',
|
||||
});
|
||||
|
||||
const plexMono = IBM_Plex_Mono({
|
||||
subsets: ['latin'],
|
||||
weight: ['400', '500', '600'],
|
||||
variable: '--font-plex-mono',
|
||||
display: 'swap',
|
||||
});
|
||||
|
||||
export const metadata: Metadata = {
|
||||
title: 'FastVideo Studio',
|
||||
icons: { icon: '/fastvideo.ico' },
|
||||
};
|
||||
|
||||
export default function RootLayout({
|
||||
children,
|
||||
}: {
|
||||
children: React.ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<html
|
||||
lang="en"
|
||||
suppressHydrationWarning
|
||||
className={`${plexSans.variable} ${plexMono.variable}`}
|
||||
>
|
||||
<head>
|
||||
{/* The studio defaults to dark; apply the stored choice before paint. */}
|
||||
<script
|
||||
id="theme-init-script"
|
||||
suppressHydrationWarning
|
||||
dangerouslySetInnerHTML={{
|
||||
__html: `(function(){var dark=true;try{dark=localStorage.getItem('theme')!=='light'}catch(e){}if(dark)document.documentElement.classList.add('dark')})()`,
|
||||
}}
|
||||
/>
|
||||
</head>
|
||||
<body className="antialiased">
|
||||
<AppShell>{children}</AppShell>
|
||||
</body>
|
||||
</html>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
import { redirect } from 'next/navigation';
|
||||
|
||||
export default function Page() {
|
||||
redirect('/finetuning');
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
import { redirect } from 'next/navigation';
|
||||
|
||||
export default function Page() {
|
||||
redirect('/inference');
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
import { fireEvent, render, screen } from '@testing-library/react';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { HeaderActionsProvider } from '@/components/shell/HeaderActionsContext';
|
||||
import { resetToDefaults, updateOption } from '@/stores/defaultOptions';
|
||||
|
||||
import SettingsPage from './page';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
getModels: vi.fn().mockResolvedValue([]),
|
||||
getSettings: vi.fn().mockResolvedValue({}),
|
||||
updateSettings: vi.fn().mockResolvedValue({}),
|
||||
}));
|
||||
|
||||
// Keep the real store (so `useStore` resolves) but spy on the mutators.
|
||||
vi.mock('@/stores/defaultOptions', async (importOriginal) => {
|
||||
const actual =
|
||||
await importOriginal<typeof import('@/stores/defaultOptions')>();
|
||||
return {
|
||||
...actual,
|
||||
updateOption: vi.fn(),
|
||||
resetToDefaults: vi.fn(),
|
||||
};
|
||||
});
|
||||
|
||||
function renderPage() {
|
||||
return render(
|
||||
<HeaderActionsProvider>
|
||||
<SettingsPage />
|
||||
</HeaderActionsProvider>,
|
||||
);
|
||||
}
|
||||
|
||||
describe('Settings page', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
});
|
||||
|
||||
it('calls updateOption when a toggle changes', () => {
|
||||
renderPage();
|
||||
fireEvent.click(
|
||||
screen.getByRole('switch', { name: 'Auto Start Job on Create' }),
|
||||
);
|
||||
expect(updateOption).toHaveBeenCalledWith('autoStartJob', true);
|
||||
});
|
||||
|
||||
it('calls updateOption when a slider changes', () => {
|
||||
renderPage();
|
||||
// First slider in DOM order is "Frames".
|
||||
const slider = screen.getAllByRole('slider')[0];
|
||||
fireEvent.keyDown(slider, { key: 'ArrowRight' });
|
||||
expect(updateOption).toHaveBeenCalledWith('numFrames', expect.any(Number));
|
||||
});
|
||||
|
||||
it('calls resetToDefaults when Reset to Defaults is clicked', () => {
|
||||
renderPage();
|
||||
fireEvent.click(screen.getByRole('button', { name: 'Reset to Defaults' }));
|
||||
expect(resetToDefaults).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,358 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import {
|
||||
FieldRow,
|
||||
NumberRow,
|
||||
SliderRow,
|
||||
ToggleRow,
|
||||
} from '@/components/form-rows';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { Card, CardContent } from '@/components/ui/card';
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { Label } from '@/components/ui/label';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { Separator } from '@/components/ui/separator';
|
||||
import { Switch } from '@/components/ui/switch';
|
||||
import { useStore } from '@/hooks/useStore';
|
||||
import { getModels, type Model } from '@/lib/api';
|
||||
import {
|
||||
defaultOptionsStore,
|
||||
resetToDefaults,
|
||||
updateOption,
|
||||
} from '@/stores/defaultOptions';
|
||||
|
||||
function TextSettingRow({
|
||||
id,
|
||||
label,
|
||||
value,
|
||||
placeholder,
|
||||
onCommit,
|
||||
}: {
|
||||
id: string;
|
||||
label: string;
|
||||
value: string;
|
||||
placeholder?: string;
|
||||
onCommit: (v: string) => void;
|
||||
}) {
|
||||
// Buffer keystrokes locally and persist on blur/Enter so each character
|
||||
// doesn't fire a settings PUT (or a localStorage write).
|
||||
const [draft, setDraft] = React.useState<string | null>(null);
|
||||
return (
|
||||
<FieldRow htmlFor={id} label={label}>
|
||||
<Input
|
||||
id={id}
|
||||
type="text"
|
||||
className="font-mono text-sm"
|
||||
value={draft ?? value}
|
||||
onChange={(e) => setDraft(e.target.value)}
|
||||
onBlur={() => {
|
||||
if (draft !== null && draft !== value) onCommit(draft);
|
||||
setDraft(null);
|
||||
}}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter') e.currentTarget.blur();
|
||||
}}
|
||||
placeholder={placeholder}
|
||||
/>
|
||||
</FieldRow>
|
||||
);
|
||||
}
|
||||
|
||||
function SelectRow({
|
||||
id,
|
||||
label,
|
||||
value,
|
||||
onChange,
|
||||
children,
|
||||
}: {
|
||||
id: string;
|
||||
label: string;
|
||||
value: string;
|
||||
onChange: (v: string) => void;
|
||||
children: React.ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<FieldRow htmlFor={id} label={label}>
|
||||
<NativeSelect
|
||||
id={id}
|
||||
value={value}
|
||||
onChange={(e) => onChange(e.target.value)}
|
||||
>
|
||||
{children}
|
||||
</NativeSelect>
|
||||
</FieldRow>
|
||||
);
|
||||
}
|
||||
|
||||
export default function SettingsPage() {
|
||||
const { options } = useStore(defaultOptionsStore);
|
||||
|
||||
const [models, setModels] = React.useState<
|
||||
Record<'t2v' | 'i2v' | 't2i', Model[]>
|
||||
>({ t2v: [], i2v: [], t2i: [] });
|
||||
|
||||
React.useEffect(() => {
|
||||
Promise.all([getModels('t2v'), getModels('i2v'), getModels('t2i')])
|
||||
.then(([t2v, i2v, t2i]) => setModels({ t2v, i2v, t2i }))
|
||||
.catch((e) => console.error('Failed to load models:', e));
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<div className="mx-auto flex w-full max-w-[850px] flex-col gap-6 px-4 pb-12 pt-6">
|
||||
<Card>
|
||||
<CardContent className="space-y-4 p-6">
|
||||
<h2 className="text-lg font-semibold">Behavior</h2>
|
||||
<div className="flex items-center justify-between gap-4">
|
||||
<Label
|
||||
htmlFor="settings-auto-start-job"
|
||||
className="pl-0.5 text-xs font-normal tracking-wide text-muted-foreground"
|
||||
>
|
||||
Auto Start Job on Create
|
||||
</Label>
|
||||
<Switch
|
||||
id="settings-auto-start-job"
|
||||
checked={options.autoStartJob}
|
||||
onCheckedChange={(v) => updateOption('autoStartJob', v)}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<Separator />
|
||||
|
||||
<h2 className="text-lg font-semibold">Paths</h2>
|
||||
<div className="space-y-4">
|
||||
<TextSettingRow
|
||||
id="settings-api-server-base-url"
|
||||
label="API Server Base URL"
|
||||
value={options.apiServerBaseUrl ?? ''}
|
||||
onCommit={(v) => updateOption('apiServerBaseUrl', v)}
|
||||
placeholder="http://localhost:8189/api"
|
||||
/>
|
||||
<TextSettingRow
|
||||
id="settings-dataset-upload-path"
|
||||
label="Dataset Upload Path"
|
||||
value={options.datasetUploadPath ?? ''}
|
||||
onCommit={(v) => updateOption('datasetUploadPath', v)}
|
||||
placeholder="outputs/ui_data/uploads/datasets"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<Separator />
|
||||
|
||||
<div className="flex items-center justify-between">
|
||||
<h2 className="text-lg font-semibold">Default Options</h2>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={resetToDefaults}
|
||||
>
|
||||
Reset to Defaults
|
||||
</Button>
|
||||
</div>
|
||||
<p className="text-sm text-muted-foreground">
|
||||
These values are used as defaults when creating new jobs.
|
||||
</p>
|
||||
|
||||
<div className="grid gap-x-3 gap-y-2 [grid-template-columns:repeat(auto-fill,minmax(160px,1fr))]">
|
||||
<SelectRow
|
||||
id="settings-default-model-t2v"
|
||||
label="Default Model (T2V)"
|
||||
value={options.defaultModelIdT2v}
|
||||
onChange={(v) => updateOption('defaultModelIdT2v', v)}
|
||||
>
|
||||
<option value="">None (select when creating job)</option>
|
||||
{models.t2v.map((model) => (
|
||||
<option key={model.id} value={model.id}>
|
||||
{model.label} ({model.id})
|
||||
</option>
|
||||
))}
|
||||
</SelectRow>
|
||||
<SelectRow
|
||||
id="settings-default-model-i2v"
|
||||
label="Default Model (I2V)"
|
||||
value={options.defaultModelIdI2v}
|
||||
onChange={(v) => updateOption('defaultModelIdI2v', v)}
|
||||
>
|
||||
<option value="">None</option>
|
||||
{models.i2v.map((model) => (
|
||||
<option key={model.id} value={model.id}>
|
||||
{model.label} ({model.id})
|
||||
</option>
|
||||
))}
|
||||
</SelectRow>
|
||||
<SelectRow
|
||||
id="settings-default-model-t2i"
|
||||
label="Default Model (T2I)"
|
||||
value={options.defaultModelIdT2i}
|
||||
onChange={(v) => updateOption('defaultModelIdT2i', v)}
|
||||
>
|
||||
<option value="">None</option>
|
||||
{models.t2i.map((model) => (
|
||||
<option key={model.id} value={model.id}>
|
||||
{model.label} ({model.id})
|
||||
</option>
|
||||
))}
|
||||
</SelectRow>
|
||||
|
||||
<SliderRow
|
||||
id="settings-num-frames"
|
||||
label="Frames"
|
||||
min={1}
|
||||
max={500}
|
||||
step={1}
|
||||
value={options.numFrames}
|
||||
onChange={(v) => updateOption('numFrames', v)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-height"
|
||||
label="Height"
|
||||
min={64}
|
||||
max={1080}
|
||||
step={16}
|
||||
value={options.height}
|
||||
onChange={(v) => updateOption('height', v)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-width"
|
||||
label="Width"
|
||||
min={64}
|
||||
max={1920}
|
||||
step={16}
|
||||
value={options.width}
|
||||
onChange={(v) => updateOption('width', v)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-num-steps"
|
||||
label="Inference Steps"
|
||||
min={1}
|
||||
max={200}
|
||||
step={1}
|
||||
value={options.numInferenceSteps}
|
||||
onChange={(v) => updateOption('numInferenceSteps', v)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-vsa-sparsity"
|
||||
label="VSA Sparsity"
|
||||
title="VSA sparsity (0–1)"
|
||||
min={0}
|
||||
max={1}
|
||||
step={0.05}
|
||||
value={options.vsaSparsity}
|
||||
onChange={(v) => updateOption('vsaSparsity', v)}
|
||||
format={(v) => v.toFixed(2)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-guidance"
|
||||
label="Guidance Scale"
|
||||
min={0}
|
||||
max={20}
|
||||
step={0.1}
|
||||
value={options.guidanceScale}
|
||||
onChange={(v) => updateOption('guidanceScale', v)}
|
||||
format={(v) => v.toFixed(1)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-guidance-rescale"
|
||||
label="Guidance Rescale"
|
||||
title="0 = disabled"
|
||||
min={0}
|
||||
max={1}
|
||||
step={0.05}
|
||||
value={options.guidanceRescale ?? 0}
|
||||
onChange={(v) => updateOption('guidanceRescale', v)}
|
||||
format={(v) => v.toFixed(2)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-tp-size"
|
||||
label="TP Size"
|
||||
title="-1 = auto"
|
||||
min={-1}
|
||||
max={8}
|
||||
step={1}
|
||||
value={options.tpSize}
|
||||
onChange={(v) => updateOption('tpSize', v)}
|
||||
format={(v) => (v === -1 ? 'Auto' : String(v))}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-sp-size"
|
||||
label="SP Size"
|
||||
title="-1 = auto"
|
||||
min={-1}
|
||||
max={8}
|
||||
step={1}
|
||||
value={options.spSize}
|
||||
onChange={(v) => updateOption('spSize', v)}
|
||||
format={(v) => (v === -1 ? 'Auto' : String(v))}
|
||||
/>
|
||||
<SliderRow
|
||||
id="settings-fps"
|
||||
label="FPS"
|
||||
min={1}
|
||||
max={60}
|
||||
step={1}
|
||||
value={options.fps ?? 24}
|
||||
onChange={(v) => updateOption('fps', v)}
|
||||
/>
|
||||
|
||||
<ToggleRow
|
||||
id="settings-dit-cpu-offload"
|
||||
label="DiT CPU Offload"
|
||||
checked={options.ditCpuOffload}
|
||||
onChange={(v) => updateOption('ditCpuOffload', v)}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="settings-text-encoder-cpu-offload"
|
||||
label="Text Encoder CPU Offload"
|
||||
checked={options.textEncoderCpuOffload}
|
||||
onChange={(v) => updateOption('textEncoderCpuOffload', v)}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="settings-use-fsdp-inference"
|
||||
label="Use FSDP Inference"
|
||||
checked={options.useFsdpInference}
|
||||
onChange={(v) => updateOption('useFsdpInference', v)}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="settings-vae-cpu-offload"
|
||||
label="VAE CPU Offload"
|
||||
checked={options.vaeCpuOffload}
|
||||
onChange={(v) => updateOption('vaeCpuOffload', v)}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="settings-image-encoder-cpu-offload"
|
||||
label="Image Encoder CPU Offload"
|
||||
checked={options.imageEncoderCpuOffload}
|
||||
onChange={(v) => updateOption('imageEncoderCpuOffload', v)}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="settings-enable-torch-compile"
|
||||
label="Torch Compile"
|
||||
checked={options.enableTorchCompile}
|
||||
onChange={(v) => updateOption('enableTorchCompile', v)}
|
||||
/>
|
||||
|
||||
<SliderRow
|
||||
id="settings-num-gpus"
|
||||
label="GPUs"
|
||||
min={1}
|
||||
max={8}
|
||||
step={1}
|
||||
value={options.numGpus}
|
||||
onChange={(v) => updateOption('numGpus', v)}
|
||||
/>
|
||||
|
||||
<NumberRow
|
||||
id="settings-seed"
|
||||
label="Seed"
|
||||
min={0}
|
||||
value={options.seed}
|
||||
onChange={(v) => updateOption('seed', v)}
|
||||
/>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
'use client';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { setCreateDatasetModalOpen } from '@/stores/createDatasetModalOpen';
|
||||
|
||||
export default function AddDatasetButton() {
|
||||
return (
|
||||
<Button type="button" onClick={() => setCreateDatasetModalOpen(true)}>
|
||||
Add Dataset
|
||||
</Button>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
import { fireEvent, render, screen } from '@testing-library/react';
|
||||
|
||||
import CreateDatasetModal from '@/components/datasets/CreateDatasetModal';
|
||||
import { createDataset } from '@/lib/api';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
createDataset: vi.fn(),
|
||||
uploadRawDataset: vi.fn(),
|
||||
}));
|
||||
|
||||
const mockedCreateDataset = vi.mocked(createDataset);
|
||||
|
||||
describe('CreateDatasetModal', () => {
|
||||
it('renders nothing when closed', () => {
|
||||
render(
|
||||
<CreateDatasetModal isOpen={false} onClose={() => {}} onSuccess={() => {}} />,
|
||||
);
|
||||
expect(screen.queryByText('Add Dataset — Raw')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('renders the form and the default JSON caption upload when open', () => {
|
||||
render(
|
||||
<CreateDatasetModal isOpen onClose={() => {}} onSuccess={() => {}} />,
|
||||
);
|
||||
expect(screen.getByText('Add Dataset — Raw')).toBeInTheDocument();
|
||||
expect(screen.getByText('Upload video files')).toBeInTheDocument();
|
||||
expect(screen.getByText('Upload videos2caption.json')).toBeInTheDocument();
|
||||
expect(
|
||||
screen.getByRole('button', { name: 'Create Dataset' }),
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('does not create a dataset when the name is empty', () => {
|
||||
const onSuccess = vi.fn();
|
||||
render(
|
||||
<CreateDatasetModal isOpen onClose={() => {}} onSuccess={onSuccess} />,
|
||||
);
|
||||
fireEvent.click(screen.getByRole('button', { name: 'Create Dataset' }));
|
||||
expect(mockedCreateDataset).not.toHaveBeenCalled();
|
||||
expect(onSuccess).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,465 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import UploadZone from '@/components/datasets/UploadZone';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { Label } from '@/components/ui/label';
|
||||
import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs';
|
||||
import { createDataset, uploadRawDataset } from '@/lib/api';
|
||||
import {
|
||||
parseCaptionCsv,
|
||||
parseVideos2Caption,
|
||||
parseVideosCaptionsTxt,
|
||||
} from '@/lib/captionParsing';
|
||||
|
||||
const ALLOWED_VIDEO_EXT = '.mp4,.webm,.avi,.mov,.mkv';
|
||||
|
||||
type CaptionFormat = 'json' | 'txt' | 'csv';
|
||||
|
||||
export interface CreateDatasetModalProps {
|
||||
isOpen: boolean;
|
||||
onClose: () => void;
|
||||
onSuccess: () => void;
|
||||
}
|
||||
|
||||
export default function CreateDatasetModal({
|
||||
isOpen,
|
||||
onClose,
|
||||
onSuccess,
|
||||
}: CreateDatasetModalProps) {
|
||||
const [name, setName] = React.useState('');
|
||||
const [isSubmitting, setIsSubmitting] = React.useState(false);
|
||||
const [rawPath, setRawPath] = React.useState('');
|
||||
const [fileNames, setFileNames] = React.useState<string[]>([]);
|
||||
const [isUploading, setIsUploading] = React.useState(false);
|
||||
const [validationError, setValidationError] = React.useState<string | null>(
|
||||
null,
|
||||
);
|
||||
const [captionFormat, setCaptionFormat] = React.useState<CaptionFormat>('json');
|
||||
const [captionMap, setCaptionMap] = React.useState<Record<
|
||||
string,
|
||||
string
|
||||
> | null>(null);
|
||||
const [captionFileName, setCaptionFileName] = React.useState<string | null>(
|
||||
null,
|
||||
);
|
||||
const [videosTxtLines, setVideosTxtLines] = React.useState<string[] | null>(
|
||||
null,
|
||||
);
|
||||
const [videosTxtFileName, setVideosTxtFileName] = React.useState<
|
||||
string | null
|
||||
>(null);
|
||||
const [captionsTxtLines, setCaptionsTxtLines] = React.useState<
|
||||
string[] | null
|
||||
>(null);
|
||||
const [captionsTxtFileName, setCaptionsTxtFileName] = React.useState<
|
||||
string | null
|
||||
>(null);
|
||||
// Bumped on every new media selection and on reset/clear; an in-flight
|
||||
// upload whose generation no longer matches is discarded, so a superseded or
|
||||
// abandoned upload can't repopulate/overwrite state.
|
||||
const uploadGeneration = React.useRef(0);
|
||||
|
||||
const txtCaptionMap = React.useMemo(() => {
|
||||
if (!captionsTxtLines || fileNames.length === 0) return null;
|
||||
const { captions, error } = parseVideosCaptionsTxt(
|
||||
videosTxtLines ?? null,
|
||||
captionsTxtLines,
|
||||
fileNames,
|
||||
);
|
||||
if (error) return null;
|
||||
return Object.keys(captions).length > 0 ? captions : null;
|
||||
}, [captionsTxtLines, fileNames, videosTxtLines]);
|
||||
|
||||
const effectiveCaptionMap =
|
||||
captionFormat === 'txt' ? txtCaptionMap : captionMap;
|
||||
|
||||
// Keep the validation message in sync with the TXT inputs.
|
||||
React.useEffect(() => {
|
||||
if (captionFormat === 'txt' && captionsTxtLines && fileNames.length > 0) {
|
||||
const hasVideosTxt =
|
||||
!!videosTxtLines &&
|
||||
videosTxtLines.length > 0 &&
|
||||
videosTxtLines.some((s) => s.trim());
|
||||
if (hasVideosTxt && videosTxtLines) {
|
||||
const { error } = parseVideosCaptionsTxt(
|
||||
videosTxtLines,
|
||||
captionsTxtLines,
|
||||
fileNames,
|
||||
);
|
||||
setValidationError(error);
|
||||
} else {
|
||||
setValidationError(null);
|
||||
}
|
||||
}
|
||||
}, [captionFormat, captionsTxtLines, fileNames, videosTxtLines]);
|
||||
|
||||
function resetState() {
|
||||
uploadGeneration.current += 1;
|
||||
setName('');
|
||||
setRawPath('');
|
||||
setFileNames([]);
|
||||
setValidationError(null);
|
||||
setCaptionFormat('json');
|
||||
setCaptionMap(null);
|
||||
setCaptionFileName(null);
|
||||
setVideosTxtLines(null);
|
||||
setVideosTxtFileName(null);
|
||||
setCaptionsTxtLines(null);
|
||||
setCaptionsTxtFileName(null);
|
||||
}
|
||||
|
||||
function handleClose() {
|
||||
if (isSubmitting) return;
|
||||
resetState();
|
||||
onClose();
|
||||
}
|
||||
|
||||
async function handleMediaChange(files: File[]) {
|
||||
const gen = (uploadGeneration.current += 1);
|
||||
setValidationError(null);
|
||||
if (files.length === 0) {
|
||||
setRawPath('');
|
||||
setFileNames([]);
|
||||
return;
|
||||
}
|
||||
setIsUploading(true);
|
||||
try {
|
||||
const res = await uploadRawDataset(files);
|
||||
if (gen !== uploadGeneration.current) return; // superseded or abandoned
|
||||
setRawPath(res.path);
|
||||
setFileNames(res.file_names);
|
||||
if (res.file_names.length === 0) {
|
||||
setValidationError(`No video files found. Allowed: ${ALLOWED_VIDEO_EXT}`);
|
||||
}
|
||||
} catch (err) {
|
||||
if (gen !== uploadGeneration.current) return;
|
||||
setRawPath('');
|
||||
setFileNames([]);
|
||||
setValidationError(err instanceof Error ? err.message : 'Upload failed');
|
||||
} finally {
|
||||
if (gen === uploadGeneration.current) setIsUploading(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleCaptionJsonChange(files: File[]) {
|
||||
setValidationError(null);
|
||||
setCaptionMap(null);
|
||||
setCaptionFileName(null);
|
||||
if (files.length === 0) return;
|
||||
const file = files[0];
|
||||
try {
|
||||
const text = await file.text();
|
||||
const { captions, error } = parseVideos2Caption(text, fileNames);
|
||||
if (error) {
|
||||
setValidationError(error);
|
||||
return;
|
||||
}
|
||||
setCaptionMap(captions);
|
||||
setCaptionFileName(file.name);
|
||||
} catch {
|
||||
setValidationError('Could not read the file.');
|
||||
}
|
||||
}
|
||||
|
||||
async function handleCaptionCsvChange(files: File[]) {
|
||||
setValidationError(null);
|
||||
setCaptionMap(null);
|
||||
setCaptionFileName(null);
|
||||
if (files.length === 0) return;
|
||||
const file = files[0];
|
||||
try {
|
||||
const text = await file.text();
|
||||
const { captions, error } = parseCaptionCsv(text, fileNames);
|
||||
if (error) {
|
||||
setValidationError(error);
|
||||
return;
|
||||
}
|
||||
setCaptionMap(captions);
|
||||
setCaptionFileName(file.name);
|
||||
} catch {
|
||||
setValidationError('Could not read the file.');
|
||||
}
|
||||
}
|
||||
|
||||
async function handleVideosTxtChange(files: File[]) {
|
||||
setValidationError(null);
|
||||
setVideosTxtLines(null);
|
||||
setVideosTxtFileName(null);
|
||||
if (files.length === 0) return;
|
||||
try {
|
||||
const text = await files[0].text();
|
||||
setVideosTxtLines(text.split(/\r?\n/).map((s) => s.trim()));
|
||||
setVideosTxtFileName(files[0].name);
|
||||
} catch {
|
||||
setValidationError('Could not read videos.txt.');
|
||||
}
|
||||
}
|
||||
|
||||
async function handleCaptionsTxtChange(files: File[]) {
|
||||
setValidationError(null);
|
||||
setCaptionsTxtLines(null);
|
||||
setCaptionsTxtFileName(null);
|
||||
if (files.length === 0) return;
|
||||
try {
|
||||
const text = await files[0].text();
|
||||
setCaptionsTxtLines(text.split(/\r?\n/).map((s) => s.trim()));
|
||||
setCaptionsTxtFileName(files[0].name);
|
||||
} catch {
|
||||
setValidationError('Could not read captions.txt.');
|
||||
}
|
||||
}
|
||||
|
||||
function handleCaptionFormatChange(format: CaptionFormat) {
|
||||
setCaptionFormat(format);
|
||||
setValidationError(null);
|
||||
setCaptionMap(null);
|
||||
setCaptionFileName(null);
|
||||
setVideosTxtLines(null);
|
||||
setVideosTxtFileName(null);
|
||||
setCaptionsTxtLines(null);
|
||||
setCaptionsTxtFileName(null);
|
||||
}
|
||||
|
||||
async function handleSubmit(e: React.FormEvent) {
|
||||
e.preventDefault();
|
||||
setValidationError(null);
|
||||
if (!name.trim()) return;
|
||||
if (!rawPath || fileNames.length === 0) {
|
||||
setValidationError('No data was found. Upload at least one video.');
|
||||
return;
|
||||
}
|
||||
if (captionFormat === 'json' || captionFormat === 'csv') {
|
||||
if (captionFileName && !captionMap) {
|
||||
setValidationError(
|
||||
'Caption file has errors. Fix or remove it before creating the dataset.',
|
||||
);
|
||||
return;
|
||||
}
|
||||
} else if (captionFormat === 'txt') {
|
||||
if (videosTxtFileName || captionsTxtFileName) {
|
||||
if (!captionsTxtFileName) {
|
||||
setValidationError('Upload captions.txt to use TXT captions.');
|
||||
return;
|
||||
}
|
||||
// Re-validate against current state rather than the stale render-time
|
||||
// `validationError`, so a videos.txt mismatch both blocks submission
|
||||
// and keeps its message visible.
|
||||
const hasVideosTxt =
|
||||
!!videosTxtLines &&
|
||||
videosTxtLines.length > 0 &&
|
||||
videosTxtLines.some((s) => s.trim());
|
||||
if (hasVideosTxt && videosTxtLines) {
|
||||
const { error } = parseVideosCaptionsTxt(
|
||||
videosTxtLines,
|
||||
captionsTxtLines ?? [],
|
||||
fileNames,
|
||||
);
|
||||
if (error) {
|
||||
setValidationError(error);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const finalCaptionMap =
|
||||
effectiveCaptionMap && Object.keys(effectiveCaptionMap).length > 0
|
||||
? effectiveCaptionMap
|
||||
: null;
|
||||
if (finalCaptionMap) {
|
||||
const missing = fileNames.filter((fn) => !(fn in finalCaptionMap));
|
||||
if (missing.length > 0) {
|
||||
const list =
|
||||
missing.length <= 5
|
||||
? missing.join(', ')
|
||||
: `${missing.slice(0, 5).join(', ')} and ${missing.length - 5} more`;
|
||||
const ok = window.confirm(
|
||||
`The caption file does not include captions for ${missing.length} video(s): ${list}. They will get empty captions. Continue?`,
|
||||
);
|
||||
if (!ok) return;
|
||||
}
|
||||
}
|
||||
|
||||
setIsSubmitting(true);
|
||||
try {
|
||||
await createDataset({
|
||||
name: name.trim(),
|
||||
upload_path: rawPath,
|
||||
file_names: fileNames,
|
||||
...(finalCaptionMap ? { captions: finalCaptionMap } : {}),
|
||||
});
|
||||
onSuccess();
|
||||
resetState();
|
||||
onClose();
|
||||
} catch (err) {
|
||||
setValidationError(
|
||||
err instanceof Error ? err.message : 'Failed to create dataset',
|
||||
);
|
||||
} finally {
|
||||
setIsSubmitting(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog
|
||||
open={isOpen}
|
||||
onOpenChange={(open) => {
|
||||
if (!open) handleClose();
|
||||
}}
|
||||
>
|
||||
<DialogContent
|
||||
aria-describedby={undefined}
|
||||
className="max-h-[90vh] w-[90vw] max-w-[850px] overflow-y-auto"
|
||||
>
|
||||
<DialogHeader>
|
||||
<DialogTitle>Add Dataset — Raw</DialogTitle>
|
||||
</DialogHeader>
|
||||
<form onSubmit={handleSubmit} autoComplete="off">
|
||||
<div className="mb-3.5 flex flex-col gap-1.5">
|
||||
<Label htmlFor="add-dataset-name">Name</Label>
|
||||
<Input
|
||||
id="add-dataset-name"
|
||||
type="text"
|
||||
value={name}
|
||||
onChange={(e) => setName(e.target.value)}
|
||||
placeholder="My dataset"
|
||||
required
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="mb-3.5 flex flex-col gap-1.5">
|
||||
<Label>Videos</Label>
|
||||
<UploadZone
|
||||
label="Upload video files"
|
||||
hint="Select files or a folder (.mp4, .webm, .avi, .mov, .mkv)"
|
||||
accept={ALLOWED_VIDEO_EXT}
|
||||
multiple
|
||||
directory
|
||||
allowBothFileAndDirectory
|
||||
value={rawPath}
|
||||
fileName={
|
||||
fileNames.length > 0 ? `${fileNames.length} file(s)` : undefined
|
||||
}
|
||||
onFiles={handleMediaChange}
|
||||
onClear={() => {
|
||||
uploadGeneration.current += 1;
|
||||
setRawPath('');
|
||||
setFileNames([]);
|
||||
setCaptionMap(null);
|
||||
setCaptionFileName(null);
|
||||
setValidationError(null);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
uploading={isUploading}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="mb-3.5 flex flex-col gap-1.5">
|
||||
<Tabs
|
||||
value={captionFormat}
|
||||
onValueChange={(value) =>
|
||||
handleCaptionFormatChange(value as CaptionFormat)
|
||||
}
|
||||
>
|
||||
<div className="mb-1.5 flex flex-wrap items-center gap-4">
|
||||
<Label>Captions (optional)</Label>
|
||||
<TabsList>
|
||||
<TabsTrigger value="json" disabled={isSubmitting}>
|
||||
JSON
|
||||
</TabsTrigger>
|
||||
<TabsTrigger value="txt" disabled={isSubmitting}>
|
||||
TXT
|
||||
</TabsTrigger>
|
||||
<TabsTrigger value="csv" disabled={isSubmitting}>
|
||||
CSV
|
||||
</TabsTrigger>
|
||||
</TabsList>
|
||||
</div>
|
||||
<TabsContent value="json">
|
||||
<UploadZone
|
||||
label="Upload videos2caption.json"
|
||||
hint="Array of { path, cap } or object mapping file names to captions"
|
||||
accept=".json,application/json"
|
||||
value={captionFileName ? '1' : ''}
|
||||
fileName={captionFileName ?? undefined}
|
||||
onFiles={handleCaptionJsonChange}
|
||||
onClear={() => {
|
||||
setCaptionMap(null);
|
||||
setCaptionFileName(null);
|
||||
setValidationError(null);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</TabsContent>
|
||||
<TabsContent value="txt">
|
||||
<div className="flex flex-wrap gap-[15px] [&>*]:min-w-[200px] [&>*]:flex-1">
|
||||
<UploadZone
|
||||
label="Upload videos.txt (optional)"
|
||||
hint="One video path per line, or leave empty to match captions to videos in alphabetical order"
|
||||
accept=".txt,text/plain"
|
||||
value={videosTxtFileName ? '1' : ''}
|
||||
fileName={videosTxtFileName ?? undefined}
|
||||
onFiles={handleVideosTxtChange}
|
||||
onClear={() => {
|
||||
setVideosTxtLines(null);
|
||||
setVideosTxtFileName(null);
|
||||
setValidationError(null);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<UploadZone
|
||||
label="Upload captions.txt"
|
||||
hint="One caption per line (same order as videos.txt or alphabetical)"
|
||||
accept=".txt,text/plain"
|
||||
value={captionsTxtFileName ? '1' : ''}
|
||||
fileName={captionsTxtFileName ?? undefined}
|
||||
onFiles={handleCaptionsTxtChange}
|
||||
onClear={() => {
|
||||
setCaptionsTxtLines(null);
|
||||
setCaptionsTxtFileName(null);
|
||||
setValidationError(null);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</div>
|
||||
</TabsContent>
|
||||
<TabsContent value="csv">
|
||||
<UploadZone
|
||||
label="Upload captions CSV"
|
||||
hint="Header: video_name, caption"
|
||||
accept=".csv,text/csv"
|
||||
value={captionFileName ? '1' : ''}
|
||||
fileName={captionFileName ?? undefined}
|
||||
onFiles={handleCaptionCsvChange}
|
||||
onClear={() => {
|
||||
setCaptionMap(null);
|
||||
setCaptionFileName(null);
|
||||
setValidationError(null);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</TabsContent>
|
||||
</Tabs>
|
||||
</div>
|
||||
|
||||
{validationError && (
|
||||
<p className="mb-2 text-sm text-destructive">{validationError}</p>
|
||||
)}
|
||||
<Button type="submit" disabled={isSubmitting}>
|
||||
{isSubmitting ? 'Creating…' : 'Create Dataset'}
|
||||
</Button>
|
||||
</form>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,105 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { fireEvent, render, screen, waitFor } from '@testing-library/react';
|
||||
|
||||
import DatasetCard from '@/components/datasets/DatasetCard';
|
||||
import { deleteDataset } from '@/lib/api';
|
||||
import type { Dataset } from '@/lib/api';
|
||||
import { setActiveDatasetId } from '@/stores/activeDataset';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
deleteDataset: vi.fn(),
|
||||
}));
|
||||
|
||||
const mockedDeleteDataset = vi.mocked(deleteDataset);
|
||||
|
||||
const dataset: Dataset = {
|
||||
id: 'ds-1',
|
||||
name: 'My Dataset',
|
||||
created_at: 0,
|
||||
file_count: 3,
|
||||
size_bytes: 2048,
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
setActiveDatasetId(null);
|
||||
});
|
||||
|
||||
describe('DatasetCard', () => {
|
||||
it('renders the name, file count and human-readable size', () => {
|
||||
render(<DatasetCard dataset={dataset} onUpdated={() => {}} />);
|
||||
expect(screen.getByText('My Dataset')).toBeInTheDocument();
|
||||
expect(screen.getByText('3 files · 2.0 KB')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('uses the singular "file" label and byte units for a small dataset', () => {
|
||||
render(
|
||||
<DatasetCard
|
||||
dataset={{ ...dataset, file_count: 1, size_bytes: 512 }}
|
||||
onUpdated={() => {}}
|
||||
/>,
|
||||
);
|
||||
expect(screen.getByText('1 file · 512 B')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('calls onSelect when the card body is clicked', () => {
|
||||
const onSelect = vi.fn();
|
||||
render(
|
||||
<DatasetCard dataset={dataset} onUpdated={() => {}} onSelect={onSelect} />,
|
||||
);
|
||||
fireEvent.click(screen.getByText('My Dataset'));
|
||||
expect(onSelect).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('deletes after confirmation and notifies the parent without selecting', async () => {
|
||||
const onUpdated = vi.fn();
|
||||
const onSelect = vi.fn();
|
||||
const confirmSpy = vi.spyOn(window, 'confirm').mockReturnValue(true);
|
||||
mockedDeleteDataset.mockResolvedValue(undefined);
|
||||
|
||||
render(
|
||||
<DatasetCard
|
||||
dataset={dataset}
|
||||
onUpdated={onUpdated}
|
||||
onSelect={onSelect}
|
||||
/>,
|
||||
);
|
||||
fireEvent.click(screen.getByRole('button', { name: 'Delete' }));
|
||||
|
||||
expect(confirmSpy).toHaveBeenCalledWith('Delete dataset "My Dataset"?');
|
||||
expect(mockedDeleteDataset).toHaveBeenCalledWith('ds-1');
|
||||
await waitFor(() => expect(onUpdated).toHaveBeenCalledTimes(1));
|
||||
expect(onSelect).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('selects on keyboard activation of the card body but not of the Delete button', () => {
|
||||
const onSelect = vi.fn();
|
||||
render(
|
||||
<DatasetCard dataset={dataset} onUpdated={() => {}} onSelect={onSelect} />,
|
||||
);
|
||||
|
||||
// Activating the Delete button must not bubble into a card selection.
|
||||
fireEvent.keyDown(screen.getByRole('button', { name: 'Delete' }), {
|
||||
key: 'Enter',
|
||||
});
|
||||
expect(onSelect).not.toHaveBeenCalled();
|
||||
|
||||
// Activating the card body itself does select.
|
||||
fireEvent.keyDown(screen.getByText('My Dataset'), { key: 'Enter' });
|
||||
expect(onSelect).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('does not delete when the user cancels the confirm dialog', () => {
|
||||
vi.spyOn(window, 'confirm').mockReturnValue(false);
|
||||
render(<DatasetCard dataset={dataset} onUpdated={() => {}} />);
|
||||
fireEvent.click(screen.getByRole('button', { name: 'Delete' }));
|
||||
expect(mockedDeleteDataset).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('applies selected styling when it is the active dataset', () => {
|
||||
setActiveDatasetId('ds-1');
|
||||
const { container } = render(
|
||||
<DatasetCard dataset={dataset} onUpdated={() => {}} />,
|
||||
);
|
||||
expect(container.firstChild).toHaveClass('border-accent-blue', 'bg-accent-blue/5');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,97 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { deleteDataset } from '@/lib/api';
|
||||
import type { Dataset } from '@/lib/api';
|
||||
import { cn } from '@/lib/utils';
|
||||
import { activeDatasetStore, setActiveDatasetId } from '@/stores/activeDataset';
|
||||
import { useStore } from '@/hooks/useStore';
|
||||
|
||||
function formatSize(sizeBytes: number): string {
|
||||
if (sizeBytes < 1024) return `${sizeBytes} B`;
|
||||
if (sizeBytes < 1024 * 1024) return `${(sizeBytes / 1024).toFixed(1)} KB`;
|
||||
if (sizeBytes < 1024 * 1024 * 1024) {
|
||||
return `${(sizeBytes / (1024 * 1024)).toFixed(1)} MB`;
|
||||
}
|
||||
return `${(sizeBytes / (1024 * 1024 * 1024)).toFixed(1)} GB`;
|
||||
}
|
||||
|
||||
export default function DatasetCard({
|
||||
dataset,
|
||||
onUpdated,
|
||||
onSelect = () => {},
|
||||
}: {
|
||||
dataset: Dataset;
|
||||
onUpdated: () => void;
|
||||
onSelect?: () => void;
|
||||
}) {
|
||||
const { activeDatasetId } = useStore(activeDatasetStore);
|
||||
const [isLoading, setIsLoading] = React.useState(false);
|
||||
|
||||
const isSelected = activeDatasetId === dataset.id;
|
||||
const fileCount = dataset.file_count ?? 0;
|
||||
const sizeLabel = formatSize(dataset.size_bytes ?? 0);
|
||||
|
||||
async function handleDelete(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading) return;
|
||||
if (!window.confirm(`Delete dataset "${dataset.name}"?`)) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await deleteDataset(dataset.id);
|
||||
if (activeDatasetStore.get().activeDatasetId === dataset.id) {
|
||||
setActiveDatasetId(null);
|
||||
}
|
||||
onUpdated();
|
||||
} catch (err) {
|
||||
window.alert(
|
||||
err instanceof Error ? err.message : 'Failed to delete dataset',
|
||||
);
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
function handleKeyDown(e: React.KeyboardEvent) {
|
||||
if ((e.target as HTMLElement).closest('button')) return;
|
||||
if (e.key === 'Enter' || e.key === ' ') {
|
||||
e.preventDefault();
|
||||
onSelect();
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
className={cn(
|
||||
'mb-3 flex cursor-pointer flex-col gap-[0.6rem] rounded-lg border border-border bg-background px-[1.15rem] py-4',
|
||||
isSelected && 'border-accent-blue bg-accent-blue/5',
|
||||
)}
|
||||
onClick={(e) => {
|
||||
if ((e.target as HTMLElement).closest('button')) return;
|
||||
onSelect();
|
||||
}}
|
||||
onKeyDown={handleKeyDown}
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
>
|
||||
<div className="flex flex-wrap items-center justify-between gap-2">
|
||||
<span className="text-[0.95rem] font-semibold">{dataset.name}</span>
|
||||
<Button
|
||||
type="button"
|
||||
variant="destructive"
|
||||
size="sm"
|
||||
onClick={handleDelete}
|
||||
disabled={isLoading}
|
||||
>
|
||||
Delete
|
||||
</Button>
|
||||
</div>
|
||||
<div className="text-sm text-muted-foreground">
|
||||
{fileCount} {fileCount === 1 ? 'file' : 'files'} · {sizeLabel}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,156 @@
|
||||
import { describe, it, expect, vi, beforeEach } from 'vitest';
|
||||
import { render, screen, fireEvent, act } from '@testing-library/react';
|
||||
|
||||
import DatasetSidebar from '@/components/datasets/DatasetSidebar';
|
||||
import * as api from '@/lib/api';
|
||||
import type { Dataset } from '@/lib/api';
|
||||
|
||||
vi.mock('@/lib/api');
|
||||
|
||||
const mockedApi = vi.mocked(api);
|
||||
|
||||
const dataset: Dataset = {
|
||||
id: 'ds-1',
|
||||
name: 'My Dataset',
|
||||
created_at: 0,
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
mockedApi.getDatasetFiles.mockResolvedValue({
|
||||
file_names: ['a.mp4', 'b.mp4'],
|
||||
captions: { 'a.mp4': 'cap a', 'b.mp4': '' },
|
||||
});
|
||||
mockedApi.getDatasetMediaUrl.mockImplementation(
|
||||
(id, fileName) => `http://test/${id}/${fileName}`,
|
||||
);
|
||||
mockedApi.updateDatasetCaption.mockResolvedValue(undefined);
|
||||
});
|
||||
|
||||
describe('DatasetSidebar', () => {
|
||||
it('lists dataset files after loading', async () => {
|
||||
render(<DatasetSidebar dataset={dataset} onClose={() => {}} />);
|
||||
|
||||
// The first file's caption is rendered once loading resolves.
|
||||
expect(await screen.findByDisplayValue('cap a')).toBeInTheDocument();
|
||||
expect(mockedApi.getDatasetFiles).toHaveBeenCalledWith('ds-1');
|
||||
|
||||
// One caption editor per returned file.
|
||||
const captionFields = screen.getAllByPlaceholderText('Caption');
|
||||
expect(captionFields).toHaveLength(2);
|
||||
|
||||
// Media URLs are requested per visible file.
|
||||
expect(mockedApi.getDatasetMediaUrl).toHaveBeenCalledWith('ds-1', 'a.mp4');
|
||||
expect(mockedApi.getDatasetMediaUrl).toHaveBeenCalledWith('ds-1', 'b.mp4');
|
||||
});
|
||||
|
||||
it('debounces caption save by 500ms', async () => {
|
||||
render(<DatasetSidebar dataset={dataset} onClose={() => {}} />);
|
||||
const textarea = await screen.findByDisplayValue('cap a');
|
||||
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
fireEvent.change(textarea, { target: { value: 'updated caption' } });
|
||||
|
||||
// Nothing saved immediately or just before the debounce window closes.
|
||||
expect(mockedApi.updateDatasetCaption).not.toHaveBeenCalled();
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(499);
|
||||
});
|
||||
expect(mockedApi.updateDatasetCaption).not.toHaveBeenCalled();
|
||||
|
||||
// Saved exactly once after the full 500ms.
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(1);
|
||||
});
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledTimes(1);
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledWith(
|
||||
'ds-1',
|
||||
'a.mp4',
|
||||
'updated caption',
|
||||
);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it('coalesces rapid edits into a single debounced save', async () => {
|
||||
render(<DatasetSidebar dataset={dataset} onClose={() => {}} />);
|
||||
const textarea = await screen.findByDisplayValue('cap a');
|
||||
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
fireEvent.change(textarea, { target: { value: 'one' } });
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(300);
|
||||
});
|
||||
fireEvent.change(textarea, { target: { value: 'two' } });
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(500);
|
||||
});
|
||||
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledTimes(1);
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledWith(
|
||||
'ds-1',
|
||||
'a.mp4',
|
||||
'two',
|
||||
);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it('debounces per file: editing another caption does not cancel a pending save', async () => {
|
||||
render(<DatasetSidebar dataset={dataset} onClose={() => {}} />);
|
||||
await screen.findByDisplayValue('cap a');
|
||||
const [textareaA, textareaB] = screen.getAllByPlaceholderText('Caption');
|
||||
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
fireEvent.change(textareaA, { target: { value: 'new cap a' } });
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(300);
|
||||
});
|
||||
// Editing b.mp4 inside a.mp4's debounce window must not drop a's save.
|
||||
fireEvent.change(textareaB, { target: { value: 'new cap b' } });
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(500);
|
||||
});
|
||||
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledTimes(2);
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledWith(
|
||||
'ds-1',
|
||||
'a.mp4',
|
||||
'new cap a',
|
||||
);
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledWith(
|
||||
'ds-1',
|
||||
'b.mp4',
|
||||
'new cap b',
|
||||
);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it('flushes a pending save on unmount instead of dropping it', async () => {
|
||||
const { unmount } = render(
|
||||
<DatasetSidebar dataset={dataset} onClose={() => {}} />,
|
||||
);
|
||||
const textarea = await screen.findByDisplayValue('cap a');
|
||||
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
fireEvent.change(textarea, { target: { value: 'edited just before close' } });
|
||||
unmount();
|
||||
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledTimes(1);
|
||||
expect(mockedApi.updateDatasetCaption).toHaveBeenCalledWith(
|
||||
'ds-1',
|
||||
'a.mp4',
|
||||
'edited just before close',
|
||||
);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,253 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { X } from 'lucide-react';
|
||||
|
||||
import DownloadCaptions from '@/components/datasets/DownloadCaptions';
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { useResizable } from '@/hooks/useResizable';
|
||||
import {
|
||||
getDatasetFiles,
|
||||
getDatasetMediaUrl,
|
||||
updateDatasetCaption,
|
||||
type Dataset,
|
||||
} from '@/lib/api';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const SIDEBAR_MIN_WIDTH = 320;
|
||||
const SIDEBAR_MAX_WIDTH = 900;
|
||||
const INITIAL_PAGE_SIZE = 24;
|
||||
const PAGE_SIZE = 24;
|
||||
const SCROLL_THRESHOLD = 200;
|
||||
|
||||
// Memoized so a caption keystroke re-renders only the edited card, not every
|
||||
// visible <video> in the grid (visibleCount grows unbounded with scrolling).
|
||||
const DatasetFileCard = React.memo(function DatasetFileCard({
|
||||
fileName,
|
||||
mediaUrl,
|
||||
caption,
|
||||
thumbLoaded,
|
||||
onCaptionChange,
|
||||
onThumbLoaded,
|
||||
}: {
|
||||
fileName: string;
|
||||
mediaUrl: string;
|
||||
caption: string;
|
||||
thumbLoaded: boolean;
|
||||
onCaptionChange: (fileName: string, value: string) => void;
|
||||
onThumbLoaded: (fileName: string) => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="relative flex flex-col overflow-hidden rounded-lg border border-border bg-background">
|
||||
{!thumbLoaded && (
|
||||
<div className="pointer-events-none absolute inset-0 flex items-center justify-center bg-background/70">
|
||||
<div className="h-6 w-6 animate-spin rounded-full border-2 border-muted-foreground/40 border-t-accent-blue" />
|
||||
</div>
|
||||
)}
|
||||
{/* eslint-disable-next-line jsx-a11y/media-has-caption */}
|
||||
<video
|
||||
src={mediaUrl}
|
||||
className="aspect-video w-full bg-border object-cover"
|
||||
muted
|
||||
autoPlay
|
||||
loop
|
||||
playsInline
|
||||
onLoadedData={() => onThumbLoaded(fileName)}
|
||||
onError={() => onThumbLoaded(fileName)}
|
||||
/>
|
||||
<Textarea
|
||||
value={caption}
|
||||
onChange={(e) => onCaptionChange(fileName, e.target.value)}
|
||||
placeholder="Caption"
|
||||
rows={2}
|
||||
className="min-h-[2.5rem] resize-y rounded-none border-0 bg-transparent p-1.5 text-xs shadow-none focus-visible:border-transparent focus-visible:ring-0"
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
});
|
||||
|
||||
export default function DatasetSidebar({
|
||||
dataset,
|
||||
onClose,
|
||||
onWidthChange,
|
||||
}: {
|
||||
dataset: Dataset;
|
||||
onClose: () => void;
|
||||
onWidthChange?: (w: number) => void;
|
||||
}) {
|
||||
const [width, setWidth] = React.useState(400);
|
||||
const [isDragging, setIsDragging] = React.useState(false);
|
||||
const [fileNames, setFileNames] = React.useState<string[]>([]);
|
||||
const [captions, setCaptions] = React.useState<Record<string, string>>({});
|
||||
const [visibleCount, setVisibleCount] = React.useState(INITIAL_PAGE_SIZE);
|
||||
const [isLoading, setIsLoading] = React.useState(true);
|
||||
const [thumbLoaded, setThumbLoaded] = React.useState<
|
||||
Record<string, boolean>
|
||||
>({});
|
||||
|
||||
// Pending debounced caption saves, keyed per file so editing one caption
|
||||
// can't cancel another file's pending save.
|
||||
const pendingSaves = React.useRef(
|
||||
new Map<string, { timer: ReturnType<typeof setTimeout>; save: () => void }>(),
|
||||
);
|
||||
const scrollRef = React.useRef<HTMLDivElement>(null);
|
||||
|
||||
React.useEffect(() => {
|
||||
onWidthChange?.(width);
|
||||
}, [width, onWidthChange]);
|
||||
|
||||
React.useEffect(() => {
|
||||
let cancelled = false;
|
||||
setIsLoading(true);
|
||||
getDatasetFiles(dataset.id)
|
||||
.then((data) => {
|
||||
if (cancelled) return;
|
||||
setFileNames(data.file_names);
|
||||
setCaptions(data.captions);
|
||||
setVisibleCount(INITIAL_PAGE_SIZE);
|
||||
setThumbLoaded({});
|
||||
})
|
||||
.catch((err) => console.error('Failed to load dataset files:', err))
|
||||
.finally(() => {
|
||||
if (!cancelled) setIsLoading(false);
|
||||
});
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [dataset.id]);
|
||||
|
||||
const flushPendingSaves = React.useCallback(() => {
|
||||
for (const { timer, save } of pendingSaves.current.values()) {
|
||||
clearTimeout(timer);
|
||||
save();
|
||||
}
|
||||
pendingSaves.current.clear();
|
||||
}, []);
|
||||
|
||||
// Flush (not drop) pending saves when the dataset changes or on unmount, so
|
||||
// an edit made within the debounce window of closing isn't lost.
|
||||
React.useEffect(() => flushPendingSaves, [dataset.id, flushPendingSaves]);
|
||||
|
||||
const { onMouseDown } = useResizable({
|
||||
edge: 'right',
|
||||
minWidth: SIDEBAR_MIN_WIDTH,
|
||||
maxWidth: SIDEBAR_MAX_WIDTH,
|
||||
getWidth: () => width,
|
||||
onWidth: setWidth,
|
||||
onDragChange: setIsDragging,
|
||||
});
|
||||
|
||||
const datasetId = dataset.id;
|
||||
const handleCaptionChange = React.useCallback(
|
||||
(fileName: string, value: string) => {
|
||||
setCaptions((prev) => ({ ...prev, [fileName]: value }));
|
||||
const pending = pendingSaves.current.get(fileName);
|
||||
if (pending) clearTimeout(pending.timer);
|
||||
const save = () => {
|
||||
updateDatasetCaption(datasetId, fileName, value).catch((err) =>
|
||||
console.error('Failed to save caption:', err),
|
||||
);
|
||||
};
|
||||
const timer = setTimeout(() => {
|
||||
pendingSaves.current.delete(fileName);
|
||||
save();
|
||||
}, 500);
|
||||
pendingSaves.current.set(fileName, { timer, save });
|
||||
},
|
||||
[datasetId],
|
||||
);
|
||||
|
||||
function handleScroll() {
|
||||
const el = scrollRef.current;
|
||||
if (!el || isLoading || visibleCount >= fileNames.length) return;
|
||||
const { scrollTop, scrollHeight, clientHeight } = el;
|
||||
const distanceFromBottom = scrollHeight - (scrollTop + clientHeight);
|
||||
if (distanceFromBottom < SCROLL_THRESHOLD) {
|
||||
setVisibleCount((c) => Math.min(c + PAGE_SIZE, fileNames.length));
|
||||
}
|
||||
}
|
||||
|
||||
// When the visible grid doesn't overflow (a wide/tall sidebar can fit the
|
||||
// first page without a scrollbar), no scroll event ever fires — so top up
|
||||
// visibleCount until it overflows or every file is shown. Re-runs on resize
|
||||
// (width) and after each page grows, otherwise files 25..N are unreachable.
|
||||
React.useEffect(() => {
|
||||
if (isLoading || visibleCount >= fileNames.length) return;
|
||||
const el = scrollRef.current;
|
||||
if (el && el.scrollHeight <= el.clientHeight) {
|
||||
setVisibleCount((c) => Math.min(c + PAGE_SIZE, fileNames.length));
|
||||
}
|
||||
}, [visibleCount, fileNames.length, isLoading, width]);
|
||||
|
||||
const markThumbLoaded = React.useCallback((fileName: string) => {
|
||||
setThumbLoaded((prev) =>
|
||||
prev[fileName] ? prev : { ...prev, [fileName]: true },
|
||||
);
|
||||
}, []);
|
||||
|
||||
const visibleFiles = fileNames.slice(0, visibleCount);
|
||||
|
||||
return (
|
||||
<aside
|
||||
className="fixed bottom-0 right-0 top-[var(--header-height)] z-50 flex max-h-[calc(100vh-var(--header-height))] min-w-[320px] shrink-0 flex-col border-l border-border bg-card"
|
||||
style={{ width, maxWidth: SIDEBAR_MAX_WIDTH }}
|
||||
>
|
||||
<div className="flex shrink-0 items-center justify-between border-b border-border px-5 py-4">
|
||||
<h2 className="m-0 min-w-0 truncate text-base font-semibold text-foreground">
|
||||
{dataset.name}
|
||||
</h2>
|
||||
<div className="flex shrink-0 items-center gap-2">
|
||||
<DownloadCaptions fileNames={fileNames} captions={captions} />
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClose}
|
||||
title="Close"
|
||||
aria-label="Close"
|
||||
className="flex items-center justify-center rounded-lg p-1.5 text-muted-foreground transition-colors hover:bg-accent hover:text-foreground"
|
||||
>
|
||||
<X className="h-[18px] w-[18px]" />
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex min-h-0 flex-1 flex-col overflow-hidden">
|
||||
<div
|
||||
ref={scrollRef}
|
||||
onScroll={handleScroll}
|
||||
className="flex-1 overflow-y-auto p-4"
|
||||
>
|
||||
{isLoading ? (
|
||||
<p className="p-8 text-center text-muted-foreground">Loading…</p>
|
||||
) : fileNames.length === 0 ? (
|
||||
<p className="p-8 text-center text-muted-foreground">
|
||||
No media files
|
||||
</p>
|
||||
) : (
|
||||
<div className="grid gap-4 [grid-template-columns:repeat(auto-fill,minmax(140px,1fr))]">
|
||||
{visibleFiles.map((fileName) => (
|
||||
<DatasetFileCard
|
||||
key={fileName}
|
||||
fileName={fileName}
|
||||
mediaUrl={getDatasetMediaUrl(dataset.id, fileName)}
|
||||
caption={captions[fileName] ?? ''}
|
||||
thumbLoaded={!!thumbLoaded[fileName]}
|
||||
onCaptionChange={handleCaptionChange}
|
||||
onThumbLoaded={markThumbLoaded}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div
|
||||
role="presentation"
|
||||
onMouseDown={onMouseDown}
|
||||
className={cn(
|
||||
'absolute bottom-0 left-0 top-0 z-[1] w-1.5 cursor-col-resize hover:bg-accent-blue/25',
|
||||
isDragging && 'bg-accent-blue/25',
|
||||
)}
|
||||
/>
|
||||
</aside>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,113 @@
|
||||
'use client';
|
||||
|
||||
import { ChevronDown } from 'lucide-react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { downloadBlob } from '@/lib/utils';
|
||||
|
||||
const MENU_ITEM =
|
||||
'block w-full cursor-pointer px-4 py-2 text-left text-sm font-medium text-foreground transition-colors hover:bg-muted disabled:cursor-not-allowed disabled:opacity-50';
|
||||
|
||||
export default function DownloadCaptions({
|
||||
fileNames,
|
||||
captions,
|
||||
}: {
|
||||
fileNames: string[];
|
||||
captions: Record<string, string>;
|
||||
}) {
|
||||
const disabled = fileNames.length === 0;
|
||||
|
||||
// Sorted lazily in the click handlers: this component re-renders with the
|
||||
// sidebar on every caption keystroke, and the sort is only needed on click.
|
||||
const sortedFileNames = () => [...fileNames].sort();
|
||||
|
||||
function handleDownloadJson() {
|
||||
const data = sortedFileNames().map((path) => ({
|
||||
path,
|
||||
cap: captions[path] ?? '',
|
||||
}));
|
||||
const blob = new Blob([JSON.stringify(data, null, 2)], {
|
||||
type: 'application/json',
|
||||
});
|
||||
downloadBlob(blob, 'videos2caption.json');
|
||||
}
|
||||
|
||||
function handleDownloadTxt() {
|
||||
const sortedNames = sortedFileNames();
|
||||
const videosContent = sortedNames.join('\n');
|
||||
const promptContent = sortedNames
|
||||
.map((fn) => captions[fn] ?? '')
|
||||
.join('\n');
|
||||
downloadBlob(
|
||||
new Blob([videosContent], { type: 'text/plain' }),
|
||||
'videos.txt',
|
||||
);
|
||||
setTimeout(() => {
|
||||
downloadBlob(
|
||||
new Blob([promptContent], { type: 'text/plain' }),
|
||||
'captions.txt',
|
||||
);
|
||||
}, 100);
|
||||
}
|
||||
|
||||
function handleDownloadCsv() {
|
||||
const escape = (s: string) =>
|
||||
s.includes('"') || s.includes(',') || s.includes('\n')
|
||||
? `"${s.replace(/"/g, '""')}"`
|
||||
: s;
|
||||
const rows = sortedFileNames().map(
|
||||
(fn) => `${escape(fn)},${escape(captions[fn] ?? '')}`,
|
||||
);
|
||||
const csv = ['video_name,caption', ...rows].join('\n');
|
||||
downloadBlob(new Blob([csv], { type: 'text/csv' }), 'captions.csv');
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="group relative inline-block">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
disabled={disabled}
|
||||
aria-haspopup="menu"
|
||||
className="gap-1.5"
|
||||
>
|
||||
Download Captions
|
||||
<ChevronDown className="h-3.5 w-3.5 opacity-85" />
|
||||
</Button>
|
||||
{!disabled && (
|
||||
<div
|
||||
role="menu"
|
||||
className="invisible absolute right-0 top-full z-[200] min-w-full -translate-y-1 pt-1 opacity-0 transition-all group-focus-within:visible group-focus-within:translate-y-0 group-focus-within:opacity-100 group-hover:visible group-hover:translate-y-0 group-hover:opacity-100"
|
||||
>
|
||||
<div className="overflow-hidden rounded-lg border border-border bg-popover py-1 shadow-lg">
|
||||
<button
|
||||
type="button"
|
||||
role="menuitem"
|
||||
className={MENU_ITEM}
|
||||
onClick={handleDownloadJson}
|
||||
>
|
||||
JSON
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
role="menuitem"
|
||||
className={MENU_ITEM}
|
||||
onClick={handleDownloadTxt}
|
||||
>
|
||||
TXT
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
role="menuitem"
|
||||
className={MENU_ITEM}
|
||||
onClick={handleDownloadCsv}
|
||||
>
|
||||
CSV
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,205 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface UploadZoneProps {
|
||||
label: string;
|
||||
hint?: string;
|
||||
accept?: string;
|
||||
multiple?: boolean;
|
||||
directory?: boolean;
|
||||
allowBothFileAndDirectory?: boolean;
|
||||
value?: string;
|
||||
fileName?: string;
|
||||
onFiles?: (files: File[]) => void;
|
||||
onClear?: () => void;
|
||||
disabled?: boolean;
|
||||
uploading?: boolean;
|
||||
}
|
||||
|
||||
const linkButtonClass =
|
||||
'cursor-pointer bg-transparent p-0 text-accent-blue hover:underline disabled:cursor-not-allowed disabled:no-underline disabled:opacity-50';
|
||||
|
||||
export default function UploadZone({
|
||||
label,
|
||||
hint,
|
||||
accept,
|
||||
multiple = false,
|
||||
directory = false,
|
||||
allowBothFileAndDirectory = false,
|
||||
value = '',
|
||||
fileName,
|
||||
onFiles,
|
||||
onClear,
|
||||
disabled = false,
|
||||
uploading = false,
|
||||
}: UploadZoneProps) {
|
||||
const fileInputRef = React.useRef<HTMLInputElement>(null);
|
||||
const directoryInputRef = React.useRef<HTMLInputElement>(null);
|
||||
|
||||
const useBoth = directory && allowBothFileAndDirectory;
|
||||
const clickable = !useBoth;
|
||||
const hasContent = !!(value || fileName);
|
||||
|
||||
// `webkitdirectory` is a DOM property without a typed JSX prop, so set it
|
||||
// imperatively. The primary input only selects folders when it is the sole
|
||||
// input; the dedicated directory input always does.
|
||||
React.useEffect(() => {
|
||||
if (fileInputRef.current) {
|
||||
fileInputRef.current.webkitdirectory = directory && !allowBothFileAndDirectory;
|
||||
}
|
||||
if (directoryInputRef.current) {
|
||||
directoryInputRef.current.webkitdirectory = true;
|
||||
}
|
||||
}, [directory, allowBothFileAndDirectory]);
|
||||
|
||||
function handleChange(e: React.ChangeEvent<HTMLInputElement>) {
|
||||
const files = e.target.files;
|
||||
if (files && files.length > 0) {
|
||||
onFiles?.(Array.from(files));
|
||||
}
|
||||
e.target.value = '';
|
||||
}
|
||||
|
||||
function handleClick() {
|
||||
if (!disabled) {
|
||||
fileInputRef.current?.click();
|
||||
}
|
||||
}
|
||||
|
||||
function handleKeyActivate(e: React.KeyboardEvent) {
|
||||
if ((e.target as HTMLElement).closest('button')) return;
|
||||
if (e.key === 'Enter' || e.key === ' ') {
|
||||
e.preventDefault();
|
||||
handleClick();
|
||||
}
|
||||
}
|
||||
|
||||
function handleDrop(e: React.DragEvent) {
|
||||
if (disabled) return;
|
||||
e.preventDefault();
|
||||
const files = e.dataTransfer.files;
|
||||
if (files && files.length > 0) {
|
||||
onFiles?.(Array.from(files));
|
||||
}
|
||||
}
|
||||
|
||||
function handleDragOver(e: React.DragEvent) {
|
||||
if (disabled) return;
|
||||
e.preventDefault();
|
||||
}
|
||||
|
||||
function clearInputs() {
|
||||
if (fileInputRef.current) fileInputRef.current.value = '';
|
||||
if (directoryInputRef.current) directoryInputRef.current.value = '';
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
className={cn(
|
||||
'flex min-h-[150px] grow flex-col items-center justify-center rounded-lg border-2 border-dashed border-border bg-muted/40 px-5 py-6 text-center transition-colors hover:border-accent-blue hover:bg-muted/60',
|
||||
clickable ? 'cursor-pointer' : 'cursor-default',
|
||||
hasContent && 'border-solid border-accent-blue',
|
||||
)}
|
||||
onClick={clickable ? handleClick : undefined}
|
||||
onKeyDown={clickable ? handleKeyActivate : undefined}
|
||||
onDragOver={handleDragOver}
|
||||
onDrop={handleDrop}
|
||||
role={clickable ? 'button' : undefined}
|
||||
tabIndex={clickable ? 0 : undefined}
|
||||
>
|
||||
<input
|
||||
ref={fileInputRef}
|
||||
type="file"
|
||||
className="hidden"
|
||||
accept={accept}
|
||||
multiple={multiple}
|
||||
onChange={handleChange}
|
||||
disabled={disabled}
|
||||
/>
|
||||
{useBoth && (
|
||||
<input
|
||||
ref={directoryInputRef}
|
||||
type="file"
|
||||
className="hidden"
|
||||
multiple
|
||||
onChange={handleChange}
|
||||
disabled={disabled}
|
||||
/>
|
||||
)}
|
||||
|
||||
<div className="mb-2 text-sm text-muted-foreground">{label}</div>
|
||||
|
||||
{!hasContent && (
|
||||
<span className="mt-1.5 text-xs text-muted-foreground">
|
||||
{uploading ? (
|
||||
'Uploading…'
|
||||
) : useBoth ? (
|
||||
<>
|
||||
<span
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
className="cursor-pointer text-accent-blue hover:underline"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
handleClick();
|
||||
}}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter' || e.key === ' ') {
|
||||
e.preventDefault();
|
||||
handleClick();
|
||||
}
|
||||
}}
|
||||
>
|
||||
Select files
|
||||
</span>
|
||||
{' · '}
|
||||
<button
|
||||
type="button"
|
||||
className={linkButtonClass}
|
||||
onClick={(e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (!disabled) directoryInputRef.current?.click();
|
||||
}}
|
||||
disabled={disabled}
|
||||
>
|
||||
Select folder
|
||||
</button>
|
||||
</>
|
||||
) : directory ? (
|
||||
'Click or drop folder'
|
||||
) : (
|
||||
'Click or drop file(s)'
|
||||
)}
|
||||
</span>
|
||||
)}
|
||||
{fileName && (
|
||||
<div className="mt-2 text-sm text-foreground">
|
||||
{fileName}
|
||||
{onClear && (
|
||||
<>
|
||||
{' · '}
|
||||
<button
|
||||
type="button"
|
||||
className={linkButtonClass}
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
onClear();
|
||||
clearInputs();
|
||||
}}
|
||||
disabled={disabled || uploading}
|
||||
>
|
||||
Clear
|
||||
</button>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{hint && <div className="mt-1.5 text-xs text-muted-foreground">{hint}</div>}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { Label } from '@/components/ui/label';
|
||||
import { Slider } from '@/components/ui/slider';
|
||||
import { Switch } from '@/components/ui/switch';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
/**
|
||||
* Labeled form-field rows shared by the Settings page and the Create Job
|
||||
* modal. They mirror the Svelte `Toggle`/`Slider` UX on top of the shadcn
|
||||
* `Switch`/`Slider`/`Input` primitives.
|
||||
*/
|
||||
|
||||
export function FieldRow({
|
||||
htmlFor,
|
||||
label,
|
||||
title,
|
||||
className,
|
||||
children,
|
||||
}: {
|
||||
htmlFor: string;
|
||||
label: string;
|
||||
title?: string;
|
||||
className?: string;
|
||||
children: React.ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<div className={cn('flex flex-col gap-1.5', className)}>
|
||||
<Label
|
||||
htmlFor={htmlFor}
|
||||
title={title}
|
||||
className="pl-0.5 text-xs font-normal tracking-wide text-muted-foreground"
|
||||
>
|
||||
{label}
|
||||
</Label>
|
||||
{children}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function SliderRow({
|
||||
id,
|
||||
label,
|
||||
title,
|
||||
min,
|
||||
max,
|
||||
step,
|
||||
value,
|
||||
onChange,
|
||||
disabled,
|
||||
format = (v) => String(v),
|
||||
}: {
|
||||
id: string;
|
||||
label: string;
|
||||
title?: string;
|
||||
min: number;
|
||||
max: number;
|
||||
step: number;
|
||||
value: number;
|
||||
/** Called once per gesture (pointer release / key press), not per drag tick. */
|
||||
onChange: (v: number) => void;
|
||||
disabled?: boolean;
|
||||
format?: (v: number) => string;
|
||||
}) {
|
||||
// Track the in-progress drag locally so `onChange` only fires on commit;
|
||||
// any external `value` change (commit landing, reset button) takes over.
|
||||
const [dragValue, setDragValue] = React.useState<number | null>(null);
|
||||
React.useEffect(() => {
|
||||
setDragValue(null);
|
||||
}, [value]);
|
||||
const shown = dragValue ?? value;
|
||||
return (
|
||||
<FieldRow htmlFor={id} label={label} title={title}>
|
||||
<div className="flex items-center gap-2">
|
||||
<Slider
|
||||
id={id}
|
||||
min={min}
|
||||
max={max}
|
||||
step={step}
|
||||
value={[shown]}
|
||||
onValueChange={(v) => setDragValue(v[0])}
|
||||
onValueCommit={(v) => onChange(v[0])}
|
||||
disabled={disabled}
|
||||
aria-label={label}
|
||||
className="min-w-0 flex-1"
|
||||
/>
|
||||
<span
|
||||
aria-hidden="true"
|
||||
className="min-w-10 shrink-0 text-right text-sm tabular-nums text-muted-foreground"
|
||||
>
|
||||
{format(shown)}
|
||||
</span>
|
||||
</div>
|
||||
</FieldRow>
|
||||
);
|
||||
}
|
||||
|
||||
export function ToggleRow({
|
||||
id,
|
||||
label,
|
||||
title,
|
||||
checked,
|
||||
onChange,
|
||||
disabled,
|
||||
}: {
|
||||
id: string;
|
||||
label: string;
|
||||
title?: string;
|
||||
checked: boolean;
|
||||
onChange: (v: boolean) => void;
|
||||
disabled?: boolean;
|
||||
}) {
|
||||
return (
|
||||
<FieldRow htmlFor={id} label={label} title={title}>
|
||||
<Switch
|
||||
id={id}
|
||||
checked={checked}
|
||||
onCheckedChange={onChange}
|
||||
disabled={disabled}
|
||||
/>
|
||||
</FieldRow>
|
||||
);
|
||||
}
|
||||
|
||||
export function NumberRow({
|
||||
id,
|
||||
label,
|
||||
title,
|
||||
min,
|
||||
max,
|
||||
step,
|
||||
value,
|
||||
onChange,
|
||||
disabled,
|
||||
}: {
|
||||
id: string;
|
||||
label: string;
|
||||
title?: string;
|
||||
min?: number;
|
||||
max?: number;
|
||||
step?: number | string;
|
||||
value: number;
|
||||
onChange: (v: number) => void;
|
||||
disabled?: boolean;
|
||||
}) {
|
||||
// Buffer the raw text so the field can be emptied while retyping; only
|
||||
// valid numbers are committed, and blur restores the last committed value.
|
||||
const [draft, setDraft] = React.useState<string | null>(null);
|
||||
React.useEffect(() => {
|
||||
setDraft(null);
|
||||
}, [value]);
|
||||
return (
|
||||
<FieldRow htmlFor={id} label={label} title={title}>
|
||||
<Input
|
||||
id={id}
|
||||
type="number"
|
||||
min={min}
|
||||
max={max}
|
||||
step={step}
|
||||
value={draft ?? value}
|
||||
onChange={(e) => {
|
||||
setDraft(e.target.value);
|
||||
const v = e.target.valueAsNumber;
|
||||
if (!Number.isNaN(v)) onChange(v);
|
||||
}}
|
||||
onBlur={() => setDraft(null)}
|
||||
disabled={disabled}
|
||||
/>
|
||||
</FieldRow>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { ChevronDown } from 'lucide-react';
|
||||
|
||||
import CreateJobModal from '@/components/jobs/CreateJobModal';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { WORKLOAD_OPTIONS } from '@/lib/jobConfig';
|
||||
import type { JobType } from '@/lib/types';
|
||||
import { triggerRefresh } from '@/stores/jobsRefresh';
|
||||
|
||||
interface CreateJobButtonProps {
|
||||
jobType: JobType;
|
||||
}
|
||||
|
||||
export default function CreateJobButton({ jobType }: CreateJobButtonProps) {
|
||||
const options = WORKLOAD_OPTIONS[jobType] ?? [];
|
||||
|
||||
const [modalOpen, setModalOpen] = React.useState(false);
|
||||
const [workloadType, setWorkloadType] = React.useState(
|
||||
options[0]?.type ?? 't2v',
|
||||
);
|
||||
|
||||
function openModal(type: string) {
|
||||
setWorkloadType(type);
|
||||
setModalOpen(true);
|
||||
}
|
||||
|
||||
function handleSuccess() {
|
||||
triggerRefresh();
|
||||
setModalOpen(false);
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<div className="group relative inline-block">
|
||||
<Button type="button" className="gap-1.5">
|
||||
Create Job
|
||||
<ChevronDown className="size-3.5 opacity-85" aria-hidden />
|
||||
</Button>
|
||||
<div
|
||||
role="menu"
|
||||
className="invisible absolute right-0 top-full z-[200] mt-1 min-w-full -translate-y-1 rounded-lg border border-border bg-popover py-1 opacity-0 shadow-lg transition-all duration-150 group-hover:visible group-hover:translate-y-0 group-hover:opacity-100"
|
||||
>
|
||||
{options.map((opt) => (
|
||||
<button
|
||||
key={opt.type}
|
||||
type="button"
|
||||
role="menuitem"
|
||||
onClick={() => openModal(opt.type)}
|
||||
className="block w-full whitespace-nowrap px-4 py-2 text-left text-sm font-medium text-popover-foreground transition-colors hover:bg-secondary"
|
||||
>
|
||||
{opt.label}
|
||||
<span className="mt-0.5 block text-xs font-normal text-muted-foreground">
|
||||
{opt.desc}
|
||||
</span>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
<CreateJobModal
|
||||
isOpen={modalOpen}
|
||||
onClose={() => setModalOpen(false)}
|
||||
onSuccess={handleSuccess}
|
||||
jobType={jobType}
|
||||
workloadType={workloadType}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,158 @@
|
||||
import * as React from 'react';
|
||||
import { render, screen, waitFor, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import CreateJobModal from './CreateJobModal';
|
||||
import { createJob, getDatasets, getModels, uploadImage } from '@/lib/api';
|
||||
import { defaultOptionsStore } from '@/stores/defaultOptions';
|
||||
import { DEFAULT_OPTIONS } from '@/lib/defaultOptions';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
createJob: vi.fn(),
|
||||
getModels: vi.fn(),
|
||||
getDatasets: vi.fn(),
|
||||
uploadImage: vi.fn(),
|
||||
getSettings: vi.fn(),
|
||||
updateSettings: vi.fn(),
|
||||
}));
|
||||
|
||||
const MODELS = [
|
||||
{ id: 'wan/t2v-1.3b', label: 'Wan T2V', type: 't2v' },
|
||||
{ id: 'wan/t2v-14b', label: 'Wan T2V Large', type: 't2v' },
|
||||
];
|
||||
|
||||
beforeEach(() => {
|
||||
// Reset the shared options store to a known baseline for test isolation.
|
||||
defaultOptionsStore.set({ options: DEFAULT_OPTIONS });
|
||||
vi.mocked(getModels).mockResolvedValue(MODELS);
|
||||
vi.mocked(getDatasets).mockResolvedValue([]);
|
||||
vi.mocked(uploadImage).mockResolvedValue({ path: '/uploads/x.png' });
|
||||
vi.mocked(createJob).mockResolvedValue({ id: 'job-1' } as never);
|
||||
});
|
||||
|
||||
function renderModal(
|
||||
overrides: Partial<React.ComponentProps<typeof CreateJobModal>> = {},
|
||||
) {
|
||||
const onClose = vi.fn();
|
||||
const onSuccess = vi.fn();
|
||||
render(
|
||||
<CreateJobModal
|
||||
isOpen
|
||||
onClose={onClose}
|
||||
onSuccess={onSuccess}
|
||||
jobType="inference"
|
||||
workloadType="t2v"
|
||||
{...overrides}
|
||||
/>,
|
||||
);
|
||||
return { onClose, onSuccess };
|
||||
}
|
||||
|
||||
describe('CreateJobModal', () => {
|
||||
it('renders the form fields for an inference job', async () => {
|
||||
renderModal();
|
||||
|
||||
expect(
|
||||
await screen.findByText('New Inference Job (T2V)'),
|
||||
).toBeInTheDocument();
|
||||
expect(screen.getByLabelText('Model')).toBeInTheDocument();
|
||||
expect(screen.getByLabelText('Prompt')).toBeInTheDocument();
|
||||
expect(screen.getByLabelText('Negative Prompt')).toBeInTheDocument();
|
||||
expect(
|
||||
screen.getByRole('button', { name: 'Create Job' }),
|
||||
).toBeInTheDocument();
|
||||
|
||||
// The model dropdown is populated once getModels resolves.
|
||||
expect(
|
||||
await screen.findByRole('option', {
|
||||
name: 'Wan T2V (wan/t2v-1.3b)',
|
||||
}),
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('seeds fields from the options store and submits an inference payload', async () => {
|
||||
// Non-default store values prove the open-time seeding effect ran (the
|
||||
// useState defaults are 50 / 480).
|
||||
defaultOptionsStore.set({
|
||||
options: { ...DEFAULT_OPTIONS, numInferenceSteps: 25, height: 720 },
|
||||
});
|
||||
|
||||
const user = userEvent.setup();
|
||||
const { onClose, onSuccess } = renderModal();
|
||||
|
||||
// Wait for models to load so the default model is selected.
|
||||
await screen.findByRole('option', { name: 'Wan T2V (wan/t2v-1.3b)' });
|
||||
|
||||
await user.type(
|
||||
screen.getByLabelText('Prompt'),
|
||||
'a raccoon in sunflowers',
|
||||
);
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
|
||||
await waitFor(() => expect(createJob).toHaveBeenCalledTimes(1));
|
||||
const payload = vi.mocked(createJob).mock.calls[0][0];
|
||||
expect(payload).toMatchObject({
|
||||
model_id: 'wan/t2v-1.3b',
|
||||
prompt: 'a raccoon in sunflowers',
|
||||
workload_type: 't2v',
|
||||
job_type: 'inference',
|
||||
num_inference_steps: 25,
|
||||
height: 720,
|
||||
num_frames: 81,
|
||||
width: 832,
|
||||
guidance_scale: 5,
|
||||
seed: 1024,
|
||||
});
|
||||
|
||||
await waitFor(() => expect(onSuccess).toHaveBeenCalledTimes(1));
|
||||
expect(onClose).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('submits a dmd_t2v distillation payload including the DMD fields', async () => {
|
||||
vi.mocked(getDatasets).mockResolvedValue([
|
||||
{ id: 'ds1', name: 'My Dataset', created_at: 0 },
|
||||
]);
|
||||
|
||||
const user = userEvent.setup();
|
||||
const { onSuccess } = renderModal({
|
||||
jobType: 'distillation',
|
||||
workloadType: 'dmd_t2v',
|
||||
});
|
||||
|
||||
// Models + datasets load asynchronously on open. The model/dataset option
|
||||
// labels also appear in the Real/Fake Score Model and Validation Dataset
|
||||
// selects, so scope each wait to the relevant select.
|
||||
await within(screen.getByLabelText('Model')).findByRole('option', {
|
||||
name: 'Wan T2V (wan/t2v-1.3b)',
|
||||
});
|
||||
const datasetSelect = screen.getByLabelText('Dataset *');
|
||||
await within(datasetSelect).findByRole('option', { name: 'My Dataset' });
|
||||
|
||||
await user.type(screen.getByLabelText('Description'), 'distill run');
|
||||
await user.selectOptions(datasetSelect, 'ds1');
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
|
||||
await waitFor(() => expect(createJob).toHaveBeenCalledTimes(1));
|
||||
const payload = vi.mocked(createJob).mock.calls[0][0];
|
||||
expect(payload).toMatchObject({
|
||||
workload_type: 'dmd_t2v',
|
||||
job_type: 'distillation',
|
||||
// The dataset id is sent; the backend resolves it to the on-disk dir.
|
||||
data_path: 'ds1',
|
||||
lora_rank: 32,
|
||||
// DMD-specific fields added to CreateJobRequest for this modal.
|
||||
dmd_use_vsa: false,
|
||||
dmd_vsa_sparsity: 0.8,
|
||||
dmd_denoising_steps: '1000,757,522',
|
||||
real_score_guidance_scale: 3.5,
|
||||
generator_update_interval: 5,
|
||||
real_score_model_path: 'wan/t2v-1.3b',
|
||||
fake_score_model_path: 'wan/t2v-1.3b',
|
||||
});
|
||||
// Inference-only keys must be absent for a training job.
|
||||
expect(payload).not.toHaveProperty('num_inference_steps');
|
||||
|
||||
await waitFor(() => expect(onSuccess).toHaveBeenCalledTimes(1));
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,823 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import {
|
||||
FieldRow,
|
||||
NumberRow,
|
||||
SliderRow,
|
||||
ToggleRow,
|
||||
} from '@/components/form-rows';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { useStore } from '@/hooks/useStore';
|
||||
import { defaultOptionsStore } from '@/stores/defaultOptions';
|
||||
import {
|
||||
createJob,
|
||||
getDatasets,
|
||||
getModels,
|
||||
uploadImage,
|
||||
type CreateJobRequest,
|
||||
type Model,
|
||||
} from '@/lib/api';
|
||||
import { getDefaultModelForWorkload } from '@/lib/defaultOptions';
|
||||
import { WORKLOAD_OPTIONS } from '@/lib/jobConfig';
|
||||
import type { JobType } from '@/lib/types';
|
||||
|
||||
export interface CreateJobModalProps {
|
||||
isOpen: boolean;
|
||||
onClose: () => void;
|
||||
onSuccess: () => void;
|
||||
jobType: JobType;
|
||||
workloadType: string;
|
||||
}
|
||||
|
||||
export default function CreateJobModal({
|
||||
isOpen,
|
||||
onClose,
|
||||
onSuccess,
|
||||
jobType,
|
||||
workloadType,
|
||||
}: CreateJobModalProps) {
|
||||
const { options } = useStore(defaultOptionsStore);
|
||||
|
||||
const isInference = jobType === 'inference';
|
||||
// Training jobs always pick from the t2v model catalogue.
|
||||
const inferenceWorkload = isInference ? workloadType : 't2v';
|
||||
|
||||
const [models, setModels] = React.useState<Model[]>([]);
|
||||
const [modelId, setModelId] = React.useState('');
|
||||
const [prompt, setPrompt] = React.useState('');
|
||||
const [imagePath, setImagePath] = React.useState('');
|
||||
const [imageFileName, setImageFileName] = React.useState('');
|
||||
const [isUploadingImage, setIsUploadingImage] = React.useState(false);
|
||||
const [negativePrompt, setNegativePrompt] = React.useState('');
|
||||
const [numInferenceSteps, setNumInferenceSteps] = React.useState(50);
|
||||
const [numFrames, setNumFrames] = React.useState(81);
|
||||
const [height, setHeight] = React.useState(480);
|
||||
const [width, setWidth] = React.useState(832);
|
||||
const [guidanceScale, setGuidanceScale] = React.useState(5);
|
||||
const [guidanceRescale, setGuidanceRescale] = React.useState(0);
|
||||
const [fps, setFps] = React.useState(24);
|
||||
const [seed, setSeed] = React.useState(1024);
|
||||
const [numGpus, setNumGpus] = React.useState(1);
|
||||
const [ditCpuOffload, setDitCpuOffload] = React.useState(false);
|
||||
const [textEncoderCpuOffload, setTextEncoderCpuOffload] =
|
||||
React.useState(false);
|
||||
const [vaeCpuOffload, setVaeCpuOffload] = React.useState(false);
|
||||
const [imageEncoderCpuOffload, setImageEncoderCpuOffload] =
|
||||
React.useState(false);
|
||||
const [useFsdpInference, setUseFsdpInference] = React.useState(false);
|
||||
const [enableTorchCompile, setEnableTorchCompile] = React.useState(false);
|
||||
const [vsaSparsity, setVsaSparsity] = React.useState(0);
|
||||
const [tpSize, setTpSize] = React.useState(-1);
|
||||
const [spSize, setSpSize] = React.useState(-1);
|
||||
const [selectedDatasetId, setSelectedDatasetId] = React.useState('');
|
||||
const [readyDatasets, setReadyDatasets] = React.useState<
|
||||
Awaited<ReturnType<typeof getDatasets>>
|
||||
>([]);
|
||||
const [maxTrainSteps, setMaxTrainSteps] = React.useState(1000);
|
||||
const [trainBatchSize, setTrainBatchSize] = React.useState(1);
|
||||
const [learningRate, setLearningRate] = React.useState(5e-5);
|
||||
const [numLatentT, setNumLatentT] = React.useState(20);
|
||||
const [selectedValidationDatasetId, setSelectedValidationDatasetId] =
|
||||
React.useState('');
|
||||
const [loraRank, setLoraRank] = React.useState(32);
|
||||
const [dmdUseVsa, setDmdUseVsa] = React.useState(false);
|
||||
const [dmdVsaSparsity, setDmdVsaSparsity] = React.useState(0.8);
|
||||
const [dmdDenoisingSteps, setDmdDenoisingSteps] =
|
||||
React.useState('1000,757,522');
|
||||
const [realScoreGuidanceScale, setRealScoreGuidanceScale] =
|
||||
React.useState(3.5);
|
||||
const [generatorUpdateInterval, setGeneratorUpdateInterval] =
|
||||
React.useState(5);
|
||||
const [realScoreModelPath, setRealScoreModelPath] = React.useState('');
|
||||
const [fakeScoreModelPath, setFakeScoreModelPath] = React.useState('');
|
||||
const [isSubmitting, setIsSubmitting] = React.useState(false);
|
||||
const [isLoadingModels, setIsLoadingModels] = React.useState(false);
|
||||
const imageInputRef = React.useRef<HTMLInputElement>(null);
|
||||
|
||||
// Seed field values from the persisted default options each time the modal
|
||||
// OPENS. A naive port of the Svelte `$effect` would re-seed on every
|
||||
// `$defaultOptions` change; we deliberately seed only on the open transition
|
||||
// so a late `initDefaultOptions()` settings refresh can't clobber the user's
|
||||
// in-progress edits or desync the (already validated) model selection.
|
||||
const justOpenedRef = React.useRef(false);
|
||||
React.useEffect(() => {
|
||||
const justOpened = isOpen && !justOpenedRef.current;
|
||||
justOpenedRef.current = isOpen;
|
||||
if (!justOpened) return;
|
||||
const opts = options;
|
||||
setNumInferenceSteps(opts.numInferenceSteps);
|
||||
setNumFrames(workloadType === 't2i' ? 1 : opts.numFrames);
|
||||
setHeight(opts.height);
|
||||
setWidth(opts.width);
|
||||
setGuidanceScale(opts.guidanceScale);
|
||||
setGuidanceRescale(opts.guidanceRescale);
|
||||
setFps(opts.fps);
|
||||
setSeed(opts.seed);
|
||||
setNumGpus(opts.numGpus);
|
||||
setDitCpuOffload(opts.ditCpuOffload);
|
||||
setTextEncoderCpuOffload(opts.textEncoderCpuOffload);
|
||||
setVaeCpuOffload(opts.vaeCpuOffload);
|
||||
setImageEncoderCpuOffload(opts.imageEncoderCpuOffload);
|
||||
setUseFsdpInference(opts.useFsdpInference);
|
||||
setEnableTorchCompile(opts.enableTorchCompile);
|
||||
setVsaSparsity(opts.vsaSparsity);
|
||||
setTpSize(opts.tpSize);
|
||||
setSpSize(opts.spSize);
|
||||
setModelId(
|
||||
getDefaultModelForWorkload(
|
||||
opts,
|
||||
inferenceWorkload as 't2v' | 'i2v' | 't2i',
|
||||
),
|
||||
);
|
||||
setImagePath('');
|
||||
setImageFileName('');
|
||||
setSelectedDatasetId('');
|
||||
setSelectedValidationDatasetId('');
|
||||
if (workloadType === 'dmd_t2v') {
|
||||
setDmdUseVsa(false);
|
||||
setDmdVsaSparsity(0.8);
|
||||
setDmdDenoisingSteps('1000,757,522');
|
||||
setRealScoreGuidanceScale(3.5);
|
||||
setGeneratorUpdateInterval(5);
|
||||
setRealScoreModelPath('');
|
||||
setFakeScoreModelPath('');
|
||||
}
|
||||
}, [isOpen, workloadType, inferenceWorkload, options]);
|
||||
|
||||
// Load the models available for this workload.
|
||||
React.useEffect(() => {
|
||||
if (!isOpen) return;
|
||||
// Ignore a superseded response so a slow fetch for a previous workload
|
||||
// can't overwrite the current workload's model list/selection.
|
||||
let stale = false;
|
||||
setIsLoadingModels(true);
|
||||
getModels(inferenceWorkload)
|
||||
.then((list) => {
|
||||
if (stale) return;
|
||||
setModels(list);
|
||||
const ids = list.map((m) => m.id);
|
||||
const opts = defaultOptionsStore.get().options;
|
||||
const defaultId = getDefaultModelForWorkload(
|
||||
opts,
|
||||
inferenceWorkload as 't2v' | 'i2v' | 't2i',
|
||||
);
|
||||
const chosen = ids.includes(defaultId) ? defaultId : (list[0]?.id ?? '');
|
||||
setModelId(chosen);
|
||||
if (workloadType === 'dmd_t2v') {
|
||||
setRealScoreModelPath(chosen);
|
||||
setFakeScoreModelPath(chosen);
|
||||
}
|
||||
})
|
||||
.catch((e) => {
|
||||
if (!stale) console.error('Failed to load models:', e);
|
||||
})
|
||||
.finally(() => {
|
||||
if (!stale) setIsLoadingModels(false);
|
||||
});
|
||||
return () => {
|
||||
stale = true;
|
||||
};
|
||||
}, [isOpen, inferenceWorkload, workloadType]);
|
||||
|
||||
// Training jobs need a dataset; load the ready datasets when relevant.
|
||||
React.useEffect(() => {
|
||||
if (isOpen && !isInference) {
|
||||
getDatasets()
|
||||
.then(setReadyDatasets)
|
||||
.catch(() => setReadyDatasets([]));
|
||||
} else {
|
||||
setReadyDatasets([]);
|
||||
}
|
||||
}, [isOpen, isInference]);
|
||||
|
||||
async function handleImageChange(e: React.ChangeEvent<HTMLInputElement>) {
|
||||
const file = e.target.files?.[0];
|
||||
if (!file) {
|
||||
setImagePath('');
|
||||
setImageFileName('');
|
||||
return;
|
||||
}
|
||||
setIsUploadingImage(true);
|
||||
setImageFileName(file.name);
|
||||
try {
|
||||
const { path } = await uploadImage(file);
|
||||
setImagePath(path);
|
||||
} catch {
|
||||
setImagePath('');
|
||||
setImageFileName('');
|
||||
} finally {
|
||||
setIsUploadingImage(false);
|
||||
}
|
||||
}
|
||||
|
||||
function clearImage() {
|
||||
setImagePath('');
|
||||
setImageFileName('');
|
||||
if (imageInputRef.current) imageInputRef.current.value = '';
|
||||
}
|
||||
|
||||
async function handleSubmit(e: React.FormEvent<HTMLFormElement>) {
|
||||
e.preventDefault();
|
||||
if (isInference && workloadType === 'i2v' && !imagePath) return;
|
||||
// Send the dataset id; the backend resolves it to the on-disk media dir.
|
||||
const effectiveDataPath = selectedDatasetId ?? '';
|
||||
if (!isInference && !selectedDatasetId) return;
|
||||
// `lora_t2v` jobs are persisted with a dedicated backend job_type that the
|
||||
// front-end JobType enum does not model; cast to keep payload parity.
|
||||
const effectiveJobType = (
|
||||
workloadType === 'lora_t2v' ? 'lora' : jobType
|
||||
) as JobType;
|
||||
setIsSubmitting(true);
|
||||
try {
|
||||
const payload: CreateJobRequest = {
|
||||
model_id: modelId,
|
||||
prompt,
|
||||
workload_type: workloadType,
|
||||
job_type: effectiveJobType,
|
||||
...(isInference
|
||||
? {
|
||||
...(workloadType === 'i2v' && imagePath
|
||||
? { image_path: imagePath }
|
||||
: {}),
|
||||
negative_prompt: negativePrompt,
|
||||
num_inference_steps: numInferenceSteps,
|
||||
num_frames: numFrames,
|
||||
height,
|
||||
width,
|
||||
guidance_scale: guidanceScale,
|
||||
guidance_rescale: guidanceRescale,
|
||||
fps,
|
||||
seed,
|
||||
num_gpus: numGpus,
|
||||
dit_cpu_offload: ditCpuOffload,
|
||||
text_encoder_cpu_offload: textEncoderCpuOffload,
|
||||
vae_cpu_offload: vaeCpuOffload,
|
||||
image_encoder_cpu_offload: imageEncoderCpuOffload,
|
||||
use_fsdp_inference: useFsdpInference,
|
||||
enable_torch_compile: enableTorchCompile,
|
||||
vsa_sparsity: vsaSparsity,
|
||||
tp_size: tpSize,
|
||||
sp_size: spSize,
|
||||
}
|
||||
: {
|
||||
data_path: effectiveDataPath.trim(),
|
||||
max_train_steps: maxTrainSteps,
|
||||
train_batch_size: trainBatchSize,
|
||||
learning_rate: learningRate,
|
||||
num_latent_t: numLatentT,
|
||||
validation_dataset_file: selectedValidationDatasetId || undefined,
|
||||
lora_rank: loraRank,
|
||||
...(workloadType === 'dmd_t2v'
|
||||
? {
|
||||
dmd_use_vsa: dmdUseVsa,
|
||||
dmd_vsa_sparsity: dmdVsaSparsity,
|
||||
dmd_denoising_steps: dmdDenoisingSteps,
|
||||
real_score_guidance_scale: realScoreGuidanceScale,
|
||||
generator_update_interval: generatorUpdateInterval,
|
||||
real_score_model_path: realScoreModelPath || modelId,
|
||||
fake_score_model_path: fakeScoreModelPath || modelId,
|
||||
}
|
||||
: {}),
|
||||
}),
|
||||
};
|
||||
await createJob(payload);
|
||||
onSuccess();
|
||||
onClose();
|
||||
} catch (err) {
|
||||
console.error('Failed to create job:', err);
|
||||
} finally {
|
||||
setIsSubmitting(false);
|
||||
}
|
||||
}
|
||||
|
||||
function handleClose() {
|
||||
if (isSubmitting) return;
|
||||
onClose();
|
||||
}
|
||||
|
||||
const workloadLabel =
|
||||
WORKLOAD_OPTIONS[jobType]?.find((o) => o.type === workloadType)?.label ?? '';
|
||||
const title = `New ${jobType.charAt(0).toUpperCase() + jobType.slice(1)} Job${
|
||||
workloadLabel ? ` (${workloadLabel})` : ''
|
||||
}`;
|
||||
|
||||
return (
|
||||
<Dialog
|
||||
open={isOpen}
|
||||
onOpenChange={(open) => {
|
||||
if (!open) handleClose();
|
||||
}}
|
||||
>
|
||||
<DialogContent
|
||||
className="max-h-[90vh] w-[90vw] max-w-[850px] overflow-y-auto"
|
||||
onEscapeKeyDown={(e) => {
|
||||
if (isSubmitting) e.preventDefault();
|
||||
}}
|
||||
onInteractOutside={(e) => {
|
||||
if (isSubmitting) e.preventDefault();
|
||||
}}
|
||||
>
|
||||
<DialogHeader>
|
||||
<DialogTitle>{title}</DialogTitle>
|
||||
</DialogHeader>
|
||||
|
||||
<form
|
||||
onSubmit={handleSubmit}
|
||||
autoComplete="off"
|
||||
className="flex flex-col gap-3.5"
|
||||
>
|
||||
<FieldRow htmlFor="modal-modelId" label="Model">
|
||||
<NativeSelect
|
||||
id="modal-modelId"
|
||||
value={modelId}
|
||||
onChange={(e) => setModelId(e.target.value)}
|
||||
required
|
||||
disabled={isSubmitting || isLoadingModels}
|
||||
>
|
||||
<option value="" disabled>
|
||||
{isLoadingModels
|
||||
? 'Loading models…'
|
||||
: models.length === 0
|
||||
? 'No models available for this workload'
|
||||
: 'Select a model…'}
|
||||
</option>
|
||||
{models.map((model) => (
|
||||
<option key={model.id} value={model.id}>
|
||||
{model.label} ({model.id})
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</FieldRow>
|
||||
|
||||
{isInference && workloadType === 'i2v' && (
|
||||
<FieldRow htmlFor="modal-image" label="Image">
|
||||
<Input
|
||||
ref={imageInputRef}
|
||||
id="modal-image"
|
||||
type="file"
|
||||
accept=".png,.jpg,.jpeg,.webp,.bmp"
|
||||
onChange={handleImageChange}
|
||||
disabled={isSubmitting || isUploadingImage}
|
||||
required
|
||||
className="h-auto py-2 file:mr-3 file:cursor-pointer file:rounded-md file:border-0 file:bg-secondary file:px-2 file:py-1 file:text-sm file:text-secondary-foreground"
|
||||
/>
|
||||
{imageFileName && (
|
||||
<span className="mt-0.5 text-xs text-muted-foreground">
|
||||
{isUploadingImage ? 'Uploading…' : imageFileName} ·{' '}
|
||||
<button
|
||||
type="button"
|
||||
onClick={clearImage}
|
||||
disabled={isSubmitting || isUploadingImage}
|
||||
className="text-accent-blue underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
Clear
|
||||
</button>
|
||||
</span>
|
||||
)}
|
||||
</FieldRow>
|
||||
)}
|
||||
|
||||
<FieldRow
|
||||
htmlFor="modal-prompt"
|
||||
label={isInference ? 'Prompt' : 'Description'}
|
||||
>
|
||||
<Textarea
|
||||
id="modal-prompt"
|
||||
value={prompt}
|
||||
onChange={(e) => setPrompt(e.target.value)}
|
||||
rows={isInference ? 3 : 2}
|
||||
placeholder={
|
||||
isInference
|
||||
? 'A curious raccoon peers through a vibrant field of yellow sunflowers…'
|
||||
: 'Brief description of this training job…'
|
||||
}
|
||||
required
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</FieldRow>
|
||||
|
||||
{isInference && (
|
||||
<FieldRow htmlFor="modal-negative-prompt" label="Negative Prompt">
|
||||
<Textarea
|
||||
id="modal-negative-prompt"
|
||||
value={negativePrompt}
|
||||
onChange={(e) => setNegativePrompt(e.target.value)}
|
||||
rows={2}
|
||||
placeholder="Optional: things to avoid in the output…"
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</FieldRow>
|
||||
)}
|
||||
|
||||
{!isInference && (
|
||||
<>
|
||||
<div className="flex gap-4">
|
||||
<FieldRow
|
||||
htmlFor="modal-dataset"
|
||||
label="Dataset *"
|
||||
className="min-w-0 flex-1"
|
||||
>
|
||||
<NativeSelect
|
||||
id="modal-dataset"
|
||||
value={selectedDatasetId}
|
||||
onChange={(e) => setSelectedDatasetId(e.target.value)}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
<option value="" disabled>
|
||||
{readyDatasets.length === 0
|
||||
? 'No datasets (add in Datasets tab)'
|
||||
: 'Select a dataset…'}
|
||||
</option>
|
||||
{readyDatasets.map((d) => (
|
||||
<option key={d.id} value={d.id}>
|
||||
{d.name}
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</FieldRow>
|
||||
<FieldRow
|
||||
htmlFor="modal-validation-dataset"
|
||||
label="Validation Dataset (optional)"
|
||||
className="min-w-0 flex-1"
|
||||
>
|
||||
<NativeSelect
|
||||
id="modal-validation-dataset"
|
||||
value={selectedValidationDatasetId}
|
||||
onChange={(e) =>
|
||||
setSelectedValidationDatasetId(e.target.value)
|
||||
}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
<option value="">None</option>
|
||||
{readyDatasets.map((d) => (
|
||||
<option key={d.id} value={d.id}>
|
||||
{d.name}
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</FieldRow>
|
||||
</div>
|
||||
|
||||
<details>
|
||||
<summary className="mb-2 cursor-pointer select-none text-sm font-medium text-accent-blue">
|
||||
Options
|
||||
</summary>
|
||||
<div className="grid grid-cols-[repeat(auto-fill,minmax(160px,1fr))] gap-x-3 gap-y-2">
|
||||
<SliderRow
|
||||
id="modal-max-train-steps"
|
||||
label="Max Train Steps"
|
||||
min={100}
|
||||
max={50000}
|
||||
step={100}
|
||||
value={maxTrainSteps}
|
||||
onChange={setMaxTrainSteps}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-train-batch-size"
|
||||
label="Train Batch Size"
|
||||
min={1}
|
||||
max={8}
|
||||
step={1}
|
||||
value={trainBatchSize}
|
||||
onChange={setTrainBatchSize}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<NumberRow
|
||||
id="modal-learning-rate"
|
||||
label="Learning Rate"
|
||||
step="1e-6"
|
||||
min={1e-6}
|
||||
max={1}
|
||||
value={learningRate}
|
||||
onChange={setLearningRate}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-num-latent-t"
|
||||
label="Num Latent T"
|
||||
min={8}
|
||||
max={40}
|
||||
step={1}
|
||||
value={numLatentT}
|
||||
onChange={setNumLatentT}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
{workloadType === 'lora_t2v' && (
|
||||
<SliderRow
|
||||
id="modal-lora-rank"
|
||||
label="LoRA Rank"
|
||||
min={8}
|
||||
max={128}
|
||||
step={8}
|
||||
value={loraRank}
|
||||
onChange={setLoraRank}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
)}
|
||||
{workloadType === 'dmd_t2v' && (
|
||||
<>
|
||||
<ToggleRow
|
||||
id="modal-dmd-use-vsa"
|
||||
label="Video Sparse Attention (VSA)"
|
||||
title="Use Video Sparse Attention for DMD"
|
||||
checked={dmdUseVsa}
|
||||
onChange={setDmdUseVsa}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
{dmdUseVsa && (
|
||||
<SliderRow
|
||||
id="modal-dmd-vsa-sparsity"
|
||||
label="VSA Sparsity"
|
||||
title="VSA sparsity (0–1)"
|
||||
min={0}
|
||||
max={1}
|
||||
step={0.05}
|
||||
value={dmdVsaSparsity}
|
||||
onChange={setDmdVsaSparsity}
|
||||
disabled={isSubmitting}
|
||||
format={(v) => v.toFixed(2)}
|
||||
/>
|
||||
)}
|
||||
<FieldRow
|
||||
htmlFor="modal-dmd-denoising-steps"
|
||||
label="DMD Denoising Steps"
|
||||
title="Comma-separated denoising steps, e.g. 1000,757,522"
|
||||
>
|
||||
<Input
|
||||
id="modal-dmd-denoising-steps"
|
||||
type="text"
|
||||
value={dmdDenoisingSteps}
|
||||
onChange={(e) => setDmdDenoisingSteps(e.target.value)}
|
||||
placeholder="1000,757,522"
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</FieldRow>
|
||||
<SliderRow
|
||||
id="modal-real-score-guidance-scale"
|
||||
label="Real Score Guidance Scale"
|
||||
min={1}
|
||||
max={10}
|
||||
step={0.1}
|
||||
value={realScoreGuidanceScale}
|
||||
onChange={setRealScoreGuidanceScale}
|
||||
disabled={isSubmitting}
|
||||
format={(v) => v.toFixed(1)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-generator-update-interval"
|
||||
label="Generator Update Interval"
|
||||
min={1}
|
||||
max={20}
|
||||
step={1}
|
||||
value={generatorUpdateInterval}
|
||||
onChange={setGeneratorUpdateInterval}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
{(
|
||||
[
|
||||
{
|
||||
id: 'modal-real-score-model',
|
||||
label: 'Real Score Model',
|
||||
value: realScoreModelPath,
|
||||
onChange: setRealScoreModelPath,
|
||||
},
|
||||
{
|
||||
id: 'modal-fake-score-model',
|
||||
label: 'Fake Score Model',
|
||||
value: fakeScoreModelPath,
|
||||
onChange: setFakeScoreModelPath,
|
||||
},
|
||||
] as const
|
||||
).map((select) => (
|
||||
<FieldRow
|
||||
key={select.id}
|
||||
htmlFor={select.id}
|
||||
label={select.label}
|
||||
>
|
||||
<NativeSelect
|
||||
id={select.id}
|
||||
value={select.value}
|
||||
onChange={(e) => select.onChange(e.target.value)}
|
||||
disabled={isSubmitting || isLoadingModels}
|
||||
>
|
||||
<option value="">Same as main model</option>
|
||||
{models.map((model) => (
|
||||
<option key={model.id} value={model.id}>
|
||||
{model.label} ({model.id})
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</FieldRow>
|
||||
))}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</details>
|
||||
</>
|
||||
)}
|
||||
|
||||
{isInference && (
|
||||
<details>
|
||||
<summary className="mb-2 cursor-pointer select-none text-sm font-medium text-accent-blue">
|
||||
Options
|
||||
</summary>
|
||||
<div className="grid grid-cols-[repeat(auto-fill,minmax(160px,1fr))] gap-x-3 gap-y-2">
|
||||
{workloadType !== 't2i' && (
|
||||
<SliderRow
|
||||
id="modal-num-frames"
|
||||
label="Frames"
|
||||
min={1}
|
||||
max={500}
|
||||
step={1}
|
||||
value={numFrames}
|
||||
onChange={setNumFrames}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
)}
|
||||
<SliderRow
|
||||
id="modal-height"
|
||||
label="Height"
|
||||
min={64}
|
||||
max={1080}
|
||||
step={16}
|
||||
value={height}
|
||||
onChange={setHeight}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-width"
|
||||
label="Width"
|
||||
min={64}
|
||||
max={1920}
|
||||
step={16}
|
||||
value={width}
|
||||
onChange={setWidth}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-num-steps"
|
||||
label="Inference Steps"
|
||||
min={1}
|
||||
max={200}
|
||||
step={1}
|
||||
value={numInferenceSteps}
|
||||
onChange={setNumInferenceSteps}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-vsa-sparsity"
|
||||
label="VSA Sparsity"
|
||||
title="VSA sparsity (0–1)"
|
||||
min={0}
|
||||
max={1}
|
||||
step={0.05}
|
||||
value={vsaSparsity}
|
||||
onChange={setVsaSparsity}
|
||||
disabled={isSubmitting}
|
||||
format={(v) => v.toFixed(2)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-guidance"
|
||||
label="Guidance Scale"
|
||||
min={0}
|
||||
max={20}
|
||||
step={0.1}
|
||||
value={guidanceScale}
|
||||
onChange={setGuidanceScale}
|
||||
disabled={isSubmitting}
|
||||
format={(v) => v.toFixed(1)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-guidance-rescale"
|
||||
label="Guidance Rescale"
|
||||
title="0 = disabled"
|
||||
min={0}
|
||||
max={1}
|
||||
step={0.05}
|
||||
value={guidanceRescale}
|
||||
onChange={setGuidanceRescale}
|
||||
disabled={isSubmitting}
|
||||
format={(v) => v.toFixed(2)}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-tp-size"
|
||||
label="TP Size"
|
||||
title="-1 = auto"
|
||||
min={-1}
|
||||
max={8}
|
||||
step={1}
|
||||
value={tpSize}
|
||||
onChange={setTpSize}
|
||||
disabled={isSubmitting}
|
||||
format={(v) => (v === -1 ? 'Auto' : String(v))}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-sp-size"
|
||||
label="SP Size"
|
||||
title="-1 = auto"
|
||||
min={-1}
|
||||
max={8}
|
||||
step={1}
|
||||
value={spSize}
|
||||
onChange={setSpSize}
|
||||
disabled={isSubmitting}
|
||||
format={(v) => (v === -1 ? 'Auto' : String(v))}
|
||||
/>
|
||||
{workloadType !== 't2i' && (
|
||||
<SliderRow
|
||||
id="modal-fps"
|
||||
label="FPS"
|
||||
min={1}
|
||||
max={60}
|
||||
step={1}
|
||||
value={fps}
|
||||
onChange={setFps}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
)}
|
||||
<ToggleRow
|
||||
id="modal-dit-cpu-offload"
|
||||
label="DiT CPU Offload"
|
||||
checked={ditCpuOffload}
|
||||
onChange={setDitCpuOffload}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="modal-text-encoder-cpu-offload"
|
||||
label="Text Encoder CPU Offload"
|
||||
checked={textEncoderCpuOffload}
|
||||
onChange={setTextEncoderCpuOffload}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="modal-use-fsdp-inference"
|
||||
label="Use FSDP Inference"
|
||||
checked={useFsdpInference}
|
||||
onChange={setUseFsdpInference}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="modal-vae-cpu-offload"
|
||||
label="VAE CPU Offload"
|
||||
checked={vaeCpuOffload}
|
||||
onChange={setVaeCpuOffload}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="modal-image-encoder-cpu-offload"
|
||||
label="Image Encoder CPU Offload"
|
||||
checked={imageEncoderCpuOffload}
|
||||
onChange={setImageEncoderCpuOffload}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="modal-enable-torch-compile"
|
||||
label="Torch Compile"
|
||||
checked={enableTorchCompile}
|
||||
onChange={setEnableTorchCompile}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<SliderRow
|
||||
id="modal-num-gpus"
|
||||
label="GPUs"
|
||||
min={1}
|
||||
max={8}
|
||||
step={1}
|
||||
value={numGpus}
|
||||
onChange={setNumGpus}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<NumberRow
|
||||
id="modal-seed"
|
||||
label="Seed"
|
||||
min={0}
|
||||
value={seed}
|
||||
onChange={setSeed}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</div>
|
||||
</details>
|
||||
)}
|
||||
|
||||
<div>
|
||||
<Button type="submit" disabled={isSubmitting}>
|
||||
{isSubmitting ? 'Creating...' : 'Create Job'}
|
||||
</Button>
|
||||
</div>
|
||||
</form>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
import { render, screen, waitFor } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import JobCard from '@/components/jobs/JobCard';
|
||||
import {
|
||||
deleteJob,
|
||||
downloadJobVideo,
|
||||
startJob,
|
||||
stopJob,
|
||||
} from '@/lib/api';
|
||||
import type { Job } from '@/lib/types';
|
||||
import { activeJobStore, setActiveJobId } from '@/stores/activeJob';
|
||||
import { makeJob as makeBaseJob } from '@/test/factories';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
startJob: vi.fn(),
|
||||
stopJob: vi.fn(),
|
||||
deleteJob: vi.fn(),
|
||||
downloadJobVideo: vi.fn(),
|
||||
}));
|
||||
|
||||
const makeJob = (overrides: Partial<Job> = {}): Job =>
|
||||
makeBaseJob({
|
||||
model_id: 'Wan2.1-T2V',
|
||||
prompt: 'a cat surfing a wave',
|
||||
created_at: 1_700_000_000,
|
||||
num_inference_steps: 50,
|
||||
num_frames: 81,
|
||||
height: 480,
|
||||
width: 832,
|
||||
guidance_scale: 5,
|
||||
seed: 42,
|
||||
num_gpus: 1,
|
||||
...overrides,
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
setActiveJobId(null);
|
||||
vi.mocked(startJob).mockResolvedValue({} as Job);
|
||||
vi.mocked(stopJob).mockResolvedValue({} as Job);
|
||||
vi.mocked(deleteJob).mockResolvedValue(undefined);
|
||||
vi.mocked(downloadJobVideo).mockResolvedValue(new Blob());
|
||||
vi.spyOn(window, 'confirm').mockReturnValue(true);
|
||||
vi.spyOn(window, 'alert').mockImplementation(() => {});
|
||||
});
|
||||
|
||||
describe('JobCard', () => {
|
||||
it('renders the model, prompt, status and inference meta', () => {
|
||||
render(<JobCard job={makeJob()} />);
|
||||
expect(screen.getByText('Wan2.1-T2V')).toBeInTheDocument();
|
||||
expect(screen.getByText('a cat surfing a wave')).toBeInTheDocument();
|
||||
expect(screen.getByText('pending')).toBeInTheDocument();
|
||||
expect(screen.getByText('81 frames')).toBeInTheDocument();
|
||||
expect(screen.getByText('480×832')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('shows the workload type (not frames) for non-inference jobs', () => {
|
||||
render(
|
||||
<JobCard
|
||||
job={makeJob({ job_type: 'finetuning', workload_type: 'lora_t2v' })}
|
||||
/>,
|
||||
);
|
||||
expect(screen.getByText('lora t2v')).toBeInTheDocument();
|
||||
expect(screen.queryByText('81 frames')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('starts a pending job and notifies the parent', async () => {
|
||||
const onJobUpdated = vi.fn();
|
||||
render(<JobCard job={makeJob({ status: 'pending' })} onJobUpdated={onJobUpdated} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Start' }));
|
||||
await waitFor(() => expect(startJob).toHaveBeenCalledWith('job-1'));
|
||||
expect(onJobUpdated).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('stops a running job', async () => {
|
||||
render(<JobCard job={makeJob({ status: 'running', started_at: Date.now() })} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Stop' }));
|
||||
await waitFor(() => expect(stopJob).toHaveBeenCalledWith('job-1'));
|
||||
});
|
||||
|
||||
it('restarts a failed job via startJob', async () => {
|
||||
render(<JobCard job={makeJob({ status: 'failed' })} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Restart' }));
|
||||
await waitFor(() => expect(startJob).toHaveBeenCalledWith('job-1'));
|
||||
});
|
||||
|
||||
it('deletes when confirmed and skips when cancelled', async () => {
|
||||
const { rerender } = render(<JobCard job={makeJob()} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Delete' }));
|
||||
await waitFor(() => expect(deleteJob).toHaveBeenCalledWith('job-1'));
|
||||
|
||||
vi.mocked(deleteJob).mockClear();
|
||||
vi.mocked(window.confirm).mockReturnValue(false);
|
||||
rerender(<JobCard job={makeJob()} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Delete' }));
|
||||
expect(deleteJob).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('downloads the video for a completed inference job', async () => {
|
||||
render(
|
||||
<JobCard
|
||||
job={makeJob({ status: 'completed', output_path: '/out/video.mp4' })}
|
||||
/>,
|
||||
);
|
||||
await userEvent.click(
|
||||
screen.getByRole('button', { name: 'Download Video' }),
|
||||
);
|
||||
await waitFor(() => expect(downloadJobVideo).toHaveBeenCalledWith('job-1'));
|
||||
});
|
||||
|
||||
it('selects the job when the card body is clicked', async () => {
|
||||
render(<JobCard job={makeJob()} />);
|
||||
await userEvent.click(screen.getByText('Wan2.1-T2V'));
|
||||
expect(activeJobStore.get().activeJobId).toBe('job-1');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,245 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { Timer } from 'lucide-react';
|
||||
|
||||
import { Badge, type BadgeProps } from '@/components/ui/badge';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { useStore } from '@/hooks/useStore';
|
||||
import {
|
||||
deleteJob,
|
||||
downloadJobVideo,
|
||||
startJob,
|
||||
stopJob,
|
||||
} from '@/lib/api';
|
||||
import type { Job } from '@/lib/types';
|
||||
import { cn, downloadBlob } from '@/lib/utils';
|
||||
import { activeJobStore, setActiveJobId } from '@/stores/activeJob';
|
||||
|
||||
interface JobCardProps {
|
||||
job: Job;
|
||||
onJobUpdated?: () => void;
|
||||
}
|
||||
|
||||
function formatDuration(seconds: number): string {
|
||||
const roundedSeconds = Math.round(seconds);
|
||||
if (roundedSeconds < 60) return `${roundedSeconds}s`;
|
||||
if (roundedSeconds < 3600) {
|
||||
const mins = Math.floor(roundedSeconds / 60);
|
||||
const secs = roundedSeconds % 60;
|
||||
return secs > 0 ? `${mins}m ${secs}s` : `${mins}m`;
|
||||
}
|
||||
const hours = Math.floor(roundedSeconds / 3600);
|
||||
const mins = Math.floor((roundedSeconds % 3600) / 60);
|
||||
return mins > 0 ? `${hours}h ${mins}m` : `${hours}h`;
|
||||
}
|
||||
|
||||
function computeElapsed(job: Job, currentTime: number): string | null {
|
||||
if (!job.started_at) return null;
|
||||
const endTime =
|
||||
job.status === 'running' ? currentTime : (job.finished_at ?? 0);
|
||||
if (!endTime && job.status !== 'running') return null;
|
||||
const startedAtMs =
|
||||
job.started_at < 1e12 ? job.started_at * 1000 : job.started_at;
|
||||
const endTimeMs = endTime < 1e12 ? endTime * 1000 : endTime;
|
||||
const elapsedSeconds = (endTimeMs - startedAtMs) / 1000;
|
||||
if (elapsedSeconds <= 0) return null;
|
||||
return formatDuration(elapsedSeconds);
|
||||
}
|
||||
|
||||
const BADGE_VARIANTS: Record<string, BadgeProps['variant']> = {
|
||||
pending: 'secondary',
|
||||
running: 'warning',
|
||||
completed: 'success',
|
||||
ready: 'success',
|
||||
failed: 'destructive',
|
||||
stopped: 'secondary',
|
||||
preprocessing: 'default',
|
||||
};
|
||||
|
||||
export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
const { activeJobId } = useStore(activeJobStore);
|
||||
const isSelected = activeJobId === job.id;
|
||||
|
||||
const [isLoading, setIsLoading] = React.useState(false);
|
||||
const [currentTime, setCurrentTime] = React.useState(() => Date.now());
|
||||
|
||||
const elapsedTime = computeElapsed(job, currentTime);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (job.status !== 'running' || !job.started_at) return;
|
||||
const interval = setInterval(() => setCurrentTime(Date.now()), 1000);
|
||||
return () => clearInterval(interval);
|
||||
}, [job.status, job.started_at]);
|
||||
|
||||
async function handleStart(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading || job.status === 'running' || job.status === 'completed')
|
||||
return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await startJob(job.id);
|
||||
onJobUpdated?.();
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to start job');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleStop(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading || job.status !== 'running') return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await stopJob(job.id);
|
||||
onJobUpdated?.();
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to stop job');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDelete(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading) return;
|
||||
if (!confirm('Delete this job?')) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await deleteJob(job.id);
|
||||
onJobUpdated?.();
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to delete job');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
function handleSelectJob(e: React.MouseEvent | React.KeyboardEvent) {
|
||||
if ((e.target as HTMLElement).closest('button')) return;
|
||||
setActiveJobId(isSelected ? null : job.id);
|
||||
}
|
||||
|
||||
function handleKeyDown(e: React.KeyboardEvent) {
|
||||
if (e.key === 'Enter' || e.key === ' ') {
|
||||
e.preventDefault();
|
||||
handleSelectJob(e);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDownloadVideo(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading || !job.output_path) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
const blob = await downloadJobVideo(job.id);
|
||||
const ext = job.output_path.endsWith('.png') ? 'png' : 'mp4';
|
||||
downloadBlob(blob, `job_${job.id}.${ext}`);
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to download video');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
role="button"
|
||||
tabIndex={0}
|
||||
onClick={handleSelectJob}
|
||||
onKeyDown={handleKeyDown}
|
||||
className={cn(
|
||||
'mb-3 flex cursor-pointer flex-col gap-2.5 rounded-lg border bg-background p-4 transition-colors last:mb-0',
|
||||
isSelected
|
||||
? 'border-accent-blue bg-accent-blue/5'
|
||||
: 'border-border hover:border-muted-foreground/40',
|
||||
)}
|
||||
>
|
||||
<div className="flex flex-wrap items-center justify-between gap-2">
|
||||
<span className="text-[0.95rem] font-semibold text-foreground">
|
||||
{job.model_id}
|
||||
</span>
|
||||
<Badge variant={BADGE_VARIANTS[job.status] ?? 'secondary'}>
|
||||
{job.status}
|
||||
</Badge>
|
||||
</div>
|
||||
<p className="max-w-full overflow-hidden text-ellipsis whitespace-nowrap text-sm text-muted-foreground">
|
||||
{job.prompt}
|
||||
</p>
|
||||
<div className="flex flex-wrap items-center gap-4 text-xs text-muted-foreground">
|
||||
{job.job_type === 'inference' ? (
|
||||
<>
|
||||
<span>{job.num_frames} frames</span>
|
||||
<span>
|
||||
{job.height}×{job.width}
|
||||
</span>
|
||||
</>
|
||||
) : (
|
||||
<span>{job.workload_type?.replace(/_/g, ' ') ?? job.job_type}</span>
|
||||
)}
|
||||
{elapsedTime && (
|
||||
<span className="inline-flex items-center gap-1">
|
||||
<Timer className="size-3.5" aria-hidden />
|
||||
{elapsedTime}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<div className="flex flex-wrap items-center gap-1.5">
|
||||
{job.status === 'running' ? (
|
||||
<Button
|
||||
size="sm"
|
||||
onClick={handleStop}
|
||||
disabled={isLoading}
|
||||
className="border-transparent bg-amber-500 text-black shadow-md hover:bg-amber-400"
|
||||
>
|
||||
Stop
|
||||
</Button>
|
||||
) : job.status === 'failed' ? (
|
||||
<Button
|
||||
size="sm"
|
||||
onClick={handleStart}
|
||||
disabled={isLoading}
|
||||
className="border-transparent bg-emerald-600 text-white shadow-md hover:bg-emerald-500"
|
||||
>
|
||||
Restart
|
||||
</Button>
|
||||
) : job.status === 'pending' || job.status === 'stopped' ? (
|
||||
<Button
|
||||
size="sm"
|
||||
onClick={handleStart}
|
||||
disabled={isLoading}
|
||||
className="border-transparent bg-emerald-600 text-white shadow-md hover:bg-emerald-500"
|
||||
>
|
||||
Start
|
||||
</Button>
|
||||
) : null}
|
||||
{job.status === 'completed' &&
|
||||
job.output_path &&
|
||||
job.job_type === 'inference' && (
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={handleDownloadVideo}
|
||||
disabled={isLoading}
|
||||
title="Download video"
|
||||
>
|
||||
Download Video
|
||||
</Button>
|
||||
)}
|
||||
<Button
|
||||
size="sm"
|
||||
variant="destructive"
|
||||
onClick={handleDelete}
|
||||
disabled={isLoading}
|
||||
>
|
||||
Delete
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
import { act, render, screen } from '@testing-library/react';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import JobDetailsSidebar from './JobDetailsSidebar';
|
||||
import { getJobLogs } from '@/lib/api';
|
||||
import type { Job } from '@/lib/types';
|
||||
import { makeJob as makeBaseJob } from '@/test/factories';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
getJobLogs: vi.fn(),
|
||||
downloadJobLog: vi.fn(),
|
||||
}));
|
||||
|
||||
const makeJob = (overrides: Partial<Job> = {}): Job =>
|
||||
makeBaseJob({
|
||||
status: 'running',
|
||||
log_file_path: '/logs/job-1.log',
|
||||
...overrides,
|
||||
});
|
||||
|
||||
describe('JobDetailsSidebar', () => {
|
||||
it('renders log lines streamed from the job log poll', async () => {
|
||||
vi.mocked(getJobLogs).mockResolvedValue({
|
||||
lines: ['boot sequence started', 'loading model weights'],
|
||||
total: 2,
|
||||
progress: 0,
|
||||
progress_msg: '',
|
||||
phase: '',
|
||||
});
|
||||
|
||||
render(
|
||||
<JobDetailsSidebar job={makeJob({ status: 'running' })} onClose={vi.fn()} />,
|
||||
);
|
||||
|
||||
expect(await screen.findByText(/boot sequence started/)).toBeInTheDocument();
|
||||
expect(screen.getByText(/loading model weights/)).toBeInTheDocument();
|
||||
expect(getJobLogs).toHaveBeenCalledWith('job-1', 0);
|
||||
});
|
||||
|
||||
it('keeps polling while the job is running', async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
vi.mocked(getJobLogs).mockResolvedValue({
|
||||
lines: [],
|
||||
total: 0,
|
||||
progress: 0,
|
||||
progress_msg: '',
|
||||
phase: '',
|
||||
});
|
||||
|
||||
render(
|
||||
<JobDetailsSidebar
|
||||
job={makeJob({ status: 'running' })}
|
||||
onClose={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
|
||||
// Flush the immediate poll fired on mount.
|
||||
await act(async () => {
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
});
|
||||
const initialCalls = vi.mocked(getJobLogs).mock.calls.length;
|
||||
|
||||
// Two 2s interval ticks should fire while the job is running.
|
||||
await act(async () => {
|
||||
await vi.advanceTimersByTimeAsync(4000);
|
||||
});
|
||||
|
||||
expect(vi.mocked(getJobLogs).mock.calls.length).toBeGreaterThan(
|
||||
initialCalls,
|
||||
);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
|
||||
it('stops polling once the job is completed', async () => {
|
||||
vi.useFakeTimers();
|
||||
try {
|
||||
vi.mocked(getJobLogs).mockResolvedValue({
|
||||
lines: ['final line'],
|
||||
total: 1,
|
||||
progress: 0,
|
||||
progress_msg: '',
|
||||
phase: '',
|
||||
});
|
||||
|
||||
render(
|
||||
<JobDetailsSidebar
|
||||
job={makeJob({ status: 'completed' })}
|
||||
onClose={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
|
||||
// The component fetches once on mount even for terminal jobs.
|
||||
await act(async () => {
|
||||
await vi.advanceTimersByTimeAsync(0);
|
||||
});
|
||||
expect(getJobLogs).toHaveBeenCalledTimes(1);
|
||||
|
||||
// No interval is registered, so advancing the clock must not re-poll.
|
||||
await act(async () => {
|
||||
await vi.advanceTimersByTimeAsync(10000);
|
||||
});
|
||||
expect(getJobLogs).toHaveBeenCalledTimes(1);
|
||||
} finally {
|
||||
vi.useRealTimers();
|
||||
}
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,208 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { X } from 'lucide-react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { useResizable } from '@/hooks/useResizable';
|
||||
import { downloadJobLog, getJobLogs } from '@/lib/api';
|
||||
import type { Job } from '@/lib/types';
|
||||
import { cn, downloadBlob } from '@/lib/utils';
|
||||
|
||||
const SIDEBAR_MIN_WIDTH = 280;
|
||||
const SIDEBAR_MAX_WIDTH = 750;
|
||||
const POLL_INTERVAL_MS = 2000;
|
||||
|
||||
export default function JobDetailsSidebar({
|
||||
job,
|
||||
onClose,
|
||||
onWidthChange,
|
||||
}: {
|
||||
job: Job;
|
||||
onClose: () => void;
|
||||
onWidthChange?: (w: number) => void;
|
||||
}) {
|
||||
const [width, setWidth] = React.useState(360);
|
||||
const [isDragging, setIsDragging] = React.useState(false);
|
||||
const [isLoading, setIsLoading] = React.useState(false);
|
||||
const [logs, setLogs] = React.useState('');
|
||||
|
||||
// Race-guard ref (mirrors the Svelte original): the cursor + accumulated
|
||||
// text live here so in-flight polls don't read stale React state. Do NOT
|
||||
// swap these for effect deps — they must update synchronously, outside
|
||||
// React's render cycle. The log is one growing string (amortized O(new)
|
||||
// appends), not an array re-copied and re-joined on every poll tick.
|
||||
const stateRef = React.useRef<{ text: string; logAfter: number }>({
|
||||
text: '',
|
||||
logAfter: 0,
|
||||
});
|
||||
const previousJobId = React.useRef<string | null>(null);
|
||||
const previousStatus = React.useRef<string | null>(null);
|
||||
const consoleRef = React.useRef<HTMLPreElement | null>(null);
|
||||
|
||||
const { onMouseDown } = useResizable({
|
||||
// Right-docked panel with the drag handle on its left edge: dragging the
|
||||
// handle left must grow the panel, which is `edge: 'right'` (matches the
|
||||
// Svelte original). `edge: 'left'` would invert the drag.
|
||||
edge: 'right',
|
||||
minWidth: SIDEBAR_MIN_WIDTH,
|
||||
maxWidth: SIDEBAR_MAX_WIDTH,
|
||||
getWidth: () => width,
|
||||
onWidth: setWidth,
|
||||
onDragChange: setIsDragging,
|
||||
});
|
||||
|
||||
React.useEffect(() => {
|
||||
onWidthChange?.(width);
|
||||
}, [width, onWidthChange]);
|
||||
|
||||
// Auto-scroll the console to the bottom whenever new lines land. Runs after
|
||||
// commit so scrollHeight reflects the freshly-rendered output.
|
||||
React.useEffect(() => {
|
||||
const el = consoleRef.current;
|
||||
if (el) el.scrollTop = el.scrollHeight;
|
||||
}, [logs]);
|
||||
|
||||
// Reset on job switch or restart. Declared before the polling effect so the
|
||||
// cursor is cleared before the next poll reads it.
|
||||
React.useEffect(() => {
|
||||
const wasTerminal =
|
||||
previousStatus.current === 'failed' ||
|
||||
previousStatus.current === 'stopped' ||
|
||||
previousStatus.current === 'completed';
|
||||
const isRestarting =
|
||||
previousJobId.current === job.id &&
|
||||
wasTerminal &&
|
||||
(job.status === 'pending' || job.status === 'running');
|
||||
|
||||
if (previousJobId.current !== job.id || isRestarting) {
|
||||
stateRef.current.text = '';
|
||||
stateRef.current.logAfter = 0;
|
||||
setLogs('');
|
||||
}
|
||||
previousJobId.current = job.id;
|
||||
previousStatus.current = job.status;
|
||||
}, [job.id, job.status]);
|
||||
|
||||
React.useEffect(() => {
|
||||
const shouldPoll = job.status === 'running' || job.status === 'pending';
|
||||
let pollInterval: ReturnType<typeof setInterval> | null = null;
|
||||
let mounted = true;
|
||||
// Effect-local lock so this job's first fetch is never blocked by a
|
||||
// previous job's in-flight poll (stale writes are dropped via `mounted`).
|
||||
let locked = false;
|
||||
|
||||
async function pollLogs() {
|
||||
if (!mounted || locked) return;
|
||||
locked = true;
|
||||
try {
|
||||
const logData = await getJobLogs(job.id, stateRef.current.logAfter);
|
||||
if (mounted && logData.lines.length > 0) {
|
||||
const chunk = logData.lines.join('\n');
|
||||
stateRef.current.text = stateRef.current.text
|
||||
? `${stateRef.current.text}\n${chunk}`
|
||||
: chunk;
|
||||
stateRef.current.logAfter = logData.total;
|
||||
setLogs(stateRef.current.text);
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('Failed to fetch logs:', e);
|
||||
} finally {
|
||||
locked = false;
|
||||
}
|
||||
}
|
||||
|
||||
pollLogs();
|
||||
if (shouldPoll) pollInterval = setInterval(pollLogs, POLL_INTERVAL_MS);
|
||||
|
||||
return () => {
|
||||
mounted = false;
|
||||
if (pollInterval) clearInterval(pollInterval);
|
||||
};
|
||||
}, [job.id, job.status]);
|
||||
|
||||
async function handleDownloadLog() {
|
||||
if (isLoading) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
const blob = await downloadJobLog(job.id);
|
||||
downloadBlob(blob, `job_${job.id}.log`);
|
||||
} catch (err) {
|
||||
console.error('Failed to download log:', err);
|
||||
alert(err instanceof Error ? err.message : 'Failed to download log');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<aside
|
||||
className="fixed bottom-0 right-0 top-[var(--header-height)] z-50 flex max-h-[calc(100vh-var(--header-height))] min-w-[280px] shrink-0 flex-col border-l border-border bg-card"
|
||||
style={{ width, maxWidth: SIDEBAR_MAX_WIDTH }}
|
||||
>
|
||||
<div className="flex items-center justify-between border-b border-border px-5 py-4">
|
||||
<h2 className="m-0 text-base font-semibold text-foreground">
|
||||
Job Details
|
||||
</h2>
|
||||
<div className="flex items-center gap-2">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={handleDownloadLog}
|
||||
disabled={isLoading || !job.log_file_path}
|
||||
title="Download log file"
|
||||
>
|
||||
Download Log
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon-sm"
|
||||
onClick={onClose}
|
||||
title="Close"
|
||||
aria-label="Close"
|
||||
>
|
||||
<X className="h-[18px] w-[18px]" />
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex min-h-0 flex-1 flex-col px-5 py-4">
|
||||
<div className="mb-2 flex items-center justify-between">
|
||||
<span className="text-xs font-semibold uppercase tracking-wider text-muted-foreground">
|
||||
Console Output
|
||||
</span>
|
||||
{job.status === 'running' && (
|
||||
<span className="text-[0.7rem] font-medium text-emerald-600 dark:text-emerald-400">
|
||||
● Live
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<pre
|
||||
ref={consoleRef}
|
||||
className="m-0 min-h-0 flex-1 overflow-auto whitespace-pre-wrap break-words rounded-lg border border-border bg-background p-3 font-mono text-xs leading-normal text-foreground"
|
||||
>
|
||||
{logs === '' ? (
|
||||
<span className="italic text-muted-foreground">
|
||||
{job.status === 'running'
|
||||
? 'Waiting for logs...'
|
||||
: 'No logs available'}
|
||||
</span>
|
||||
) : (
|
||||
logs
|
||||
)}
|
||||
</pre>
|
||||
</div>
|
||||
|
||||
<div
|
||||
role="presentation"
|
||||
onMouseDown={onMouseDown}
|
||||
className={cn(
|
||||
'absolute bottom-0 left-0 top-0 z-[1] w-1.5 cursor-col-resize hover:bg-accent-blue/25',
|
||||
isDragging && 'bg-accent-blue/25',
|
||||
)}
|
||||
/>
|
||||
</aside>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,82 @@
|
||||
import { act, render, screen, waitFor } from '@testing-library/react';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import JobQueue from '@/components/jobs/JobQueue';
|
||||
import { getJobsList } from '@/lib/api';
|
||||
import type { Job, JobType } from '@/lib/types';
|
||||
import { setActiveJobId } from '@/stores/activeJob';
|
||||
import { triggerRefresh } from '@/stores/jobsRefresh';
|
||||
import { makeJob as makeBaseJob } from '@/test/factories';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
getJobsList: vi.fn(),
|
||||
startJob: vi.fn(),
|
||||
stopJob: vi.fn(),
|
||||
deleteJob: vi.fn(),
|
||||
downloadJobVideo: vi.fn(),
|
||||
}));
|
||||
|
||||
const makeJob = (overrides: Partial<Job> = {}): Job =>
|
||||
makeBaseJob({
|
||||
model_id: 'Wan2.1-T2V',
|
||||
status: 'completed',
|
||||
created_at: 1_700_000_000,
|
||||
num_inference_steps: 50,
|
||||
num_frames: 81,
|
||||
height: 480,
|
||||
width: 832,
|
||||
guidance_scale: 5,
|
||||
seed: 42,
|
||||
num_gpus: 1,
|
||||
...overrides,
|
||||
});
|
||||
|
||||
beforeEach(() => {
|
||||
setActiveJobId(null);
|
||||
vi.mocked(getJobsList).mockResolvedValue([]);
|
||||
});
|
||||
|
||||
describe('JobQueue', () => {
|
||||
it('shows an empty placeholder and fetches for the single job type', async () => {
|
||||
render(<JobQueue jobType="inference" />);
|
||||
expect(
|
||||
await screen.findByText('No inference jobs yet. Create one above.'),
|
||||
).toBeInTheDocument();
|
||||
expect(getJobsList).toHaveBeenCalledWith('inference');
|
||||
});
|
||||
|
||||
it('renders a JobCard for each fetched job', async () => {
|
||||
vi.mocked(getJobsList).mockResolvedValue([
|
||||
makeJob({ id: 'a', model_id: 'Model-A' }),
|
||||
]);
|
||||
render(<JobQueue jobType="distillation" />);
|
||||
expect(await screen.findByText('Model-A')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('merges all job types when jobTypesForList is provided', async () => {
|
||||
vi.mocked(getJobsList).mockImplementation((t?: JobType) =>
|
||||
Promise.resolve(
|
||||
t === ('lora' as JobType)
|
||||
? [makeJob({ id: 'l', model_id: 'Lora-Model', created_at: 2 })]
|
||||
: [makeJob({ id: 'f', model_id: 'Full-Model', created_at: 1 })],
|
||||
),
|
||||
);
|
||||
render(
|
||||
<JobQueue
|
||||
jobType="finetuning"
|
||||
jobTypesForList={['finetuning', 'lora'] as JobType[]}
|
||||
/>,
|
||||
);
|
||||
expect(await screen.findByText('Lora-Model')).toBeInTheDocument();
|
||||
expect(screen.getByText('Full-Model')).toBeInTheDocument();
|
||||
expect(getJobsList).toHaveBeenCalledWith('finetuning');
|
||||
expect(getJobsList).toHaveBeenCalledWith('lora');
|
||||
});
|
||||
|
||||
it('refetches when a refresh is triggered', async () => {
|
||||
render(<JobQueue jobType="inference" />);
|
||||
await waitFor(() => expect(getJobsList).toHaveBeenCalledTimes(1));
|
||||
act(() => triggerRefresh());
|
||||
await waitFor(() => expect(getJobsList).toHaveBeenCalledTimes(2));
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,141 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import JobCard from '@/components/jobs/JobCard';
|
||||
import { useStore } from '@/hooks/useStore';
|
||||
import { getJobsList } from '@/lib/api';
|
||||
import type { Job, JobType } from '@/lib/types';
|
||||
import {
|
||||
activeJobStore,
|
||||
setActiveJob,
|
||||
setActiveJobId,
|
||||
} from '@/stores/activeJob';
|
||||
import { jobsRefreshStore } from '@/stores/jobsRefresh';
|
||||
|
||||
interface JobQueueProps {
|
||||
jobType: JobType;
|
||||
jobTypesForList?: JobType[];
|
||||
}
|
||||
|
||||
// Jobs are flat objects of primitives, so a shallow compare detects "nothing
|
||||
// changed" across poll responses (which are referentially fresh every fetch).
|
||||
function jobsShallowEqual(a: Job | null, b: Job | null): boolean {
|
||||
if (a === b) return true;
|
||||
if (!a || !b) return false;
|
||||
const aKeys = Object.keys(a) as (keyof Job)[];
|
||||
return (
|
||||
aKeys.length === Object.keys(b).length &&
|
||||
aKeys.every((k) => a[k] === b[k])
|
||||
);
|
||||
}
|
||||
|
||||
export default function JobQueue({ jobType, jobTypesForList }: JobQueueProps) {
|
||||
const [jobs, setJobs] = React.useState<Job[]>([]);
|
||||
const { nonce } = useStore(jobsRefreshStore);
|
||||
const { activeJobId } = useStore(activeJobStore);
|
||||
|
||||
// Stable primitive key so the memoized list below keeps its identity when an
|
||||
// inline array prop (e.g. ['finetuning', 'lora']) gets a fresh reference each
|
||||
// render, which would otherwise re-run the fetch effects needlessly.
|
||||
const typesKey = (jobTypesForList ?? [jobType]).join(',');
|
||||
const typesToFetch = React.useMemo<JobType[]>(
|
||||
() => jobTypesForList ?? [jobType],
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
[typesKey],
|
||||
);
|
||||
|
||||
// Guard the poll against slow responses: `inFlight` lets the interval skip
|
||||
// a tick instead of stacking requests, and the sequence counter drops
|
||||
// out-of-order responses so a delayed older payload can't overwrite a newer
|
||||
// job list (direct refetches always run and supersede in-flight polls).
|
||||
const fetchSeq = React.useRef(0);
|
||||
const inFlight = React.useRef(false);
|
||||
|
||||
const fetchJobs = React.useCallback(async () => {
|
||||
const seq = ++fetchSeq.current;
|
||||
inFlight.current = true;
|
||||
try {
|
||||
let next: Job[];
|
||||
if (typesToFetch.length === 1) {
|
||||
next = await getJobsList(typesToFetch[0]);
|
||||
} else {
|
||||
const results = await Promise.all(
|
||||
typesToFetch.map((t) => getJobsList(t)),
|
||||
);
|
||||
next = results
|
||||
.flat()
|
||||
.sort(
|
||||
(a, b) =>
|
||||
new Date(b.created_at ?? 0).getTime() -
|
||||
new Date(a.created_at ?? 0).getTime(),
|
||||
);
|
||||
}
|
||||
if (seq === fetchSeq.current) setJobs(next);
|
||||
} catch (e) {
|
||||
console.error('Failed to fetch jobs:', e);
|
||||
} finally {
|
||||
if (seq === fetchSeq.current) inFlight.current = false;
|
||||
}
|
||||
}, [typesKey]);
|
||||
|
||||
// Keep the latest fetchJobs reachable from the polling interval without
|
||||
// tearing it down/recreating it on every fetchJobs identity change.
|
||||
const fetchJobsRef = React.useRef(fetchJobs);
|
||||
React.useEffect(() => {
|
||||
fetchJobsRef.current = fetchJobs;
|
||||
}, [fetchJobs]);
|
||||
|
||||
// Initial fetch + refetch whenever an external refresh is triggered.
|
||||
React.useEffect(() => {
|
||||
fetchJobs();
|
||||
}, [fetchJobs, nonce]);
|
||||
|
||||
// Poll every second while any job is running/pending; stop otherwise.
|
||||
const hasActive = jobs.some(
|
||||
(j) => j.status === 'running' || j.status === 'pending',
|
||||
);
|
||||
React.useEffect(() => {
|
||||
if (!hasActive) return;
|
||||
const interval = setInterval(() => {
|
||||
if (!inFlight.current) fetchJobsRef.current();
|
||||
}, 1000);
|
||||
return () => clearInterval(interval);
|
||||
}, [hasActive]);
|
||||
|
||||
// Keep the active job in sync with the selected id, and void the selection
|
||||
// if its job disappeared (e.g. deleted). Skip the store write when nothing
|
||||
// changed — each poll returns fresh objects, and an unconditional write
|
||||
// would re-render every subscriber (shell, sidebars, cards) once a second.
|
||||
React.useEffect(() => {
|
||||
if (!activeJobId) {
|
||||
if (activeJobStore.get().activeJob) setActiveJob(null);
|
||||
return;
|
||||
}
|
||||
const activeJob = jobs.find((j) => j.id === activeJobId) ?? null;
|
||||
if (!jobsShallowEqual(activeJob, activeJobStore.get().activeJob)) {
|
||||
setActiveJob(activeJob);
|
||||
}
|
||||
if (!activeJob) setActiveJobId(null);
|
||||
}, [activeJobId, jobs]);
|
||||
|
||||
const multiType = typesToFetch.length > 1;
|
||||
|
||||
return (
|
||||
<main className="mx-auto flex w-full max-w-[850px] flex-col gap-6 px-4 pb-12">
|
||||
<section className="p-6">
|
||||
<div>
|
||||
{jobs.length === 0 ? (
|
||||
<p className="py-8 text-center text-muted-foreground">
|
||||
No {multiType ? 'jobs' : `${jobType} jobs`} yet. Create one above.
|
||||
</p>
|
||||
) : (
|
||||
jobs.map((job) => (
|
||||
<JobCard key={job.id} job={job} onJobUpdated={fetchJobs} />
|
||||
))
|
||||
)}
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,88 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { usePathname } from 'next/navigation';
|
||||
|
||||
import DatasetSidebar from '@/components/datasets/DatasetSidebar';
|
||||
import Header from '@/components/shell/Header';
|
||||
import { HeaderActionsProvider } from '@/components/shell/HeaderActionsContext';
|
||||
import PrimarySidebar from '@/components/shell/PrimarySidebar';
|
||||
import JobDetailsSidebar from '@/components/jobs/JobDetailsSidebar';
|
||||
import { Toaster } from '@/components/ui/sonner';
|
||||
import { useStore } from '@/hooks/useStore';
|
||||
import {
|
||||
activeDatasetStore,
|
||||
setActiveDatasetId,
|
||||
} from '@/stores/activeDataset';
|
||||
import { activeJobStore, setActiveJobId } from '@/stores/activeJob';
|
||||
import { initDefaultOptions } from '@/stores/defaultOptions';
|
||||
|
||||
const JOB_ROUTES = ['/inference', '/finetuning', '/distillation'];
|
||||
|
||||
export function AppShell({ children }: { children: React.ReactNode }) {
|
||||
const pathname = usePathname();
|
||||
const { activeJob } = useStore(activeJobStore);
|
||||
const { activeDataset } = useStore(activeDatasetStore);
|
||||
|
||||
const [primaryWidth, setPrimaryWidth] = React.useState(220);
|
||||
const [secondaryWidth, setSecondaryWidth] = React.useState(0);
|
||||
|
||||
const jobSidebarOpen = JOB_ROUTES.includes(pathname) && activeJob != null;
|
||||
const datasetSidebarOpen =
|
||||
pathname === '/datasets' && activeDataset != null;
|
||||
const secondaryOpen = jobSidebarOpen || datasetSidebarOpen;
|
||||
|
||||
React.useEffect(() => {
|
||||
initDefaultOptions();
|
||||
}, []);
|
||||
|
||||
React.useEffect(() => {
|
||||
function handleKeyDown(e: KeyboardEvent) {
|
||||
if (e.key === 'Escape' && !document.querySelector('[data-modal]')) {
|
||||
if (activeJobStore.get().activeJob) setActiveJobId(null);
|
||||
if (activeDatasetStore.get().activeDataset) setActiveDatasetId(null);
|
||||
}
|
||||
}
|
||||
document.addEventListener('keydown', handleKeyDown);
|
||||
return () => document.removeEventListener('keydown', handleKeyDown);
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<HeaderActionsProvider>
|
||||
<Header />
|
||||
<div
|
||||
className="flex overflow-hidden"
|
||||
style={{
|
||||
marginTop: 'var(--header-height)',
|
||||
height: 'calc(100vh - var(--header-height))',
|
||||
}}
|
||||
>
|
||||
<PrimarySidebar onWidthChange={setPrimaryWidth} />
|
||||
<main
|
||||
className="flex min-w-0 flex-1 flex-col overflow-auto"
|
||||
style={{
|
||||
marginLeft: primaryWidth,
|
||||
marginRight: secondaryOpen ? secondaryWidth : 0,
|
||||
}}
|
||||
>
|
||||
{children}
|
||||
</main>
|
||||
{jobSidebarOpen && activeJob && (
|
||||
<JobDetailsSidebar
|
||||
job={activeJob}
|
||||
onClose={() => setActiveJobId(null)}
|
||||
onWidthChange={setSecondaryWidth}
|
||||
/>
|
||||
)}
|
||||
{datasetSidebarOpen && activeDataset && (
|
||||
<DatasetSidebar
|
||||
dataset={activeDataset}
|
||||
onClose={() => setActiveDatasetId(null)}
|
||||
onWidthChange={setSecondaryWidth}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
<Toaster />
|
||||
</HeaderActionsProvider>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
'use client';
|
||||
|
||||
import { usePathname } from 'next/navigation';
|
||||
|
||||
import { useHeaderActions } from '@/components/shell/HeaderActionsContext';
|
||||
import { ThemeToggle } from '@/components/ui/theme-toggle';
|
||||
|
||||
const TAB_TITLES: Record<string, string> = {
|
||||
'/inference': 'Jobs',
|
||||
'/finetuning': 'Jobs',
|
||||
'/distillation': 'Jobs',
|
||||
'/datasets': 'Datasets',
|
||||
'/gallery': 'Gallery',
|
||||
'/gpus': 'GPUs',
|
||||
'/settings': 'Settings',
|
||||
};
|
||||
|
||||
export default function Header() {
|
||||
const pathname = usePathname();
|
||||
const { actions } = useHeaderActions();
|
||||
const title = TAB_TITLES[pathname] ?? 'FastVideo';
|
||||
|
||||
return (
|
||||
<header className="fixed inset-x-0 top-0 z-[100] flex h-[var(--header-height)] items-center gap-6 border-b border-border bg-background/80 px-6 backdrop-blur">
|
||||
{/* eslint-disable-next-line @next/next/no-img-element */}
|
||||
<img
|
||||
src="/logo.svg"
|
||||
alt="FastVideo Logo"
|
||||
width={100}
|
||||
height={42}
|
||||
className="block h-[42px] w-[100px]"
|
||||
/>
|
||||
<h1 className="m-0 flex-1 text-xl font-semibold tracking-tight">
|
||||
{title}
|
||||
</h1>
|
||||
<div className="flex items-center gap-3">
|
||||
{actions}
|
||||
<ThemeToggle />
|
||||
</div>
|
||||
</header>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
interface HeaderActionsContextValue {
|
||||
actions: React.ReactNode;
|
||||
setActions: (node: React.ReactNode) => void;
|
||||
}
|
||||
|
||||
const HeaderActionsContext =
|
||||
React.createContext<HeaderActionsContextValue | null>(null);
|
||||
|
||||
export function HeaderActionsProvider({
|
||||
children,
|
||||
}: {
|
||||
children: React.ReactNode;
|
||||
}) {
|
||||
const [actions, setActions] = React.useState<React.ReactNode>(null);
|
||||
const value = React.useMemo(() => ({ actions, setActions }), [actions]);
|
||||
return (
|
||||
<HeaderActionsContext.Provider value={value}>
|
||||
{children}
|
||||
</HeaderActionsContext.Provider>
|
||||
);
|
||||
}
|
||||
|
||||
export function useHeaderActions(): HeaderActionsContextValue {
|
||||
const ctx = React.useContext(HeaderActionsContext);
|
||||
if (!ctx) {
|
||||
throw new Error(
|
||||
'useHeaderActions must be used within a HeaderActionsProvider',
|
||||
);
|
||||
}
|
||||
return ctx;
|
||||
}
|
||||
|
||||
/**
|
||||
* Declaratively publish the current page's header actions: renders nothing,
|
||||
* registers `children` in the header on mount and clears them on unmount.
|
||||
* Pages without actions simply don't render it.
|
||||
*/
|
||||
export function HeaderActions({ children }: { children: React.ReactNode }) {
|
||||
const { setActions } = useHeaderActions();
|
||||
// Mount-only on purpose: pages pass inline JSX, which is referentially new
|
||||
// every render and would re-register per render if it were a dependency.
|
||||
React.useEffect(() => {
|
||||
setActions(children);
|
||||
return () => setActions(null);
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [setActions]);
|
||||
return null;
|
||||
}
|
||||
@@ -0,0 +1,176 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import Link from 'next/link';
|
||||
import { usePathname } from 'next/navigation';
|
||||
|
||||
import { useResizable } from '@/hooks/useResizable';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const SIDEBAR_MIN_WIDTH = 100;
|
||||
const SIDEBAR_MAX_WIDTH = 300;
|
||||
const SIDEBAR_COLLAPSED_WIDTH = 0;
|
||||
const SIDEBAR_COLLAPSED_VISIBLE_WIDTH = 60;
|
||||
|
||||
const JOB_ROUTES = [
|
||||
{ href: '/inference', label: 'Inference' },
|
||||
{ href: '/finetuning', label: 'Finetuning' },
|
||||
{ href: '/distillation', label: 'Distillation' },
|
||||
] as const;
|
||||
|
||||
const TAB_BASE =
|
||||
'block px-5 py-[0.65rem] text-left text-sm text-muted-foreground transition-colors hover:bg-accent/60 hover:text-foreground';
|
||||
const TAB_ACTIVE = 'bg-accent-blue/10 font-medium text-accent-blue';
|
||||
|
||||
export default function PrimarySidebar({
|
||||
onWidthChange,
|
||||
}: {
|
||||
onWidthChange?: (w: number) => void;
|
||||
}) {
|
||||
const pathname = usePathname();
|
||||
const [width, setWidth] = React.useState(220);
|
||||
const [isCollapsed, setIsCollapsed] = React.useState(false);
|
||||
const [isDragging, setIsDragging] = React.useState(false);
|
||||
const [jobsOpen, setJobsOpen] = React.useState(true);
|
||||
|
||||
const effectiveWidth = isCollapsed ? SIDEBAR_COLLAPSED_WIDTH : width;
|
||||
const layoutWidth = isCollapsed ? SIDEBAR_COLLAPSED_VISIBLE_WIDTH : width;
|
||||
const isJobsActive = JOB_ROUTES.some((r) => pathname === r.href);
|
||||
|
||||
React.useEffect(() => {
|
||||
onWidthChange?.(layoutWidth);
|
||||
}, [layoutWidth, onWidthChange]);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (JOB_ROUTES.some((r) => pathname === r.href)) {
|
||||
setJobsOpen(true);
|
||||
}
|
||||
}, [pathname]);
|
||||
|
||||
const { onMouseDown } = useResizable({
|
||||
edge: 'left',
|
||||
minWidth: SIDEBAR_MIN_WIDTH,
|
||||
maxWidth: SIDEBAR_MAX_WIDTH,
|
||||
getWidth: () => width,
|
||||
onWidth: setWidth,
|
||||
onDragChange: setIsDragging,
|
||||
});
|
||||
|
||||
return (
|
||||
<aside
|
||||
className="fixed bottom-0 left-0 top-[var(--header-height)] z-50 flex max-h-[calc(100vh-var(--header-height))] shrink-0 flex-col border-r border-border bg-card"
|
||||
style={{ width: effectiveWidth }}
|
||||
>
|
||||
{!isCollapsed && (
|
||||
<nav className="flex flex-col py-2">
|
||||
<div className="flex flex-col">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setJobsOpen((v) => !v)}
|
||||
aria-expanded={jobsOpen}
|
||||
aria-haspopup="true"
|
||||
className={cn(
|
||||
TAB_BASE,
|
||||
'flex w-full cursor-pointer items-center justify-between',
|
||||
isJobsActive && TAB_ACTIVE,
|
||||
)}
|
||||
>
|
||||
<span>Jobs</span>
|
||||
<svg
|
||||
viewBox="0 0 24 24"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth={2}
|
||||
className={cn(
|
||||
'h-4 w-4 shrink-0 opacity-85 transition-transform',
|
||||
jobsOpen && 'rotate-180',
|
||||
)}
|
||||
>
|
||||
<path d="M6 9l6 6 6-6" />
|
||||
</svg>
|
||||
</button>
|
||||
{jobsOpen && (
|
||||
<div className="mb-1 ml-4 flex flex-col border-l-2 border-border pl-2">
|
||||
{JOB_ROUTES.map((route) => (
|
||||
<Link
|
||||
key={route.href}
|
||||
href={route.href}
|
||||
className={cn(
|
||||
TAB_BASE,
|
||||
'px-4 py-2 text-[0.85rem]',
|
||||
pathname === route.href && TAB_ACTIVE,
|
||||
)}
|
||||
>
|
||||
{route.label}
|
||||
</Link>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
<Link
|
||||
href="/datasets"
|
||||
className={cn(TAB_BASE, pathname === '/datasets' && TAB_ACTIVE)}
|
||||
>
|
||||
Datasets
|
||||
</Link>
|
||||
<Link
|
||||
href="/gallery"
|
||||
className={cn(TAB_BASE, pathname === '/gallery' && TAB_ACTIVE)}
|
||||
>
|
||||
Gallery
|
||||
</Link>
|
||||
<Link
|
||||
href="/gpus"
|
||||
className={cn(TAB_BASE, pathname === '/gpus' && TAB_ACTIVE)}
|
||||
>
|
||||
GPUs
|
||||
</Link>
|
||||
<Link
|
||||
href="/settings"
|
||||
className={cn(TAB_BASE, pathname === '/settings' && TAB_ACTIVE)}
|
||||
>
|
||||
Settings
|
||||
</Link>
|
||||
</nav>
|
||||
)}
|
||||
|
||||
<div
|
||||
className={cn(
|
||||
'absolute bottom-0 p-2',
|
||||
isCollapsed ? '-right-[60px] top-0' : 'right-0',
|
||||
)}
|
||||
>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setIsCollapsed((v) => !v)}
|
||||
title={isCollapsed ? 'Expand sidebar' : 'Collapse sidebar'}
|
||||
className={cn(
|
||||
'flex items-center justify-center rounded-lg text-muted-foreground transition-colors hover:bg-accent hover:text-foreground',
|
||||
isCollapsed ? 'p-3' : 'p-2',
|
||||
)}
|
||||
>
|
||||
<svg
|
||||
viewBox="0 0 24 24"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth={2}
|
||||
className="h-[18px] w-[18px]"
|
||||
>
|
||||
<path d={isCollapsed ? 'M9 18l6-6-6-6' : 'M15 18l-6-6 6-6'} />
|
||||
</svg>
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{!isCollapsed && (
|
||||
<div
|
||||
role="presentation"
|
||||
onMouseDown={onMouseDown}
|
||||
className={cn(
|
||||
'absolute bottom-0 right-0 top-0 z-[1] w-1.5 cursor-col-resize hover:bg-accent-blue/25',
|
||||
isDragging && 'bg-accent-blue/25',
|
||||
)}
|
||||
/>
|
||||
)}
|
||||
</aside>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,76 @@
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import GpuGrid from './GpuGrid';
|
||||
import { getGpus } from '@/lib/api';
|
||||
import type { GpuSnapshot } from '@/lib/api';
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
getGpus: vi.fn(),
|
||||
}));
|
||||
|
||||
const SNAPSHOT: GpuSnapshot = {
|
||||
available: true,
|
||||
error: null,
|
||||
gpus: [
|
||||
{
|
||||
index: 0,
|
||||
name: 'NVIDIA B200',
|
||||
utilization: 62,
|
||||
memory_used_mib: 61_440,
|
||||
memory_total_mib: 183_359,
|
||||
temperature_c: 41,
|
||||
power_watts: 312.4,
|
||||
power_limit_watts: 1000,
|
||||
},
|
||||
{
|
||||
index: 1,
|
||||
name: 'NVIDIA B200',
|
||||
utilization: 0,
|
||||
memory_used_mib: 1_024,
|
||||
memory_total_mib: 183_359,
|
||||
temperature_c: null,
|
||||
power_watts: null,
|
||||
power_limit_watts: null,
|
||||
},
|
||||
],
|
||||
};
|
||||
|
||||
beforeEach(() => {
|
||||
vi.mocked(getGpus).mockResolvedValue(SNAPSHOT);
|
||||
});
|
||||
|
||||
describe('GpuGrid', () => {
|
||||
it('renders a card per GPU with utilization and memory', async () => {
|
||||
render(<GpuGrid />);
|
||||
|
||||
expect(await screen.findAllByText('NVIDIA B200')).toHaveLength(2);
|
||||
expect(screen.getByText('GPU 0')).toBeInTheDocument();
|
||||
expect(screen.getByText('GPU 1')).toBeInTheDocument();
|
||||
expect(screen.getByText('62%')).toBeInTheDocument();
|
||||
expect(screen.getByText('60.0 GiB / 179.1 GiB')).toBeInTheDocument();
|
||||
// Optional sensors render only when present.
|
||||
expect(screen.getByText('41°C')).toBeInTheDocument();
|
||||
expect(screen.getByText('312 W / 1000 W')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('shows the backend-reported reason when telemetry is unavailable', async () => {
|
||||
vi.mocked(getGpus).mockResolvedValue({
|
||||
available: false,
|
||||
gpus: [],
|
||||
error: 'NVML Shared Library Not Found',
|
||||
});
|
||||
render(<GpuGrid />);
|
||||
expect(
|
||||
await screen.findByText(/GPU telemetry unavailable: NVML/),
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('explains when the API server is unreachable', async () => {
|
||||
vi.mocked(getGpus).mockRejectedValue(new Error('network down'));
|
||||
render(<GpuGrid />);
|
||||
expect(
|
||||
await screen.findByText(/Could not reach the API server/),
|
||||
).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,164 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Card, CardContent } from '@/components/ui/card';
|
||||
import { getGpus, type GpuInfo, type GpuSnapshot } from '@/lib/api';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const POLL_INTERVAL_MS = 3000;
|
||||
|
||||
function formatGib(mib: number): string {
|
||||
return `${(mib / 1024).toFixed(1)} GiB`;
|
||||
}
|
||||
|
||||
function Meter({
|
||||
label,
|
||||
percent,
|
||||
detail,
|
||||
warnAt,
|
||||
}: {
|
||||
label: string;
|
||||
percent: number;
|
||||
detail: string;
|
||||
/** Turn the fill rose at this percentage (e.g. VRAM pressure). */
|
||||
warnAt?: number;
|
||||
}) {
|
||||
const clamped = Math.max(0, Math.min(100, percent));
|
||||
return (
|
||||
<div className="flex flex-col gap-1">
|
||||
<div className="flex items-baseline justify-between gap-2 text-xs">
|
||||
<span className="text-muted-foreground">{label}</span>
|
||||
<span className="font-medium tabular-nums text-foreground">
|
||||
{detail}
|
||||
</span>
|
||||
</div>
|
||||
<div
|
||||
role="meter"
|
||||
aria-label={label}
|
||||
aria-valuenow={Math.round(clamped)}
|
||||
aria-valuemin={0}
|
||||
aria-valuemax={100}
|
||||
className="h-1.5 overflow-hidden rounded-full bg-muted"
|
||||
>
|
||||
<div
|
||||
className={cn(
|
||||
'h-full rounded-full bg-accent-blue transition-[width] duration-500',
|
||||
warnAt !== undefined && clamped >= warnAt && 'bg-rose-500',
|
||||
)}
|
||||
style={{ width: `${clamped}%` }}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function GpuCard({ gpu }: { gpu: GpuInfo }) {
|
||||
const memPercent =
|
||||
gpu.memory_total_mib > 0
|
||||
? (gpu.memory_used_mib / gpu.memory_total_mib) * 100
|
||||
: 0;
|
||||
return (
|
||||
<Card>
|
||||
<CardContent className="flex flex-col gap-4 p-5">
|
||||
<div className="flex items-baseline justify-between gap-2">
|
||||
<span className="min-w-0 truncate text-sm font-semibold">
|
||||
{gpu.name}
|
||||
</span>
|
||||
<span className="shrink-0 text-xs font-medium uppercase tracking-wider text-muted-foreground">
|
||||
GPU {gpu.index}
|
||||
</span>
|
||||
</div>
|
||||
<Meter
|
||||
label="Utilization"
|
||||
percent={gpu.utilization}
|
||||
detail={`${gpu.utilization}%`}
|
||||
/>
|
||||
<Meter
|
||||
label="Memory"
|
||||
percent={memPercent}
|
||||
warnAt={90}
|
||||
detail={`${formatGib(gpu.memory_used_mib)} / ${formatGib(gpu.memory_total_mib)}`}
|
||||
/>
|
||||
<div className="flex flex-wrap gap-x-5 gap-y-1 text-xs tabular-nums text-muted-foreground">
|
||||
{gpu.temperature_c != null && <span>{gpu.temperature_c}°C</span>}
|
||||
{gpu.power_watts != null && (
|
||||
<span>
|
||||
{Math.round(gpu.power_watts)} W
|
||||
{gpu.power_limit_watts != null &&
|
||||
` / ${Math.round(gpu.power_limit_watts)} W`}
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
);
|
||||
}
|
||||
|
||||
export default function GpuGrid() {
|
||||
const [snapshot, setSnapshot] = React.useState<GpuSnapshot | null>(null);
|
||||
const [fetchError, setFetchError] = React.useState(false);
|
||||
|
||||
React.useEffect(() => {
|
||||
let mounted = true;
|
||||
let inFlight = false;
|
||||
|
||||
async function poll() {
|
||||
if (inFlight) return;
|
||||
inFlight = true;
|
||||
try {
|
||||
const next = await getGpus();
|
||||
if (mounted) {
|
||||
setSnapshot(next);
|
||||
setFetchError(false);
|
||||
}
|
||||
} catch {
|
||||
if (mounted) setFetchError(true);
|
||||
} finally {
|
||||
inFlight = false;
|
||||
}
|
||||
}
|
||||
|
||||
poll();
|
||||
const interval = setInterval(poll, POLL_INTERVAL_MS);
|
||||
return () => {
|
||||
mounted = false;
|
||||
clearInterval(interval);
|
||||
};
|
||||
}, []);
|
||||
|
||||
if (fetchError && !snapshot) {
|
||||
return (
|
||||
<p className="py-8 text-center text-muted-foreground">
|
||||
Could not reach the API server. GPU status needs the studio API server
|
||||
running.
|
||||
</p>
|
||||
);
|
||||
}
|
||||
if (!snapshot) {
|
||||
return <p className="py-8 text-center text-muted-foreground">Loading…</p>;
|
||||
}
|
||||
if (!snapshot.available) {
|
||||
return (
|
||||
<p className="py-8 text-center text-muted-foreground">
|
||||
GPU telemetry unavailable
|
||||
{snapshot.error ? `: ${snapshot.error}` : '.'}
|
||||
</p>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-4">
|
||||
{fetchError && (
|
||||
<p className="rounded-md border border-amber-500/40 bg-amber-500/10 px-3 py-2 text-sm text-amber-600 dark:text-amber-400">
|
||||
Lost contact with the API server — showing the last known values.
|
||||
</p>
|
||||
)}
|
||||
<div className="grid gap-4 [grid-template-columns:repeat(auto-fill,minmax(280px,1fr))]">
|
||||
{snapshot.gpus.map((gpu) => (
|
||||
<GpuCard key={gpu.index} gpu={gpu} />
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { cva, type VariantProps } from 'class-variance-authority';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const badgeVariants = cva(
|
||||
'inline-flex items-center rounded-full border px-2.5 py-1 text-[11px] font-semibold uppercase tracking-[0.16em] transition-colors',
|
||||
{
|
||||
variants: {
|
||||
variant: {
|
||||
default:
|
||||
'border-blue-400/35 bg-blue-500/15 text-blue-900 dark:text-blue-100',
|
||||
secondary: 'border-border bg-secondary text-secondary-foreground',
|
||||
outline: 'border-border bg-transparent text-muted-foreground',
|
||||
success:
|
||||
'border-emerald-400/30 bg-emerald-500/15 text-emerald-800 dark:text-emerald-100',
|
||||
warning:
|
||||
'border-amber-400/30 bg-amber-500/15 text-amber-800 dark:text-amber-100',
|
||||
destructive:
|
||||
'border-rose-400/30 bg-rose-500/15 text-rose-800 dark:text-rose-100',
|
||||
},
|
||||
},
|
||||
defaultVariants: {
|
||||
variant: 'default',
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
export interface BadgeProps
|
||||
extends React.HTMLAttributes<HTMLDivElement>,
|
||||
VariantProps<typeof badgeVariants> {}
|
||||
|
||||
function Badge({ className, variant, ...props }: BadgeProps) {
|
||||
return <div className={cn(badgeVariants({ variant }), className)} {...props} />;
|
||||
}
|
||||
|
||||
export { Badge, badgeVariants };
|
||||
@@ -0,0 +1,45 @@
|
||||
"use client";
|
||||
|
||||
import * as React from "react";
|
||||
import { Slot } from "@radix-ui/react-slot";
|
||||
import { cva, type VariantProps } from "class-variance-authority";
|
||||
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
const buttonVariants = cva(
|
||||
"inline-flex items-center justify-center gap-2 whitespace-nowrap rounded-xl border !text-sm !font-semibold transition-colors duration-150 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-sky-400/40 disabled:pointer-events-none disabled:cursor-not-allowed disabled:opacity-50",
|
||||
{
|
||||
variants: {
|
||||
variant: {
|
||||
default: "border-slate-700 bg-slate-800 text-white shadow-md hover:bg-slate-700 dark:border-slate-300 dark:bg-slate-300 dark:text-slate-900 dark:hover:bg-slate-200",
|
||||
secondary: "border-input bg-card text-card-foreground hover:bg-accent",
|
||||
outline: "border-input bg-secondary text-secondary-foreground hover:bg-accent",
|
||||
ghost: "border-transparent bg-transparent text-foreground hover:bg-accent",
|
||||
destructive: "border-rose-500/60 bg-rose-600/90 text-white hover:bg-rose-500",
|
||||
},
|
||||
size: {
|
||||
default: "h-10 px-4 py-2",
|
||||
sm: "h-9 rounded-lg px-3 !text-xs",
|
||||
lg: "h-11 px-5 !text-sm",
|
||||
icon: "size-10",
|
||||
"icon-sm": "size-8",
|
||||
},
|
||||
},
|
||||
defaultVariants: {
|
||||
variant: "default",
|
||||
size: "default",
|
||||
},
|
||||
},
|
||||
);
|
||||
|
||||
export interface ButtonProps extends React.ButtonHTMLAttributes<HTMLButtonElement>, VariantProps<typeof buttonVariants> {
|
||||
asChild?: boolean;
|
||||
}
|
||||
|
||||
const Button = React.forwardRef<HTMLButtonElement, ButtonProps>(({ className, variant, size, asChild = false, ...props }, ref) => {
|
||||
const Comp = asChild ? Slot : "button";
|
||||
return <Comp className={cn(buttonVariants({ variant, size, className }))} ref={ref} {...props} />;
|
||||
});
|
||||
Button.displayName = "Button";
|
||||
|
||||
export { Button, buttonVariants };
|
||||
@@ -0,0 +1,84 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const Card = React.forwardRef<HTMLDivElement, React.HTMLAttributes<HTMLDivElement>>(
|
||||
({ className, ...props }, ref) => (
|
||||
<div
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'rounded-2xl border border-border bg-card text-card-foreground shadow-sm',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
),
|
||||
);
|
||||
Card.displayName = 'Card';
|
||||
|
||||
const CardHeader = React.forwardRef<
|
||||
HTMLDivElement,
|
||||
React.HTMLAttributes<HTMLDivElement>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<div
|
||||
ref={ref}
|
||||
className={cn('flex flex-col gap-1.5 p-6', className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
CardHeader.displayName = 'CardHeader';
|
||||
|
||||
const CardTitle = React.forwardRef<
|
||||
HTMLParagraphElement,
|
||||
React.HTMLAttributes<HTMLHeadingElement>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<h3
|
||||
ref={ref}
|
||||
className={cn('text-lg font-semibold leading-none tracking-tight', className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
CardTitle.displayName = 'CardTitle';
|
||||
|
||||
const CardDescription = React.forwardRef<
|
||||
HTMLParagraphElement,
|
||||
React.HTMLAttributes<HTMLParagraphElement>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<p
|
||||
ref={ref}
|
||||
className={cn('text-sm text-muted-foreground', className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
CardDescription.displayName = 'CardDescription';
|
||||
|
||||
const CardContent = React.forwardRef<
|
||||
HTMLDivElement,
|
||||
React.HTMLAttributes<HTMLDivElement>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<div ref={ref} className={cn('p-6 pt-0', className)} {...props} />
|
||||
));
|
||||
CardContent.displayName = 'CardContent';
|
||||
|
||||
const CardFooter = React.forwardRef<
|
||||
HTMLDivElement,
|
||||
React.HTMLAttributes<HTMLDivElement>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<div
|
||||
ref={ref}
|
||||
className={cn('flex items-center p-6 pt-0', className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
CardFooter.displayName = 'CardFooter';
|
||||
|
||||
export {
|
||||
Card,
|
||||
CardHeader,
|
||||
CardFooter,
|
||||
CardTitle,
|
||||
CardDescription,
|
||||
CardContent,
|
||||
};
|
||||
@@ -0,0 +1,117 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import * as DialogPrimitive from '@radix-ui/react-dialog';
|
||||
import { X } from 'lucide-react';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const Dialog = DialogPrimitive.Root;
|
||||
const DialogTrigger = DialogPrimitive.Trigger;
|
||||
const DialogPortal = DialogPrimitive.Portal;
|
||||
const DialogClose = DialogPrimitive.Close;
|
||||
|
||||
const DialogOverlay = React.forwardRef<
|
||||
React.ElementRef<typeof DialogPrimitive.Overlay>,
|
||||
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Overlay>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<DialogPrimitive.Overlay
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'fixed inset-0 z-50 bg-black/70 backdrop-blur-sm data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
DialogOverlay.displayName = DialogPrimitive.Overlay.displayName;
|
||||
|
||||
const DialogContent = React.forwardRef<
|
||||
React.ElementRef<typeof DialogPrimitive.Content>,
|
||||
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Content>
|
||||
>(({ className, children, ...props }, ref) => (
|
||||
<DialogPortal>
|
||||
<DialogOverlay />
|
||||
<DialogPrimitive.Content
|
||||
ref={ref}
|
||||
data-modal=""
|
||||
className={cn(
|
||||
'fixed left-1/2 top-1/2 z-50 grid w-full max-w-lg -translate-x-1/2 -translate-y-1/2 gap-4 rounded-2xl border border-border bg-card p-6 text-card-foreground shadow-lg duration-200 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
{children}
|
||||
<DialogPrimitive.Close className="absolute right-4 top-4 rounded-lg p-1 text-muted-foreground opacity-70 transition-opacity hover:bg-secondary hover:opacity-100 focus:outline-none focus:ring-2 focus:ring-sky-400/40 disabled:pointer-events-none">
|
||||
<X className="h-4 w-4" />
|
||||
<span className="sr-only">Close</span>
|
||||
</DialogPrimitive.Close>
|
||||
</DialogPrimitive.Content>
|
||||
</DialogPortal>
|
||||
));
|
||||
DialogContent.displayName = DialogPrimitive.Content.displayName;
|
||||
|
||||
const DialogHeader = ({
|
||||
className,
|
||||
...props
|
||||
}: React.HTMLAttributes<HTMLDivElement>) => (
|
||||
<div
|
||||
className={cn('flex flex-col gap-1.5 text-left', className)}
|
||||
{...props}
|
||||
/>
|
||||
);
|
||||
DialogHeader.displayName = 'DialogHeader';
|
||||
|
||||
const DialogFooter = ({
|
||||
className,
|
||||
...props
|
||||
}: React.HTMLAttributes<HTMLDivElement>) => (
|
||||
<div
|
||||
className={cn(
|
||||
'flex flex-col-reverse gap-2 sm:flex-row sm:justify-end',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
);
|
||||
DialogFooter.displayName = 'DialogFooter';
|
||||
|
||||
const DialogTitle = React.forwardRef<
|
||||
React.ElementRef<typeof DialogPrimitive.Title>,
|
||||
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Title>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<DialogPrimitive.Title
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'text-lg font-semibold leading-none tracking-tight',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
DialogTitle.displayName = DialogPrimitive.Title.displayName;
|
||||
|
||||
const DialogDescription = React.forwardRef<
|
||||
React.ElementRef<typeof DialogPrimitive.Description>,
|
||||
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Description>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<DialogPrimitive.Description
|
||||
ref={ref}
|
||||
className={cn('text-sm text-muted-foreground', className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
DialogDescription.displayName = DialogPrimitive.Description.displayName;
|
||||
|
||||
export {
|
||||
Dialog,
|
||||
DialogPortal,
|
||||
DialogOverlay,
|
||||
DialogTrigger,
|
||||
DialogClose,
|
||||
DialogContent,
|
||||
DialogHeader,
|
||||
DialogFooter,
|
||||
DialogTitle,
|
||||
DialogDescription,
|
||||
};
|
||||
@@ -0,0 +1,22 @@
|
||||
"use client";
|
||||
|
||||
import * as React from "react";
|
||||
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
const Input = React.forwardRef<HTMLInputElement, React.ComponentProps<"input">>(({ className, type, ...props }, ref) => {
|
||||
return (
|
||||
<input
|
||||
type={type}
|
||||
className={cn(
|
||||
"flex h-10 w-full rounded-xl border border-input bg-card/60 px-3 py-2 text-sm text-foreground shadow-sm backdrop-blur-md transition-colors placeholder:text-muted-foreground focus-visible:border-sky-400/70 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-sky-400/25 disabled:cursor-not-allowed disabled:opacity-50",
|
||||
className,
|
||||
)}
|
||||
ref={ref}
|
||||
{...props}
|
||||
/>
|
||||
);
|
||||
});
|
||||
Input.displayName = "Input";
|
||||
|
||||
export { Input };
|
||||
@@ -0,0 +1,26 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import * as LabelPrimitive from '@radix-ui/react-label';
|
||||
import { cva, type VariantProps } from 'class-variance-authority';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const labelVariants = cva(
|
||||
'text-sm font-medium leading-none text-foreground peer-disabled:cursor-not-allowed peer-disabled:opacity-70',
|
||||
);
|
||||
|
||||
const Label = React.forwardRef<
|
||||
React.ElementRef<typeof LabelPrimitive.Root>,
|
||||
React.ComponentPropsWithoutRef<typeof LabelPrimitive.Root> &
|
||||
VariantProps<typeof labelVariants>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<LabelPrimitive.Root
|
||||
ref={ref}
|
||||
className={cn(labelVariants(), className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
Label.displayName = LabelPrimitive.Root.displayName;
|
||||
|
||||
export { Label };
|
||||
@@ -0,0 +1,24 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const NativeSelect = React.forwardRef<
|
||||
HTMLSelectElement,
|
||||
React.ComponentProps<'select'>
|
||||
>(({ className, children, ...props }, ref) => (
|
||||
<select
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'flex h-10 w-full appearance-none rounded-xl border border-input bg-card px-3 py-2 text-sm text-foreground shadow-sm transition-colors focus-visible:border-sky-400/70 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-sky-400/25 disabled:cursor-not-allowed disabled:opacity-50',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
{children}
|
||||
</select>
|
||||
));
|
||||
NativeSelect.displayName = 'NativeSelect';
|
||||
|
||||
export { NativeSelect };
|
||||
@@ -0,0 +1,45 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import * as ScrollAreaPrimitive from '@radix-ui/react-scroll-area';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const ScrollArea = React.forwardRef<
|
||||
React.ElementRef<typeof ScrollAreaPrimitive.Root>,
|
||||
React.ComponentPropsWithoutRef<typeof ScrollAreaPrimitive.Root>
|
||||
>(({ className, children, ...props }, ref) => (
|
||||
<ScrollAreaPrimitive.Root
|
||||
ref={ref}
|
||||
className={cn('relative overflow-hidden', className)}
|
||||
{...props}
|
||||
>
|
||||
<ScrollAreaPrimitive.Viewport className="h-full w-full rounded-[inherit]">
|
||||
{children}
|
||||
</ScrollAreaPrimitive.Viewport>
|
||||
<ScrollBar />
|
||||
<ScrollAreaPrimitive.Corner />
|
||||
</ScrollAreaPrimitive.Root>
|
||||
));
|
||||
ScrollArea.displayName = ScrollAreaPrimitive.Root.displayName;
|
||||
|
||||
const ScrollBar = React.forwardRef<
|
||||
React.ElementRef<typeof ScrollAreaPrimitive.ScrollAreaScrollbar>,
|
||||
React.ComponentPropsWithoutRef<typeof ScrollAreaPrimitive.ScrollAreaScrollbar>
|
||||
>(({ className, orientation = 'vertical', ...props }, ref) => (
|
||||
<ScrollAreaPrimitive.ScrollAreaScrollbar
|
||||
ref={ref}
|
||||
orientation={orientation}
|
||||
className={cn(
|
||||
'flex touch-none select-none p-0.5 transition-colors',
|
||||
orientation === 'vertical' ? 'h-full w-2.5' : 'h-2.5 flex-col',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
<ScrollAreaPrimitive.ScrollAreaThumb className="relative flex-1 rounded-full bg-border" />
|
||||
</ScrollAreaPrimitive.ScrollAreaScrollbar>
|
||||
));
|
||||
ScrollBar.displayName = ScrollAreaPrimitive.ScrollAreaScrollbar.displayName;
|
||||
|
||||
export { ScrollArea, ScrollBar };
|
||||
@@ -0,0 +1,158 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import * as SelectPrimitive from '@radix-ui/react-select';
|
||||
import { Check, ChevronDown, ChevronUp } from 'lucide-react';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const Select = SelectPrimitive.Root;
|
||||
const SelectGroup = SelectPrimitive.Group;
|
||||
const SelectValue = SelectPrimitive.Value;
|
||||
|
||||
const SelectTrigger = React.forwardRef<
|
||||
React.ElementRef<typeof SelectPrimitive.Trigger>,
|
||||
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Trigger>
|
||||
>(({ className, children, ...props }, ref) => (
|
||||
<SelectPrimitive.Trigger
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'flex h-10 w-full items-center justify-between gap-2 rounded-xl border border-input bg-card px-3 py-2 text-sm text-foreground shadow-sm outline-none transition-colors placeholder:text-muted-foreground focus:border-sky-400/70 focus:ring-2 focus:ring-sky-400/25 disabled:cursor-not-allowed disabled:opacity-50 [&>span]:line-clamp-1',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
{children}
|
||||
<SelectPrimitive.Icon asChild>
|
||||
<ChevronDown className="h-4 w-4 text-muted-foreground" />
|
||||
</SelectPrimitive.Icon>
|
||||
</SelectPrimitive.Trigger>
|
||||
));
|
||||
SelectTrigger.displayName = SelectPrimitive.Trigger.displayName;
|
||||
|
||||
const SelectScrollUpButton = React.forwardRef<
|
||||
React.ElementRef<typeof SelectPrimitive.ScrollUpButton>,
|
||||
React.ComponentPropsWithoutRef<typeof SelectPrimitive.ScrollUpButton>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<SelectPrimitive.ScrollUpButton
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'flex cursor-default items-center justify-center py-1 text-muted-foreground',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
<ChevronUp className="h-4 w-4" />
|
||||
</SelectPrimitive.ScrollUpButton>
|
||||
));
|
||||
SelectScrollUpButton.displayName =
|
||||
SelectPrimitive.ScrollUpButton.displayName;
|
||||
|
||||
const SelectScrollDownButton = React.forwardRef<
|
||||
React.ElementRef<typeof SelectPrimitive.ScrollDownButton>,
|
||||
React.ComponentPropsWithoutRef<typeof SelectPrimitive.ScrollDownButton>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<SelectPrimitive.ScrollDownButton
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'flex cursor-default items-center justify-center py-1 text-muted-foreground',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
<ChevronDown className="h-4 w-4" />
|
||||
</SelectPrimitive.ScrollDownButton>
|
||||
));
|
||||
SelectScrollDownButton.displayName =
|
||||
SelectPrimitive.ScrollDownButton.displayName;
|
||||
|
||||
const SelectContent = React.forwardRef<
|
||||
React.ElementRef<typeof SelectPrimitive.Content>,
|
||||
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Content>
|
||||
>(({ className, children, position = 'popper', ...props }, ref) => (
|
||||
<SelectPrimitive.Portal>
|
||||
<SelectPrimitive.Content
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'relative z-50 max-h-80 min-w-[8rem] overflow-hidden rounded-2xl border border-border bg-popover text-popover-foreground shadow-lg backdrop-blur-sm',
|
||||
position === 'popper' &&
|
||||
'data-[side=bottom]:translate-y-1 data-[side=left]:-translate-x-1 data-[side=right]:translate-x-1 data-[side=top]:-translate-y-1',
|
||||
className,
|
||||
)}
|
||||
position={position}
|
||||
{...props}
|
||||
>
|
||||
<SelectScrollUpButton />
|
||||
<SelectPrimitive.Viewport
|
||||
className={cn(
|
||||
'p-1',
|
||||
position === 'popper' &&
|
||||
'h-[var(--radix-select-trigger-height)] w-full min-w-[var(--radix-select-trigger-width)]',
|
||||
)}
|
||||
>
|
||||
{children}
|
||||
</SelectPrimitive.Viewport>
|
||||
<SelectScrollDownButton />
|
||||
</SelectPrimitive.Content>
|
||||
</SelectPrimitive.Portal>
|
||||
));
|
||||
SelectContent.displayName = SelectPrimitive.Content.displayName;
|
||||
|
||||
const SelectLabel = React.forwardRef<
|
||||
React.ElementRef<typeof SelectPrimitive.Label>,
|
||||
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Label>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<SelectPrimitive.Label
|
||||
ref={ref}
|
||||
className={cn('px-2 py-1.5 text-xs font-semibold text-muted-foreground', className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
SelectLabel.displayName = SelectPrimitive.Label.displayName;
|
||||
|
||||
const SelectItem = React.forwardRef<
|
||||
React.ElementRef<typeof SelectPrimitive.Item>,
|
||||
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Item>
|
||||
>(({ className, children, ...props }, ref) => (
|
||||
<SelectPrimitive.Item
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'relative flex w-full cursor-default select-none items-center rounded-xl py-2 pl-8 pr-3 text-sm text-foreground outline-none data-[disabled]:pointer-events-none data-[disabled]:opacity-50 data-[highlighted]:bg-accent data-[highlighted]:text-accent-foreground',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
<span className="absolute left-2 flex h-3.5 w-3.5 items-center justify-center">
|
||||
<SelectPrimitive.ItemIndicator>
|
||||
<Check className="h-4 w-4 text-accent-blue" />
|
||||
</SelectPrimitive.ItemIndicator>
|
||||
</span>
|
||||
<SelectPrimitive.ItemText>{children}</SelectPrimitive.ItemText>
|
||||
</SelectPrimitive.Item>
|
||||
));
|
||||
SelectItem.displayName = SelectPrimitive.Item.displayName;
|
||||
|
||||
const SelectSeparator = React.forwardRef<
|
||||
React.ElementRef<typeof SelectPrimitive.Separator>,
|
||||
React.ComponentPropsWithoutRef<typeof SelectPrimitive.Separator>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<SelectPrimitive.Separator
|
||||
ref={ref}
|
||||
className={cn('-mx-1 my-1 h-px bg-border', className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
SelectSeparator.displayName = SelectPrimitive.Separator.displayName;
|
||||
|
||||
export {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectGroup,
|
||||
SelectItem,
|
||||
SelectLabel,
|
||||
SelectScrollDownButton,
|
||||
SelectScrollUpButton,
|
||||
SelectSeparator,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
};
|
||||
@@ -0,0 +1,31 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import * as SeparatorPrimitive from '@radix-ui/react-separator';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const Separator = React.forwardRef<
|
||||
React.ElementRef<typeof SeparatorPrimitive.Root>,
|
||||
React.ComponentPropsWithoutRef<typeof SeparatorPrimitive.Root>
|
||||
>(
|
||||
(
|
||||
{ className, orientation = 'horizontal', decorative = true, ...props },
|
||||
ref,
|
||||
) => (
|
||||
<SeparatorPrimitive.Root
|
||||
ref={ref}
|
||||
decorative={decorative}
|
||||
orientation={orientation}
|
||||
className={cn(
|
||||
'shrink-0 bg-border',
|
||||
orientation === 'horizontal' ? 'h-px w-full' : 'h-full w-px',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
),
|
||||
);
|
||||
Separator.displayName = SeparatorPrimitive.Root.displayName;
|
||||
|
||||
export { Separator };
|
||||
@@ -0,0 +1,28 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import * as SliderPrimitive from '@radix-ui/react-slider';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const Slider = React.forwardRef<
|
||||
React.ElementRef<typeof SliderPrimitive.Root>,
|
||||
React.ComponentPropsWithoutRef<typeof SliderPrimitive.Root>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<SliderPrimitive.Root
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'relative flex w-full touch-none select-none items-center',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
<SliderPrimitive.Track className="relative h-1.5 w-full grow overflow-hidden rounded-full bg-border">
|
||||
<SliderPrimitive.Range className="absolute h-full bg-accent-blue" />
|
||||
</SliderPrimitive.Track>
|
||||
<SliderPrimitive.Thumb className="block h-4 w-4 rounded-full border-2 border-accent-blue bg-accent-blue shadow transition-colors hover:border-accent-blue/80 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-sky-400/40 disabled:pointer-events-none disabled:opacity-50" />
|
||||
</SliderPrimitive.Root>
|
||||
));
|
||||
Slider.displayName = SliderPrimitive.Root.displayName;
|
||||
|
||||
export { Slider };
|
||||
@@ -0,0 +1,25 @@
|
||||
'use client';
|
||||
|
||||
import { Toaster as Sonner, type ToasterProps } from 'sonner';
|
||||
|
||||
export function Toaster(props: ToasterProps) {
|
||||
return (
|
||||
<Sonner
|
||||
theme="dark"
|
||||
position="bottom-center"
|
||||
richColors={true}
|
||||
toastOptions={{
|
||||
classNames: {
|
||||
toast:
|
||||
'border border-border bg-card text-card-foreground shadow-lg',
|
||||
description: 'text-muted-foreground',
|
||||
actionButton:
|
||||
'bg-primary text-primary-foreground hover:bg-primary/90',
|
||||
cancelButton:
|
||||
'bg-secondary text-secondary-foreground hover:bg-secondary/80',
|
||||
},
|
||||
}}
|
||||
{...props}
|
||||
/>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import * as SwitchPrimitives from '@radix-ui/react-switch';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const Switch = React.forwardRef<
|
||||
React.ElementRef<typeof SwitchPrimitives.Root>,
|
||||
React.ComponentPropsWithoutRef<typeof SwitchPrimitives.Root>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<SwitchPrimitives.Root
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'peer inline-flex h-5 w-9 shrink-0 cursor-pointer items-center rounded-full border border-border transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-sky-400/40 disabled:cursor-not-allowed disabled:opacity-50 data-[state=checked]:border-accent-blue data-[state=checked]:bg-accent-blue data-[state=unchecked]:bg-background',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
>
|
||||
<SwitchPrimitives.Thumb
|
||||
className={cn(
|
||||
'pointer-events-none block h-3.5 w-3.5 rounded-full bg-muted-foreground shadow-lg ring-0 transition-transform data-[state=checked]:translate-x-4 data-[state=checked]:bg-white data-[state=unchecked]:translate-x-0.5',
|
||||
)}
|
||||
/>
|
||||
</SwitchPrimitives.Root>
|
||||
));
|
||||
Switch.displayName = SwitchPrimitives.Root.displayName;
|
||||
|
||||
export { Switch };
|
||||
@@ -0,0 +1,52 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import * as TabsPrimitive from '@radix-ui/react-tabs';
|
||||
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const Tabs = TabsPrimitive.Root;
|
||||
|
||||
const TabsList = React.forwardRef<
|
||||
React.ElementRef<typeof TabsPrimitive.List>,
|
||||
React.ComponentPropsWithoutRef<typeof TabsPrimitive.List>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<TabsPrimitive.List
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'inline-flex items-center justify-center rounded-lg border border-border bg-background p-0.5 text-muted-foreground',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
TabsList.displayName = TabsPrimitive.List.displayName;
|
||||
|
||||
const TabsTrigger = React.forwardRef<
|
||||
React.ElementRef<typeof TabsPrimitive.Trigger>,
|
||||
React.ComponentPropsWithoutRef<typeof TabsPrimitive.Trigger>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<TabsPrimitive.Trigger
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'inline-flex items-center justify-center whitespace-nowrap rounded-md px-3 py-1.5 text-sm font-medium text-muted-foreground transition-colors hover:text-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-sky-400/40 disabled:pointer-events-none disabled:opacity-50 data-[state=active]:bg-card data-[state=active]:text-foreground data-[state=active]:shadow-sm',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
TabsTrigger.displayName = TabsPrimitive.Trigger.displayName;
|
||||
|
||||
const TabsContent = React.forwardRef<
|
||||
React.ElementRef<typeof TabsPrimitive.Content>,
|
||||
React.ComponentPropsWithoutRef<typeof TabsPrimitive.Content>
|
||||
>(({ className, ...props }, ref) => (
|
||||
<TabsPrimitive.Content
|
||||
ref={ref}
|
||||
className={cn('mt-2 focus-visible:outline-none', className)}
|
||||
{...props}
|
||||
/>
|
||||
));
|
||||
TabsContent.displayName = TabsPrimitive.Content.displayName;
|
||||
|
||||
export { Tabs, TabsList, TabsTrigger, TabsContent };
|
||||
@@ -0,0 +1,21 @@
|
||||
"use client";
|
||||
|
||||
import * as React from "react";
|
||||
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
const Textarea = React.forwardRef<HTMLTextAreaElement, React.ComponentProps<"textarea">>(({ className, ...props }, ref) => {
|
||||
return (
|
||||
<textarea
|
||||
className={cn(
|
||||
"flex min-h-24 w-full resize-y rounded-xl border border-input bg-card px-3 py-2 text-sm text-foreground shadow-sm transition-colors placeholder:text-muted-foreground focus-visible:border-sky-400/70 focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-sky-400/25 disabled:cursor-not-allowed disabled:opacity-50",
|
||||
className,
|
||||
)}
|
||||
ref={ref}
|
||||
{...props}
|
||||
/>
|
||||
);
|
||||
});
|
||||
Textarea.displayName = "Textarea";
|
||||
|
||||
export { Textarea };
|
||||
@@ -0,0 +1,59 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { Moon, Sun } from 'lucide-react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
|
||||
/**
|
||||
* Light/dark switch (ported from Dreamverse). The studio defaults to dark —
|
||||
* layout.tsx applies the stored choice before paint; this button just
|
||||
* mirrors and persists it.
|
||||
*/
|
||||
function ThemeToggle({ className }: { className?: string }) {
|
||||
const [dark, setDark] = React.useState(true);
|
||||
|
||||
React.useEffect(() => {
|
||||
setDark(document.documentElement.classList.contains('dark'));
|
||||
}, []);
|
||||
|
||||
const transitionTimer =
|
||||
React.useRef<ReturnType<typeof setTimeout>>(undefined);
|
||||
|
||||
function toggle() {
|
||||
const next = !dark;
|
||||
setDark(next);
|
||||
|
||||
clearTimeout(transitionTimer.current);
|
||||
document.documentElement.classList.add('theme-transition');
|
||||
document.documentElement.classList.toggle('dark', next);
|
||||
|
||||
transitionTimer.current = setTimeout(() => {
|
||||
document.documentElement.classList.remove('theme-transition');
|
||||
}, 350);
|
||||
|
||||
try {
|
||||
localStorage.setItem('theme', next ? 'dark' : 'light');
|
||||
} catch {
|
||||
// storage unavailable
|
||||
}
|
||||
}
|
||||
|
||||
React.useEffect(() => {
|
||||
return () => clearTimeout(transitionTimer.current);
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<Button
|
||||
variant="outline"
|
||||
size="icon-sm"
|
||||
onClick={toggle}
|
||||
aria-label={dark ? 'Switch to light mode' : 'Switch to dark mode'}
|
||||
className={className}
|
||||
>
|
||||
{dark ? <Sun className="size-4" /> : <Moon className="size-4" />}
|
||||
</Button>
|
||||
);
|
||||
}
|
||||
|
||||
export { ThemeToggle };
|
||||
@@ -0,0 +1,84 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
export type ResizableEdge = 'left' | 'right';
|
||||
|
||||
export interface UseResizableOptions {
|
||||
/** Which edge is fixed: "left" = drag right grows; "right" = drag left grows. */
|
||||
edge: ResizableEdge;
|
||||
minWidth: number;
|
||||
maxWidth: number;
|
||||
/** Return current width (e.g. from parent state). */
|
||||
getWidth: () => number;
|
||||
/** Called with the new width while the user drags. */
|
||||
onWidth: (width: number) => void;
|
||||
/** Optional: notified when a drag starts/stops. */
|
||||
onDragChange?: (dragging: boolean) => void;
|
||||
}
|
||||
|
||||
export interface UseResizableResult {
|
||||
onMouseDown: (event: React.MouseEvent) => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* React port of the Svelte `resizable` action. Spread the returned
|
||||
* `onMouseDown` onto a drag handle; the parent owns the width state.
|
||||
*/
|
||||
export function useResizable(
|
||||
options: UseResizableOptions,
|
||||
): UseResizableResult {
|
||||
const optionsRef = React.useRef(options);
|
||||
optionsRef.current = options;
|
||||
|
||||
const dragStart = React.useRef({ x: 0, width: 0 });
|
||||
const dragging = React.useRef(false);
|
||||
|
||||
const onMouseMove = React.useCallback((e: MouseEvent) => {
|
||||
const opts = optionsRef.current;
|
||||
const delta = e.clientX - dragStart.current.x;
|
||||
const sign = opts.edge === 'left' ? 1 : -1;
|
||||
const newWidth = Math.min(
|
||||
opts.maxWidth,
|
||||
Math.max(opts.minWidth, dragStart.current.width + sign * delta),
|
||||
);
|
||||
opts.onWidth(newWidth);
|
||||
}, []);
|
||||
|
||||
const onMouseUp = React.useCallback(() => {
|
||||
dragging.current = false;
|
||||
optionsRef.current.onDragChange?.(false);
|
||||
document.body.style.cursor = '';
|
||||
document.body.style.userSelect = '';
|
||||
document.removeEventListener('mousemove', onMouseMove);
|
||||
document.removeEventListener('mouseup', onMouseUp);
|
||||
}, [onMouseMove]);
|
||||
|
||||
const onMouseDown = React.useCallback(
|
||||
(e: React.MouseEvent) => {
|
||||
e.preventDefault();
|
||||
const opts = optionsRef.current;
|
||||
dragStart.current = { x: e.clientX, width: opts.getWidth() };
|
||||
dragging.current = true;
|
||||
opts.onDragChange?.(true);
|
||||
document.body.style.cursor = 'col-resize';
|
||||
document.body.style.userSelect = 'none';
|
||||
document.addEventListener('mousemove', onMouseMove);
|
||||
document.addEventListener('mouseup', onMouseUp);
|
||||
},
|
||||
[onMouseMove, onMouseUp],
|
||||
);
|
||||
|
||||
React.useEffect(() => {
|
||||
return () => {
|
||||
if (dragging.current) {
|
||||
document.body.style.cursor = '';
|
||||
document.body.style.userSelect = '';
|
||||
document.removeEventListener('mousemove', onMouseMove);
|
||||
document.removeEventListener('mouseup', onMouseUp);
|
||||
}
|
||||
};
|
||||
}, [onMouseMove, onMouseUp]);
|
||||
|
||||
return { onMouseDown };
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
'use client';
|
||||
|
||||
import { useSyncExternalStore } from 'react';
|
||||
|
||||
import type { ManagedStore } from '@/stores/createManagedStore';
|
||||
|
||||
export function useStore<T>(
|
||||
store: ManagedStore<T>,
|
||||
): T {
|
||||
return useSyncExternalStore(
|
||||
store.subscribe,
|
||||
store.get,
|
||||
store.getServerSnapshot,
|
||||
);
|
||||
}
|
||||
@@ -1,76 +0,0 @@
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
/**
|
||||
* Svelte action: attach horizontal resize behavior to a handle element.
|
||||
* Use on the resize-handle div inside a sidebar; the parent holds width state.
|
||||
*/
|
||||
|
||||
export type ResizableEdge = "left" | "right";
|
||||
|
||||
export interface ResizableOptions {
|
||||
/** Which edge is fixed: "left" = drag right increases width, "right" = drag left increases width */
|
||||
edge: ResizableEdge;
|
||||
minWidth: number;
|
||||
maxWidth: number;
|
||||
/** Return current width (e.g. from parent state) */
|
||||
getWidth: () => number;
|
||||
/** Called with new width when user drags */
|
||||
onWidth: (width: number) => void;
|
||||
/** Optional: called when drag starts/stops so parent can show active state */
|
||||
onDragChange?: (dragging: boolean) => void;
|
||||
}
|
||||
|
||||
export function resizable(
|
||||
node: HTMLElement,
|
||||
options: ResizableOptions,
|
||||
): { update?: (opts: ResizableOptions) => void; destroy?: () => void } {
|
||||
let dragStart = { x: 0, width: 0 };
|
||||
let listenersAttached = false;
|
||||
|
||||
function onMouseMove(e: MouseEvent) {
|
||||
const delta = e.clientX - dragStart.x;
|
||||
const sign = options.edge === "left" ? 1 : -1;
|
||||
const newWidth = Math.min(
|
||||
options.maxWidth,
|
||||
Math.max(options.minWidth, dragStart.width + sign * delta),
|
||||
);
|
||||
options.onWidth(newWidth);
|
||||
}
|
||||
|
||||
function onMouseUp() {
|
||||
listenersAttached = false;
|
||||
options.onDragChange?.(false);
|
||||
document.body.style.cursor = "";
|
||||
document.body.style.userSelect = "";
|
||||
document.removeEventListener("mousemove", onMouseMove);
|
||||
document.removeEventListener("mouseup", onMouseUp);
|
||||
}
|
||||
|
||||
function onMouseDown(e: MouseEvent) {
|
||||
e.preventDefault();
|
||||
dragStart = { x: e.clientX, width: options.getWidth() };
|
||||
listenersAttached = true;
|
||||
options.onDragChange?.(true);
|
||||
document.body.style.cursor = "col-resize";
|
||||
document.body.style.userSelect = "none";
|
||||
document.addEventListener("mousemove", onMouseMove);
|
||||
document.addEventListener("mouseup", onMouseUp);
|
||||
}
|
||||
|
||||
node.addEventListener("mousedown", onMouseDown);
|
||||
|
||||
return {
|
||||
update(newOptions: ResizableOptions) {
|
||||
options = newOptions;
|
||||
},
|
||||
destroy() {
|
||||
node.removeEventListener("mousedown", onMouseDown);
|
||||
if (listenersAttached) {
|
||||
options.onDragChange?.(false);
|
||||
document.body.style.cursor = "";
|
||||
document.body.style.userSelect = "";
|
||||
document.removeEventListener("mousemove", onMouseMove);
|
||||
document.removeEventListener("mouseup", onMouseUp);
|
||||
}
|
||||
},
|
||||
};
|
||||
}
|
||||
@@ -1,30 +1,52 @@
|
||||
// SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
import { env } from "$env/dynamic/public";
|
||||
import type { Job, JobType } from "$lib/types";
|
||||
import {
|
||||
STORAGE_KEY,
|
||||
loadDefaultOptions,
|
||||
type DefaultOptions,
|
||||
} from "./defaultOptions";
|
||||
import type { Job, JobType } from "./types";
|
||||
|
||||
const DEFAULT_API_BASE_URL = "http://localhost:8189/api";
|
||||
|
||||
// getApiBaseUrl runs on every request and in render paths (media URL
|
||||
// builders), so cache the configured URL keyed on the raw stored options
|
||||
// string instead of re-parsing the options JSON each call.
|
||||
let cachedRaw: string | null = null;
|
||||
let cachedConfigured = "";
|
||||
|
||||
export function getApiBaseUrl(): string {
|
||||
// 1) Check user-configured value from local storage (Settings page)
|
||||
// 1) Check user-configured value from the persisted options (Settings
|
||||
// page). The settings sync persists the full options object, so the key
|
||||
// holds an empty string until the user sets a URL — treat that as unset
|
||||
// and fall through to the env/default below.
|
||||
if (typeof window !== "undefined") {
|
||||
try {
|
||||
const stored = window.localStorage.getItem("fastvideo-default-options");
|
||||
if (stored) {
|
||||
const parsed = JSON.parse(stored) as { apiServerBaseUrl?: string };
|
||||
return parsed.apiServerBaseUrl!.trim();
|
||||
const raw = window.localStorage.getItem(STORAGE_KEY);
|
||||
if (raw !== cachedRaw) {
|
||||
cachedRaw = raw;
|
||||
cachedConfigured = stripTrailingSlash(
|
||||
loadDefaultOptions().apiServerBaseUrl.trim(),
|
||||
);
|
||||
}
|
||||
if (cachedConfigured) return cachedConfigured;
|
||||
} catch {
|
||||
// Ignore storage / JSON errors and fall back to env/default
|
||||
// Ignore storage errors and fall back to env/default
|
||||
}
|
||||
}
|
||||
|
||||
// 2) Fall back to env var if set, then hardcoded default
|
||||
const fromEnv = process.env.NEXT_PUBLIC_API_BASE_URL;
|
||||
const apiUrl =
|
||||
typeof env.PUBLIC_API_BASE_URL === "string" && env.PUBLIC_API_BASE_URL
|
||||
? env.PUBLIC_API_BASE_URL
|
||||
: DEFAULT_API_BASE_URL;
|
||||
return apiUrl;
|
||||
typeof fromEnv === "string" && fromEnv ? fromEnv : DEFAULT_API_BASE_URL;
|
||||
return stripTrailingSlash(apiUrl);
|
||||
}
|
||||
|
||||
// The base URL is used as a prefix in `${base}/jobs` template strings, so a
|
||||
// trailing slash (a natural thing to paste into Settings) would produce `//`
|
||||
// and 404 every request.
|
||||
function stripTrailingSlash(url: string): string {
|
||||
return url.replace(/\/+$/, "");
|
||||
}
|
||||
|
||||
/** Full URL for streaming a job's video/image output (for <video> or <img> src). */
|
||||
@@ -64,41 +86,44 @@ export interface CreateJobRequest {
|
||||
vsa_sparsity?: number;
|
||||
tp_size?: number;
|
||||
sp_size?: number;
|
||||
// DMD distillation extras (sent by CreateJobModal)
|
||||
dmd_use_vsa?: boolean;
|
||||
dmd_vsa_sparsity?: number;
|
||||
dmd_denoising_steps?: string;
|
||||
real_score_guidance_scale?: number;
|
||||
generator_update_interval?: number;
|
||||
real_score_model_path?: string;
|
||||
fake_score_model_path?: string;
|
||||
}
|
||||
|
||||
export interface Model {
|
||||
id: string;
|
||||
label: string;
|
||||
type: string;
|
||||
}
|
||||
|
||||
export interface Settings {
|
||||
defaultModelId: string;
|
||||
defaultModelIdT2v: string;
|
||||
defaultModelIdI2v: string;
|
||||
defaultModelIdT2i: string;
|
||||
numInferenceSteps: number;
|
||||
numFrames: number;
|
||||
height: number;
|
||||
width: number;
|
||||
guidanceScale: number;
|
||||
guidanceRescale: number;
|
||||
fps: number;
|
||||
seed: number;
|
||||
numGpus: number;
|
||||
ditCpuOffload: boolean;
|
||||
textEncoderCpuOffload: boolean;
|
||||
vaeCpuOffload: boolean;
|
||||
imageEncoderCpuOffload: boolean;
|
||||
useFsdpInference: boolean;
|
||||
enableTorchCompile: boolean;
|
||||
vsaSparsity: number;
|
||||
tpSize: number;
|
||||
spSize: number;
|
||||
autoStartJob: boolean;
|
||||
datasetUploadPath: string;
|
||||
export interface GpuInfo {
|
||||
index: number;
|
||||
name: string;
|
||||
utilization: number;
|
||||
memory_used_mib: number;
|
||||
memory_total_mib: number;
|
||||
temperature_c: number | null;
|
||||
power_watts: number | null;
|
||||
power_limit_watts: number | null;
|
||||
}
|
||||
|
||||
export interface GpuSnapshot {
|
||||
available: boolean;
|
||||
gpus: GpuInfo[];
|
||||
error: string | null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Server-persisted settings: everything in DefaultOptions except
|
||||
* apiServerBaseUrl, which is purely local (per-browser).
|
||||
*/
|
||||
export type Settings = Omit<DefaultOptions, "apiServerBaseUrl">;
|
||||
|
||||
// MARK: - API Functions
|
||||
|
||||
export async function getSettings(): Promise<Settings> {
|
||||
@@ -177,6 +202,15 @@ export async function getModels(workloadType?: string): Promise<Model[]> {
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export async function getGpus(): Promise<GpuSnapshot> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/gpus`);
|
||||
if (!response.ok) {
|
||||
throw new Error("Failed to fetch GPU status");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export async function getJobsList(jobType?: JobType): Promise<Job[]> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const url = jobType
|
||||
@@ -204,15 +238,6 @@ export async function createJob(job: CreateJobRequest): Promise<Job> {
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export async function getJobDetails(id: string): Promise<Job> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/${id}`);
|
||||
if (!response.ok) {
|
||||
throw new Error("Failed to fetch job");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export async function startJob(id: string): Promise<Job> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/${id}/start`, {
|
||||
@@ -326,15 +351,6 @@ export async function getDatasets(): Promise<Dataset[]> {
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export async function getDataset(id: string): Promise<Dataset> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/datasets/${id}`);
|
||||
if (!response.ok) {
|
||||
throw new Error("Failed to fetch dataset");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export async function createDataset(
|
||||
data: CreateDatasetRequest,
|
||||
): Promise<Dataset> {
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user