Compare commits
60
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c59cf79ea4 | ||
|
|
fab9b3159c | ||
|
|
d377f463f1 | ||
|
|
01050277ea | ||
|
|
a943220c11 | ||
|
|
556ac7088e | ||
|
|
e7456f1b75 | ||
|
|
c993d7393e | ||
|
|
7f83164233 | ||
|
|
4e52f47d1e | ||
|
|
e19913f6e9 | ||
|
|
2413a57651 | ||
|
|
7bb76b5ec9 | ||
|
|
0bd19a976b | ||
|
|
40b93784d2 | ||
|
|
33d3478bad | ||
|
|
3d8ac9d14b | ||
|
|
aaef49bfc6 | ||
|
|
cf6a00b9be | ||
|
|
1ae39562dd | ||
|
|
26064193e2 | ||
|
|
8446fc003e | ||
|
|
a28f2bab4b | ||
|
|
f82d8be4bf | ||
|
|
8e1775183e | ||
|
|
620bc36dc4 | ||
|
|
29ff16ec96 | ||
|
|
8f9d76a80d | ||
|
|
a4d9a75e2c | ||
|
|
b2db0c0a13 | ||
|
|
6aa7d8a278 | ||
|
|
ccc9014430 | ||
|
|
a159b63c67 | ||
|
|
ac48bb3cd1 | ||
|
|
3987b9ddcd | ||
|
|
c7da2f5d60 | ||
|
|
39ae1decc0 | ||
|
|
1aed667377 | ||
|
|
c1612ff397 | ||
|
|
a534ba20a0 | ||
|
|
e9bbaca07d | ||
|
|
9bfa585448 | ||
|
|
b2062556a9 | ||
|
|
c9c5585758 | ||
|
|
9212f4f218 | ||
|
|
6388db815b | ||
|
|
7a4285189f | ||
|
|
a837fe841a | ||
|
|
f9e3680f11 | ||
|
|
98f761ec45 | ||
|
|
c041318f2c | ||
|
|
604e0205a4 | ||
|
|
13213395b4 | ||
|
|
46afee5998 | ||
|
|
c488fa1211 | ||
|
|
d3cff517cd | ||
|
|
2f3d407406 | ||
|
|
bcffa4026e | ||
|
|
6d6a10be7a | ||
|
|
73dd105f3d |
@@ -0,0 +1,99 @@
|
||||
---
|
||||
name: ci-runner
|
||||
description: Work on FastVideo's Slurm-only, change-aware GPU CI lanes, static Buildkite graph, trusted ci-runner policy, lane scripts, and GB200 validation.
|
||||
---
|
||||
|
||||
# Slinky Slurm CI lanes
|
||||
|
||||
FastVideo's `ci-runner` Buildkite queue is the control plane for all active
|
||||
GPU CI. A private host-owned dispatcher leases GPUs from the Slinky Slurm tray
|
||||
and runs the immutable PR SHA inside an isolated Enroot container. Buildkite
|
||||
pipeline upload and Slurm submission occur on the login plane; every test
|
||||
payload executes on Slurm compute.
|
||||
|
||||
The files under `fastvideo/tests/modal/` and `.buildkite/scripts/pr_test.sh`
|
||||
are dormant rollback code. Never add an active Buildkite or slash-command
|
||||
route to them. `pr_test.sh` must continue to reject Buildkite invocations.
|
||||
|
||||
The private operator bundle is deliberately outside this repository because
|
||||
it contains site paths and credentials. See
|
||||
`docs/contributing/ci_architecture.md`; this skill covers the repository half
|
||||
and the coordination contract with that bundle.
|
||||
|
||||
## Invariants
|
||||
|
||||
- `.buildkite/pipeline.yml` contains exactly one static step for every active
|
||||
GPU lane. Each step pins a unique key and label, a 90-minute timeout, the
|
||||
trusted `/opt/fastvideo-ci-runner/run-ci` command (`run-unit` is the one
|
||||
compatibility wrapper), step-level internal `TEST_TYPE`, and
|
||||
`queue: "ci-runner"`.
|
||||
- Active CI contains no `pr_test.sh` command, Modal invocation, default queue,
|
||||
Buildkite plugin, `soft_fail`, or job-controlled artifact glob.
|
||||
- The six Fastcheck lanes use `:microscope:` labels. Full-Suite-only lanes use
|
||||
`:test_tube:` or `:bar_chart:` so direct reruns update the right aggregate.
|
||||
- SSIM and vanilla training request all four GPUs. Keep both in the
|
||||
`fastvideo/slinky/whole-tray` Buildkite concurrency group with a limit of one
|
||||
so the second job does not consume an agent or command timeout while waiting
|
||||
for the same tray.
|
||||
- `/test full` schedules all twenty lanes. `/merge`, `ready`, and new pushes to
|
||||
ready PRs use the trusted base-branch planner in
|
||||
`.github/scripts/plan_merge_ci.py`: automatic Fastcheck remains the universal
|
||||
six-lane baseline, and the merge build adds only path-relevant integration
|
||||
lanes. Unknown source/build paths fail closed to all fourteen additive lanes.
|
||||
The trusted uploader still normalizes and validates the complete static graph
|
||||
before Buildkite evaluates its plan conditions.
|
||||
- Focused merge builds may pass allowlisted golden-gate and SSIM test basenames.
|
||||
The private host validates the lane plan and basenames before staging them,
|
||||
and the in-container scripts validate them again. Direct `/test ssim`,
|
||||
explicit `/test full`, and the weekly main-branch schedule run the complete
|
||||
SSIM matrix.
|
||||
- The trusted uploader serves exactly three entry pipelines:
|
||||
`pr-fastcheck` for automatic PR builds, `ci` for slash-command/ready-label
|
||||
API builds, and `fastvideo-performance-lane` for the weekly schedule. Keep
|
||||
incoming GitHub webhook processing disabled on `ci` so it cannot duplicate
|
||||
`pr-fastcheck` on every PR update.
|
||||
- Test payloads live in `.buildkite/scripts/unit_test.sh` or executable
|
||||
`.buildkite/scripts/lanes/<lane>.sh`. Backend policy (GPU count, extras,
|
||||
secrets, kernel build, artifacts) stays in the agent-owned lane table.
|
||||
- Tests must preserve an inherited `MASTER_PORT`. Packed containers share the
|
||||
tray network namespace, so the private runner assigns a distinct port range
|
||||
per GPU lease and the SSIM scheduler assigns task offsets within its range.
|
||||
- The ARM64 runner image includes the pinned FA4 CuTe overlay validated on
|
||||
GB200. Keep SSIM at `FASTVIDEO_FA4=1` because its references were seeded with
|
||||
FA4; keep lanes with FA2 baselines at `FASTVIDEO_FA4=0`. A runner image change
|
||||
must revalidate both the FA4 import and an actual GB200 forward kernel.
|
||||
- `fastvideo/tests/ssim/ci_runner.py` is the active four-GPU SSIM scheduler.
|
||||
New SSIM files are discovered through `REQUIRED_GPUS` and
|
||||
`*_MODEL_TO_PARAMS`; do not wire them through the dormant Modal scheduler.
|
||||
- The host policy fail-closes unknown tuples. A repository-side lane change is
|
||||
inert until the operator updates the private lane table and uploader policy
|
||||
in the same rollout.
|
||||
|
||||
## Adding or changing a lane
|
||||
|
||||
1. Read the closest `AGENTS.md` and the domain-specific testing guide.
|
||||
2. Add or update the executable lane payload under `.buildkite/scripts/`.
|
||||
Keep it deterministic and free of host-specific paths or credential fetches.
|
||||
3. Add the static pipeline step and canonical `/test <name>` mapping. Keep the
|
||||
`<name>-ci` alias only when compatibility requires it.
|
||||
4. Add its source/test path ownership to `.github/scripts/plan_merge_ci.py`.
|
||||
Prefer the narrowest correctness-preserving lane set; leave unknown paths
|
||||
fail-closed. Extend `fastvideo/tests/contract/test_ci_test_collection.py`,
|
||||
`test_merge_ci_plan.py`, and focused CPU-only scheduler/policy tests.
|
||||
5. Coordinate the private lane row: GPU count (1-4), wall time, script, scope
|
||||
pairs, step key, command, HF cache/token, tracking mode, extras, attention
|
||||
backend policy, kernel policy, and artifact relay. Active training lanes
|
||||
keep W&B offline and do not stage a W&B credential.
|
||||
6. Update the trusted pipeline-uploader schema. A mismatch must reject the
|
||||
pipeline rather than silently skip a lane.
|
||||
7. Run `pre-commit run --files <changed paths>`, the planner's representative
|
||||
diff matrix, contract tests, private driver tests, and a real GB200 canary.
|
||||
Multi-GPU, hardware-reference, training, performance, and SSIM changes need
|
||||
their own target-hardware evidence.
|
||||
|
||||
## Rollback
|
||||
|
||||
Rollback the Slurm routing/configuration change or pause the `ci-runner` queue.
|
||||
Do not silently reactivate Modal. A manual Modal experiment requires the
|
||||
explicit local opt-in documented in `ci_architecture.md`; returning it to
|
||||
production CI needs a separate reviewed decision.
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
name: reseed-ssim-references
|
||||
description: Re-seed HF reference videos for a single existing SSIM test on Modal L40S. Always backs up current refs locally first, regenerates on Modal, pauses for the user to eyeball before-vs-after quality, then overwrites the targeted `<model_id>` subtree on `FastVideo/ssim-reference-videos` with `--force`. Use when an intentional code change (model port fix, attention backend swap, kernel upgrade, hyperparameter change) has invalidated existing refs and they need to be regenerated. Pairs with `seed-ssim-references`, which is for first-time seeding only.
|
||||
description: Re-seed HF reference videos for a single existing SSIM test on Modal L40S. Always backs up current refs locally first, regenerates on Modal, pauses for the user to eyeball before-vs-after quality, then overwrites the targeted model subtree on `FastVideo/ssim-reference-videos` with `--force`. Use when an intentional code change (model port fix, attention backend swap, kernel upgrade, hyperparameter change) has invalidated existing refs and they need to be regenerated. Pairs with `seed-ssim-references`, which is for first-time seeding only.
|
||||
---
|
||||
|
||||
# Re-seed SSIM Reference Videos
|
||||
@@ -13,7 +13,7 @@ on HF — the old refs are overwritten — so the skill always:
|
||||
|
||||
1. Confirms intent with a one-liner the user has to type.
|
||||
2. Downloads the existing refs as a local, timestamped backup.
|
||||
3. Regenerates on Modal L40S (same code path that CI uses).
|
||||
3. Regenerates through the manual legacy Modal L40S maintenance path.
|
||||
4. Pauses for a side-by-side eyeball of backup vs new mp4s.
|
||||
5. Uploads with `--force`, scoped to the single `--model-id`.
|
||||
6. Reminds the user to keep the backup until the PR lands.
|
||||
@@ -51,8 +51,9 @@ harder to recover from than failing closed.
|
||||
|
||||
Hardcoded:
|
||||
|
||||
- Modal GPU: **L40S** (matches CI; re-seeding from another SKU produces refs
|
||||
that L40S CI cannot match).
|
||||
- Modal GPU: **L40S**. This is a manual reference-maintenance target, not the
|
||||
active Slurm CI compute path; changing the SKU also changes the historical
|
||||
`L40S_reference_videos` contract.
|
||||
- Quality tier: **`default`**. `full_quality` is a separate, deliberate
|
||||
operation.
|
||||
- HF repo: `FastVideo/ssim-reference-videos` (override via
|
||||
|
||||
@@ -35,7 +35,8 @@ The skill is run **manually**, once per new test. Before invoking it, the user
|
||||
has already sanity-tested the new test locally — it launches `VideoGenerator`
|
||||
and writes an artefact without crashing (the missing-reference assertion at
|
||||
the end is expected). The skill does not re-test locally; it goes straight
|
||||
to Modal L40S (which is what CI uses).
|
||||
to the manual legacy Modal L40S reference-maintenance target. Active CI runs
|
||||
on the Slinky Slurm cluster and only consumes the resulting references.
|
||||
|
||||
## When to use
|
||||
|
||||
@@ -61,7 +62,8 @@ Prompt the user for it if they didn't supply it.
|
||||
|
||||
Everything else is fixed:
|
||||
|
||||
- Modal runner GPU: **L40S** (hardcoded in `fastvideo/tests/modal/ssim_test.py`).
|
||||
- Modal maintenance GPU: **L40S** (hardcoded in
|
||||
`fastvideo/tests/modal/ssim_test.py`; this is not the active CI compute path).
|
||||
- Device folder: `L40S_reference_videos`.
|
||||
- Quality tier: `default` (the tier CI runs). The `full_quality` tier is not
|
||||
seeded by this skill.
|
||||
|
||||
+461
-528
File diff suppressed because it is too large
Load Diff
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the OpenAI-compatible API lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest ./fastvideo/tests/entrypoints/test_openai_api_integration.py -vs
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the distillation-DMD lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest ./fastvideo/tests/training/distill/test_distill_dmd.py -vs
|
||||
Executable
+87
@@ -0,0 +1,87 @@
|
||||
#!/usr/bin/env bash
|
||||
# DreamVerse needs a GPU for import-time device resolution, but it does not
|
||||
# build or exercise fastvideo-kernel. A checksummed Node archive is installed
|
||||
# in the disposable Slurm container because the shared CI image is
|
||||
# Python/CUDA focused.
|
||||
set -euo pipefail
|
||||
|
||||
node_version=v22.23.2
|
||||
case $(uname -m) in
|
||||
aarch64 | arm64)
|
||||
node_arch=arm64
|
||||
node_archive_sha256=013b59cfd2819703a6f4a14ab891fc46fc2a4e3f5bcd92de3fb4929b43e35b30
|
||||
;;
|
||||
x86_64 | amd64)
|
||||
node_arch=x64
|
||||
node_archive_sha256=b294a556e639d64338823920e5866c21c02741742d2e1529ee1a225c1ec9252a
|
||||
;;
|
||||
*)
|
||||
echo "Unsupported architecture for DreamVerse Node runtime: $(uname -m)" >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
node_archive="node-${node_version}-linux-${node_arch}.tar.gz"
|
||||
node_runtime_root=$(mktemp -d -t fastvideo-node.XXXXXX)
|
||||
node_archive_path="${node_runtime_root}/${node_archive}"
|
||||
node_install_dir="${node_runtime_root}/${node_archive%.tar.gz}"
|
||||
curl --proto '=https' --tlsv1.2 --retry 5 --retry-all-errors \
|
||||
--location --fail --silent --show-error \
|
||||
"https://nodejs.org/dist/${node_version}/${node_archive}" \
|
||||
--output "$node_archive_path"
|
||||
printf '%s %s\n' "$node_archive_sha256" "$node_archive_path" | sha256sum --check --status
|
||||
tar -xzf "$node_archive_path" -C "$node_runtime_root"
|
||||
export PATH="${node_install_dir}/bin:${PATH}"
|
||||
node --version
|
||||
npm --version
|
||||
|
||||
export PYTHONPATH="$(pwd)/apps/dreamverse${PYTHONPATH:+:$PYTHONPATH}"
|
||||
pytest apps/dreamverse/dreamverse/tests -q
|
||||
|
||||
cd apps/dreamverse/web
|
||||
npm ci
|
||||
npm run typecheck
|
||||
npm test
|
||||
machine_arch=$(uname -m)
|
||||
if [[ $machine_arch =~ ^(aarch64|arm64)$ ]]; then
|
||||
npx playwright install --with-deps chromium firefox
|
||||
else
|
||||
npx playwright install --with-deps chromium webkit firefox
|
||||
fi
|
||||
|
||||
master_port=${MASTER_PORT:-7959}
|
||||
BACKEND_PORT=${BACKEND_PORT:-$((master_port + 50))}
|
||||
python -m uvicorn dreamverse.mock_server:app --host 127.0.0.1 --port "$BACKEND_PORT" &
|
||||
mock_server_pid=$!
|
||||
cleanup() {
|
||||
kill "$mock_server_pid" 2>/dev/null || true
|
||||
wait "$mock_server_pid" 2>/dev/null || true
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
for _ in {1..30}; do
|
||||
curl -fsS "http://127.0.0.1:$BACKEND_PORT/healthz" && break
|
||||
sleep 1
|
||||
done
|
||||
curl -fsS "http://127.0.0.1:$BACKEND_PORT/healthz"
|
||||
|
||||
if [[ $machine_arch =~ ^(aarch64|arm64)$ ]]; then
|
||||
# Playwright WebKit traps before opening a page on Linux ARM64, and its
|
||||
# bundled Chromium lacks the H.264/AAC codecs used by the fMP4 assertions.
|
||||
# Firefox covers every flow, including streaming. Chromium and its mobile
|
||||
# profile still cover all codec-independent UI behavior on GB200.
|
||||
BACKEND_HOST=127.0.0.1 BACKEND_PORT="$BACKEND_PORT" CI=1 \
|
||||
npm run e2e -- --project=firefox
|
||||
BACKEND_HOST=127.0.0.1 BACKEND_PORT="$BACKEND_PORT" CI=1 \
|
||||
npm run e2e -- \
|
||||
--project=chromium \
|
||||
--project=mobile-chromium \
|
||||
--grep-invert='streams, plays, and surfaces a downloadable clip|starts a new project and switches back to the prior session|saved projects persist across a page reload'
|
||||
else
|
||||
BACKEND_HOST=127.0.0.1 BACKEND_PORT="$BACKEND_PORT" CI=1 \
|
||||
npm run e2e -- \
|
||||
--project=chromium \
|
||||
--project=webkit \
|
||||
--project=firefox \
|
||||
--project=mobile-safari \
|
||||
--project=mobile-chromium
|
||||
fi
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the encoder lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest ./fastvideo/tests/encoders -vs
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the evaluation lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest ./fastvideo/tests/eval -vs
|
||||
Executable
+35
@@ -0,0 +1,35 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the golden-gate lane. Environment (HF_HOME
|
||||
# and authentication) is the runner's responsibility.
|
||||
set -euo pipefail
|
||||
|
||||
golden_root=./fastvideo/tests/golden_gate
|
||||
selected=${FASTVIDEO_GOLDEN_TEST_FILES-}
|
||||
if [ -z "$selected" ]; then
|
||||
if [ "${TEST_SCOPE:-}" = merge ]; then
|
||||
echo "Missing FASTVIDEO_GOLDEN_TEST_FILES for merge scope" >&2
|
||||
exit 2
|
||||
fi
|
||||
selected=all
|
||||
fi
|
||||
if [ "$selected" = all ]; then
|
||||
exec pytest "$golden_root" -xvs
|
||||
fi
|
||||
|
||||
[[ $selected =~ ^test_[a-z0-9_]+\.py(,test_[a-z0-9_]+\.py)*$ ]] || {
|
||||
echo "Invalid FASTVIDEO_GOLDEN_TEST_FILES selection" >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
IFS=, read -r -a golden_files <<< "$selected"
|
||||
golden_paths=()
|
||||
for golden_file in "${golden_files[@]}"; do
|
||||
golden_path="$golden_root/$golden_file"
|
||||
[ -f "$golden_path" ] || {
|
||||
echo "Selected golden test does not exist: $golden_file" >&2
|
||||
exit 2
|
||||
}
|
||||
golden_paths+=("$golden_path")
|
||||
done
|
||||
|
||||
exec pytest "${golden_paths[@]}" -xvs
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the LoRA-inference lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest ./fastvideo/tests/inference/lora/test_lora_inference_similarity.py -vs
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the VMoBA-inference lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec python fastvideo/tests/inference/vmoba/test_vmoba_inference.py
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the custom-kernel lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest fastvideo-kernel/tests/ -vs
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the LoRA-extraction lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest ./fastvideo/tests/lora_extraction/test_lora_extraction.py -vs
|
||||
Executable
+52
@@ -0,0 +1,52 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm performance lane. Reports are written outside the checkout
|
||||
# so the trusted host driver can upload them after untrusted code exits.
|
||||
set -uo pipefail
|
||||
|
||||
export PERFORMANCE_TRACKING_ROOT=/tmp/perf-tracking
|
||||
export PERF_REPORTS_DIR=/workspace/artifacts/performance
|
||||
mkdir -p "$PERF_REPORTS_DIR"
|
||||
|
||||
if [[ ${BUILDKITE_PULL_REQUEST:-false} =~ ^[1-9][0-9]*$ ]]; then
|
||||
export PERF_RUN_SOURCE=pr
|
||||
export PERF_UPLOAD_POLICY=pass
|
||||
elif [ "${BUILDKITE_BRANCH:-}" = main ] \
|
||||
&& { [ "${BUILDKITE_SOURCE:-}" = schedule ] || [ "${TEST_SCOPE:-}" = full ]; }; then
|
||||
export PERF_RUN_SOURCE=scheduled_main
|
||||
export PERF_UPLOAD_POLICY=always
|
||||
elif [ "${TEST_SCOPE:-}" = direct ]; then
|
||||
export PERF_RUN_SOURCE=unknown
|
||||
export PERF_UPLOAD_POLICY=pass
|
||||
else
|
||||
export PERF_RUN_SOURCE=unknown
|
||||
export PERF_UPLOAD_POLICY=never
|
||||
fi
|
||||
|
||||
nvidia-smi \
|
||||
--query-gpu=index,timestamp,clocks.sm,clocks.max.sm,power.draw,power.limit,temperature.gpu \
|
||||
--format=csv -l 10 > "$PERF_REPORTS_DIR/gpu_telemetry.csv" 2>/dev/null &
|
||||
telemetry_pid=$!
|
||||
cleanup() {
|
||||
kill "$telemetry_pid" 2>/dev/null || true
|
||||
wait "$telemetry_pid" 2>/dev/null || true
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
pytest ./fastvideo/tests/performance -vs
|
||||
pytest_rc=$?
|
||||
compare_rc=0
|
||||
if [ "$pytest_rc" -eq 0 ] || [ "$PERF_UPLOAD_POLICY" = always ]; then
|
||||
PERF_PYTEST_RC=$pytest_rc python ./fastvideo/tests/performance/compare_baseline.py
|
||||
compare_rc=$?
|
||||
fi
|
||||
python ./fastvideo/tests/performance/dashboard.py || true
|
||||
cp -f fastvideo/tests/performance/results/*.json "$PERF_REPORTS_DIR/" 2>/dev/null || true
|
||||
|
||||
echo "--- GPU telemetry (clocks.sm vs clocks.max.sm reveals capped hosts) ---"
|
||||
cat "$PERF_REPORTS_DIR/gpu_telemetry.csv" || true
|
||||
|
||||
final_rc=$pytest_rc
|
||||
if [ "$final_rc" -eq 0 ]; then
|
||||
final_rc=$compare_rc
|
||||
fi
|
||||
exit "$final_rc"
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the self-forcing lane.
|
||||
set -euo pipefail
|
||||
|
||||
export WANDB_MODE=offline
|
||||
exec pytest ./fastvideo/tests/training/self-forcing/test_self_forcing.py -vs
|
||||
Executable
+40
@@ -0,0 +1,40 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical four-GPU SSIM lane for the Slinky Slurm worker.
|
||||
set -euo pipefail
|
||||
|
||||
args=()
|
||||
if [ "${FASTVIDEO_SSIM_BOOTSTRAP_MODE:-0}" = 1 ]; then
|
||||
args+=(--bootstrap-mode)
|
||||
fi
|
||||
selected=${FASTVIDEO_SSIM_TEST_FILES-}
|
||||
if [ -z "$selected" ]; then
|
||||
if [ "${TEST_SCOPE:-}" = merge ]; then
|
||||
echo "Missing FASTVIDEO_SSIM_TEST_FILES for merge scope" >&2
|
||||
exit 2
|
||||
fi
|
||||
selected=all
|
||||
fi
|
||||
if [ "$selected" != all ]; then
|
||||
[[ $selected =~ ^test_[a-z0-9_]+\.py(,test_[a-z0-9_]+\.py)*$ ]] || {
|
||||
echo "Invalid FASTVIDEO_SSIM_TEST_FILES selection" >&2
|
||||
exit 2
|
||||
}
|
||||
IFS=, read -r -a ssim_files <<< "$selected"
|
||||
for ssim_file in "${ssim_files[@]}"; do
|
||||
args+=(--test-file "$ssim_file")
|
||||
done
|
||||
fi
|
||||
|
||||
# MoGe's utils3d dependency builds glcontext from source on ARM64. The current
|
||||
# runner image predates the baked-in X11 headers below, so keep this guarded
|
||||
# bootstrap until every deployed image digest contains libx11-dev.
|
||||
if [ ! -f /usr/include/X11/Xlib.h ]; then
|
||||
apt-get -o Acquire::Retries=5 update
|
||||
apt-get -o Acquire::Retries=5 install -y --no-install-recommends libx11-dev
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
fi
|
||||
|
||||
uv pip install git+https://github.com/microsoft/MoGe.git
|
||||
uv pip install k_diffusion einops_exts alias_free_torch torchsde
|
||||
|
||||
exec python fastvideo/tests/ssim/ci_runner.py "${args[@]}"
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the modular training-framework lane.
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest ./fastvideo/tests/train/models ./fastvideo/tests/train/methods -vs
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the legacy vanilla-training lane.
|
||||
set -euo pipefail
|
||||
|
||||
export WANDB_MODE=offline
|
||||
exec pytest ./fastvideo/tests/training/Vanilla -srP
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the legacy LoRA-training lane.
|
||||
set -euo pipefail
|
||||
|
||||
export WANDB_MODE=offline
|
||||
exec pytest ./fastvideo/tests/training/lora/test_lora_training.py -srP
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the legacy VSA-training lane.
|
||||
set -euo pipefail
|
||||
|
||||
export WANDB_MODE=offline
|
||||
exec pytest ./fastvideo/tests/training/VSA -srP
|
||||
Executable
+9
@@ -0,0 +1,9 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the transformer lane.
|
||||
set -euo pipefail
|
||||
|
||||
# The existing block reference records an absent FASTVIDEO_FA4 (FA2). Keep
|
||||
# that reference identity; the component lane also selects FA2 explicitly.
|
||||
env -u FASTVIDEO_FA4 pytest ./fastvideo/tests/golden_gate/test_wan_t2v.py -xvs
|
||||
pytest ./fastvideo/tests/golden_gate/test_wan_causal.py -xvs
|
||||
exec pytest ./fastvideo/tests/transformers -vs
|
||||
Executable
+6
@@ -0,0 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
# Canonical Slurm CI selection for the VAE lane.
|
||||
set -euo pipefail
|
||||
|
||||
pytest ./fastvideo/tests/golden_gate/test_wan_vae.py -xvs
|
||||
exec pytest ./fastvideo/tests/vaes -vs
|
||||
@@ -1,6 +1,19 @@
|
||||
#!/bin/bash
|
||||
set -uo pipefail
|
||||
|
||||
# DORMANT ROLLBACK ONLY. Active CI is Slurm-only and pipeline.yml never calls
|
||||
# this launcher. Refuse every Buildkite invocation even if a stale step or
|
||||
# operator typo reaches this file; local rollback experiments require an
|
||||
# explicit opt-in.
|
||||
if [ -n "${BUILDKITE:-}" ]; then
|
||||
echo "Legacy Modal CI is disabled; use the Slinky Slurm runner." >&2
|
||||
exit 2
|
||||
fi
|
||||
if [ "${FASTVIDEO_ENABLE_LEGACY_MODAL_CI:-0}" != 1 ]; then
|
||||
echo "Legacy Modal CI is dormant. Set FASTVIDEO_ENABLE_LEGACY_MODAL_CI=1 only for a manual rollback test." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
log() {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] $1"
|
||||
}
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
exec pytest \
|
||||
./fastvideo/tests/api/ \
|
||||
./fastvideo/tests/contract/ \
|
||||
./fastvideo/tests/dataset/ \
|
||||
./fastvideo/tests/workflow/ \
|
||||
./fastvideo/tests/entrypoints/ \
|
||||
./fastvideo/tests/loader/ \
|
||||
./fastvideo/tests/pipelines/ \
|
||||
./fastvideo/tests/platforms/ \
|
||||
./fastvideo/tests/train/ \
|
||||
./fastvideo/tests/stages/ \
|
||||
./fastvideo/tests/ops/ \
|
||||
./fastvideo/tests/worker/ \
|
||||
./fastvideo/tests/training/test_trackers.py \
|
||||
./fastvideo/tests/attention/test_sdpa_metadata_mask_contract.py \
|
||||
./fastvideo/tests/modal/test_kernel_build_cache.py \
|
||||
./fastvideo/tests/modal/test_pr_test.py \
|
||||
./fastvideo/tests/modal/test_ssim_test.py \
|
||||
--ignore=./fastvideo/tests/entrypoints/test_openai_api_integration.py \
|
||||
--ignore=./fastvideo/tests/train/models \
|
||||
--ignore=./fastvideo/tests/train/methods \
|
||||
-vs
|
||||
@@ -8,10 +8,10 @@ PR TITLE: Must start with a type tag, e.g.:
|
||||
MERGE WORKFLOW:
|
||||
1. Ensure pre-commit passes and you have at least 1 approval
|
||||
2. Comment /merge (or add the "ready" label) to enter the Merge Queue
|
||||
3. Full Test Suite runs automatically on a staging branch → auto-merge on success
|
||||
3. A path-aware merge gate runs only relevant integration tests → auto-merge on success
|
||||
|
||||
ON-DEMAND TESTING (write access required):
|
||||
/test full — Full Test Suite /test ssim — SSIM regression
|
||||
/test full — Explicit all-lane run /test ssim — Full SSIM regression
|
||||
/test training — Training pipeline /test encoder — Encoder tests
|
||||
/test transformer — Transformer tests /test vae — VAE tests
|
||||
/test kernel — CUDA kernel tests /test unit — Unit tests
|
||||
|
||||
@@ -1,14 +1,14 @@
|
||||
#!/usr/bin/env bash
|
||||
# Gate the expensive Buildkite full suite on the cheap GitHub checks.
|
||||
# Gate the path-aware Buildkite merge plan on the cheap GitHub checks.
|
||||
#
|
||||
# Polls the workflow runs for the PR head commit and only exits 0 once the
|
||||
# watched cheap workflows (pre-commit, docs build) have succeeded, so the
|
||||
# 'ready' label cannot burn ~20 GPU lanes on a head that a cheap check has
|
||||
# already doomed.
|
||||
# 'ready' label cannot burn path-selected GPU lanes on a head that a cheap
|
||||
# check has already doomed.
|
||||
#
|
||||
# Semantics:
|
||||
# - watched run completed with a bad conclusion -> exit 1 (fail CLOSED:
|
||||
# no full suite; the next push re-arms via the 'synchronize' trigger)
|
||||
# no merge gate; the next push re-arms via the 'synchronize' trigger)
|
||||
# - watched run cancelled -> still pending: the docs
|
||||
# workflow's repo-global 'pages' concurrency group cancels runs superseded
|
||||
# by unrelated pushes, so 'cancelled' is not a verdict on this PR
|
||||
@@ -29,7 +29,7 @@ set -euo pipefail
|
||||
: "${PR_NUMBER:?PR_NUMBER (pull request number) is required}"
|
||||
: "${GITHUB_REPOSITORY:?GITHUB_REPOSITORY is required}"
|
||||
|
||||
# Workflow-level `name:` values that must be green before the full suite
|
||||
# Workflow-level `name:` values that must be green before the merge gate
|
||||
# may start. "Deploy Documentation" is path-filtered on PRs, so its run may
|
||||
# legitimately never exist; pre-commit always runs, so it must appear.
|
||||
WATCHED_NAMES='["pre-commit", "Deploy Documentation"]'
|
||||
@@ -56,7 +56,7 @@ recheck_ready_label() {
|
||||
if pr_json=$(gh_api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" 2>/dev/null); then
|
||||
if ! jq -e '[.labels[]?.name] | index("ready")' <<<"$pr_json" >/dev/null 2>&1; then
|
||||
echo "::error::PR #${PR_NUMBER} no longer has the 'ready' label —" \
|
||||
"NOT triggering the Buildkite full suite. Re-add the label to re-arm."
|
||||
"NOT triggering the Buildkite merge gate. Re-add the label to re-arm."
|
||||
exit 1
|
||||
fi
|
||||
else
|
||||
@@ -84,7 +84,7 @@ while true; do
|
||||
| map(.name) | join(", ")' <<<"$state")
|
||||
if [ -n "$failed" ]; then
|
||||
echo "::error::Cheap check(s) failed on ${PR_SHA}: ${failed}." \
|
||||
"NOT triggering the Buildkite full suite. Push a fix (the 'ready'" \
|
||||
"NOT triggering the Buildkite merge gate. Push a fix (the 'ready'" \
|
||||
"label re-arms on every push), or re-run the failed check and then" \
|
||||
"re-run this workflow."
|
||||
exit 1
|
||||
@@ -97,7 +97,7 @@ while true; do
|
||||
if [ "$pending" -eq 0 ]; then
|
||||
if [ -z "$missing" ]; then
|
||||
recheck_ready_label
|
||||
echo "All watched cheap checks are green — full suite may proceed."
|
||||
echo "All watched cheap checks are green — merge gate may proceed."
|
||||
exit 0
|
||||
fi
|
||||
case "$missing" in
|
||||
@@ -119,14 +119,14 @@ while true; do
|
||||
echo "::warning::GitHub API error querying workflow runs for ${PR_SHA} (attempt ${api_fails}/3)."
|
||||
if [ "$api_fails" -ge 3 ]; then
|
||||
recheck_ready_label
|
||||
echo "::warning::FAILING OPEN: cannot query GitHub check status — triggering the full suite WITHOUT the cheap-check gate."
|
||||
echo "::warning::FAILING OPEN: cannot query GitHub check status — triggering the merge gate WITHOUT the cheap-check gate."
|
||||
exit 0
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$elapsed" -ge "$MAX_WAIT_SECS" ]; then
|
||||
recheck_ready_label
|
||||
echo "::warning::FAILING OPEN: watched checks still pending after $(( MAX_WAIT_SECS / 60 )) min${missing:+ (never appeared: ${missing})} — triggering the full suite anyway."
|
||||
echo "::warning::FAILING OPEN: watched checks still pending after $(( MAX_WAIT_SECS / 60 )) min${missing:+ (never appeared: ${missing})} — triggering the merge gate anyway."
|
||||
exit 0
|
||||
fi
|
||||
sleep "$POLL_SECS"
|
||||
|
||||
@@ -0,0 +1,582 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Select the additive GPU integration lanes needed by a PR diff.
|
||||
|
||||
Fastcheck is the universal six-lane baseline and is intentionally not repeated
|
||||
here. This planner selects only the more expensive merge-gate lanes. Unknown
|
||||
source/build paths fail closed to the complete integration set, while explicit
|
||||
documentation and repository-metadata paths require no additional GPU work.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import fnmatch
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import TextIO
|
||||
|
||||
MERGE_LANES = (
|
||||
"golden-gate",
|
||||
"ssim",
|
||||
"lora-inference",
|
||||
"lora-extraction",
|
||||
"training",
|
||||
"distillation",
|
||||
"self-forcing",
|
||||
"lora-training",
|
||||
"training-vsa",
|
||||
"inference-vmoba",
|
||||
"performance",
|
||||
"api-server",
|
||||
"train-framework",
|
||||
"eval",
|
||||
)
|
||||
|
||||
LANE_SCRIPT_TO_KEY = {
|
||||
"api_server.sh": "api-server",
|
||||
"distillation_dmd.sh": "distillation",
|
||||
"eval.sh": "eval",
|
||||
"golden_gate.sh": "golden-gate",
|
||||
"inference_lora.sh": "lora-inference",
|
||||
"inference_vmoba.sh": "inference-vmoba",
|
||||
"lora_extraction.sh": "lora-extraction",
|
||||
"performance.sh": "performance",
|
||||
"self_forcing.sh": "self-forcing",
|
||||
"ssim.sh": "ssim",
|
||||
"train_framework.sh": "train-framework",
|
||||
"training.sh": "training",
|
||||
"training_lora.sh": "lora-training",
|
||||
"training_vsa.sh": "training-vsa",
|
||||
}
|
||||
|
||||
FASTCHECK_LANE_SCRIPTS = {
|
||||
"dreamverse.sh",
|
||||
"encoder.sh",
|
||||
"kernel_tests.sh",
|
||||
"transformer.sh",
|
||||
"vae.sh",
|
||||
}
|
||||
|
||||
LEGACY_TRAINING_LANES = (
|
||||
"training",
|
||||
"distillation",
|
||||
"self-forcing",
|
||||
"lora-training",
|
||||
"training-vsa",
|
||||
)
|
||||
|
||||
ALL_TRAINING_LANES = (*LEGACY_TRAINING_LANES, "train-framework")
|
||||
|
||||
SSIM_SMOKE_TESTS = (
|
||||
"test_flux_t2i_similarity.py",
|
||||
"test_wan_t2v_similarity.py",
|
||||
)
|
||||
|
||||
SAFE_PATTERNS = (
|
||||
"*.md",
|
||||
"*.rst",
|
||||
".agents/**",
|
||||
".claude/**",
|
||||
".codex/**",
|
||||
".github/ISSUE_TEMPLATE/**",
|
||||
".github/PULL_REQUEST_TEMPLATE.md",
|
||||
".github/dependabot.yml",
|
||||
".github/mergify.yml",
|
||||
".github/scripts/**",
|
||||
".github/workflows/**",
|
||||
".buildkite/scripts/pre_commit.sh",
|
||||
".git-blame-ignore-revs",
|
||||
".gitattributes",
|
||||
".gitignore",
|
||||
".pre-commit-config.yaml",
|
||||
"AGENTS.md",
|
||||
"CITATION.cff",
|
||||
"CODE_OF_CONDUCT.md",
|
||||
"CONTRIBUTING.md",
|
||||
"LICENSE",
|
||||
"NOTICE",
|
||||
"__init__.py",
|
||||
"collect_env.py",
|
||||
"SECURITY.md",
|
||||
"assets/**",
|
||||
"comfyui/**",
|
||||
"docs/**",
|
||||
"examples/**",
|
||||
"mkdocs.yml",
|
||||
"requirements-mkdocs.in",
|
||||
"requirements-mkdocs.txt",
|
||||
"scripts/**",
|
||||
"tests/__init__.py",
|
||||
"tests/local_tests/**",
|
||||
)
|
||||
|
||||
ALL_IMPACT_PATTERNS = (
|
||||
".buildkite/pipeline.yml",
|
||||
"docker/**",
|
||||
"pyproject.toml",
|
||||
"requirements*.txt",
|
||||
"setup.cfg",
|
||||
"setup.py",
|
||||
"uv.lock",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FamilyCoverage:
|
||||
pattern: re.Pattern[str]
|
||||
golden_tests: tuple[str, ...]
|
||||
ssim_tests: tuple[str, ...]
|
||||
|
||||
|
||||
FAMILY_COVERAGE = (
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])dreamx(_world)?([/_.-]|$)"),
|
||||
("test_dreamx.py", ),
|
||||
("test_dreamx_world_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])flux[_-]?2([/_.-]|$)"),
|
||||
("test_flux2_klein.py", ),
|
||||
("test_flux2_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])flux(?![_-]?2)([/_.-]|$)"),
|
||||
("test_flux.py", ),
|
||||
("test_flux_t2i_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])(hunyuan)?gamecraft([/_.-]|$)"),
|
||||
("test_gamecraft.py", ),
|
||||
("test_gamecraft_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])gen3c([/_.-]|$)"),
|
||||
("test_gen3c.py", ),
|
||||
("test_gen3c_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])glm[_-]?image([/_.-]|$)"),
|
||||
("test_glm_image.py", ),
|
||||
("test_glm_image_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])kandinsky[_-]?5([/_.-]|$)"),
|
||||
("test_kandinsky5.py", ),
|
||||
("test_kandinsky5_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])lingbot([a-z0-9_-]*)([/_.-]|$)"),
|
||||
("test_lingbot.py", ),
|
||||
("test_lingbot_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])longcat([/_.-]|$)"),
|
||||
("test_longcat.py", ),
|
||||
("test_longcat_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])ltx[_-]?2([/_.-]|$)"),
|
||||
("test_ltx2.py", ),
|
||||
("test_ltx2_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])matrixgame[_-]?2([/_.-]|$)"),
|
||||
("test_matrixgame.py", ),
|
||||
("test_matrixgame2_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])matrixgame[_-]?3([/_.-]|$)"),
|
||||
("test_matrixgame.py", ),
|
||||
("test_matrixgame3_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])minimax[_-]?h3([/_.-]|$)"),
|
||||
("test_minimax_h3_t2v.py", ),
|
||||
("test_minimax_h3_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])sd[_-]?3([._-]?5)?([/_.-]|$)"),
|
||||
("test_sd35.py", ),
|
||||
("test_sd35_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])stable[_-]?audio([/_.-]|$)"),
|
||||
("test_stable_audio.py", ),
|
||||
("test_stable_audio_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])turbo(diffusion)?([/_.-]|$)"),
|
||||
(),
|
||||
("test_turbodiffusion_similarity.py", ),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])wan(video|vae)?([/_.-]|$)"),
|
||||
("test_wan_t2v.py", "test_wan_vae.py", "test_wan_causal.py", "test_wan_denoising.py"),
|
||||
(
|
||||
"test_causal_similarity.py",
|
||||
"test_wan_i2v_similarity.py",
|
||||
"test_wan_t2v_similarity.py",
|
||||
),
|
||||
),
|
||||
FamilyCoverage(
|
||||
re.compile(r"(^|[/_.-])z[_-]?image([/_.-]|$)"),
|
||||
("test_zimage.py", ),
|
||||
("test_zimage_similarity.py", ),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class MergePlan:
|
||||
lanes: set[str] = field(default_factory=set)
|
||||
golden_tests: set[str] = field(default_factory=set)
|
||||
ssim_tests: set[str] = field(default_factory=set)
|
||||
golden_all: bool = False
|
||||
ssim_all: bool = False
|
||||
reasons: list[str] = field(default_factory=list)
|
||||
|
||||
def add_lanes(self, *lanes: str, reason: str) -> None:
|
||||
unknown = set(lanes) - set(MERGE_LANES)
|
||||
if unknown:
|
||||
raise ValueError(f"Unknown merge lanes: {sorted(unknown)}")
|
||||
self.lanes.update(lanes)
|
||||
self.reasons.append(reason)
|
||||
|
||||
def add_golden(self, tests: tuple[str, ...], reason: str) -> None:
|
||||
self.add_lanes("golden-gate", reason=reason)
|
||||
self.golden_tests.update(tests)
|
||||
|
||||
def add_ssim(self, tests: tuple[str, ...], reason: str) -> None:
|
||||
self.add_lanes("ssim", reason=reason)
|
||||
self.ssim_tests.update(tests)
|
||||
|
||||
def require_all(self, reason: str) -> None:
|
||||
self.lanes.update(MERGE_LANES)
|
||||
self.golden_all = True
|
||||
self.ssim_all = True
|
||||
self.reasons.append(reason)
|
||||
|
||||
def ordered_lanes(self) -> tuple[str, ...]:
|
||||
return tuple(lane for lane in MERGE_LANES if lane in self.lanes)
|
||||
|
||||
def encoded_lanes(self) -> str:
|
||||
lanes = self.ordered_lanes()
|
||||
return "," + ",".join(lanes or ("none", )) + ","
|
||||
|
||||
def encoded_golden_tests(self) -> str:
|
||||
if "golden-gate" not in self.lanes:
|
||||
return "none"
|
||||
if self.golden_all or not self.golden_tests:
|
||||
return "all"
|
||||
return ",".join(sorted(self.golden_tests))
|
||||
|
||||
def encoded_ssim_tests(self) -> str:
|
||||
if "ssim" not in self.lanes:
|
||||
return "none"
|
||||
if self.ssim_all or not self.ssim_tests:
|
||||
return "all"
|
||||
return ",".join(sorted(self.ssim_tests))
|
||||
|
||||
|
||||
def _matches_any(path: str, patterns: tuple[str, ...]) -> bool:
|
||||
return any(fnmatch.fnmatchcase(path, pattern) for pattern in patterns)
|
||||
|
||||
|
||||
def _family_coverage(path: str) -> tuple[set[str], set[str]]:
|
||||
normalized = path.lower()
|
||||
golden: set[str] = set()
|
||||
ssim: set[str] = set()
|
||||
for family in FAMILY_COVERAGE:
|
||||
if family.pattern.search(normalized):
|
||||
golden.update(family.golden_tests)
|
||||
ssim.update(family.ssim_tests)
|
||||
# Select the component actually touched, including compatibility paths.
|
||||
# Family configs/pipeline wiring can affect all four Wan gates.
|
||||
if re.search(r"(^|[/_.-])wan(video|vae)?([/_.-]|$)", normalized):
|
||||
if (normalized.endswith(("/wan/vae.py", "/wan/vae_config.py", "/vaes/wanvae.py"))
|
||||
or normalized.endswith("/wan/stages/conditioning.py")):
|
||||
golden = {"test_wan_vae.py"}
|
||||
elif normalized.endswith(("/wan/causal_transformer.py", "/dits/causal_wanvideo.py",
|
||||
"/wan/stages/causal_denoising.py")):
|
||||
golden = {"test_wan_causal.py"}
|
||||
elif (normalized == "fastvideo/models/dits/wanvideo.py"
|
||||
or normalized.endswith(("/wan/transformer.py", "/wan/stages/denoising.py", "/wan/stages/dmd.py"))):
|
||||
golden = {"test_wan_t2v.py", "test_wan_denoising.py"}
|
||||
return golden, ssim
|
||||
|
||||
|
||||
def _select_output_coverage(plan: MergePlan, path: str) -> None:
|
||||
golden, ssim = _family_coverage(path)
|
||||
if golden:
|
||||
plan.add_golden(tuple(sorted(golden)), reason=f"model-family golden coverage: {path}")
|
||||
else:
|
||||
plan.golden_all = True
|
||||
plan.add_lanes("golden-gate", reason=f"shared output golden coverage: {path}")
|
||||
if ssim:
|
||||
plan.add_ssim(tuple(sorted(ssim)), reason=f"model-family SSIM coverage: {path}")
|
||||
else:
|
||||
plan.add_ssim(SSIM_SMOKE_TESTS, reason=f"shared output SSIM smoke coverage: {path}")
|
||||
|
||||
|
||||
def classify_paths(paths: list[str]) -> MergePlan:
|
||||
plan = MergePlan()
|
||||
normalized_paths: list[str] = []
|
||||
for raw_path in paths:
|
||||
path = raw_path.strip()
|
||||
while path.startswith("./"):
|
||||
path = path[2:]
|
||||
if path:
|
||||
normalized_paths.append(path)
|
||||
normalized_paths = sorted(set(normalized_paths))
|
||||
if not normalized_paths:
|
||||
plan.require_all("changed-file list was empty; failing closed")
|
||||
return plan
|
||||
|
||||
for path in normalized_paths:
|
||||
if path == "__FASTVIDEO_CI_PLAN_ALL__":
|
||||
plan.require_all("changed-file API failed; failing closed")
|
||||
continue
|
||||
|
||||
if path in {"requirements-mkdocs.in", "requirements-mkdocs.txt"}:
|
||||
plan.reasons.append(f"documentation dependencies need no GPU integration: {path}")
|
||||
continue
|
||||
|
||||
if _matches_any(path, ALL_IMPACT_PATTERNS):
|
||||
plan.require_all(f"cross-cutting build/runtime surface: {path}")
|
||||
continue
|
||||
|
||||
lane_script_prefix = ".buildkite/scripts/lanes/"
|
||||
if path.startswith(lane_script_prefix):
|
||||
script_name = Path(path).name
|
||||
lane = LANE_SCRIPT_TO_KEY.get(script_name)
|
||||
if lane is None:
|
||||
if script_name in FASTCHECK_LANE_SCRIPTS:
|
||||
plan.reasons.append(f"covered by automatic Fastcheck lane: {path}")
|
||||
else:
|
||||
plan.require_all(f"unknown lane script: {path}")
|
||||
elif lane == "golden-gate":
|
||||
plan.golden_all = True
|
||||
plan.add_lanes(lane, reason=f"golden lane implementation: {path}")
|
||||
elif lane == "ssim":
|
||||
plan.ssim_all = True
|
||||
plan.add_lanes(lane, reason=f"SSIM lane implementation: {path}")
|
||||
else:
|
||||
plan.add_lanes(lane, reason=f"lane implementation: {path}")
|
||||
continue
|
||||
|
||||
if path.startswith("fastvideo/tests/golden_gate/"):
|
||||
name = Path(path).name
|
||||
if name.startswith("test_") and name.endswith(".py"):
|
||||
plan.add_golden((name, ), reason=f"changed golden test: {path}")
|
||||
elif name in {"AGENTS.md", "README.md"}:
|
||||
plan.reasons.append(f"golden documentation only: {path}")
|
||||
else:
|
||||
plan.golden_all = True
|
||||
plan.add_lanes("golden-gate", reason=f"shared golden harness/reference: {path}")
|
||||
continue
|
||||
|
||||
if path.startswith("fastvideo/tests/ssim/"):
|
||||
name = Path(path).name
|
||||
if name.startswith("test_") and name.endswith(".py"):
|
||||
plan.add_ssim((name, ), reason=f"changed SSIM test: {path}")
|
||||
elif path.endswith((".py", ".json", ".pt", ".png", ".mp4")):
|
||||
plan.ssim_all = True
|
||||
plan.add_lanes("ssim", reason=f"shared SSIM harness/reference: {path}")
|
||||
continue
|
||||
|
||||
if path.startswith("fastvideo/tests/performance/") or path.startswith(".buildkite/performance-benchmarks/"):
|
||||
plan.add_lanes("performance", reason=f"performance coverage: {path}")
|
||||
continue
|
||||
if path.startswith(("fastvideo/performance/", "fastvideo/performance_dashboard/",
|
||||
"apps/performance_dashboard/")):
|
||||
plan.add_lanes("performance", reason=f"performance implementation: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/benchmarks/"):
|
||||
if "/mlx_" in path or Path(path).name.startswith("mlx_"):
|
||||
plan.reasons.append(f"covered by the path-filtered macOS MLX workflow: {path}")
|
||||
else:
|
||||
plan.add_lanes("performance", reason=f"benchmark implementation: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/tests/eval/") or path.startswith("fastvideo/eval/"):
|
||||
plan.add_lanes("eval", reason=f"evaluation coverage: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/third_party/eval/"):
|
||||
plan.add_lanes("eval", reason=f"vendored evaluation implementation: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/tests/lora_extraction/") or path.startswith("scripts/lora_extraction/"):
|
||||
plan.add_lanes("lora-extraction", reason=f"LoRA extraction coverage: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/tests/inference/lora/"):
|
||||
plan.add_lanes("lora-inference", reason=f"LoRA inference coverage: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/tests/inference/vmoba/"):
|
||||
plan.add_lanes("inference-vmoba", reason=f"VMoBA inference coverage: {path}")
|
||||
continue
|
||||
if path.startswith(("fastvideo/dataset/", "fastvideo/workflow/", "fastvideo/pipelines/preprocess/",
|
||||
"fastvideo/pipelines/training/")):
|
||||
plan.add_lanes(*ALL_TRAINING_LANES, reason=f"shared data/training input surface: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/tests/train/") or path.startswith("fastvideo/train/"):
|
||||
plan.add_lanes("train-framework", reason=f"modular training coverage: {path}")
|
||||
continue
|
||||
|
||||
if path.startswith("fastvideo/tests/training/"):
|
||||
lowered = path.lower()
|
||||
if "/vanilla/" in lowered:
|
||||
plan.add_lanes("training", reason=f"vanilla training coverage: {path}")
|
||||
elif "/distill/" in lowered:
|
||||
plan.add_lanes("distillation", reason=f"distillation coverage: {path}")
|
||||
elif "/self-forcing/" in lowered:
|
||||
plan.add_lanes("self-forcing", reason=f"self-forcing coverage: {path}")
|
||||
elif "/lora/" in lowered:
|
||||
plan.add_lanes("lora-training", reason=f"LoRA training coverage: {path}")
|
||||
elif "/vsa/" in lowered:
|
||||
plan.add_lanes("training-vsa", reason=f"VSA training coverage: {path}")
|
||||
else:
|
||||
plan.add_lanes(*LEGACY_TRAINING_LANES, reason=f"shared legacy training coverage: {path}")
|
||||
continue
|
||||
|
||||
if path.startswith("fastvideo/training/"):
|
||||
lowered = path.lower()
|
||||
if "self_forcing" in lowered:
|
||||
plan.add_lanes("self-forcing", reason=f"self-forcing implementation: {path}")
|
||||
elif "distill" in lowered:
|
||||
plan.add_lanes("distillation", reason=f"distillation implementation: {path}")
|
||||
elif "lora" in lowered:
|
||||
plan.add_lanes("lora-training", reason=f"LoRA training implementation: {path}")
|
||||
else:
|
||||
plan.add_lanes(*LEGACY_TRAINING_LANES, reason=f"shared legacy training implementation: {path}")
|
||||
continue
|
||||
|
||||
lowered = path.lower()
|
||||
if "vmoba" in lowered and path.startswith(("fastvideo/", ".buildkite/")):
|
||||
plan.add_lanes("inference-vmoba", reason=f"VMoBA implementation: {path}")
|
||||
plan.add_golden(("test_wan_t2v.py", ), reason=f"VMoBA end-to-end coverage: {path}")
|
||||
continue
|
||||
if "lora" in lowered and path.startswith("fastvideo/"):
|
||||
plan.add_lanes(
|
||||
"lora-inference",
|
||||
"lora-extraction",
|
||||
"lora-training",
|
||||
reason=f"shared LoRA implementation: {path}",
|
||||
)
|
||||
_select_output_coverage(plan, path)
|
||||
continue
|
||||
|
||||
if path.startswith("fastvideo/entrypoints/") or path.startswith("fastvideo/api/"):
|
||||
plan.add_lanes("api-server", reason=f"API/entrypoint integration: {path}")
|
||||
if "openai" not in lowered and "/cli/" not in lowered:
|
||||
_select_output_coverage(plan, path)
|
||||
continue
|
||||
if path.startswith("fastvideo/worker/"):
|
||||
plan.add_lanes("api-server", reason=f"worker/API integration: {path}")
|
||||
_select_output_coverage(plan, path)
|
||||
continue
|
||||
if path.startswith("fastvideo/distributed/"):
|
||||
plan.add_lanes(
|
||||
"training",
|
||||
"train-framework",
|
||||
reason=f"distributed runtime integration: {path}",
|
||||
)
|
||||
_select_output_coverage(plan, path)
|
||||
continue
|
||||
if path.startswith(("fastvideo/hooks/", "fastvideo/platforms/", "fastvideo/third_party/")):
|
||||
_select_output_coverage(plan, path)
|
||||
continue
|
||||
if path.startswith(("fastvideo/models/", "fastvideo/pipelines/", "fastvideo/configs/",
|
||||
"fastvideo/layers/", "fastvideo/attention/")):
|
||||
_select_output_coverage(plan, path)
|
||||
continue
|
||||
if path in {
|
||||
"fastvideo/fastvideo_args.py",
|
||||
"fastvideo/forward_context.py",
|
||||
"fastvideo/image_processor.py",
|
||||
"fastvideo/registry.py",
|
||||
"fastvideo/utils.py",
|
||||
}:
|
||||
_select_output_coverage(plan, path)
|
||||
continue
|
||||
if path.startswith("fastvideo/mlx_runtime/"):
|
||||
plan.reasons.append(f"covered by the path-filtered macOS MLX workflow: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/logging_utils/") or path in {
|
||||
"fastvideo/__init__.py",
|
||||
"fastvideo/envs.py",
|
||||
"fastvideo/logger.py",
|
||||
"fastvideo/profiler.py",
|
||||
"fastvideo/version.py",
|
||||
}:
|
||||
plan.reasons.append(f"covered by automatic Fastcheck: {path}")
|
||||
continue
|
||||
if path.startswith(("fastvideo-kernel/", "csrc/")):
|
||||
plan.add_golden(("test_wan_t2v.py", ), reason=f"kernel integration smoke: {path}")
|
||||
plan.add_ssim(("test_wan_t2v_similarity.py", ), reason=f"kernel numerical smoke: {path}")
|
||||
continue
|
||||
|
||||
if path.startswith("apps/dreamverse/"):
|
||||
# DreamVerse is already one of the six automatic Fastcheck lanes.
|
||||
plan.reasons.append(f"covered by automatic DreamVerse Fastcheck: {path}")
|
||||
continue
|
||||
if path.startswith("fastvideo/tests/"):
|
||||
# The automatic unit/component Fastcheck lanes own the remaining
|
||||
# package tests. Domain-specific expensive test roots were handled
|
||||
# above.
|
||||
plan.reasons.append(f"covered by automatic Fastcheck: {path}")
|
||||
continue
|
||||
if path in {".buildkite/scripts/unit_test.sh", ".buildkite/scripts/pr_test.sh"}:
|
||||
plan.reasons.append(f"covered by automatic unit Fastcheck: {path}")
|
||||
continue
|
||||
if _matches_any(path, SAFE_PATTERNS):
|
||||
plan.reasons.append(f"no additional GPU integration needed: {path}")
|
||||
continue
|
||||
|
||||
plan.require_all(f"unclassified path; failing closed: {path}")
|
||||
|
||||
return plan
|
||||
|
||||
|
||||
def _write_github_output(output: TextIO, plan: MergePlan) -> None:
|
||||
output.write(f"merge_test_plan={plan.encoded_lanes()}\n")
|
||||
output.write(f"merge_golden_tests={plan.encoded_golden_tests()}\n")
|
||||
output.write(f"merge_ssim_tests={plan.encoded_ssim_tests()}\n")
|
||||
output.write(f"merge_plan_label={','.join(plan.ordered_lanes()) or 'none'}\n")
|
||||
|
||||
|
||||
def _write_summary(output: TextIO, plan: MergePlan) -> None:
|
||||
output.write("## Change-aware merge test plan\n\n")
|
||||
output.write("| Selection | Value |\n|---|---|\n")
|
||||
output.write(f"| Additional Slurm lanes | `{','.join(plan.ordered_lanes()) or 'none'}` |\n")
|
||||
output.write(f"| Golden tests | `{plan.encoded_golden_tests()}` |\n")
|
||||
output.write(f"| SSIM tests | `{plan.encoded_ssim_tests()}` |\n\n")
|
||||
output.write("Fastcheck remains the universal six-lane baseline.\n")
|
||||
|
||||
|
||||
def parse_args() -> argparse.Namespace:
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--paths-file", type=Path, required=True)
|
||||
parser.add_argument("--github-output", type=Path)
|
||||
parser.add_argument("--summary-file", type=Path)
|
||||
return parser.parse_args()
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = parse_args()
|
||||
paths = args.paths_file.read_text(encoding="utf-8").splitlines()
|
||||
plan = classify_paths(paths)
|
||||
print(f"MERGE_TEST_PLAN={plan.encoded_lanes()}")
|
||||
print(f"MERGE_GOLDEN_TESTS={plan.encoded_golden_tests()}")
|
||||
print(f"MERGE_SSIM_TESTS={plan.encoded_ssim_tests()}")
|
||||
for reason in plan.reasons:
|
||||
print(f"- {reason}")
|
||||
if args.github_output:
|
||||
with args.github_output.open("a", encoding="utf-8") as output:
|
||||
_write_github_output(output, plan)
|
||||
if args.summary_file:
|
||||
with args.summary_file.open("a", encoding="utf-8") as output:
|
||||
_write_summary(output, plan)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -53,7 +53,7 @@ PC_PENDING='{"name": "pre-commit", "id": 1, "status": "in_progress", "conclusion
|
||||
DOCS_OK='{"name": "Deploy Documentation", "id": 2, "status": "completed", "conclusion": "success"}'
|
||||
DOCS_BAD='{"name": "Deploy Documentation", "id": 2, "status": "completed", "conclusion": "failure"}'
|
||||
DOCS_CANCELLED='{"name": "Deploy Documentation", "id": 2, "status": "completed", "conclusion": "cancelled"}'
|
||||
OTHER='{"name": "Trigger Full Suite", "id": 3, "status": "in_progress", "conclusion": null}'
|
||||
OTHER='{"name": "Trigger Merge Gate", "id": 3, "status": "in_progress", "conclusion": null}'
|
||||
NULL_NAME='{"name": null, "id": 4, "status": "completed", "conclusion": "failure"}'
|
||||
PC_OK_RERUN='{"name": "pre-commit", "id": 5, "status": "completed", "conclusion": "success"}'
|
||||
|
||||
|
||||
@@ -190,6 +190,7 @@ jobs:
|
||||
if: ${{ !inputs.push_by_digest }}
|
||||
run: |
|
||||
echo "✅ Python ${{ inputs.python_version }} image successfully built and pushed to ${{ steps.image.outputs.name }}:${{ inputs.tag_suffix }}-sha-${GITHUB_SHA::7}"
|
||||
echo "Digest: ${{ steps.build-push.outputs.digest }}"
|
||||
echo "To run tests with this image, manually trigger the 'Run Tests' workflow."
|
||||
|
||||
- name: Digest success message
|
||||
|
||||
@@ -26,29 +26,48 @@ jobs:
|
||||
per_page: 100,
|
||||
});
|
||||
|
||||
const bkStatuses = data.statuses.filter(
|
||||
s => s.context.startsWith('buildkite/ci/')
|
||||
);
|
||||
|
||||
const FASTCHECK_PREFIX = 'buildkite/ci/microscope-';
|
||||
// Buildkite derives the GitHub context prefix from the label emoji.
|
||||
// Keep hard Full Suite lanes in test-tube/bar-chart namespaces and
|
||||
// Fastcheck lanes in microscope so targeted reruns cannot clear the
|
||||
// wrong aggregate status. Automatic PR jobs use pr-fastcheck while
|
||||
// slash-command and Full Suite jobs use ci; normalize the suffix
|
||||
// and keep the newest status for each logical lane.
|
||||
const FASTCHECK_PREFIXES = [
|
||||
'buildkite/pr-fastcheck/microscope-',
|
||||
'buildkite/ci/microscope-',
|
||||
];
|
||||
const FULL_SUITE_PREFIXES = [
|
||||
'buildkite/ci/test-tube-',
|
||||
'buildkite/ci/bar-chart-',
|
||||
];
|
||||
|
||||
const fastcheck = bkStatuses.filter(
|
||||
s => s.context.startsWith(FASTCHECK_PREFIX)
|
||||
);
|
||||
const fullSuite = bkStatuses.filter(
|
||||
s => FULL_SUITE_PREFIXES.some(p => s.context.startsWith(p))
|
||||
);
|
||||
function newestByLane(prefixes) {
|
||||
const statuses = new Map();
|
||||
for (const status of data.statuses) {
|
||||
const prefix = prefixes.find(p => status.context.startsWith(p));
|
||||
if (!prefix) continue;
|
||||
const lane = status.context.slice(prefix.length);
|
||||
const previous = statuses.get(lane);
|
||||
if (!previous || Date.parse(status.updated_at) > Date.parse(previous.updated_at)) {
|
||||
statuses.set(lane, status);
|
||||
}
|
||||
}
|
||||
return statuses;
|
||||
}
|
||||
|
||||
if (
|
||||
fastcheck.length > 0
|
||||
&& fastcheck.every(s => s.state === 'success')
|
||||
) {
|
||||
const fastcheck = newestByLane(FASTCHECK_PREFIXES);
|
||||
const fullSuiteOnly = newestByLane(FULL_SUITE_PREFIXES);
|
||||
const fastcheckPassed =
|
||||
fastcheck.size === 6
|
||||
&& [...fastcheck.values()].every(s => s.state === 'success');
|
||||
const fullSuitePassed =
|
||||
fastcheckPassed
|
||||
&& fullSuiteOnly.size === 14
|
||||
&& [...fullSuiteOnly.values()].every(s => s.state === 'success');
|
||||
|
||||
if (fastcheckPassed) {
|
||||
core.info(
|
||||
`All ${fastcheck.length} fastcheck tests passed — updating fastcheck-passed`
|
||||
`All ${fastcheck.size} fastcheck tests passed — updating fastcheck-passed`
|
||||
);
|
||||
await github.rest.repos.createCommitStatus({
|
||||
owner: context.repo.owner,
|
||||
@@ -56,17 +75,13 @@ jobs:
|
||||
sha,
|
||||
state: 'success',
|
||||
context: 'fastcheck-passed',
|
||||
description:
|
||||
`All ${fastcheck.length} fastcheck tests passed`,
|
||||
description: `All ${fastcheck.size} fastcheck tests passed`,
|
||||
});
|
||||
}
|
||||
|
||||
if (
|
||||
fullSuite.length > 0
|
||||
&& fullSuite.every(s => s.state === 'success')
|
||||
) {
|
||||
if (fullSuitePassed) {
|
||||
core.info(
|
||||
`All ${fullSuite.length} full suite tests passed — updating full-suite-passed`
|
||||
'All 20 full suite tests passed — updating full-suite-passed'
|
||||
);
|
||||
await github.rest.repos.createCommitStatus({
|
||||
owner: context.repo.owner,
|
||||
@@ -74,7 +89,6 @@ jobs:
|
||||
sha,
|
||||
state: 'success',
|
||||
context: 'full-suite-passed',
|
||||
description:
|
||||
`All ${fullSuite.length} full suite tests passed`,
|
||||
description: 'All 20 full suite tests passed',
|
||||
});
|
||||
}
|
||||
|
||||
@@ -8,6 +8,8 @@ on:
|
||||
- "fastvideo/mlx_runtime/**"
|
||||
- "fastvideo/tests/mlx/**"
|
||||
- "fastvideo/tests/platforms/test_mps_vsa_error.py"
|
||||
- "fastvideo/tests/platforms/test_cpu_sdpa.py"
|
||||
- "fastvideo/platforms/cpu.py"
|
||||
- "fastvideo/platforms/mps.py"
|
||||
- "fastvideo/platforms/__init__.py"
|
||||
- "fastvideo/__init__.py"
|
||||
@@ -78,6 +80,13 @@ jobs:
|
||||
fastvideo/tests/mlx/test_mlx_dit_parity.py \
|
||||
fastvideo/tests/mlx/test_mlx_compile_parity.py \
|
||||
fastvideo/tests/mlx/test_mlx_checkpoint.py \
|
||||
fastvideo/tests/mlx/test_mlx_checkpoint_compat.py \
|
||||
fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_parity.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa_regressions.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_fast_mode.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_fast_spatial.py \
|
||||
fastvideo/tests/mlx/test_mlx_fastwan_benchmark.py \
|
||||
fastvideo/tests/mlx/test_taehv_decode.py \
|
||||
fastvideo/tests/mlx/test_frame_upsample.py \
|
||||
@@ -90,7 +99,8 @@ jobs:
|
||||
fastvideo/tests/mlx/test_mlx_rife_interpolation.py::test_rife_download_unavailable_has_specific_error \
|
||||
fastvideo/tests/mlx/test_mlx_rife_interpolation.py::test_rife_backend_regression_is_not_skip_eligible \
|
||||
fastvideo/tests/platforms/test_mps_vsa_error.py \
|
||||
-q
|
||||
fastvideo/tests/platforms/test_cpu_sdpa.py \
|
||||
-v -s -o faulthandler_timeout=120
|
||||
|
||||
# Same tests on MLX's CPU backend. Hosted macOS runners are scarce and
|
||||
# slower to schedule; this Linux job gives fast PR signal on the identical
|
||||
@@ -134,6 +144,13 @@ jobs:
|
||||
fastvideo/tests/mlx/test_mlx_dit_parity.py \
|
||||
fastvideo/tests/mlx/test_mlx_compile_parity.py \
|
||||
fastvideo/tests/mlx/test_mlx_checkpoint.py \
|
||||
fastvideo/tests/mlx/test_mlx_checkpoint_compat.py \
|
||||
fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_parity.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_vsa_regressions.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_fast_mode.py \
|
||||
fastvideo/tests/mlx/test_mlx_minimax_h3_fast_spatial.py \
|
||||
fastvideo/tests/mlx/test_mlx_fastwan_benchmark.py \
|
||||
fastvideo/tests/mlx/test_taehv_decode.py \
|
||||
fastvideo/tests/mlx/test_frame_upsample.py \
|
||||
@@ -146,4 +163,5 @@ jobs:
|
||||
fastvideo/tests/mlx/test_mlx_rife_interpolation.py::test_rife_download_unavailable_has_specific_error \
|
||||
fastvideo/tests/mlx/test_mlx_rife_interpolation.py::test_rife_backend_regression_is_not_skip_eligible \
|
||||
fastvideo/tests/platforms/test_mps_vsa_error.py \
|
||||
-q
|
||||
fastvideo/tests/platforms/test_cpu_sdpa.py \
|
||||
-v -s -o faulthandler_timeout=120
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
name: Scheduled Full SSIM
|
||||
|
||||
on:
|
||||
schedule:
|
||||
- cron: "0 5 * * 0"
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
trigger:
|
||||
if: github.repository == 'hao-ai-lab/FastVideo'
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Trigger weekly full SSIM on Slinky Slurm
|
||||
env:
|
||||
BUILDKITE_API_TOKEN: ${{ secrets.BUILDKITE_API_TOKEN }}
|
||||
SOURCE_SHA: ${{ github.sha }}
|
||||
SOURCE_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
BK_ORG: ${{ vars.BUILDKITE_ORG_SLUG }}
|
||||
BK_PIPELINE: ${{ vars.BUILDKITE_PIPELINE_SLUG }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
curl -sS --fail-with-body -X POST \
|
||||
"https://api.buildkite.com/v2/organizations/${BK_ORG}/pipelines/${BK_PIPELINE}/builds" \
|
||||
-H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data-raw "$(jq -n \
|
||||
--arg commit "$SOURCE_SHA" \
|
||||
--arg branch "$SOURCE_BRANCH" \
|
||||
'{
|
||||
commit: $commit,
|
||||
branch: $branch,
|
||||
message: "Weekly full SSIM on Slinky Slurm",
|
||||
ignore_pipeline_branch_filters: true,
|
||||
env: {
|
||||
TEST_SCOPE: "scheduled",
|
||||
FULL_SUITE: "false",
|
||||
TEST_TYPE: "ssim",
|
||||
PR_NUMBER: "false",
|
||||
PR_TITLE: "Scheduled full SSIM"
|
||||
}
|
||||
}')"
|
||||
@@ -33,7 +33,6 @@ jobs:
|
||||
core.setOutput('has_write', String(hasWrite));
|
||||
|
||||
- name: Add ready label and react
|
||||
id: label
|
||||
if: steps.perm.outputs.has_write == 'true'
|
||||
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
|
||||
with:
|
||||
@@ -48,47 +47,6 @@ jobs:
|
||||
comment_id: context.payload.comment.id,
|
||||
content: 'rocket',
|
||||
});
|
||||
const { data: pr } = await github.rest.pulls.get({ owner, repo, pull_number: prNumber });
|
||||
core.setOutput('pr_sha', pr.head.sha);
|
||||
core.setOutput('pr_branch', pr.head.ref);
|
||||
core.setOutput('pr_number', String(prNumber));
|
||||
core.setOutput('pr_title', pr.title);
|
||||
|
||||
- name: Trigger Full Suite
|
||||
if: steps.perm.outputs.has_write == 'true'
|
||||
env:
|
||||
BUILDKITE_API_TOKEN: ${{ secrets.BUILDKITE_API_TOKEN }}
|
||||
PR_SHA: ${{ steps.label.outputs.pr_sha }}
|
||||
PR_BRANCH: ${{ steps.label.outputs.pr_branch }}
|
||||
PR_NUMBER: ${{ steps.label.outputs.pr_number }}
|
||||
PR_TITLE: ${{ steps.label.outputs.pr_title }}
|
||||
BK_ORG: ${{ vars.BUILDKITE_ORG_SLUG }}
|
||||
BK_PIPELINE: ${{ vars.BUILDKITE_PIPELINE_SLUG }}
|
||||
run: |
|
||||
curl -sS --fail-with-body -X POST \
|
||||
"https://api.buildkite.com/v2/organizations/${BK_ORG}/pipelines/${BK_PIPELINE}/builds" \
|
||||
-H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
|
||||
-H "Content-Type: application/json" \
|
||||
--data-raw "$(jq -n \
|
||||
--arg commit "$PR_SHA" \
|
||||
--arg branch "$PR_BRANCH" \
|
||||
--arg message "Full Suite for PR #${PR_NUMBER} (via /merge)" \
|
||||
--arg pr_title "$PR_TITLE" \
|
||||
--argjson pr_id "$PR_NUMBER" \
|
||||
'{
|
||||
commit: $commit,
|
||||
branch: $branch,
|
||||
message: $message,
|
||||
ignore_pipeline_branch_filters: true,
|
||||
pull_request_id: $pr_id,
|
||||
pull_request_base_branch: "main",
|
||||
env: {
|
||||
TEST_SCOPE: "full",
|
||||
FULL_SUITE: "true",
|
||||
PR_NUMBER: ($pr_id | tostring),
|
||||
PR_TITLE: $pr_title
|
||||
}
|
||||
}')"
|
||||
|
||||
parse-command:
|
||||
if: >-
|
||||
@@ -129,7 +87,7 @@ jobs:
|
||||
set -euo pipefail
|
||||
TEST_NAME=$(echo "$COMMENT" | grep -oP '(?<=/test\s)\S+' | head -1 || true)
|
||||
|
||||
VALID="encoder vae transformer kernel unit dreamverse ssim golden-gate training lora-inference lora-training lora-extraction distillation self-forcing vsa vmoba performance api train-framework eval full fastcheck pre-commit"
|
||||
VALID="encoder vae transformer kernel unit dreamverse ssim golden-gate training lora-inference lora-training lora-extraction distillation self-forcing vsa vmoba performance api train-framework eval unit-ci kernel-ci dreamverse-ci ssim-ci golden-gate-ci encoder-ci vae-ci transformer-ci lora-inference-ci lora-training-ci lora-extraction-ci training-ci distillation-ci self-forcing-ci vsa-ci vmoba-ci performance-ci api-ci train-framework-ci eval-ci full fastcheck pre-commit"
|
||||
if [ -z "$TEST_NAME" ] || ! echo "$VALID" | grep -qw "$TEST_NAME"; then
|
||||
echo "Unknown test: '$TEST_NAME'. Valid: $VALID"
|
||||
exit 1
|
||||
@@ -137,7 +95,17 @@ jobs:
|
||||
|
||||
declare -A MAP=(
|
||||
[encoder]=encoder [vae]=vae [transformer]=transformer
|
||||
[kernel]=kernel_tests [unit]=unit_test [dreamverse]=dreamverse_app
|
||||
[kernel]=kernel_tests [unit]=unit_test [unit-ci]=unit_test_ci
|
||||
[kernel-ci]=kernel_tests_ci [dreamverse-ci]=dreamverse_app_ci
|
||||
[ssim-ci]=ssim_ci [vmoba-ci]=inference_vmoba_ci
|
||||
[golden-gate-ci]=golden_gate_ci [training-ci]=training_ci
|
||||
[encoder-ci]=encoder_ci [vae-ci]=vae_ci [transformer-ci]=transformer_ci
|
||||
[lora-inference-ci]=inference_lora_ci [lora-training-ci]=training_lora_ci
|
||||
[lora-extraction-ci]=lora_extraction_ci [distillation-ci]=distillation_dmd_ci
|
||||
[self-forcing-ci]=self_forcing_ci [vsa-ci]=training_vsa_ci
|
||||
[performance-ci]=performance_ci [api-ci]=api_server_ci
|
||||
[train-framework-ci]=train_framework_ci [eval-ci]=eval_ci
|
||||
[dreamverse]=dreamverse_app
|
||||
[ssim]=ssim [golden-gate]=golden_gate [training]=training
|
||||
[lora-inference]=inference_lora [lora-training]=training_lora
|
||||
[lora-extraction]=lora_extraction
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
name: Trigger Full Suite
|
||||
name: Trigger Merge Gate
|
||||
|
||||
on:
|
||||
pull_request_target:
|
||||
@@ -10,7 +10,7 @@ permissions:
|
||||
actions: read
|
||||
|
||||
concurrency:
|
||||
group: full-suite-${{ github.event.pull_request.number }}
|
||||
group: merge-gate-${{ github.event.pull_request.number }}
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
@@ -34,29 +34,72 @@ jobs:
|
||||
});
|
||||
const hasReady = pr.labels.some(l => l.name === 'ready');
|
||||
core.setOutput('has_ready', String(hasReady));
|
||||
if (!hasReady) core.info('No ready label — skipping Full Suite trigger.');
|
||||
core.setOutput('changed_files', String(pr.changed_files));
|
||||
if (!hasReady) core.info('No ready label — skipping merge-gate trigger.');
|
||||
|
||||
- name: Cancel previous Buildkite builds
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
env:
|
||||
BUILDKITE_API_TOKEN: ${{ secrets.BUILDKITE_API_TOKEN }}
|
||||
PR_BRANCH: ${{ github.event.pull_request.head.ref }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
run: |
|
||||
# Find running builds for this branch with TEST_SCOPE=full and cancel them
|
||||
builds=$(curl -sS -H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
|
||||
"https://api.buildkite.com/v2/organizations/${{ vars.BUILDKITE_ORG_SLUG }}/pipelines/${{ vars.BUILDKITE_PIPELINE_SLUG }}/builds?branch=${PR_BRANCH}&state=running,scheduled" \
|
||||
| jq -r '.[] | select(try (.env.TEST_SCOPE == "full") catch false) | .number')
|
||||
# Match both branch and PR number: forks can reuse the same branch name.
|
||||
builds=$(curl -sS --get -H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
|
||||
--data-urlencode "branch=$PR_BRANCH" \
|
||||
--data-urlencode "state=running,scheduled" \
|
||||
"https://api.buildkite.com/v2/organizations/${{ vars.BUILDKITE_ORG_SLUG }}/pipelines/${{ vars.BUILDKITE_PIPELINE_SLUG }}/builds" \
|
||||
| jq -r --arg pr_number "$PR_NUMBER" \
|
||||
'.[] | select((.env.TEST_SCOPE? == "merge") and (.env.PR_NUMBER? == $pr_number)) | .number')
|
||||
for build_num in $builds; do
|
||||
echo "Cancelling Buildkite build #$build_num"
|
||||
curl -sS -X PUT -H "Authorization: Bearer $BUILDKITE_API_TOKEN" \
|
||||
"https://api.buildkite.com/v2/organizations/${{ vars.BUILDKITE_ORG_SLUG }}/pipelines/${{ vars.BUILDKITE_PIPELINE_SLUG }}/builds/${build_num}/cancel"
|
||||
done
|
||||
|
||||
# Checks out the BASE branch (default for pull_request_target), so PR
|
||||
# authors cannot tamper with the gate script.
|
||||
- name: Checkout gate script
|
||||
# Check out the immutable BASE SHA: pull_request_target must never run a
|
||||
# planner or gate script from the untrusted PR head.
|
||||
- name: Checkout trusted merge planner
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
||||
with:
|
||||
ref: ${{ github.event.pull_request.base.sha }}
|
||||
persist-credentials: false
|
||||
|
||||
- name: Collect changed paths
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
EXPECTED_CHANGED_FILES: ${{ steps.check.outputs.changed_files }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
changed_json="$RUNNER_TEMP/merge-changed-files.json"
|
||||
changed_paths="$RUNNER_TEMP/merge-changed-paths.txt"
|
||||
if gh api --paginate --slurp \
|
||||
"repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?per_page=100" \
|
||||
> "$changed_json"; then
|
||||
observed=$(jq '[.[][] | .filename] | unique | length' "$changed_json")
|
||||
if [ "$observed" = "$EXPECTED_CHANGED_FILES" ]; then
|
||||
jq -r '.[][] | .filename, (.previous_filename // empty)' "$changed_json" \
|
||||
| sort -u > "$changed_paths"
|
||||
else
|
||||
echo "::warning::Changed-file API returned $observed of $EXPECTED_CHANGED_FILES paths; selecting all merge lanes."
|
||||
echo '__FASTVIDEO_CI_PLAN_ALL__' > "$changed_paths"
|
||||
fi
|
||||
else
|
||||
echo "::warning::Changed-file API failed; selecting all merge lanes."
|
||||
echo '__FASTVIDEO_CI_PLAN_ALL__' > "$changed_paths"
|
||||
fi
|
||||
|
||||
- name: Select minimal merge tests
|
||||
id: plan
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
run: |
|
||||
python3 .github/scripts/plan_merge_ci.py \
|
||||
--paths-file "$RUNNER_TEMP/merge-changed-paths.txt" \
|
||||
--github-output "$GITHUB_OUTPUT" \
|
||||
--summary-file "$GITHUB_STEP_SUMMARY"
|
||||
|
||||
- name: Wait for pre-commit and docs build
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
@@ -66,7 +109,7 @@ jobs:
|
||||
PR_NUMBER: ${{ github.event.pull_request.number }}
|
||||
run: bash .github/scripts/gate_full_suite.sh
|
||||
|
||||
- name: Trigger Buildkite Full Suite
|
||||
- name: Trigger Buildkite merge gate
|
||||
if: steps.check.outputs.has_ready == 'true'
|
||||
env:
|
||||
BUILDKITE_API_TOKEN: ${{ secrets.BUILDKITE_API_TOKEN }}
|
||||
@@ -76,6 +119,10 @@ jobs:
|
||||
PR_TITLE: ${{ github.event.pull_request.title }}
|
||||
BK_ORG: ${{ vars.BUILDKITE_ORG_SLUG }}
|
||||
BK_PIPELINE: ${{ vars.BUILDKITE_PIPELINE_SLUG }}
|
||||
MERGE_TEST_PLAN: ${{ steps.plan.outputs.merge_test_plan }}
|
||||
MERGE_GOLDEN_TESTS: ${{ steps.plan.outputs.merge_golden_tests }}
|
||||
MERGE_SSIM_TESTS: ${{ steps.plan.outputs.merge_ssim_tests }}
|
||||
MERGE_PLAN_LABEL: ${{ steps.plan.outputs.merge_plan_label }}
|
||||
run: |
|
||||
curl -sS --fail-with-body -X POST \
|
||||
"https://api.buildkite.com/v2/organizations/${BK_ORG}/pipelines/${BK_PIPELINE}/builds" \
|
||||
@@ -84,8 +131,11 @@ jobs:
|
||||
--data-raw "$(jq -n \
|
||||
--arg commit "$PR_SHA" \
|
||||
--arg branch "$PR_BRANCH" \
|
||||
--arg message "Full Suite for PR #${PR_NUMBER}" \
|
||||
--arg message "Merge gate [${MERGE_PLAN_LABEL}] for PR #${PR_NUMBER}" \
|
||||
--arg pr_title "$PR_TITLE" \
|
||||
--arg merge_test_plan "$MERGE_TEST_PLAN" \
|
||||
--arg merge_golden_tests "$MERGE_GOLDEN_TESTS" \
|
||||
--arg merge_ssim_tests "$MERGE_SSIM_TESTS" \
|
||||
--argjson pr_id "$PR_NUMBER" \
|
||||
'{
|
||||
commit: $commit,
|
||||
@@ -95,8 +145,11 @@ jobs:
|
||||
pull_request_id: $pr_id,
|
||||
pull_request_base_branch: "main",
|
||||
env: {
|
||||
TEST_SCOPE: "full",
|
||||
TEST_SCOPE: "merge",
|
||||
FULL_SUITE: "true",
|
||||
MERGE_TEST_PLAN: $merge_test_plan,
|
||||
MERGE_GOLDEN_TESTS: $merge_golden_tests,
|
||||
MERGE_SSIM_TESTS: $merge_ssim_tests,
|
||||
PR_NUMBER: ($pr_id | tostring),
|
||||
PR_TITLE: $pr_title
|
||||
}
|
||||
|
||||
@@ -38,17 +38,17 @@ jobs:
|
||||
|
||||
**How our CI works:**
|
||||
|
||||
PRs run a two-tier CI system:
|
||||
PRs run a three-tier CI system:
|
||||
1. **Pre-commit** — formatting (yapf), linting (ruff), type checking (mypy). Runs immediately on every PR.
|
||||
2. **Fastcheck** — core GPU tests (encoders, VAEs, transformers, kernels, unit tests). Runs automatically via Buildkite on relevant file changes (~10-15 min).
|
||||
3. **Full Suite** — integration tests, training pipelines, SSIM regression. Runs only when a reviewer adds the `ready` label.
|
||||
2. **Fastcheck** — six core GPU lanes run automatically via Buildkite (~10-15 min).
|
||||
3. **Merge gate** — a reviewer adds `ready`; changed paths select only the relevant integration, training, golden, or SSIM coverage.
|
||||
|
||||
**Before your PR is reviewed:**
|
||||
- [ ] `pre-commit run --all-files` passes locally
|
||||
- [ ] You've added or updated tests for your changes
|
||||
- [ ] The PR description explains what and why
|
||||
|
||||
If pre-commit fails, a bot comment will explain how to fix it. Fastcheck and Full Suite results appear in the Checks section below.
|
||||
If pre-commit fails, a bot comment will explain how to fix it. Fastcheck and merge-gate results appear in the Checks section below.
|
||||
|
||||
**Useful links:**
|
||||
- [Contributing Guide](https://hao-ai-lab.github.io/FastVideo/contributing/overview/)
|
||||
|
||||
@@ -13,6 +13,11 @@ on:
|
||||
required: false
|
||||
default: false
|
||||
type: boolean
|
||||
build_ci_runner_image:
|
||||
description: 'Build the ARM64 CUDA 13 CI runner image (sm_100)'
|
||||
required: false
|
||||
default: false
|
||||
type: boolean
|
||||
# Auto-rebuild the CUDA images when a repository-controlled image input
|
||||
# changes on main. This includes the trusted SM89 kernel artifact's source,
|
||||
# metadata/key helper, ABI dependency metadata, and build orchestration.
|
||||
@@ -198,6 +203,28 @@ jobs:
|
||||
docker buildx imagetools create "${TAG_ARGS[@]}" "${IMAGE_REFS[@]}"
|
||||
docker buildx imagetools inspect "${TAGS[0]}"
|
||||
|
||||
# The CI runner is ARM64 like DGX Spark, but targets sm_100 rather than sm_121.
|
||||
# Publish a single-architecture variant so the self-hosted CI runner can reuse
|
||||
# the exact prebuilt kernel instead of compiling it in every job.
|
||||
build-ci-runner-image:
|
||||
if: ${{ (github.event_name == 'push' && github.repository == 'hao-ai-lab/FastVideo') || github.event.inputs.build_ci_runner_image == 'true' }}
|
||||
uses: ./.github/workflows/_template-build-image.yml
|
||||
with:
|
||||
python_version: '3.12'
|
||||
dockerfile_path: docker/Dockerfile
|
||||
tag_suffix: py3.12-cuda13.0.0-sm100
|
||||
runner: ubuntu-24.04-arm
|
||||
architecture: arm64
|
||||
build_args: |
|
||||
PYTHON_VERSION=3.12
|
||||
CUDA_VERSION=13.0.0
|
||||
UV_TORCH_BACKEND=cu130
|
||||
TORCH_CUDA_ARCH_LIST=10.0
|
||||
CMAKE_BUILD_PARALLEL_LEVEL=1
|
||||
FLASH_ATTN_WHEEL_TAG=cu130torch2.12
|
||||
FLASH_ATTN_WHEEL_RELEASE_ARM64=https://github.com/mjun0812/flash-attention-prebuild-wheels/releases/download/v0.9.22
|
||||
secrets: inherit
|
||||
|
||||
# Dreamverse matrix: {backend, UI} x {12.6.3, 13.0.0}, Python 3.12. Torch backend
|
||||
# matches the base CUDA (cu126 / cu130). Keep these images amd64-only until the
|
||||
# required FA4 dependency stack is available and validated on arm64.
|
||||
|
||||
@@ -62,8 +62,9 @@ jobs:
|
||||
cuda-version: '13.0.0'
|
||||
torch-cuda-short: 'cu130'
|
||||
platform:
|
||||
# x86_64 builds the full cu126 + cu130 set (cu130 ships the consumer
|
||||
# Blackwell sm_120a FP4 kernels).
|
||||
# x86_64 builds the full cu126 + cu130 set. cu130 ships the
|
||||
# data-center Blackwell sm_100a/sm_103a VSA and consumer sm_120a FP4
|
||||
# kernels.
|
||||
- os: ubuntu-22.04
|
||||
arch: x86_64
|
||||
wheel-plat: manylinux_2_35_x86_64
|
||||
@@ -124,7 +125,7 @@ jobs:
|
||||
- name: Install dependencies (GCC, Clang, CUDA Paths, Git)
|
||||
run: |
|
||||
sudo apt update
|
||||
sudo apt install -y git patchelf gcc-11 g++-11 clang-11
|
||||
sudo apt install -y git gcc-11 g++-11 clang-11
|
||||
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 100 --slave /usr/bin/g++ g++ /usr/bin/g++-11
|
||||
|
||||
# Allow Git to Access Safe Directory
|
||||
@@ -168,7 +169,8 @@ jobs:
|
||||
# covers sm_120a; turbodiffusion covers sm_100a+sm_120a. The sm_100 FP4
|
||||
# forward is the FA4 CuTe DSL path in the fastvideo package (PR #1221),
|
||||
# JIT-compiled at runtime — not built into this wheel.
|
||||
# * x86_64 cu130 = Hopper TK + consumer Blackwell sm_120a FP4.
|
||||
# * x86_64 cu130 = Hopper TK + data-center Blackwell sm_100a/sm_103a VSA
|
||||
# + consumer Blackwell sm_120a FP4.
|
||||
# * x86_64 cu126 = Hopper TK only (older drivers; CUDA < 12.8 has no FP4).
|
||||
# The per-arch split in CMakeLists pins the FP4 targets to sm_120a and builds
|
||||
# the main extension for the full arch list. CMAKE_BUILD_PARALLEL_LEVEL caps
|
||||
@@ -178,7 +180,7 @@ jobs:
|
||||
export CMAKE_ARGS="${CMAKE_ARGS:-} -DFASTVIDEO_KERNEL_BUILD_TK=OFF -DFASTVIDEO_KERNEL_BUILD_ATTN_QAT_INFER=ON"
|
||||
export CMAKE_BUILD_PARALLEL_LEVEL=1
|
||||
elif [ "${{ matrix.torch-cuda.torch-cuda-short }}" = "cu130" ]; then
|
||||
export TORCH_CUDA_ARCH_LIST="9.0a;12.0a"
|
||||
export TORCH_CUDA_ARCH_LIST="9.0a;10.0a;12.0a"
|
||||
export CMAKE_ARGS="${CMAKE_ARGS:-} -DFASTVIDEO_KERNEL_BUILD_TK=ON -DFASTVIDEO_KERNEL_BUILD_ATTN_QAT_INFER=ON -DCMAKE_CUDA_ARCHITECTURES=90a"
|
||||
# A single FP4 TU (attn_qat_infer) can use ~8-12 GB on its own, so serialize.
|
||||
export CMAKE_BUILD_PARALLEL_LEVEL=1
|
||||
@@ -194,7 +196,11 @@ jobs:
|
||||
python -m build --wheel --outdir dist
|
||||
|
||||
# Fix the wheel to be manylinux compliant
|
||||
uv pip install --system auditwheel
|
||||
# Ubuntu 22.04 ships patchelf 0.14.3, while current auditwheel
|
||||
# requires at least 0.14.5. Use the stable PyPI binary on both
|
||||
# x86_64 and aarch64 release runners.
|
||||
uv pip install --system auditwheel patchelf==0.17.2.4
|
||||
patchelf --version
|
||||
# Point auditwheel at torch libs, but do not vendor them into the wheel.
|
||||
TORCH_LIB_DIR=$(python - <<'PY'
|
||||
import os
|
||||
@@ -211,7 +217,8 @@ jobs:
|
||||
--exclude libtorch.so \
|
||||
--exclude libc10.so \
|
||||
--exclude libc10_cuda.so \
|
||||
--exclude libtorch_python.so
|
||||
--exclude libtorch_python.so \
|
||||
--exclude libnccl.so.2
|
||||
# Move fixed wheels back to dist for upload consistency
|
||||
rm dist/*.whl
|
||||
mv fixed_dist/*.whl dist/
|
||||
|
||||
@@ -55,6 +55,8 @@ eggs/
|
||||
|
||||
# MkDocs documentation
|
||||
site/
|
||||
docs/assets/cookbook-serving.json
|
||||
examples/serving/clients/node_modules/
|
||||
docs/getting_started/examples/
|
||||
docs/examples/
|
||||
docs/inference/examples/
|
||||
@@ -133,6 +135,9 @@ fastvideo/tests/ssim/reference_videos/**
|
||||
!fastvideo/tests/ssim/reference_videos/**/*.mp4
|
||||
!fastvideo/tests/ssim/reference_videos/**/*.png
|
||||
|
||||
# Local H3 MLX kernel / exactness benches (JSON, logs, frames, videos)
|
||||
.kernel_bench/
|
||||
|
||||
# Editor logs and local Python version pins (accidentally committed)
|
||||
*.nvimlog
|
||||
.nvimlog
|
||||
|
||||
@@ -9,7 +9,7 @@ exclude: |
|
||||
tests/.*|
|
||||
scripts/.*|
|
||||
fastvideo/dataset/.*|
|
||||
fastvideo/models/.*|
|
||||
fastvideo/models/(?!wan/(config|vae_config|pipeline_config|definition|__init__)\.py$).*|
|
||||
^apps/dreamverse/web/.*|
|
||||
examples/.*|
|
||||
\.agents/.*|
|
||||
|
||||
@@ -3,12 +3,14 @@
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
| <a href="https://hao-ai-lab.github.io/FastVideo"><b>Documentation</b></a> | <a href="https://hao-ai-lab.github.io/FastVideo/inference/inference_quick_start/"><b> Quick Start</b></a> | <a href="https://github.com/hao-ai-lab/FastVideo/discussions/982" target="_blank"><b>Weekly Dev Meeting</b></a> | 🟣💬 <a href="https://join.slack.com/t/fastvideo/shared_invite/zt-3f4lao1uq-u~Ipx6Lt4J27AlD2y~IdLQ" target="_blank"> <b>Slack</b> </a> | 🟣💬 <a href="https://github.com/hao-ai-lab/FastVideo/discussions/1097" target="_blank"> <b> WeChat </b> </a> |
|
||||
| <a href="https://hao-ai-lab.github.io/FastVideo"><b>Documentation</b></a> | <a href="https://haoailab.com/FastVideo/cookbook/"><b>Cookbook</b></a> | <a href="https://hao-ai-lab.github.io/FastVideo/inference/inference_quick_start/"><b> Quick Start</b></a> | <a href="https://github.com/hao-ai-lab/FastVideo/discussions/982" target="_blank"><b>Weekly Dev Meeting</b></a> | 🟣💬 <a href="https://join.slack.com/t/fastvideo/shared_invite/zt-3f4lao1uq-u~Ipx6Lt4J27AlD2y~IdLQ" target="_blank"> <b>Slack</b> </a> | 🟣💬 <a href="https://github.com/hao-ai-lab/FastVideo/discussions/1097" target="_blank"> <b> WeChat </b> </a> |
|
||||
</p>
|
||||
|
||||
**FastVideo is a unified post-training and real-time inference framework for accelerated video generation.**
|
||||
|
||||
## NEWS
|
||||
- `2026/09/01`: FastH3 now runs locally on Apple Silicon through MLX and on NVIDIA DGX Spark through CUDA 13, including two-Spark inference. Follow the [FastH3 recipes](https://haoailab.com/FastVideo/cookbook/minimax-h3/) and read the [Blog](https://haoailab.com/blogs/fasth3-local/).
|
||||
- `2026/08/27`: [FastH3 Preview v1](https://haoailab.com/blogs/fasth3-preview/) is an open-weight 4-step sparse-distilled MiniMax-H3 model for synchronized video-and-audio generation, developed in collaboration with [Nuva Lab](https://nuvalab.ai/) and the [NVIDIA FastGen team](https://github.com/NVlabs/FastGen). Download the recommended [VSA / Data-Free weights](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree), or see the [full FastH3 collection](https://huggingface.co/collections/FastVideo/fastvideo-fasth3).
|
||||
- `2026/08/19`: FastVideo now supports MLX on Apple Silicon with [FastMetal-QAD](https://huggingface.co/collections/FastVideo/fastmetal), a family of 1.3B, 5B, and 14B models optimized for Mac—follow the [Apple Silicon guide](https://hao-ai-lab.github.io/FastVideo/getting_started/installation/mps/) and read the [Blog](https://haoailab.com/blogs/fastmetal/).
|
||||
- `2026/06/23`: Release FastWan-QAD: 5s of Video generated in 1.8s E2E. See the [FastWan-QAD models](https://huggingface.co/FastVideo/FastWan-QAD-FP8-1.3B), [Attn-QAT training guide](https://haoailab.com/FastVideo/training/attn_qat/), and [blog](https://haoailab.com/blogs/fastwan-qad/).
|
||||
- `2026/03/17`: Release demo: Into the Dreamverse: Vibe Directing in FastVideo, check out the [Blog](https://haoailab.com/blogs/dreamverse/).
|
||||
@@ -63,9 +65,10 @@ UV_TORCH_BACKEND=cu126 uv pip install fastvideo
|
||||
Use `UV_TORCH_BACKEND=cu130` on CUDA 13. Apple silicon users should follow the
|
||||
[MPS installation guide](https://hao-ai-lab.github.io/FastVideo/getting_started/installation/mps/).
|
||||
|
||||
> **On an Apple Silicon Mac?** FastVideo runs FastWan text-to-video natively
|
||||
> through an MLX runtime — a 5-second 480p clip generated locally, no cloud,
|
||||
> no discrete GPU. Install with `uv pip install -e '.[mlx]'` and follow the
|
||||
> **On an Apple Silicon Mac?** FastVideo runs FastMetal-QAD through an MLX
|
||||
> runtime. Install with `uv pip install -e '.[mlx]'`, download
|
||||
> [`FastVideo/FastMetal-1.3B-QAD`](https://huggingface.co/FastVideo/FastMetal-1.3B-QAD),
|
||||
> and follow the
|
||||
> [Apple Silicon guide](https://hao-ai-lab.github.io/FastVideo/getting_started/installation/mps/).
|
||||
|
||||
Please see our [docs](https://hao-ai-lab.github.io/FastVideo/getting_started/installation/) for more detailed installation instructions.
|
||||
|
||||
@@ -97,13 +97,33 @@ dreamverse-server --port 8009
|
||||
dreamverse-mock-server --port 8009
|
||||
```
|
||||
|
||||
### Run Dreamverse with FastH3
|
||||
|
||||
Select the VSA data-free FastH3 Preview profile when you start the backend:
|
||||
|
||||
```bash
|
||||
DREAMVERSE_MODEL_ID=fast-h3 dreamverse-server --port 8009
|
||||
```
|
||||
|
||||
The `fast-h3` profile uses four visible GPUs by default. It loads the `MiniMaxAI/MiniMax-H3` base checkpoint and the
|
||||
`vsa-datafree/adapter_model.safetensors` adapter from
|
||||
`FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA`. Each request generates a 124-frame, 768×1344 video with
|
||||
synchronized audio and five sigma-grid points. Dreamverse uses the last frame of each segment as first-frame
|
||||
conditioning for the following segment.
|
||||
|
||||
Set `CUDA_VISIBLE_DEVICES` when you need to choose the four physical GPUs:
|
||||
|
||||
```bash
|
||||
CUDA_VISIBLE_DEVICES=0,1,2,3 DREAMVERSE_MODEL_ID=fast-h3 dreamverse-server --port 8009
|
||||
```
|
||||
|
||||
> **Expect a slow first boot.** With `torch.compile` and startup warmup enabled
|
||||
> (the default), the backend compiles the segment 1 and segment 2 inference
|
||||
> paths before it reports ready — this can take **tens of minutes on a cold
|
||||
> cache**, regardless of how you deploy (local, server, Docker, or Modal).
|
||||
> `/healthz` responds as soon as the process is up; `/readyz` stays `503` until
|
||||
> warmup finishes. For a faster, uncompiled startup while testing, set
|
||||
> `FASTVIDEO_ENABLE_STARTUP_WARMUP=0` before starting the backend.
|
||||
> warmup finishes. To defer compilation until the first generated request while
|
||||
> testing, set `FASTVIDEO_ENABLE_STARTUP_WARMUP=0` before starting the backend.
|
||||
|
||||
## Frontend Setup
|
||||
|
||||
@@ -219,6 +239,7 @@ selection, and mock-server behavior:
|
||||
pytest apps/dreamverse/dreamverse/tests/test_config.py \
|
||||
apps/dreamverse/dreamverse/tests/test_entrypoints.py \
|
||||
apps/dreamverse/dreamverse/tests/test_gpu_pool.py \
|
||||
apps/dreamverse/dreamverse/tests/test_minimax_h3_generation.py \
|
||||
apps/dreamverse/dreamverse/tests/test_mock_server.py -q
|
||||
```
|
||||
|
||||
|
||||
+12
-1
@@ -139,7 +139,18 @@ session.
|
||||
- startup warmup
|
||||
- user join/leave commands
|
||||
- `USER_STEP` execution for each segment
|
||||
- continuation state between segments
|
||||
- generation-command routing and stream-result delivery
|
||||
|
||||
Model generation has a separate ownership boundary inside each GPU process:
|
||||
|
||||
- `apps/dreamverse/dreamverse/generation_worker.py` selects the backend that the active model profile declares and owns
|
||||
the backend lifecycle.
|
||||
- `apps/dreamverse/dreamverse/ltx2_generation.py` owns LTX-2 generator configuration, video and audio continuation, and
|
||||
runtime LoRA application.
|
||||
- `apps/dreamverse/dreamverse/minimax_h3_generation.py` owns the VSA data-free FastH3 adapter, FastH3 generator and
|
||||
request configuration, and last-frame continuation through MiniMax H3 first-frame conditioning.
|
||||
- `apps/dreamverse/dreamverse/generation_contracts.py` defines the decoded media and stream-trimming result that both
|
||||
model backends return to `apps/dreamverse/dreamverse/gpu_pool.py`.
|
||||
|
||||
`apps/dreamverse/dreamverse/prompt_enhancer.py` manages:
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Benchmark the LTX-2 generation pipeline driven by the dreamverse Python SDK path.
|
||||
|
||||
Mirrors how ``apps/dreamverse/dreamverse/video_generation.py`` constructs
|
||||
Mirrors how ``apps/dreamverse/dreamverse/ltx2_generation.py`` constructs
|
||||
``GeneratorConfig`` and calls ``VideoGenerator.generate()``, then
|
||||
captures per-stage timings via the ``FASTVIDEO_STAGE_LOGGING=1`` log
|
||||
hooks (same mechanism as ``FastVideo-internal/examples/inference/basic/
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
_REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
_SERVER_ROOT = Path(__file__).resolve().parent
|
||||
@@ -55,16 +56,34 @@ FRONTEND_STATIC_DIR_CANDIDATES = _resolve_frontend_static_dir_candidates()
|
||||
MODEL_REGISTRY = {
|
||||
"fast-ltx2": {
|
||||
"name": "FastLTX2",
|
||||
"generation_backend": "ltx2",
|
||||
"default_sp_size": 1,
|
||||
"model_path": "FastVideo/LTX2-Distilled-Diffusers",
|
||||
"config_model_path": "FastVideo/LTX2-Distilled-Diffusers",
|
||||
"lora_repo": "FastVideo/LTX2-OmniNFT-LoRA",
|
||||
},
|
||||
"fast-ltx23": {
|
||||
"name": "FastLTX23",
|
||||
"generation_backend": "ltx2",
|
||||
"default_sp_size": 1,
|
||||
"model_path": "FastVideo/LTX-2.3-Distilled-Diffusers",
|
||||
"config_model_path": "FastVideo/LTX-2.3-Distilled-Diffusers",
|
||||
"lora_repo": "FastVideo/LTX-2.3-OmniNFT-LoRA",
|
||||
},
|
||||
"fast-h3": {
|
||||
"name": "FastH3",
|
||||
"generation_backend": "minimax_h3",
|
||||
"default_sp_size": 4,
|
||||
"model_path": "MiniMaxAI/MiniMax-H3",
|
||||
"adapter_repo": "FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA",
|
||||
"adapter_filename": "vsa-datafree/adapter_model.safetensors",
|
||||
"attention_backend": "VIDEO_SPARSE_ATTN_H3",
|
||||
"height": 768,
|
||||
"width": 1344,
|
||||
"num_frames": 124,
|
||||
"num_inference_steps": 5,
|
||||
"seed": 1000,
|
||||
},
|
||||
}
|
||||
|
||||
DEFAULT_MODEL_ID = "fast-ltx2"
|
||||
@@ -171,7 +190,7 @@ def _optional_env(*names: str) -> str | None:
|
||||
DEVTOOLS_ENABLED = _env_bool("FASTVIDEO_ENABLE_DEVTOOLS", False)
|
||||
PROMPT_SAFETY_ENABLED = _env_bool("FASTVIDEO_ENABLE_PROMPT_SAFETY", False)
|
||||
DREAMVERSE_MAX_AUTOTUNE = _env_bool("DREAMVERSE_MAX_AUTOTUNE", True)
|
||||
DREAMVERSE_SP_SIZE = max(1, _env_int("DREAMVERSE_SP_SIZE", 1))
|
||||
DREAMVERSE_SP_SIZE = max(1, _env_int("DREAMVERSE_SP_SIZE", cast(int, MODEL_CONFIG["default_sp_size"])))
|
||||
|
||||
DREAMVERSE_MODEL_PATH = (os.getenv("DREAMVERSE_MODEL_PATH", "").strip() or None)
|
||||
if DREAMVERSE_MODEL_PATH:
|
||||
@@ -213,7 +232,7 @@ def _resolve_lora_spec(spec: str) -> str | None:
|
||||
if not spec:
|
||||
return None
|
||||
if spec.lower() == "omninft":
|
||||
return MODEL_CONFIG.get("lora_repo")
|
||||
return cast(str | None, MODEL_CONFIG.get("lora_repo"))
|
||||
if spec.lower() in AVAILABLE_LORAS:
|
||||
return AVAILABLE_LORAS[spec.lower()]["repo"]
|
||||
return spec
|
||||
|
||||
@@ -0,0 +1,46 @@
|
||||
"""Shared contract between DreamVerse generation backends and GPU workers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Protocol
|
||||
|
||||
|
||||
@dataclass
|
||||
class StepResult:
|
||||
"""Decoded media and stream-trimming metadata for one DreamVerse segment."""
|
||||
|
||||
frames: list
|
||||
audio: Any
|
||||
audio_sample_rate: int | None
|
||||
timings: dict[str, float]
|
||||
head_trim_frames: int
|
||||
head_trim_audio_frames: int
|
||||
|
||||
|
||||
class GenerationBackend(Protocol):
|
||||
"""Model-owned generation operations used by one GPU worker process."""
|
||||
|
||||
def initialize(self, model_config: dict | None = None) -> None:
|
||||
...
|
||||
|
||||
def shutdown(self) -> None:
|
||||
...
|
||||
|
||||
def clear_conditioning(self) -> None:
|
||||
...
|
||||
|
||||
def generate_step(
|
||||
self,
|
||||
prompt: str,
|
||||
segment_idx: int,
|
||||
image_path: str | None,
|
||||
reset_conditioning: bool,
|
||||
) -> StepResult:
|
||||
...
|
||||
|
||||
def warmup(self, prompt: str) -> dict[str, float]:
|
||||
...
|
||||
|
||||
def apply_lora_stack(self, stack: list[tuple[str, float]]) -> tuple[str | None, str | None]:
|
||||
...
|
||||
@@ -0,0 +1,96 @@
|
||||
"""Select and own one model-specific generation backend per GPU process."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dreamverse.config import MODEL_CONFIG
|
||||
from dreamverse.generation_contracts import GenerationBackend, StepResult
|
||||
|
||||
|
||||
def _create_generation_backend(backend_name: str, gpu_id: int) -> GenerationBackend:
|
||||
"""Construct the backend that owns the selected model family's behavior."""
|
||||
if backend_name == "ltx2":
|
||||
from dreamverse.ltx2_generation import LTX2GenerationBackend
|
||||
|
||||
return LTX2GenerationBackend(gpu_id)
|
||||
if backend_name == "minimax_h3":
|
||||
from dreamverse.minimax_h3_generation import MiniMaxH3GenerationBackend
|
||||
|
||||
return MiniMaxH3GenerationBackend(gpu_id)
|
||||
raise ValueError(f"Unsupported DreamVerse generation backend: {backend_name!r}")
|
||||
|
||||
|
||||
class VideoGenerationWorker:
|
||||
"""Delegate GPU lifecycle and generation calls to the active model backend."""
|
||||
|
||||
def __init__(self, gpu_id: int):
|
||||
self.gpu_id = gpu_id
|
||||
self.model_config: dict = dict(MODEL_CONFIG)
|
||||
self.backend_name: str | None = None
|
||||
self.backend: GenerationBackend | None = None
|
||||
|
||||
def initialize(self, model_config: dict | None = None) -> None:
|
||||
"""Load the requested model through its generation backend.
|
||||
|
||||
Model selection belongs here so the GPU process and streaming layers
|
||||
use one stable media contract without importing model-specific code.
|
||||
"""
|
||||
requested_model_config = dict(model_config) if model_config is not None else dict(self.model_config)
|
||||
backend_name = requested_model_config.get("generation_backend")
|
||||
if not isinstance(backend_name, str) or not backend_name:
|
||||
raise ValueError("DreamVerse model configuration requires `generation_backend`.")
|
||||
|
||||
candidate_backend = self.backend
|
||||
if candidate_backend is None or self.backend_name != backend_name:
|
||||
if candidate_backend is not None:
|
||||
candidate_backend.shutdown()
|
||||
candidate_backend = _create_generation_backend(backend_name, self.gpu_id)
|
||||
|
||||
try:
|
||||
candidate_backend.initialize(requested_model_config)
|
||||
except Exception:
|
||||
try:
|
||||
candidate_backend.shutdown()
|
||||
except Exception as shutdown_error:
|
||||
print(f"[GPU {self.gpu_id}] Backend cleanup after initialization failure: {shutdown_error}")
|
||||
self.backend = None
|
||||
self.backend_name = None
|
||||
raise
|
||||
|
||||
self.model_config = requested_model_config
|
||||
self.backend = candidate_backend
|
||||
self.backend_name = backend_name
|
||||
|
||||
def _require_backend(self) -> GenerationBackend:
|
||||
"""Return the initialized backend or fail before processing a command."""
|
||||
if self.backend is None:
|
||||
raise RuntimeError("Generation backend is not initialized.")
|
||||
return self.backend
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""Release model resources owned by the selected backend."""
|
||||
if self.backend is not None:
|
||||
self.backend.shutdown()
|
||||
|
||||
def clear_conditioning(self) -> None:
|
||||
self._require_backend().clear_conditioning()
|
||||
|
||||
def generate_step(
|
||||
self,
|
||||
prompt: str,
|
||||
segment_idx: int,
|
||||
image_path: str | None,
|
||||
reset_conditioning: bool,
|
||||
) -> StepResult:
|
||||
"""Generate one segment through the selected model backend."""
|
||||
return self._require_backend().generate_step(
|
||||
prompt,
|
||||
segment_idx,
|
||||
image_path,
|
||||
reset_conditioning,
|
||||
)
|
||||
|
||||
def warmup(self, prompt: str) -> dict[str, float]:
|
||||
return self._require_backend().warmup(prompt)
|
||||
|
||||
def apply_lora_stack(self, stack: list[tuple[str, float]]) -> tuple[str | None, str | None]:
|
||||
return self._require_backend().apply_lora_stack(stack)
|
||||
@@ -12,7 +12,7 @@ from enum import Enum
|
||||
from multiprocessing import Process, Queue
|
||||
|
||||
from dreamverse.config import (
|
||||
DEFAULT_MODEL_ID,
|
||||
ACTIVE_MODEL_ID,
|
||||
DREAMVERSE_SP_SIZE,
|
||||
MODEL_REGISTRY,
|
||||
STARTUP_WARMUP_ENABLED,
|
||||
@@ -54,7 +54,7 @@ from dreamverse.worker_ipc import (
|
||||
def _parse_requested_gpu_limit() -> int | None:
|
||||
raw_value = os.getenv("FASTVIDEO_GPU_COUNT", "").strip().lower()
|
||||
if not raw_value:
|
||||
return 1
|
||||
return DREAMVERSE_SP_SIZE
|
||||
if raw_value == "all":
|
||||
return None
|
||||
try:
|
||||
@@ -164,12 +164,12 @@ def gpu_worker_process(
|
||||
os.environ["CUDA_VISIBLE_DEVICES"] = cuda_device
|
||||
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "FLASH_ATTN"
|
||||
|
||||
from dreamverse.video_generation import VideoGenerationWorker
|
||||
from dreamverse.generation_worker import VideoGenerationWorker
|
||||
|
||||
worker = VideoGenerationWorker(gpu_id)
|
||||
|
||||
def event_loop(first_cmd: Command = None):
|
||||
"""Blocking event loop for LTX2; dispatches user commands."""
|
||||
"""Block on generation commands after the model is initialized."""
|
||||
print(f"[GPU {gpu_id}] Entering event loop")
|
||||
|
||||
def handle_command(cmd: Command):
|
||||
@@ -435,7 +435,7 @@ class GPUSlot:
|
||||
self._response_reader_task: asyncio.Task | None = None
|
||||
self._active: bool = False
|
||||
self._reader_lock: asyncio.Lock | None = None
|
||||
self.current_model_id: str = DEFAULT_MODEL_ID
|
||||
self.current_model_id: str | None = ACTIVE_MODEL_ID
|
||||
self.shared_stream_buffer = None
|
||||
self.shared_stream_buffer_size = SHARED_STREAM_BUFFER_BYTES
|
||||
|
||||
@@ -690,7 +690,7 @@ class GPUSlot:
|
||||
async def join_user(self, user_id: str, model_id: str = None) -> JoinAck:
|
||||
"""Add a user to this GPU."""
|
||||
if model_id is None:
|
||||
model_id = DEFAULT_MODEL_ID
|
||||
model_id = ACTIVE_MODEL_ID
|
||||
|
||||
# Reload model if a different one is requested
|
||||
if model_id != self.current_model_id and model_id in MODEL_REGISTRY:
|
||||
@@ -705,16 +705,23 @@ class GPUSlot:
|
||||
self.connected_users.clear()
|
||||
|
||||
model_config = MODEL_REGISTRY[model_id]
|
||||
reload_response = await self._send_command(Command(CommandType.RELOAD_MODEL,
|
||||
payload=ReloadModelPayload(model_config=model_config),
|
||||
user_id="__reload__"),
|
||||
timeout=600.0)
|
||||
try:
|
||||
reload_response = await self._send_command(Command(
|
||||
CommandType.RELOAD_MODEL,
|
||||
payload=ReloadModelPayload(model_config=model_config),
|
||||
user_id="__reload__"),
|
||||
timeout=600.0)
|
||||
except Exception:
|
||||
self.current_model_id = None
|
||||
raise
|
||||
match reload_response:
|
||||
case ReloadAck():
|
||||
pass
|
||||
case WorkerError(message=msg):
|
||||
self.current_model_id = None
|
||||
raise RuntimeError(f"Model reload failed: {msg}")
|
||||
case _:
|
||||
self.current_model_id = None
|
||||
raise RuntimeError(f"Unexpected reload response: "
|
||||
f"{type(reload_response).__name__}")
|
||||
|
||||
|
||||
+4
-22
@@ -1,9 +1,9 @@
|
||||
"""LTX2 model lifecycle and continuation conditioning.
|
||||
"""LTX-2 model lifecycle and continuation conditioning.
|
||||
|
||||
Runs inside a GPU worker subprocess. Owns the model, the audio
|
||||
encoder, and the per-session continuation state carried across
|
||||
segments. Callers must set ``os.environ["CUDA_VISIBLE_DEVICES"]``
|
||||
before constructing ``VideoGenerationWorker`` — all ``fastvideo.*``
|
||||
before constructing ``LTX2GenerationBackend`` — all ``fastvideo.*``
|
||||
imports are deferred to method bodies so nothing touches CUDA at
|
||||
module import time.
|
||||
"""
|
||||
@@ -14,9 +14,6 @@ import gc
|
||||
import os
|
||||
import re
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
|
||||
@@ -35,6 +32,7 @@ from dreamverse.config import (
|
||||
DREAMVERSE_LORA_STACK,
|
||||
_resolve_lora_spec,
|
||||
)
|
||||
from dreamverse.generation_contracts import StepResult
|
||||
|
||||
# Multi-frame decoded continuation defaults from
|
||||
# examples/inference/basic/basic_ltx2_distilled_video_continuation.py.
|
||||
@@ -80,22 +78,6 @@ def _reset_lora_registry(worker) -> dict:
|
||||
return {"status": "lora_registry_reset"}
|
||||
|
||||
|
||||
@dataclass
|
||||
class StepResult:
|
||||
"""Output of one generation step.
|
||||
|
||||
``head_trim_frames`` / ``head_trim_audio_frames`` are derived here
|
||||
so downstream AV streaming never needs to import conditioning
|
||||
constants.
|
||||
"""
|
||||
frames: list
|
||||
audio: Any
|
||||
audio_sample_rate: int | None
|
||||
timings: dict
|
||||
head_trim_frames: int
|
||||
head_trim_audio_frames: int
|
||||
|
||||
|
||||
class ContinuationState:
|
||||
"""Per-session video + audio conditioning carried across segments."""
|
||||
|
||||
@@ -202,7 +184,7 @@ class ContinuationState:
|
||||
self.audio_latents = latents.detach().clone().cpu()
|
||||
|
||||
|
||||
class VideoGenerationWorker:
|
||||
class LTX2GenerationBackend:
|
||||
"""Single-GPU LTX2 generator with continuation state.
|
||||
|
||||
Caller must set ``os.environ["CUDA_VISIBLE_DEVICES"]`` before
|
||||
@@ -0,0 +1,297 @@
|
||||
"""FastH3 model lifecycle and first-frame continuation for DreamVerse."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import gc
|
||||
import os
|
||||
import time
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
|
||||
from dreamverse.config import DREAMVERSE_SP_SIZE
|
||||
from dreamverse.generation_contracts import StepResult
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from PIL.Image import Image
|
||||
|
||||
|
||||
def _required_config_str(model_config: dict, field_name: str) -> str:
|
||||
"""Read one required non-empty string from a DreamVerse model profile."""
|
||||
value = model_config.get(field_name)
|
||||
if not isinstance(value, str) or not value.strip():
|
||||
raise ValueError(f"FastH3 model configuration requires `{field_name}`.")
|
||||
return value.strip()
|
||||
|
||||
|
||||
class MiniMaxH3GenerationBackend:
|
||||
"""Run the VSA data-free FastH3 adapter and retain one continuation frame."""
|
||||
|
||||
def __init__(self, gpu_id: int):
|
||||
self.gpu_id = gpu_id
|
||||
self.generator: Any | None = None
|
||||
self.model_config: dict = {}
|
||||
self.continuation_image: Image | None = None
|
||||
|
||||
def _gpu_mem(self) -> str:
|
||||
allocated_gib = torch.cuda.memory_allocated() / 1024**3
|
||||
reserved_gib = torch.cuda.memory_reserved() / 1024**3
|
||||
return f"alloc={allocated_gib:.2f}GiB, reserved={reserved_gib:.2f}GiB"
|
||||
|
||||
@staticmethod
|
||||
def _configure_environment(attention_backend: str) -> None:
|
||||
"""Apply the fixed boot-time switches from the FastH3 reference recipe."""
|
||||
os.environ.update({
|
||||
"FASTVIDEO_ATTENTION_BACKEND": attention_backend,
|
||||
"FASTVIDEO_FA4": "1",
|
||||
"FASTVIDEO_MINIMAX_H3_FUSIONS": "all",
|
||||
"FASTVIDEO_VSA_SM100A": "0",
|
||||
})
|
||||
os.environ.pop("FASTVIDEO_INFERENCE_TORCH_COMPILE", None)
|
||||
|
||||
def initialize(self, model_config: dict | None = None) -> None:
|
||||
"""Download the fixed Preview adapter and load the FastH3 generator.
|
||||
|
||||
The model profile owns the base checkpoint, adapter file, attention
|
||||
backend, and generation geometry. The backend translates that profile
|
||||
into FastVideo's typed generator configuration.
|
||||
"""
|
||||
if model_config is not None:
|
||||
self.model_config = dict(model_config)
|
||||
if not self.model_config:
|
||||
raise ValueError("FastH3 initialization requires a model configuration.")
|
||||
|
||||
if self.generator is not None:
|
||||
self.generator.shutdown()
|
||||
self.generator = None
|
||||
gc.collect()
|
||||
torch.cuda.empty_cache()
|
||||
|
||||
self.clear_conditioning()
|
||||
model_path = _required_config_str(self.model_config, "model_path")
|
||||
adapter_repo = _required_config_str(self.model_config, "adapter_repo")
|
||||
adapter_filename = _required_config_str(self.model_config, "adapter_filename")
|
||||
attention_backend = _required_config_str(self.model_config, "attention_backend")
|
||||
self._configure_environment(attention_backend)
|
||||
|
||||
from huggingface_hub import hf_hub_download
|
||||
|
||||
from fastvideo import VideoGenerator
|
||||
from fastvideo.api import (
|
||||
CompileConfig,
|
||||
ComponentConfig,
|
||||
EngineConfig,
|
||||
GeneratorConfig,
|
||||
OffloadConfig,
|
||||
ParallelismConfig,
|
||||
PipelineSelection,
|
||||
)
|
||||
|
||||
adapter_path = hf_hub_download(repo_id=adapter_repo, filename=adapter_filename)
|
||||
experimental = {
|
||||
"attention_backend": attention_backend,
|
||||
"inference_torch_compile": attention_backend == "FLASH_ATTN",
|
||||
"vae_parallel_decode": True,
|
||||
"vae_parallel_decode_strategy": "gather",
|
||||
}
|
||||
if attention_backend == "VIDEO_SPARSE_ATTN_H3":
|
||||
experimental.update({
|
||||
"VSA_sparsity": 0.9,
|
||||
"VSA_tile_size": 64,
|
||||
})
|
||||
generator_config = GeneratorConfig(
|
||||
model_path=model_path,
|
||||
pipeline=PipelineSelection(
|
||||
components=ComponentConfig(lora_path=adapter_path, lora_strength=1.0),
|
||||
experimental=experimental,
|
||||
),
|
||||
engine=EngineConfig(
|
||||
num_gpus=DREAMVERSE_SP_SIZE,
|
||||
parallelism=ParallelismConfig(tp_size=1, sp_size=DREAMVERSE_SP_SIZE),
|
||||
offload=OffloadConfig(
|
||||
dit=False,
|
||||
dit_layerwise=False,
|
||||
text_encoder=True,
|
||||
image_encoder=True,
|
||||
vae=True,
|
||||
pin_cpu_memory=True,
|
||||
),
|
||||
compile=CompileConfig(enabled=False, vae_enabled=True),
|
||||
use_fsdp_inference=False,
|
||||
),
|
||||
)
|
||||
|
||||
print(f"[GPU {self.gpu_id}] Loading FastH3 model: {model_path}")
|
||||
print(f"[GPU {self.gpu_id}] FastH3 adapter: {adapter_repo}/{adapter_filename}")
|
||||
print(f"[GPU {self.gpu_id}] Before model load: {self._gpu_mem()}")
|
||||
self.generator = VideoGenerator.from_config(generator_config)
|
||||
print(f"[GPU {self.gpu_id}] FastH3 loaded: {self._gpu_mem()} (warmup pending)")
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""Release the FastVideo generator and cached continuation image."""
|
||||
self.clear_conditioning()
|
||||
if self.generator is not None:
|
||||
self.generator.shutdown()
|
||||
self.generator = None
|
||||
|
||||
def clear_conditioning(self) -> None:
|
||||
"""Release the first-frame image retained for the next segment."""
|
||||
if self.continuation_image is not None:
|
||||
self.continuation_image.close()
|
||||
self.continuation_image = None
|
||||
|
||||
@staticmethod
|
||||
def _load_rgb_image(image_path: str) -> Image:
|
||||
"""Load an image into an independent RGB buffer with no open file handle."""
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(image_path) as image:
|
||||
return image.convert("RGB").copy()
|
||||
|
||||
def _select_conditioning_image(
|
||||
self,
|
||||
segment_idx: int,
|
||||
image_path: str | None,
|
||||
reset_conditioning: bool,
|
||||
) -> tuple[Image | None, bool]:
|
||||
"""Select the initial upload or retained last frame for one segment."""
|
||||
if reset_conditioning:
|
||||
self.clear_conditioning()
|
||||
if segment_idx > 1 and self.continuation_image is not None:
|
||||
return self.continuation_image.copy(), True
|
||||
if segment_idx > 1 and not reset_conditioning:
|
||||
raise RuntimeError(f"FastH3 segment {segment_idx} requires a retained continuation frame.")
|
||||
if segment_idx == 1 and image_path:
|
||||
return self._load_rgb_image(image_path), False
|
||||
return None, False
|
||||
|
||||
def _build_request(self, prompt: str, conditioning_image: Image | None):
|
||||
"""Build the typed FastVideo request owned by the FastH3 profile."""
|
||||
from fastvideo.api import GenerationRequest, InputConfig, OutputConfig, SamplingConfig
|
||||
|
||||
return GenerationRequest(
|
||||
prompt=prompt,
|
||||
negative_prompt="",
|
||||
inputs=InputConfig(pil_image=conditioning_image),
|
||||
sampling=SamplingConfig(
|
||||
height=int(self.model_config["height"]),
|
||||
width=int(self.model_config["width"]),
|
||||
num_frames=int(self.model_config["num_frames"]),
|
||||
fps=24,
|
||||
num_inference_steps=int(self.model_config["num_inference_steps"]),
|
||||
guidance_scale=1.0,
|
||||
batch_cfg=False,
|
||||
seed=int(self.model_config["seed"]),
|
||||
),
|
||||
output=OutputConfig(save_video=False, return_frames=True),
|
||||
)
|
||||
|
||||
def _save_continuation_frame(self, frames: list) -> None:
|
||||
"""Retain the last decoded frame as first-frame conditioning."""
|
||||
from PIL import Image
|
||||
|
||||
self.clear_conditioning()
|
||||
self.continuation_image = Image.fromarray(np.ascontiguousarray(frames[-1])).convert("RGB")
|
||||
|
||||
def generate_step(
|
||||
self,
|
||||
prompt: str,
|
||||
segment_idx: int,
|
||||
image_path: str | None,
|
||||
reset_conditioning: bool,
|
||||
) -> StepResult:
|
||||
"""Generate one synchronized FastH3 segment and retain its last frame.
|
||||
|
||||
Later segments use MiniMax H3's first-frame-to-video path. The first
|
||||
conditioned frame and its matching audio duration are trimmed before
|
||||
streaming so adjacent segments do not duplicate media.
|
||||
"""
|
||||
if self.generator is None:
|
||||
raise RuntimeError("FastH3 generator is not initialized.")
|
||||
conditioning_image, uses_continuation = self._select_conditioning_image(
|
||||
segment_idx,
|
||||
image_path,
|
||||
reset_conditioning,
|
||||
)
|
||||
request = self._build_request(prompt, conditioning_image)
|
||||
started = time.perf_counter()
|
||||
try:
|
||||
result = self.generator.generate(request)
|
||||
finally:
|
||||
if conditioning_image is not None:
|
||||
conditioning_image.close()
|
||||
torch.cuda.synchronize()
|
||||
generation_ms = (time.perf_counter() - started) * 1000.0
|
||||
|
||||
if isinstance(result, list):
|
||||
raise RuntimeError("FastH3 returned multiple results for one DreamVerse segment.")
|
||||
frames = result.frames
|
||||
if not isinstance(frames, list) or not frames:
|
||||
raise RuntimeError("FastH3 generation did not return decoded frames.")
|
||||
audio = result.audio
|
||||
audio_sample_rate = result.audio_sample_rate
|
||||
if audio is not None and audio_sample_rate is None:
|
||||
raise RuntimeError("FastH3 returned audio without an audio sample rate.")
|
||||
|
||||
save_started = time.perf_counter()
|
||||
self._save_continuation_frame(frames)
|
||||
save_conditioning_ms = (time.perf_counter() - save_started) * 1000.0
|
||||
timings = {
|
||||
"generation_ms": generation_ms,
|
||||
"generation_time_ms": float(result.generation_time or 0.0) * 1000.0,
|
||||
"save_conditioning_ms": save_conditioning_ms,
|
||||
"e2e_latency_ms": (time.perf_counter() - started) * 1000.0,
|
||||
}
|
||||
trim_frames = 1 if uses_continuation else 0
|
||||
print(f"[GPU {self.gpu_id}] FastH3 segment {segment_idx}: "
|
||||
f"{len(frames)} frames, gen={generation_ms:.0f}ms, "
|
||||
f"save_conditioning={save_conditioning_ms:.0f}ms, "
|
||||
f"e2e={timings['e2e_latency_ms']:.0f}ms")
|
||||
return StepResult(
|
||||
frames=frames,
|
||||
audio=audio,
|
||||
audio_sample_rate=audio_sample_rate,
|
||||
timings=timings,
|
||||
head_trim_frames=trim_frames,
|
||||
head_trim_audio_frames=trim_frames,
|
||||
)
|
||||
|
||||
def warmup(self, prompt: str) -> dict[str, float]:
|
||||
"""Compile the FastH3 text and first-frame paths before readiness."""
|
||||
warmup_prompt = (prompt or "").strip()
|
||||
if not warmup_prompt:
|
||||
raise RuntimeError("Startup warmup prompt must be non-empty.")
|
||||
print(f"[GPU {self.gpu_id}] FastH3 startup warmup starting "
|
||||
"(synthetic segments: text-to-video, first-frame-to-video)")
|
||||
started = time.perf_counter()
|
||||
text_result = self.generate_step(
|
||||
warmup_prompt,
|
||||
segment_idx=1,
|
||||
image_path=None,
|
||||
reset_conditioning=True,
|
||||
)
|
||||
first_frame_result = self.generate_step(
|
||||
warmup_prompt,
|
||||
segment_idx=2,
|
||||
image_path=None,
|
||||
reset_conditioning=False,
|
||||
)
|
||||
total_ms = (time.perf_counter() - started) * 1000.0
|
||||
self.clear_conditioning()
|
||||
text_ms = float(text_result.timings.get("e2e_latency_ms", 0.0))
|
||||
first_frame_ms = float(first_frame_result.timings.get("e2e_latency_ms", 0.0))
|
||||
print(f"[GPU {self.gpu_id}] FastH3 startup warmup complete: "
|
||||
f"text_to_video={text_ms:.0f}ms, "
|
||||
f"first_frame_to_video={first_frame_ms:.0f}ms, "
|
||||
f"total={total_ms:.0f}ms")
|
||||
return {
|
||||
"warmup_text_to_video_ms": text_ms,
|
||||
"warmup_first_frame_to_video_ms": first_frame_ms,
|
||||
"warmup_total_ms": total_ms,
|
||||
}
|
||||
|
||||
def apply_lora_stack(self, stack: list[tuple[str, float]]) -> tuple[str | None, str | None]:
|
||||
"""Reject runtime LoRA mutation because FastH3 uses one startup adapter."""
|
||||
del stack
|
||||
raise RuntimeError("FastH3 uses its fixed startup adapter and does not support runtime LoRA changes.")
|
||||
@@ -30,7 +30,7 @@ from dreamverse.session_init_image import cleanup_session_init_image, persist_se
|
||||
from dreamverse.worker_ipc import MediaChunk, MediaComplete, MediaInit
|
||||
|
||||
from dreamverse.config import (
|
||||
DEFAULT_MODEL_ID,
|
||||
ACTIVE_MODEL_ID,
|
||||
GENERATION_SEGMENT_CAP,
|
||||
PROMPT_AUTO_SLEEP_MS,
|
||||
PROMPT_AUTO_TIMEOUT_MS,
|
||||
@@ -264,7 +264,7 @@ class SessionController:
|
||||
timeout_task = asyncio.create_task(session_timeout())
|
||||
|
||||
# Join the engine on this GPU.
|
||||
await slot.join_user(client_id, model_id=DEFAULT_MODEL_ID)
|
||||
await slot.join_user(client_id, model_id=ACTIVE_MODEL_ID)
|
||||
|
||||
# Notify client they're connected to a GPU.
|
||||
await ws_send_json({
|
||||
|
||||
@@ -2,13 +2,14 @@ from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
from types import ModuleType
|
||||
|
||||
import pytest
|
||||
|
||||
SERVER_DIR = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def _load_config_module():
|
||||
def _load_config_module() -> ModuleType:
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"server_config_test_module",
|
||||
SERVER_DIR / "config.py",
|
||||
@@ -20,7 +21,7 @@ def _load_config_module():
|
||||
return module
|
||||
|
||||
|
||||
def _set_required_prompt_keys(monkeypatch):
|
||||
def _set_required_prompt_keys(monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
monkeypatch.setenv("CEREBRAS_API_KEY", "cerebras-key")
|
||||
monkeypatch.setenv("GROQ_API_KEY", "groq-key")
|
||||
|
||||
@@ -150,3 +151,38 @@ def test_config_rejects_invalid_prompt_provider(monkeypatch):
|
||||
|
||||
with pytest.raises(RuntimeError, match="Invalid FASTVIDEO_PROMPT_PROVIDER"):
|
||||
_load_config_module()
|
||||
|
||||
|
||||
def test_config_registers_vsa_datafree_fasth3_profile(monkeypatch):
|
||||
"""The FastH3 registry entry owns the complete fixed Preview recipe."""
|
||||
_set_required_prompt_keys(monkeypatch)
|
||||
|
||||
module = _load_config_module()
|
||||
|
||||
assert module.MODEL_REGISTRY["fast-h3"] == {
|
||||
"name": "FastH3",
|
||||
"generation_backend": "minimax_h3",
|
||||
"default_sp_size": 4,
|
||||
"model_path": "MiniMaxAI/MiniMax-H3",
|
||||
"adapter_repo": "FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA",
|
||||
"adapter_filename": "vsa-datafree/adapter_model.safetensors",
|
||||
"attention_backend": "VIDEO_SPARSE_ATTN_H3",
|
||||
"height": 768,
|
||||
"width": 1344,
|
||||
"num_frames": 124,
|
||||
"num_inference_steps": 5,
|
||||
"seed": 1000,
|
||||
}
|
||||
|
||||
|
||||
def test_config_uses_fasth3_sequence_parallel_default(monkeypatch):
|
||||
"""Selecting FastH3 defaults DreamVerse to its four-GPU topology."""
|
||||
_set_required_prompt_keys(monkeypatch)
|
||||
monkeypatch.setenv("DREAMVERSE_MODEL_ID", "fast-h3")
|
||||
monkeypatch.delenv("DREAMVERSE_SP_SIZE", raising=False)
|
||||
|
||||
module = _load_config_module()
|
||||
|
||||
assert module.ACTIVE_MODEL_ID == "fast-h3"
|
||||
assert module.MODEL_CONFIG["generation_backend"] == "minimax_h3"
|
||||
assert module.DREAMVERSE_SP_SIZE == 4
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
from dreamverse.generation_worker import _create_generation_backend
|
||||
from dreamverse.ltx2_generation import LTX2GenerationBackend
|
||||
|
||||
|
||||
def test_create_generation_backend_ltx2_module_import():
|
||||
backend = _create_generation_backend("ltx2", gpu_id=3)
|
||||
|
||||
assert isinstance(backend, LTX2GenerationBackend)
|
||||
assert backend.gpu_id == 3
|
||||
@@ -63,6 +63,14 @@ def test_get_available_gpus_defaults_to_first_visible_device(monkeypatch):
|
||||
assert gpu_pool.get_available_gpus() == [3]
|
||||
|
||||
|
||||
def test_get_available_gpus_defaults_to_active_model_sequence_parallel_size(monkeypatch):
|
||||
monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "0,1,2,3,4")
|
||||
monkeypatch.delenv("FASTVIDEO_GPU_COUNT", raising=False)
|
||||
monkeypatch.setattr(gpu_pool, "DREAMVERSE_SP_SIZE", 4)
|
||||
|
||||
assert gpu_pool.get_available_gpus() == [0, 1, 2, 3]
|
||||
|
||||
|
||||
def test_get_available_gpus_rejects_invalid_gpu_count(monkeypatch):
|
||||
monkeypatch.delenv("CUDA_VISIBLE_DEVICES", raising=False)
|
||||
monkeypatch.setenv("FASTVIDEO_GPU_COUNT", "zero")
|
||||
@@ -71,6 +79,23 @@ def test_get_available_gpus_rejects_invalid_gpu_count(monkeypatch):
|
||||
gpu_pool.get_available_gpus()
|
||||
|
||||
|
||||
def test_join_user_failed_reload_marks_model_uninitialized(monkeypatch):
|
||||
"""A failed model reload forces the next join to reload a model."""
|
||||
slot = gpu_pool.GPUSlot(gpu_id=0, cuda_device="0")
|
||||
slot.current_model_id = "fast-ltx2"
|
||||
|
||||
async def fake_send_command(command, timeout):
|
||||
del command, timeout
|
||||
return gpu_pool.WorkerError(user_id="__reload__", message="load failed")
|
||||
|
||||
monkeypatch.setattr(slot, "_send_command", fake_send_command)
|
||||
|
||||
with pytest.raises(RuntimeError, match="Model reload failed"):
|
||||
asyncio.run(slot.join_user("client-id", model_id="fast-h3"))
|
||||
|
||||
assert slot.current_model_id is None
|
||||
|
||||
|
||||
def test_send_command_raises_on_worker_death():
|
||||
"""A worker that consumes a command and exits without replying must
|
||||
surface as RuntimeError via sentinel detection, not after the long
|
||||
@@ -92,9 +117,9 @@ def test_send_command_raises_on_worker_death():
|
||||
ready = resp_q.get(timeout=30.0)
|
||||
assert ready == "READY"
|
||||
|
||||
async def runner():
|
||||
async def runner() -> None:
|
||||
slot = gpu_pool.GPUSlot(gpu_id=0, cuda_device="0")
|
||||
slot.process = proc
|
||||
slot.process = proc # type: ignore[assignment]
|
||||
slot.command_queue = cmd_q
|
||||
slot.response_queue = resp_q
|
||||
|
||||
|
||||
@@ -17,11 +17,11 @@ FORBIDDEN_PREFIXES = (
|
||||
)
|
||||
ALLOWED_INTERNAL_IMPORTS = {
|
||||
(
|
||||
"video_generation.py",
|
||||
"ltx2_generation.py",
|
||||
"fastvideo.models.audio.ltx2_audio_processing",
|
||||
),
|
||||
(
|
||||
"video_generation.py",
|
||||
"ltx2_generation.py",
|
||||
"fastvideo.models.loader.component_loader",
|
||||
),
|
||||
}
|
||||
|
||||
@@ -0,0 +1,247 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
import dreamverse.generation_worker as generation_worker
|
||||
from dreamverse.minimax_h3_generation import MiniMaxH3GenerationBackend
|
||||
|
||||
|
||||
FASTH3_MODEL_CONFIG = {
|
||||
"name": "FastH3",
|
||||
"generation_backend": "minimax_h3",
|
||||
"default_sp_size": 4,
|
||||
"model_path": "MiniMaxAI/MiniMax-H3",
|
||||
"adapter_repo": "FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA",
|
||||
"adapter_filename": "vsa-datafree/adapter_model.safetensors",
|
||||
"attention_backend": "VIDEO_SPARSE_ATTN_H3",
|
||||
"height": 768,
|
||||
"width": 1344,
|
||||
"num_frames": 124,
|
||||
"num_inference_steps": 5,
|
||||
"seed": 1000,
|
||||
}
|
||||
|
||||
|
||||
class _RecordingGenerator:
|
||||
"""Record typed requests and return small synchronized media fixtures."""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.requests: list[Any] = []
|
||||
self.conditioning_pixels: list[np.ndarray | None] = []
|
||||
|
||||
def generate(self, request):
|
||||
"""Capture the request and return two tiny video frames with audio."""
|
||||
self.requests.append(request)
|
||||
conditioning_image = request.inputs.pil_image
|
||||
self.conditioning_pixels.append(
|
||||
None if conditioning_image is None else np.asarray(conditioning_image).copy())
|
||||
frames = [
|
||||
np.full((2, 3, 3), 10, dtype=np.uint8),
|
||||
np.full((2, 3, 3), 20, dtype=np.uint8),
|
||||
]
|
||||
return SimpleNamespace(
|
||||
frames=frames,
|
||||
audio=np.zeros((2, 16), dtype=np.float32),
|
||||
audio_sample_rate=44100,
|
||||
generation_time=0.25,
|
||||
)
|
||||
|
||||
|
||||
def test_initialize_builds_vsa_datafree_fasth3_generator(monkeypatch):
|
||||
"""Initialization translates the DreamVerse profile into typed FastVideo config."""
|
||||
from fastvideo import VideoGenerator
|
||||
|
||||
captured = {}
|
||||
fake_generator = SimpleNamespace(shutdown=lambda: None)
|
||||
|
||||
def fake_from_config(config):
|
||||
captured["config"] = config
|
||||
return fake_generator
|
||||
|
||||
def fake_download(**kwargs):
|
||||
captured["download"] = kwargs
|
||||
return f"/models/{kwargs['filename']}"
|
||||
|
||||
monkeypatch.setattr("huggingface_hub.hf_hub_download", fake_download)
|
||||
monkeypatch.setattr(VideoGenerator, "from_config", fake_from_config)
|
||||
monkeypatch.setattr("dreamverse.minimax_h3_generation.DREAMVERSE_SP_SIZE", 4)
|
||||
monkeypatch.setenv("FASTVIDEO_ATTENTION_BACKEND", "test-attention")
|
||||
monkeypatch.setenv("FASTVIDEO_FA4", "0")
|
||||
monkeypatch.setenv("FASTVIDEO_MINIMAX_H3_FUSIONS", "0")
|
||||
monkeypatch.setenv("FASTVIDEO_VSA_SM100A", "1")
|
||||
monkeypatch.setenv("FASTVIDEO_INFERENCE_TORCH_COMPILE", "1")
|
||||
|
||||
backend = MiniMaxH3GenerationBackend(gpu_id=0)
|
||||
monkeypatch.setattr(backend, "_gpu_mem", lambda: "alloc=0.00GiB, reserved=0.00GiB")
|
||||
backend.initialize(FASTH3_MODEL_CONFIG)
|
||||
|
||||
config = captured["config"]
|
||||
assert captured["download"] == {
|
||||
"repo_id": "FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA",
|
||||
"filename": "vsa-datafree/adapter_model.safetensors",
|
||||
}
|
||||
assert config.model_path == "MiniMaxAI/MiniMax-H3"
|
||||
assert config.pipeline.components.lora_path.endswith("vsa-datafree/adapter_model.safetensors")
|
||||
assert config.pipeline.components.lora_strength == 1.0
|
||||
assert config.pipeline.experimental == {
|
||||
"attention_backend": "VIDEO_SPARSE_ATTN_H3",
|
||||
"inference_torch_compile": False,
|
||||
"vae_parallel_decode": True,
|
||||
"vae_parallel_decode_strategy": "gather",
|
||||
"VSA_sparsity": 0.9,
|
||||
"VSA_tile_size": 64,
|
||||
}
|
||||
assert config.engine.num_gpus == 4
|
||||
assert config.engine.parallelism.tp_size == 1
|
||||
assert config.engine.parallelism.sp_size == 4
|
||||
assert config.engine.offload.dit is False
|
||||
assert config.engine.offload.dit_layerwise is False
|
||||
assert config.engine.offload.text_encoder is True
|
||||
assert config.engine.offload.vae is True
|
||||
assert config.engine.compile.vae_enabled is True
|
||||
assert config.engine.use_fsdp_inference is False
|
||||
assert os.environ["FASTVIDEO_ATTENTION_BACKEND"] == "VIDEO_SPARSE_ATTN_H3"
|
||||
assert os.environ["FASTVIDEO_FA4"] == "1"
|
||||
assert os.environ["FASTVIDEO_MINIMAX_H3_FUSIONS"] == "all"
|
||||
assert os.environ["FASTVIDEO_VSA_SM100A"] == "0"
|
||||
assert "FASTVIDEO_INFERENCE_TORCH_COMPILE" not in os.environ
|
||||
|
||||
|
||||
def test_initialize_selects_declared_generation_backend(monkeypatch):
|
||||
"""The GPU worker constructs the backend that the active model profile declares."""
|
||||
from unittest.mock import Mock
|
||||
|
||||
selected_backend = Mock()
|
||||
monkeypatch.setattr(
|
||||
generation_worker,
|
||||
"_create_generation_backend",
|
||||
lambda backend_name, gpu_id: selected_backend,
|
||||
)
|
||||
worker = generation_worker.VideoGenerationWorker(gpu_id=3)
|
||||
|
||||
worker.initialize(FASTH3_MODEL_CONFIG)
|
||||
|
||||
assert worker.backend_name == "minimax_h3"
|
||||
assert worker.backend is selected_backend
|
||||
selected_backend.initialize.assert_called_once_with(FASTH3_MODEL_CONFIG)
|
||||
|
||||
|
||||
def test_initialize_failure_clears_backend_ownership(monkeypatch):
|
||||
"""A failed family change leaves the GPU worker explicitly uninitialized."""
|
||||
ltx_backend = SimpleNamespace(initialize=lambda config: None, shutdown=lambda: None)
|
||||
|
||||
def fail_initialize(config):
|
||||
del config
|
||||
raise RuntimeError("load failed")
|
||||
|
||||
fasth3_backend = SimpleNamespace(
|
||||
initialize=fail_initialize,
|
||||
shutdown=lambda: None,
|
||||
)
|
||||
backends = {
|
||||
"ltx2": ltx_backend,
|
||||
"minimax_h3": fasth3_backend,
|
||||
}
|
||||
monkeypatch.setattr(
|
||||
generation_worker,
|
||||
"_create_generation_backend",
|
||||
lambda backend_name, gpu_id: backends[backend_name],
|
||||
)
|
||||
worker = generation_worker.VideoGenerationWorker(gpu_id=3)
|
||||
worker.initialize({"generation_backend": "ltx2"})
|
||||
|
||||
with pytest.raises(RuntimeError, match="load failed"):
|
||||
worker.initialize(FASTH3_MODEL_CONFIG)
|
||||
|
||||
assert worker.backend is None
|
||||
assert worker.backend_name is None
|
||||
assert worker.model_config == {"generation_backend": "ltx2"}
|
||||
|
||||
|
||||
def test_generate_step_uses_last_frame_for_continuation(monkeypatch):
|
||||
"""A later segment receives the prior segment's last decoded frame."""
|
||||
backend = MiniMaxH3GenerationBackend(gpu_id=0)
|
||||
backend.model_config = dict(FASTH3_MODEL_CONFIG)
|
||||
backend.generator = _RecordingGenerator()
|
||||
monkeypatch.setattr("dreamverse.minimax_h3_generation.torch.cuda.synchronize", lambda: None)
|
||||
|
||||
first_result = backend.generate_step(
|
||||
"first prompt",
|
||||
segment_idx=1,
|
||||
image_path=None,
|
||||
reset_conditioning=True,
|
||||
)
|
||||
second_result = backend.generate_step(
|
||||
"second prompt",
|
||||
segment_idx=2,
|
||||
image_path=None,
|
||||
reset_conditioning=False,
|
||||
)
|
||||
|
||||
first_request = backend.generator.requests[0]
|
||||
assert first_request.inputs.pil_image is None
|
||||
assert first_request.negative_prompt == ""
|
||||
assert first_request.sampling.height == 768
|
||||
assert first_request.sampling.width == 1344
|
||||
assert first_request.sampling.num_frames == 124
|
||||
assert first_request.sampling.num_inference_steps == 5
|
||||
assert first_request.sampling.fps == 24
|
||||
assert first_request.sampling.guidance_scale == 1.0
|
||||
assert first_request.sampling.batch_cfg is False
|
||||
assert first_request.sampling.seed == 1000
|
||||
assert first_request.output.save_video is False
|
||||
assert first_request.output.return_frames is True
|
||||
assert backend.generator.conditioning_pixels[1].tolist() == np.full((2, 3, 3), 20).tolist()
|
||||
assert first_result.head_trim_frames == 0
|
||||
assert first_result.head_trim_audio_frames == 0
|
||||
assert second_result.head_trim_frames == 1
|
||||
assert second_result.head_trim_audio_frames == 1
|
||||
assert second_result.audio_sample_rate == 44100
|
||||
|
||||
|
||||
def test_generate_step_reset_uses_text_to_video_path(monkeypatch):
|
||||
"""Resetting continuation produces an unconditioned text-to-video request."""
|
||||
backend = MiniMaxH3GenerationBackend(gpu_id=0)
|
||||
backend.model_config = dict(FASTH3_MODEL_CONFIG)
|
||||
backend.generator = _RecordingGenerator()
|
||||
monkeypatch.setattr("dreamverse.minimax_h3_generation.torch.cuda.synchronize", lambda: None)
|
||||
|
||||
backend.generate_step("first prompt", 1, None, True)
|
||||
reset_result = backend.generate_step("reset prompt", 2, None, True)
|
||||
|
||||
assert backend.generator.requests[-1].inputs.pil_image is None
|
||||
assert reset_result.head_trim_frames == 0
|
||||
assert reset_result.head_trim_audio_frames == 0
|
||||
|
||||
|
||||
def test_generate_step_missing_continuation_frame(monkeypatch):
|
||||
"""A later segment fails when no reset or retained frame defines its input."""
|
||||
backend = MiniMaxH3GenerationBackend(gpu_id=0)
|
||||
backend.model_config = dict(FASTH3_MODEL_CONFIG)
|
||||
backend.generator = _RecordingGenerator()
|
||||
|
||||
with pytest.raises(RuntimeError, match="requires a retained continuation frame"):
|
||||
backend.generate_step("later prompt", 2, None, False)
|
||||
|
||||
assert backend.generator.requests == []
|
||||
|
||||
|
||||
def test_warmup_exercises_text_and_first_frame_paths(monkeypatch):
|
||||
"""Warmup covers both request shapes used by a DreamVerse session."""
|
||||
backend = MiniMaxH3GenerationBackend(gpu_id=0)
|
||||
backend.model_config = dict(FASTH3_MODEL_CONFIG)
|
||||
backend.generator = _RecordingGenerator()
|
||||
monkeypatch.setattr("dreamverse.minimax_h3_generation.torch.cuda.synchronize", lambda: None)
|
||||
|
||||
timings = backend.warmup("warmup prompt")
|
||||
|
||||
assert backend.generator.conditioning_pixels[0] is None
|
||||
assert backend.generator.conditioning_pixels[1] is not None
|
||||
assert backend.continuation_image is None
|
||||
assert "warmup_text_to_video_ms" in timings
|
||||
assert "warmup_first_frame_to_video_ms" in timings
|
||||
@@ -70,7 +70,7 @@
|
||||
<mxCell id="dispatcher" value="command dispatcher

gpu_worker_process() branches on
CommandType; asserts payload type

INIT / WARMUP / RELOAD_MODEL
USER_JOIN / USER_STEP / USER_LEAVE
SHUTDOWN" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;fontSize=11;align=left;spacingLeft=10;spacingTop=8;fontStyle=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="120" y="1120" width="240" height="120" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="do_step" value="VideoGenerationWorker.generate_step()
video_generation.py:380

reads + updates ContinuationState,
calls generator" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=11;align=left;spacingLeft=10;spacingTop=8;fontStyle=1;" parent="1" vertex="1">
|
||||
<mxCell id="do_step" value="VideoGenerationWorker.generate_step()
ltx2_generation.py:380

reads + updates ContinuationState,
calls generator" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=11;align=left;spacingLeft=10;spacingTop=8;fontStyle=1;" parent="1" vertex="1">
|
||||
<mxGeometry x="460" y="1120" width="240" height="120" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="stream_av" value="stream_fmp4()
av_streaming.py:121

trims overlap, pipes to ffmpeg,
publishes StreamInit / StreamChunk /
StreamComplete via callback" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#b1d8d7;strokeColor=#23445d;fontSize=11;align=left;spacingLeft=10;spacingTop=8;fontStyle=1;" parent="1" vertex="1">
|
||||
@@ -79,13 +79,13 @@
|
||||
<mxCell id="Ot8BU52QTIb4EhyRSe7I-2" value="" style="edgeStyle=none;html=1;" parent="1" source="generator" target="Ot8BU52QTIb4EhyRSe7I-1" edge="1">
|
||||
<mxGeometry relative="1" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="generator" value="VideoGenerator (fastvideo)

LTX2 DiT + refine upsampler
FP4 quant, torch.compile

owned by VideoGenerationWorker
video_generation.py:211" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=11;" parent="1" vertex="1">
|
||||
<mxCell id="generator" value="VideoGenerator (fastvideo)

LTX2 DiT + refine upsampler
FP4 quant, torch.compile

owned by VideoGenerationWorker
ltx2_generation.py:211" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=11;" parent="1" vertex="1">
|
||||
<mxGeometry x="460" y="1300" width="240" height="100" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="ffmpeg" value="ffmpeg subprocess

libx264 / *_nvenc
fragmented mp4" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#d79b00;fontSize=11;" parent="1" vertex="1">
|
||||
<mxGeometry x="800" y="1300" width="260" height="100" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="caches" value="ContinuationState
video_generation.py:89

• video_images: list[PIL.Image]
• audio_latents: torch.Tensor (CPU)

carried across segments" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=11;align=left;spacingLeft=10;spacingTop=8;" parent="1" vertex="1">
|
||||
<mxCell id="caches" value="ContinuationState
ltx2_generation.py:89

• video_images: list[PIL.Image]
• audio_latents: torch.Tensor (CPU)

carried across segments" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e1d5e7;strokeColor=#9673a6;fontSize=11;align=left;spacingLeft=10;spacingTop=8;" parent="1" vertex="1">
|
||||
<mxGeometry x="120" y="1300" width="240" height="100" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="e_cp" value="acquire" style="edgeStyle=orthogonalEdgeStyle;rounded=0;html=1;strokeColor=#6c8ebf;endArrow=classic;fontSize=11;exitX=0.5;exitY=1;exitDx=0;exitDy=0;entryX=0.5;entryY=0;entryDx=0;entryDy=0;" parent="1" source="client" target="pool" edge="1">
|
||||
@@ -250,7 +250,7 @@
|
||||
<mxPoint x="690" y="880"/>
|
||||
</Array>
|
||||
</mxCell>
|
||||
<mxCell id="legend" value="Legend

■ blue client / external
■ green main-process pool/slot
 (methods — italic label)
■ yellow containers (routing state)
■ red IPC primitives (mp.Queue, mp.RawArray)

Worker subprocess modules:
■ orange gpu_pool.py (dispatcher)
■ lavender video_generation.py
■ teal av_streaming.py
■ gray worker_ipc.py (shared types)

Flow:
 client → pool → slot
 → _send_command(_tagged) → command_queue
 → dispatcher → generate_step()
 → stream_fmp4() → ffmpeg
 → shared_buf + response_queue
 → _response_reader → futures / stream_queues
 → client awaits (via main.py AV loop)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#999999;fontSize=11;align=left;spacingLeft=10;spacingTop=8;" parent="1" vertex="1">
|
||||
<mxCell id="legend" value="Legend

■ blue client / external
■ green main-process pool/slot
 (methods — italic label)
■ yellow containers (routing state)
■ red IPC primitives (mp.Queue, mp.RawArray)

Worker subprocess modules:
■ orange gpu_pool.py (dispatcher)
■ lavender ltx2_generation.py
■ teal av_streaming.py
■ gray worker_ipc.py (shared types)

Flow:
 client → pool → slot
 → _send_command(_tagged) → command_queue
 → dispatcher → generate_step()
 → stream_fmp4() → ffmpeg
 → shared_buf + response_queue
 → _response_reader → futures / stream_queues
 → client awaits (via main.py AV loop)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#f5f5f5;strokeColor=#999999;fontSize=11;align=left;spacingLeft=10;spacingTop=8;" parent="1" vertex="1">
|
||||
<mxGeometry x="39" y="-200" width="270" height="380" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="Ot8BU52QTIb4EhyRSe7I-1" value="FastVideo video_generator" style="whiteSpace=wrap;html=1;fontSize=11;fillColor=#e1d5e7;strokeColor=#9673a6;rounded=1;" parent="1" vertex="1">
|
||||
@@ -389,10 +389,10 @@
|
||||
<mxCell id="cw2" value="from fastvideo.entrypoints.video_generator import VideoGenerator
from fastvideo.models.dits.ltx2 import DEFAULT_LTX2_AUDIO_*

** Dreamverse reaches into fastvideo internals here **" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe0b2;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
|
||||
<mxGeometry x="675" y="695" width="550" height="60" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="cw3" value="on Command(INIT):
 VideoGenerationWorker.initialize() (video_generation.py:247)
 maybe_download_model(model_id)
 VideoGenerator.from_pretrained(path, FP4Config, PipelineConfig)
 load audio VAE, resolve refine upsampler
 resp_q.put(InitAck(success=True))" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
|
||||
<mxCell id="cw3" value="on Command(INIT):
 VideoGenerationWorker.initialize() (ltx2_generation.py:247)
 maybe_download_model(model_id)
 VideoGenerator.from_pretrained(path, FP4Config, PipelineConfig)
 load audio VAE, resolve refine upsampler
 resp_q.put(InitAck(success=True))" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
|
||||
<mxGeometry x="675" y="765" width="550" height="95" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="cw4" value="on Command(WARMUP) with WarmupPayload:
 VideoGenerationWorker.warmup(payload.prompt) (video_generation.py:518)
 two synthetic segments prime caches + torch.compile
 resp_q.put(WarmupComplete(timings=...))" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
|
||||
<mxCell id="cw4" value="on Command(WARMUP) with WarmupPayload:
 VideoGenerationWorker.warmup(payload.prompt) (ltx2_generation.py:518)
 two synthetic segments prime caches + torch.compile
 resp_q.put(WarmupComplete(timings=...))" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffffff;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
|
||||
<mxGeometry x="675" y="870" width="550" height="55" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="cw5" value="enter main worker loop → waits for JOIN_USER / USER_STEP / LEAVE" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#c8e6c9;strokeColor=#388e3c;fontSize=11;fontStyle=1;fontFamily=monospace;" parent="1" vertex="1">
|
||||
@@ -534,7 +534,7 @@
|
||||
<mxPoint x="1040" y="1610" as="targetPoint"/>
|
||||
</mxGeometry>
|
||||
</mxCell>
|
||||
<mxCell id="dm11a" value="10a. worker runs:
VideoGenerationWorker.generate_step()
 (video_generation.py:380)
 → generator.generate_video()
 → updates ContinuationState
then stream_fmp4() (av_streaming.py:121)
 → ffmpeg (rawvideo+wav → fmp4)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe0b2;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
|
||||
<mxCell id="dm11a" value="10a. worker runs:
VideoGenerationWorker.generate_step()
 (ltx2_generation.py:380)
 → generator.generate_video()
 → updates ContinuationState
then stream_fmp4() (av_streaming.py:121)
 → ffmpeg (rawvideo+wav → fmp4)" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#ffe0b2;strokeColor=#d79b00;fontSize=10;align=left;spacingLeft=8;fontFamily=monospace;" parent="1" vertex="1">
|
||||
<mxGeometry x="955" y="1640" width="180" height="70" as="geometry"/>
|
||||
</mxCell>
|
||||
<mxCell id="dm11" value="10b. resp_q.put(MediaInit / MediaChunk / MediaComplete / StepComplete)" style="endArrow=classic;html=1;strokeColor=#b85450;fontSize=10;labelBackgroundColor=#ffffff;" parent="1" edge="1">
|
||||
|
||||
File diff suppressed because one or more lines are too long
|
Before Width: | Height: | Size: 85 KiB After Width: | Height: | Size: 85 KiB |
@@ -16,6 +16,9 @@ The UI currently supports:
|
||||
```
|
||||
apps/fastvideo_studio/
|
||||
├── server.py / job_runner.py / database.py # FastAPI backend + job lifecycle
|
||||
├── job_queue.py # Queue scheduling rules + last-frame references (GPU-free)
|
||||
├── merge.py # Join a scene's clips into one video (ffmpeg stream copy, GPU-free)
|
||||
├── trim.py # Cut a clip to an in/out range, non-destructively (GPU-free)
|
||||
├── mock_server.py # In-memory API mock for e2e tests
|
||||
├── models/ # Pydantic request models (shared with the mock)
|
||||
├── training_config.py # Studio workloads → fastvideo/train YAML configs
|
||||
@@ -88,6 +91,93 @@ Running it this way allows you to pass command line parameters.
|
||||
The API server defaults to port 8189. To configure this, you can edit `.env.local` file.
|
||||
Refer to `.env.example` for reference.
|
||||
|
||||
### Job queue
|
||||
|
||||
`POST /api/jobs/queue` puts jobs in a queue; the server runs them oldest-first,
|
||||
`--max-concurrent-jobs` at a time (default 1). Start on a job while that many
|
||||
are already running is refused; queue it instead. Queued jobs survive a server
|
||||
restart and carry on.
|
||||
|
||||
Inference jobs run on that many long-lived worker threads. The model's GPU
|
||||
processes are killed when the thread that started them exits, so a worker that
|
||||
never exits is what keeps the model loaded from one job to the next (it loads
|
||||
once, not per job). A worker holds one model at a time: a job with a different
|
||||
model or settings replaces it, and a failed or stopped job unloads it so the next
|
||||
one starts clean. Each extra worker loads its own copy of the model.
|
||||
|
||||
A job can start from how another job's video ended, two ways:
|
||||
|
||||
* **Last frame** -- a reference whose source is `job-last-frame:<job id>`. A
|
||||
hard, exact opening frame: the prompt is rewritten to say the shot begins
|
||||
from it.
|
||||
* **Last clip** -- a reference whose source is `job-last-clip:<job id>`, and
|
||||
`media_type: "video"`. The trailing ~1s of the source job's video instead of
|
||||
a single still, so the model has real motion to continue rather than a pose
|
||||
to guess it from. Costs one of H3's 3 video-reference slots instead of one
|
||||
of its 9 image slots.
|
||||
|
||||
Either is extracted when the job runs, so the reference can be added before
|
||||
that job has finished, and the queue holds the job until it has. If that job
|
||||
fails, is stopped, or is deleted, the waiting job fails with a message saying
|
||||
so (and so does anything waiting on *it*); fix the earlier job and queue them
|
||||
again. Starting such a job directly while its source is unfinished is refused
|
||||
with a 409.
|
||||
|
||||
### Merging a scene
|
||||
|
||||
**Merge scene** on the Scenes page joins a scene's clips, in the order shown (the
|
||||
take chosen for each), into one video under `<output-dir>/merged/`, then plays it
|
||||
and offers a download. It is enabled once every clip has finished, and refuses
|
||||
otherwise, so a scene missing a clip can't pass for the whole thing.
|
||||
|
||||
The clips are copied, not re-encoded (about a second for 15 clips, no quality
|
||||
loss). That needs them to share size, frame rate and audio, which clips from one
|
||||
scene do; if one differs, the merge says which. Each clip is cut at the end of its
|
||||
last video frame, because H3's audio runs a few milliseconds longer and would
|
||||
otherwise leave a small gap at every join. It uses `ffmpeg` from `PATH`,
|
||||
`$FASTVIDEO_FFMPEG_BIN`, or the copy bundled with `imageio-ffmpeg`.
|
||||
|
||||
### Editing a clip's video
|
||||
|
||||
**Edit video** on a finished clip in Scenes cuts it to a chosen in/out range and
|
||||
adjusts its brightness, contrast and saturation, in place -- other jobs'
|
||||
last-frame/last-clip references, and a later merge, all just read the job's
|
||||
`output_path`, so nothing else needs to know a clip was edited. Frame-accurate
|
||||
(re-encoded, not stream-copied), so the cut isn't limited to the source's
|
||||
keyframes. Audio is cut to the same range and carried over too (re-encoded to
|
||||
AAC regardless of the source's codec), unless the source has none.
|
||||
|
||||
Color grading is per **section**, not the whole clip at once, like a razor/blade
|
||||
tool: **Split at current time** cuts the kept range wherever the preview is
|
||||
paused, and each resulting section gets its own brightness, contrast and
|
||||
saturation sliders, next to a before/after preview of a still frame
|
||||
(**Preview** on a section seeks to its start and captures that frame; **Use
|
||||
current frame** captures wherever the preview is paused instead). That preview
|
||||
is computed entirely in the browser (`src/lib/grading.ts` mirrors `trim.py`'s
|
||||
`_apply_grade` pixel math) as the sliders move, so it never waits on the
|
||||
server -- only **Apply** does that, to actually re-render the file. **Previous
|
||||
frame**/**Next frame** step the preview by exactly one frame (at the clip's
|
||||
own fps), for lining up a cut precisely -- the native scrub bar alone isn't
|
||||
frame-accurate. **Merge with next** undoes a cut, keeping the earlier of the
|
||||
two sections' grade.
|
||||
Sections are always contiguous and cover the whole kept range -- there's no
|
||||
reordering or gaps, only where to cut and what to grade each side; a plain
|
||||
range trim with no cuts is just the one-section case. Audio is one continuous
|
||||
track regardless of how many sections there are.
|
||||
|
||||
The untouched original is always kept alongside it the first time a clip is
|
||||
edited, and **every edit always re-renders from that original** with the full
|
||||
set of values shown in the dialog -- range and sections together, in one pass.
|
||||
That's what lets adjusting either one leave the other in place instead of each
|
||||
one silently discarding the other. The job itself records the range/sections
|
||||
of its last edit (`edit_start_seconds`/`edit_end_seconds`/`edit_segments`,
|
||||
alongside `output_path`), so reopening the dialog -- even after closing it or
|
||||
reloading the page -- shows exactly what produced the current video, not a
|
||||
guess: a never-edited clip shows one neutral section covering its full length;
|
||||
an edited one shows every section that made it, cuts and start offset
|
||||
included. **Restore original** undoes every edit in one step and resets those
|
||||
recorded values back to one neutral section too.
|
||||
|
||||
### API Endpoints
|
||||
|
||||
| Method | Path | Description |
|
||||
@@ -98,6 +188,15 @@ Refer to `.env.example` for reference.
|
||||
| `POST` | `/api/jobs` | Create a new job |
|
||||
| `POST` | `/api/jobs/{id}/start` | Start a pending/stopped/failed job |
|
||||
| `POST` | `/api/jobs/{id}/stop` | Request a running job to stop |
|
||||
| `POST` | `/api/jobs/queue` | Queue jobs (`{"job_ids": [...]}`), in order |
|
||||
| `POST` | `/api/jobs/{id}/queue` | Queue one job |
|
||||
| `POST` | `/api/jobs/{id}/dequeue` | Take a queued job back out of the queue |
|
||||
| `POST` | `/api/jobs/{id}/last-frame` | Save the job's last frame, for another job to start from |
|
||||
| `POST` | `/api/jobs/{id}/last-clip` | Save the job's trailing ~1s, for another job to continue from |
|
||||
| `POST` | `/api/scenes/merge` | Join finished jobs' videos (`{"job_ids": [...], "name": ""}`) in order |
|
||||
| `POST` | `/api/jobs/{id}/trim` | Cut a job's video to a range and/or grade it section by section (`{"start_seconds": 0, "end_seconds": null, "segments": [{"end_seconds": null, "brightness": 0, "contrast": 1, "saturation": 1}]}`), in place |
|
||||
| `POST` | `/api/jobs/{id}/restore-video`| Undo every edit, back to the original video |
|
||||
| `GET` | `/api/merged/{filename}` | Stream a merged scene video |
|
||||
| `DELETE` | `/api/jobs/{id}` | Delete a job |
|
||||
| `GET` | `/api/jobs/{id}/video` | Stream the generated video/image |
|
||||
| `GET` | `/api/jobs/{id}/logs` | Get job logs (polling, supports `?after=`) |
|
||||
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import logging
|
||||
import json
|
||||
import sqlite3
|
||||
import threading
|
||||
from pathlib import Path
|
||||
@@ -46,8 +47,13 @@ DEFAULT_SETTINGS: dict[str, Any] = {
|
||||
|
||||
|
||||
def _sqlite_row_get(row: sqlite3.Row, key: str, default: Any) -> Any:
|
||||
"""Like dict.get for sqlite3.Row (Row has no .get on Python 3.10)."""
|
||||
return row[key] if key in row else default # noqa: SIM401
|
||||
"""Like dict.get for sqlite3.Row (Row has no .get on Python 3.10).
|
||||
|
||||
NOTE: `key in row` tests Row *values*, not column names, so the membership
|
||||
check has to go through .keys() -- otherwise every lookup falls back to the
|
||||
default and jobs restored from the database lose their stored fields.
|
||||
"""
|
||||
return row[key] if key in row.keys() else default # noqa: SIM401, SIM118
|
||||
|
||||
|
||||
def _get_db_path(data_dir: Path) -> Path:
|
||||
@@ -83,6 +89,9 @@ def _migrate_db(conn: sqlite3.Connection) -> None:
|
||||
_add_column_if_missing(conn, "jobs", "fps", "INTEGER", "24")
|
||||
_add_column_if_missing(conn, "jobs", "workload_type", "TEXT", "'t2v'")
|
||||
_add_column_if_missing(conn, "jobs", "image_path", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "name", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "last_image_path", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "references_json", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "job_type", "TEXT", "'inference'")
|
||||
_add_column_if_missing(conn, "jobs", "data_path", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "max_train_steps", "INTEGER", "1000")
|
||||
@@ -101,6 +110,15 @@ def _migrate_db(conn: sqlite3.Connection) -> None:
|
||||
_add_column_if_missing(conn, "jobs", "generator_update_interval", "INTEGER", "5")
|
||||
_add_column_if_missing(conn, "jobs", "real_score_model_path", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "fake_score_model_path", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "queued_at", "REAL", "NULL")
|
||||
_add_column_if_missing(conn, "jobs", "edit_start_seconds", "REAL", "0.0")
|
||||
_add_column_if_missing(conn, "jobs", "edit_end_seconds", "REAL", "NULL")
|
||||
# Superseded by edit_segments_json (a list of razor-cut sections, each with
|
||||
# its own grade); left in place, unused, rather than risking a DROP COLUMN.
|
||||
_add_column_if_missing(conn, "jobs", "edit_brightness", "REAL", "0.0")
|
||||
_add_column_if_missing(conn, "jobs", "edit_contrast", "REAL", "1.0")
|
||||
_add_column_if_missing(conn, "jobs", "edit_saturation", "REAL", "1.0")
|
||||
_add_column_if_missing(conn, "jobs", "edit_segments_json", "TEXT", "'[]'")
|
||||
# Settings table
|
||||
_add_column_if_missing(conn, "settings", "vae_cpu_offload", "INTEGER", "0")
|
||||
_add_column_if_missing(conn, "settings", "image_encoder_cpu_offload", "INTEGER", "0")
|
||||
@@ -242,7 +260,8 @@ class Database:
|
||||
self._execute(
|
||||
"""
|
||||
INSERT INTO jobs (
|
||||
id, model_id, prompt, workload_type, image_path, job_type, status,
|
||||
id, model_id, name, prompt, workload_type, image_path,
|
||||
last_image_path, references_json, job_type, status,
|
||||
created_at, started_at, finished_at, error, output_path, log_file_path,
|
||||
num_inference_steps, num_frames, height, width, guidance_scale,
|
||||
guidance_rescale, fps, seed, num_gpus, dit_cpu_offload,
|
||||
@@ -254,14 +273,17 @@ class Database:
|
||||
dmd_use_vsa, dmd_vsa_sparsity, dmd_denoising_steps,
|
||||
real_score_guidance_scale,
|
||||
generator_update_interval, real_score_model_path, fake_score_model_path
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
job["id"],
|
||||
job["model_id"],
|
||||
job.get("name", ""),
|
||||
job["prompt"],
|
||||
job.get("workload_type", "t2v"),
|
||||
job.get("image_path", ""),
|
||||
job.get("last_image_path", ""),
|
||||
json.dumps(job.get("references") or []),
|
||||
job.get("job_type", "inference"),
|
||||
job["status"],
|
||||
job["created_at"],
|
||||
@@ -313,6 +335,7 @@ class Database:
|
||||
return
|
||||
allowed = {
|
||||
"status",
|
||||
"queued_at",
|
||||
"started_at",
|
||||
"finished_at",
|
||||
"error",
|
||||
@@ -332,6 +355,43 @@ class Database:
|
||||
self._execute(sql, tuple(vals))
|
||||
self._commit()
|
||||
|
||||
#: Job fields stored under a different column name, and how they are encoded.
|
||||
_CONFIG_COLUMNS = {"references": "references_json", "edit_segments": "edit_segments_json"}
|
||||
_JSON_LIST_CONFIG_FIELDS = {"references", "edit_segments"}
|
||||
_BOOL_CONFIG_FIELDS = {
|
||||
"dit_cpu_offload",
|
||||
"text_encoder_cpu_offload",
|
||||
"vae_cpu_offload",
|
||||
"image_encoder_cpu_offload",
|
||||
"use_fsdp_inference",
|
||||
"enable_torch_compile",
|
||||
"dmd_use_vsa",
|
||||
}
|
||||
|
||||
def update_job_config(self, job_id: str, updates: dict[str, Any]) -> None:
|
||||
"""Persist edits to a job's configuration (prompt, references, sizes...).
|
||||
|
||||
Fields the table has no column for are skipped, as they are on insert.
|
||||
"""
|
||||
columns = _get_table_columns(self._conn(), "jobs")
|
||||
cols = []
|
||||
vals: list[Any] = []
|
||||
for key, value in updates.items():
|
||||
column = self._CONFIG_COLUMNS.get(key, key)
|
||||
if column not in columns or column == "id":
|
||||
continue
|
||||
if key in self._JSON_LIST_CONFIG_FIELDS:
|
||||
value = json.dumps(value or [])
|
||||
elif key in self._BOOL_CONFIG_FIELDS:
|
||||
value = 1 if value else 0
|
||||
cols.append(f"{column} = ?")
|
||||
vals.append(value)
|
||||
if not cols:
|
||||
return
|
||||
vals.append(job_id)
|
||||
self._execute(f"UPDATE jobs SET {', '.join(cols)} WHERE id = ?", tuple(vals))
|
||||
self._commit()
|
||||
|
||||
def delete_job(self, job_id: str) -> bool:
|
||||
"""Delete a job. Returns True if a row was deleted."""
|
||||
cur = self._execute("DELETE FROM jobs WHERE id = ?", (job_id, ))
|
||||
@@ -524,8 +584,31 @@ def _row_to_dataset(row: sqlite3.Row) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _row_to_job(row: sqlite3.Row) -> dict[str, Any]:
|
||||
class _ColumnRow:
|
||||
"""A sqlite3.Row whose ``in`` tests column names.
|
||||
|
||||
sqlite3.Row's own ``in`` tests *values*, so ``"fps" in row`` is always False
|
||||
and a job restored from the database silently lost fps, negative_prompt and
|
||||
the other optional columns.
|
||||
"""
|
||||
|
||||
def __init__(self, row: sqlite3.Row):
|
||||
self._row = row
|
||||
self._columns = set(row.keys())
|
||||
|
||||
def __getitem__(self, key: str) -> Any:
|
||||
return self._row[key]
|
||||
|
||||
def __contains__(self, key: object) -> bool:
|
||||
return key in self._columns
|
||||
|
||||
def keys(self) -> list[str]:
|
||||
return self._row.keys()
|
||||
|
||||
|
||||
def _row_to_job(sqlite_row: sqlite3.Row) -> dict[str, Any]:
|
||||
"""Convert a DB row to job dict (snake_case, matching Job.to_dict)."""
|
||||
row = _ColumnRow(sqlite_row)
|
||||
int_defaults = {
|
||||
"max_train_steps": 1000,
|
||||
"train_batch_size": 1,
|
||||
@@ -540,12 +623,16 @@ def _row_to_job(row: sqlite3.Row) -> dict[str, Any]:
|
||||
result = {
|
||||
"id": row["id"],
|
||||
"model_id": row["model_id"],
|
||||
"name": _sqlite_row_get(row, "name", "") or "",
|
||||
"prompt": row["prompt"],
|
||||
"workload_type": _sqlite_row_get(row, "workload_type", "t2v"),
|
||||
"image_path": _sqlite_row_get(row, "image_path", "") or "",
|
||||
"last_image_path": _sqlite_row_get(row, "last_image_path", "") or "",
|
||||
"references": _sqlite_row_get(row, "references_json", "") or "",
|
||||
"job_type": _sqlite_row_get(row, "job_type", "inference"),
|
||||
"status": row["status"],
|
||||
"created_at": row["created_at"],
|
||||
"queued_at": row["queued_at"] if "queued_at" in row else None,
|
||||
"started_at": row["started_at"],
|
||||
"finished_at": row["finished_at"],
|
||||
"error": row["error"],
|
||||
@@ -598,6 +685,13 @@ def _row_to_job(row: sqlite3.Row) -> dict[str, Any]:
|
||||
for col in ("dmd_denoising_steps", "real_score_model_path", "fake_score_model_path"):
|
||||
if col in row:
|
||||
result[col] = (row[col] or "") or ""
|
||||
if "edit_start_seconds" in row:
|
||||
start_val: Any = row["edit_start_seconds"]
|
||||
result["edit_start_seconds"] = float(start_val) if start_val is not None else 0.0
|
||||
if "edit_end_seconds" in row:
|
||||
end_val: Any = row["edit_end_seconds"]
|
||||
result["edit_end_seconds"] = float(end_val) if end_val is not None else None
|
||||
result["edit_segments"] = _sqlite_row_get(row, "edit_segments_json", "") or ""
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Save the end of a finished clip, so the next clip can continue from it.
|
||||
|
||||
Two forms: the final frame alone (a still, for a hard "opens on this exact
|
||||
picture" continuation), or a short trailing clip (a few hundred milliseconds of
|
||||
video, attached as an H3 video reference so the model has actual motion to
|
||||
continue rather than guessing it from one still). Kept apart from ``server.py``
|
||||
because it needs nothing from FastVideo -- only PyAV -- and so can be exercised
|
||||
without a GPU.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from fractions import Fraction
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
VIDEO_EXTENSIONS = {".mp4", ".webm", ".avi", ".mov", ".mkv"}
|
||||
|
||||
|
||||
class FrameError(Exception):
|
||||
"""A request that can't be satisfied, carrying the HTTP status to report."""
|
||||
|
||||
def __init__(self, status_code: int, detail: str):
|
||||
super().__init__(detail)
|
||||
self.status_code = status_code
|
||||
self.detail = detail
|
||||
|
||||
|
||||
def save_last_frame(video_path: str, out_path: str) -> None:
|
||||
"""Decode ``video_path`` and write its final frame to ``out_path`` as a PNG.
|
||||
|
||||
Clips are a few hundred frames, so decoding through to the end is simpler
|
||||
and more reliable than seeking (which lands on keyframes, not the last frame).
|
||||
"""
|
||||
import av
|
||||
|
||||
tmp_path = f"{out_path}.tmp.png"
|
||||
with av.open(video_path) as container:
|
||||
last = None
|
||||
for frame in container.decode(container.streams.video[0]):
|
||||
last = frame
|
||||
if last is None:
|
||||
raise ValueError("The video has no frames.")
|
||||
last.to_image().save(tmp_path)
|
||||
os.replace(tmp_path, out_path) # atomic, so a reader never sees a half-written file
|
||||
|
||||
|
||||
def last_frame_for_job(job: Any, upload_dir: str) -> str:
|
||||
"""Path of a PNG of the last frame of ``job``'s video, extracting it if needed.
|
||||
|
||||
Raises :class:`FrameError` with the status the API should return.
|
||||
"""
|
||||
status = getattr(job.status, "value", job.status)
|
||||
if status != "completed" or not job.output_path:
|
||||
raise FrameError(404, "No output available for this job")
|
||||
if not os.path.isfile(job.output_path):
|
||||
raise FrameError(404, "Output file not found on disk")
|
||||
if Path(job.output_path).suffix.lower() not in VIDEO_EXTENSIONS:
|
||||
raise FrameError(400, "This job's output is not a video")
|
||||
if not upload_dir:
|
||||
raise FrameError(503, "Upload directory not configured")
|
||||
|
||||
out_dir = os.path.join(upload_dir, "last_frames")
|
||||
os.makedirs(out_dir, exist_ok=True)
|
||||
out_path = os.path.abspath(os.path.join(out_dir, f"last_frame_{job.id}.png"))
|
||||
# Reuse an earlier extraction unless the video has been rewritten since.
|
||||
if not (os.path.isfile(out_path) and os.path.getmtime(out_path) >= os.path.getmtime(job.output_path)):
|
||||
try:
|
||||
save_last_frame(job.output_path, out_path)
|
||||
except Exception as e:
|
||||
raise FrameError(500, f"Could not read the last frame: {e}") from e
|
||||
return out_path
|
||||
|
||||
|
||||
DEFAULT_LAST_CLIP_SECONDS = 1.0
|
||||
|
||||
|
||||
def save_last_clip(video_path: str, out_path: str, seconds: float = DEFAULT_LAST_CLIP_SECONDS) -> None:
|
||||
"""Decode the final ``seconds`` of ``video_path`` and re-encode them to ``out_path``.
|
||||
|
||||
Re-encoded rather than stream-copied: the source's keyframes rarely align
|
||||
with an arbitrary cut ``seconds`` from the end, and a copy can only start on
|
||||
one. Silent -- the trailing video is for motion continuity; voice continuity
|
||||
already comes from each job's own audio reference.
|
||||
"""
|
||||
import av
|
||||
|
||||
if seconds <= 0:
|
||||
raise ValueError(f"seconds must be positive, got {seconds}.")
|
||||
|
||||
tmp_path = f"{out_path}.tmp.mp4"
|
||||
with av.open(video_path) as src:
|
||||
video_stream = src.streams.video[0]
|
||||
fps = float(video_stream.average_rate or video_stream.guessed_rate or 24)
|
||||
frames = list(src.decode(video_stream))
|
||||
if not frames:
|
||||
raise ValueError("The video has no frames.")
|
||||
keep = max(1, round(seconds * fps))
|
||||
tail = frames[-keep:]
|
||||
|
||||
with av.open(tmp_path, mode="w") as out:
|
||||
out_stream = out.add_stream("h264", rate=round(fps))
|
||||
out_stream.width, out_stream.height = tail[0].width, tail[0].height
|
||||
out_stream.pix_fmt = "yuv420p"
|
||||
# Each frame needs its own increasing pts at a known time base, or PyAV
|
||||
# stamps every frame identically (pts=0), which corrupts the muxed
|
||||
# container's average-frame-rate metadata -- H3's reference pipeline
|
||||
# trusts that metadata to resample the clip, so a wrong rate there
|
||||
# silently collapses a short clip down to as little as one frame.
|
||||
frame_time_base = Fraction(1, round(fps))
|
||||
for i, frame in enumerate(tail):
|
||||
frame.pts = i
|
||||
frame.time_base = frame_time_base
|
||||
for packet in out_stream.encode(frame):
|
||||
out.mux(packet)
|
||||
for packet in out_stream.encode():
|
||||
out.mux(packet)
|
||||
os.replace(tmp_path, out_path) # atomic, so a reader never sees a half-written file
|
||||
|
||||
|
||||
def last_clip_for_job(job: Any, upload_dir: str, seconds: float = DEFAULT_LAST_CLIP_SECONDS) -> str:
|
||||
"""Path of a short video of the end of ``job``'s video, extracting it if needed.
|
||||
|
||||
Raises :class:`FrameError` with the status the API should return.
|
||||
"""
|
||||
status = getattr(job.status, "value", job.status)
|
||||
if status != "completed" or not job.output_path:
|
||||
raise FrameError(404, "No output available for this job")
|
||||
if not os.path.isfile(job.output_path):
|
||||
raise FrameError(404, "Output file not found on disk")
|
||||
if Path(job.output_path).suffix.lower() not in VIDEO_EXTENSIONS:
|
||||
raise FrameError(400, "This job's output is not a video")
|
||||
if not upload_dir:
|
||||
raise FrameError(503, "Upload directory not configured")
|
||||
|
||||
out_dir = os.path.join(upload_dir, "last_clips")
|
||||
os.makedirs(out_dir, exist_ok=True)
|
||||
out_path = os.path.abspath(os.path.join(out_dir, f"last_clip_{job.id}.mp4"))
|
||||
if not (os.path.isfile(out_path) and os.path.getmtime(out_path) >= os.path.getmtime(job.output_path)):
|
||||
try:
|
||||
save_last_clip(job.output_path, out_path, seconds)
|
||||
except Exception as e:
|
||||
raise FrameError(500, f"Could not read the end of the clip: {e}") from e
|
||||
return out_path
|
||||
@@ -0,0 +1,208 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Scheduling rules for queued jobs, and references to another job's ending.
|
||||
|
||||
Kept apart from ``job_runner.py`` because it is pure -- it works on plain
|
||||
snapshots and needs neither FastVideo nor a GPU -- so it can be tested anywhere.
|
||||
|
||||
A job can start from how another job's video ended -- either its last frame (a
|
||||
still, for a hard "opens on this exact picture" continuation) or its last
|
||||
second or so (a short video, so the model has actual motion to continue, not
|
||||
just a pose to guess motion from). Neither exists until the other job
|
||||
finishes, so the reference is *deferred*: its source is ``job-last-frame:<job
|
||||
id>`` or ``job-last-clip:<job id>``, resolved to a real file only when the job
|
||||
runs. That link is also the queue's dependency graph: a queued job waits until
|
||||
every job it takes an ending from has completed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Iterable, Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Protocol
|
||||
|
||||
DEFERRED_LAST_FRAME_PREFIX = "job-last-frame:"
|
||||
DEFERRED_LAST_CLIP_PREFIX = "job-last-clip:"
|
||||
_DEFERRED_PREFIXES = (DEFERRED_LAST_FRAME_PREFIX, DEFERRED_LAST_CLIP_PREFIX)
|
||||
|
||||
QUEUED = "queued"
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
PENDING = "pending"
|
||||
FAILED = "failed"
|
||||
STOPPED = "stopped"
|
||||
|
||||
|
||||
def deferred_last_frame_source(job_id: str) -> str:
|
||||
return f"{DEFERRED_LAST_FRAME_PREFIX}{job_id}"
|
||||
|
||||
|
||||
def deferred_last_clip_source(job_id: str) -> str:
|
||||
return f"{DEFERRED_LAST_CLIP_PREFIX}{job_id}"
|
||||
|
||||
|
||||
def dependency_of(reference: Mapping[str, Any] | None) -> str | None:
|
||||
"""The job a deferred reference takes its ending from, or None for any other."""
|
||||
source = (reference or {}).get("source")
|
||||
if not isinstance(source, str):
|
||||
return None
|
||||
for prefix in _DEFERRED_PREFIXES:
|
||||
if source.startswith(prefix):
|
||||
return source[len(prefix):] or None
|
||||
return None
|
||||
|
||||
|
||||
def dependencies(references: Iterable[Mapping[str, Any]] | None) -> list[str]:
|
||||
"""Job ids a job needs finished first, in reference order, without repeats."""
|
||||
found: list[str] = []
|
||||
for ref in references or []:
|
||||
dep = dependency_of(ref)
|
||||
if dep and dep not in found:
|
||||
found.append(dep)
|
||||
return found
|
||||
|
||||
|
||||
class JobView(Protocol):
|
||||
"""The parts of a job the scheduler looks at."""
|
||||
|
||||
id: str
|
||||
references: Any
|
||||
queued_at: float | None
|
||||
|
||||
@property
|
||||
def status(self) -> Any: # a JobStatus (str enum) or plain str
|
||||
...
|
||||
|
||||
|
||||
def _status(job: Any) -> str:
|
||||
return str(getattr(job.status, "value", job.status))
|
||||
|
||||
|
||||
def circular_dependency(jobs: Mapping[str, JobView], job_id: str) -> list[str] | None:
|
||||
"""The chain of job ids if ``job_id`` (transitively) waits on itself, else None."""
|
||||
|
||||
def walk(current: str, path: list[str]) -> list[str] | None:
|
||||
job = jobs.get(current)
|
||||
if job is None:
|
||||
return None
|
||||
for dep in dependencies(job.references):
|
||||
if dep == job_id:
|
||||
return [*path, current, dep]
|
||||
if dep in path or dep == current:
|
||||
continue
|
||||
found = walk(dep, [*path, current])
|
||||
if found:
|
||||
return found
|
||||
return None
|
||||
|
||||
return walk(job_id, [])
|
||||
|
||||
|
||||
def _label(job: Any) -> str:
|
||||
name = getattr(job, "name", "") or ""
|
||||
return f"'{name}'" if name else job.id
|
||||
|
||||
|
||||
@dataclass
|
||||
class Plan:
|
||||
"""What to do now: jobs to launch, and jobs that can never run."""
|
||||
|
||||
start: list[str] = field(default_factory=list)
|
||||
fail: dict[str, str] = field(default_factory=dict) # job id -> reason
|
||||
|
||||
|
||||
def plan_dispatch(jobs: Mapping[str, JobView], max_concurrent: int) -> Plan:
|
||||
"""Decide which queued jobs launch now and which have become impossible.
|
||||
|
||||
* Queued jobs run oldest-queued first, at most ``max_concurrent`` at a time
|
||||
(counting jobs already running).
|
||||
* A job whose frame source hasn't completed *waits*, and doesn't hold up
|
||||
jobs behind it that don't need it.
|
||||
* A job whose frame source failed, was stopped, or was deleted can never
|
||||
run, so it fails -- and so does anything waiting on *it*.
|
||||
* A job whose frame source is idle (neither queued nor running) keeps
|
||||
waiting: the user may still queue it.
|
||||
"""
|
||||
plan = Plan()
|
||||
status = {jid: _status(job) for jid, job in jobs.items()}
|
||||
queued = sorted(
|
||||
(job for job in jobs.values() if status[job.id] == QUEUED),
|
||||
key=lambda job: (job.queued_at or 0.0, job.id),
|
||||
)
|
||||
|
||||
# Failures cascade down a chain, so repeat until nothing new fails.
|
||||
changed = True
|
||||
while changed:
|
||||
changed = False
|
||||
for job in queued:
|
||||
if job.id in plan.fail:
|
||||
continue
|
||||
for dep_id in dependencies(job.references):
|
||||
dep = jobs.get(dep_id)
|
||||
if dep is None:
|
||||
plan.fail[job.id] = "Waiting on the ending of a job that was deleted."
|
||||
elif dep_id in plan.fail or status[dep_id] in (FAILED, STOPPED):
|
||||
reason = "failed" if dep_id in plan.fail or status[dep_id] == FAILED else "was stopped"
|
||||
plan.fail[job.id] = f"Needs the ending of {_label(dep)}, which {reason}."
|
||||
else:
|
||||
continue
|
||||
status[job.id] = FAILED
|
||||
changed = True
|
||||
break
|
||||
|
||||
slots = max(max_concurrent, 1) - sum(1 for s in status.values() if s == RUNNING)
|
||||
for job in queued:
|
||||
if slots <= 0:
|
||||
break
|
||||
if job.id in plan.fail:
|
||||
continue
|
||||
if all(status.get(dep) == COMPLETED for dep in dependencies(job.references)):
|
||||
plan.start.append(job.id)
|
||||
slots -= 1
|
||||
return plan
|
||||
|
||||
|
||||
class UnresolvedReference(ValueError):
|
||||
"""A deferred reference can't be turned into a real file."""
|
||||
|
||||
|
||||
def resolve_references(
|
||||
references: list[dict[str, Any]] | None,
|
||||
get_job: Callable[[str], Any],
|
||||
last_frame: Callable[[Any], str],
|
||||
last_clip: Callable[[Any], str] | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Copy of ``references`` with each deferred reference made real.
|
||||
|
||||
``last_frame(job)`` and ``last_clip(job)`` each return the path of an
|
||||
extracted file and may raise (see ``frames.FrameError``); the message is
|
||||
passed on. ``last_clip`` is optional so callers that never attach a
|
||||
trailing-clip reference (e.g. the mock server) don't need to supply one --
|
||||
it is only required if a ``job-last-clip:`` reference is actually present.
|
||||
Other references are returned unchanged, and the stored list is never
|
||||
modified, so a re-run resolves again against the source job's current
|
||||
output.
|
||||
"""
|
||||
resolved: list[dict[str, Any]] = []
|
||||
for ref in references or []:
|
||||
source = (ref or {}).get("source")
|
||||
prefix = next((p for p in _DEFERRED_PREFIXES if isinstance(source, str) and source.startswith(p)), None)
|
||||
if prefix is None:
|
||||
resolved.append(ref)
|
||||
continue
|
||||
dep_id = source[len(prefix):] or None
|
||||
extract = last_frame if prefix == DEFERRED_LAST_FRAME_PREFIX else last_clip
|
||||
what = "the last frame of" if prefix == DEFERRED_LAST_FRAME_PREFIX else "the ending of"
|
||||
if extract is None:
|
||||
raise UnresolvedReference("This server can't resolve a trailing-clip reference yet.")
|
||||
dep = get_job(dep_id) if dep_id else None
|
||||
if dep is None:
|
||||
raise UnresolvedReference(f"A reference needs {what} a job that no longer exists.")
|
||||
if _status(dep) != COMPLETED:
|
||||
raise UnresolvedReference(f"A reference needs {what} {_label(dep)}, which hasn't finished "
|
||||
f"(it is {_status(dep)}).")
|
||||
try:
|
||||
path = extract(dep)
|
||||
except Exception as exc: # FrameError carries a readable .detail
|
||||
raise UnresolvedReference(str(getattr(exc, "detail", None) or exc)) from exc
|
||||
resolved.append({**ref, "source": path})
|
||||
return resolved
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,223 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Join a scene's clips, in order, into one video.
|
||||
|
||||
Kept apart from ``server.py`` because it needs nothing from FastVideo -- only
|
||||
PyAV (to read each clip's format) and an ``ffmpeg`` binary -- so it can be
|
||||
exercised without a GPU.
|
||||
|
||||
Clips from one scene share codec, size and frame rate, so they are joined by
|
||||
copying the streams rather than re-encoding: it takes about a second, loses no
|
||||
quality, and keeps picture and sound in step. Each clip is cut off at the end of
|
||||
its last video frame, because the audio track runs a few milliseconds past it and
|
||||
would otherwise open a gap between clips.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
VIDEO_EXTENSIONS = {".mp4", ".mov", ".mkv", ".webm", ".avi"}
|
||||
MERGED_DIRNAME = "merged"
|
||||
_FILENAME_RE = re.compile(r"^[\w.-]+\.mp4$")
|
||||
|
||||
|
||||
class MergeError(Exception):
|
||||
"""A merge that can't be done, carrying the HTTP status to report."""
|
||||
|
||||
def __init__(self, status_code: int, detail: str):
|
||||
super().__init__(detail)
|
||||
self.status_code = status_code
|
||||
self.detail = detail
|
||||
|
||||
|
||||
def find_ffmpeg() -> str | None:
|
||||
"""An ``ffmpeg`` to run: $FASTVIDEO_FFMPEG_BIN, then PATH, then imageio-ffmpeg's bundled one."""
|
||||
configured = os.getenv("FASTVIDEO_FFMPEG_BIN")
|
||||
if configured and (shutil.which(configured) or os.path.isfile(configured)):
|
||||
return shutil.which(configured) or configured
|
||||
on_path = shutil.which("ffmpeg")
|
||||
if on_path:
|
||||
return on_path
|
||||
try:
|
||||
import imageio_ffmpeg
|
||||
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ClipInfo:
|
||||
path: str
|
||||
video_codec: str
|
||||
width: int
|
||||
height: int
|
||||
fps: float
|
||||
pix_fmt: str
|
||||
video_seconds: float
|
||||
audio: tuple[str, int, int] | None # codec, sample rate, channels
|
||||
|
||||
def describe(self) -> str:
|
||||
audio = f"{self.audio[0]} {self.audio[1]} Hz x{self.audio[2]}" if self.audio else "no audio"
|
||||
return f"{self.width}x{self.height} {self.fps:.2f} fps {self.video_codec}/{self.pix_fmt}, {audio}"
|
||||
|
||||
def signature(self) -> tuple[Any, ...]:
|
||||
return (self.video_codec, self.width, self.height, round(self.fps, 3), self.pix_fmt, self.audio)
|
||||
|
||||
|
||||
def probe_clip(path: str) -> ClipInfo:
|
||||
"""Read what ``check_compatible`` needs from a video file."""
|
||||
import av
|
||||
|
||||
try:
|
||||
with av.open(path) as container:
|
||||
video = next((s for s in container.streams if s.type == "video"), None)
|
||||
if video is None:
|
||||
raise MergeError(409, f"{os.path.basename(path)} has no video stream.")
|
||||
audio = next((s for s in container.streams if s.type == "audio"), None)
|
||||
fps = float(video.average_rate or video.guessed_rate or 0)
|
||||
if video.duration is not None:
|
||||
seconds = float(video.duration * video.time_base)
|
||||
elif container.duration is not None:
|
||||
seconds = container.duration / 1_000_000
|
||||
else:
|
||||
seconds = (video.frames or 0) / fps if fps else 0.0
|
||||
if seconds <= 0:
|
||||
raise MergeError(409, f"{os.path.basename(path)} has no playable video.")
|
||||
return ClipInfo(
|
||||
path=path,
|
||||
video_codec=video.codec_context.name,
|
||||
width=video.codec_context.width,
|
||||
height=video.codec_context.height,
|
||||
fps=fps,
|
||||
pix_fmt=video.codec_context.pix_fmt or "",
|
||||
video_seconds=seconds,
|
||||
audio=((audio.codec_context.name, audio.codec_context.sample_rate, audio.codec_context.channels)
|
||||
if audio else None),
|
||||
)
|
||||
except MergeError:
|
||||
raise
|
||||
except Exception as exc: # av.error.* on a corrupt or unreadable file
|
||||
raise MergeError(409, f"Could not read {os.path.basename(path)}: {exc}") from exc
|
||||
|
||||
|
||||
def check_compatible(infos: list[ClipInfo], labels: list[str]) -> None:
|
||||
"""Refuse clips that can't be joined by copying, naming the first that differs."""
|
||||
first = infos[0]
|
||||
for info, label in zip(infos[1:], labels[1:], strict=True):
|
||||
if info.signature() != first.signature():
|
||||
raise MergeError(
|
||||
409,
|
||||
f"{label} ({info.describe()}) doesn't match {labels[0]} ({first.describe()}). "
|
||||
"Clips need the same size, frame rate and audio to be joined.",
|
||||
)
|
||||
|
||||
|
||||
def _concat_list(infos: list[ClipInfo]) -> str:
|
||||
lines = ["ffconcat version 1.0"]
|
||||
for info in infos:
|
||||
escaped = info.path.replace("'", "'\\''")
|
||||
lines += [f"file '{escaped}'", f"outpoint {info.video_seconds:.6f}"]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def merge_clips(paths: list[str], out_path: str, labels: list[str] | None = None, ffmpeg: str | None = None) -> float:
|
||||
"""Join ``paths`` in order into ``out_path``. Returns the merged length in seconds."""
|
||||
if not paths:
|
||||
raise MergeError(400, "No clips to merge.")
|
||||
labels = labels or [os.path.basename(p) for p in paths]
|
||||
ffmpeg = ffmpeg or find_ffmpeg()
|
||||
if not ffmpeg:
|
||||
raise MergeError(503, "ffmpeg isn't available on the server. Install it or set FASTVIDEO_FFMPEG_BIN.")
|
||||
|
||||
infos = [probe_clip(p) for p in paths]
|
||||
check_compatible(infos, labels)
|
||||
|
||||
os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
|
||||
tmp_out = f"{out_path}.tmp.mp4"
|
||||
list_fd, list_path = tempfile.mkstemp(suffix=".ffconcat", text=True)
|
||||
try:
|
||||
with os.fdopen(list_fd, "w", encoding="utf-8") as f:
|
||||
f.write(_concat_list(infos))
|
||||
result = subprocess.run(
|
||||
[ffmpeg, "-y", "-loglevel", "error", "-f", "concat", "-safe", "0", "-i", list_path, "-c", "copy",
|
||||
"-movflags", "+faststart", tmp_out],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=600,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
tail = " ".join(result.stderr.strip().splitlines()[-3:])
|
||||
raise MergeError(500, f"ffmpeg failed: {tail or 'unknown error'}")
|
||||
os.replace(tmp_out, out_path) # atomic, so a reader never sees a half-written file
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
raise MergeError(500, "ffmpeg took too long.") from exc
|
||||
finally:
|
||||
for leftover in (list_path, tmp_out):
|
||||
try:
|
||||
os.remove(leftover)
|
||||
except OSError:
|
||||
pass
|
||||
return sum(i.video_seconds for i in infos)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MergeResult:
|
||||
filename: str
|
||||
path: str
|
||||
clips: int
|
||||
seconds: float
|
||||
|
||||
|
||||
def _safe_stem(name: str) -> str:
|
||||
return re.sub(r"[^\w.-]+", "-", name).strip("-.")[:60] or "scene"
|
||||
|
||||
|
||||
def _label(job: Any) -> str:
|
||||
name = (getattr(job, "name", "") or "").strip()
|
||||
return f"'{name}'" if name else str(job.id)
|
||||
|
||||
|
||||
def merge_jobs(jobs: list[Any], out_dir: str, name: str = "") -> MergeResult:
|
||||
"""Join the finished videos of ``jobs`` (in the order given) into ``<out_dir>/merged``.
|
||||
|
||||
Every job must be completed with a video on disk; otherwise nothing is made,
|
||||
so a scene missing a clip can't be mistaken for the whole thing.
|
||||
"""
|
||||
if not jobs:
|
||||
raise MergeError(400, "No clips to merge.")
|
||||
paths: list[str] = []
|
||||
for job in jobs:
|
||||
status = getattr(job.status, "value", job.status)
|
||||
if status != "completed" or not job.output_path:
|
||||
raise MergeError(409, f"{_label(job)} hasn't finished ({status}), so the scene can't be merged yet.")
|
||||
if not os.path.isfile(job.output_path):
|
||||
raise MergeError(409, f"{_label(job)}'s video is missing from disk.")
|
||||
if Path(job.output_path).suffix.lower() not in VIDEO_EXTENSIONS:
|
||||
raise MergeError(409, f"{_label(job)}'s output isn't a video.")
|
||||
paths.append(job.output_path)
|
||||
|
||||
filename = f"{_safe_stem(name)}-{time.strftime('%Y%m%d-%H%M%S')}.mp4"
|
||||
merged_dir = os.path.join(out_dir, MERGED_DIRNAME)
|
||||
out_path = os.path.join(merged_dir, filename)
|
||||
seconds = merge_clips(paths, out_path, labels=[_label(j) for j in jobs])
|
||||
return MergeResult(filename=filename, path=out_path, clips=len(paths), seconds=seconds)
|
||||
|
||||
|
||||
def merged_path(out_dir: str, filename: str) -> str:
|
||||
"""Path of a merged video by filename, refusing anything that isn't one."""
|
||||
if not _FILENAME_RE.match(filename):
|
||||
raise MergeError(404, "Merged video not found.")
|
||||
path = os.path.join(out_dir, MERGED_DIRNAME, filename)
|
||||
if not os.path.isfile(path):
|
||||
raise MergeError(404, "Merged video not found.")
|
||||
return path
|
||||
@@ -4,8 +4,8 @@ In-memory mock of the FastVideo Studio API (``server.py``).
|
||||
|
||||
This mock implements the same ``/api`` routes and response shapes as the real
|
||||
FastAPI server but keeps everything in memory and never touches the FastVideo
|
||||
library, a GPU, or a database. It is meant to back the Playwright e2e suite so
|
||||
the Next.js frontend can be exercised end-to-end without a real backend.
|
||||
library or a GPU. It is meant to back the Playwright e2e suite so the Next.js
|
||||
frontend can be exercised end-to-end without a real backend.
|
||||
|
||||
Job lifecycle is simulated by recording a start timestamp and *computing* the
|
||||
status on read: a started job reports ``running`` for a few seconds and then
|
||||
@@ -13,14 +13,24 @@ flips to ``completed`` with an ``output_path``, so polling the job list / logs
|
||||
shows progression. Generated media is a tiny 1s ``testsrc`` MP4 built lazily
|
||||
with ffmpeg and cached.
|
||||
|
||||
Pass ``--data-dir`` to seed jobs/datasets/settings from a real server's
|
||||
persisted SQLite database instead of the built-in fake demo data, so you can
|
||||
browse real job history through the mock without a GPU. This is a read-only
|
||||
snapshot taken once at startup: nothing created or changed while the mock
|
||||
runs is ever written back to that database, so it's safe to point at the
|
||||
same ``--data-dir`` a real ``server.py`` is using.
|
||||
|
||||
Usage (from the ``apps/`` directory)::
|
||||
|
||||
PYTHONPATH=.. python -m fastvideo_studio.mock_server --port 8190
|
||||
PYTHONPATH=.. python -m fastvideo_studio.mock_server --port 8190 --data-dir ../outputs/ui_data
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import shutil
|
||||
@@ -29,22 +39,36 @@ import tempfile
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import Annotated, Any
|
||||
|
||||
from fastapi import FastAPI, File, HTTPException, UploadFile
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, PlainTextResponse
|
||||
|
||||
from fastvideo_studio.database import default_settings_dict
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, SettingsUpdate, UpdateCaptionRequest,
|
||||
from fastvideo_studio.database import Database, default_settings_dict
|
||||
from fastvideo_studio.job_queue import circular_dependency, dependencies, plan_dispatch
|
||||
from fastvideo_studio.merge import MergeError, merge_jobs, merged_path
|
||||
from fastvideo_studio.trim import BRIGHTNESS_RANGE, CONTRAST_RANGE, SATURATION_RANGE
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, GradeSegment, MergeSceneRequest,
|
||||
QueueJobsRequest, SettingsUpdate, TrimRequest, UpdateCaptionRequest,
|
||||
model_label)
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(name)s: %(message)s")
|
||||
logger = logging.getLogger("fastvideo.studio.mock")
|
||||
|
||||
# --- Config -----------------------------------------------------------------
|
||||
|
||||
# How long a started job stays "running" before it flips to "completed".
|
||||
COMPLETE_AFTER_SECONDS = 3.0
|
||||
FFMPEG_BIN = shutil.which(os.getenv("FASTVIDEO_FFMPEG_BIN", "ffmpeg"))
|
||||
|
||||
# Marks a job dict as a frozen row loaded from a real --data-dir snapshot
|
||||
# (see _load_snapshot_from_db) rather than something this process is
|
||||
# simulating. Stripped from API responses by _public_job.
|
||||
_FROM_SNAPSHOT_KEY = "_from_db_snapshot"
|
||||
|
||||
# A small catalogue of fake models keyed by workload type. Mirrors the real
|
||||
# server's {id, label} shape (label derived from the HF-style path).
|
||||
_MODELS_BY_WORKLOAD: dict[str, list[str]] = {
|
||||
@@ -54,6 +78,10 @@ _MODELS_BY_WORKLOAD: dict[str, list[str]] = {
|
||||
],
|
||||
"i2v": [
|
||||
"Wan-AI/Wan2.1-I2V-14B-480P-Diffusers",
|
||||
# Real registered id (fastvideo/registry.py) -- needed so the frontend's
|
||||
# `modelId.toLowerCase().includes('minimax-h3')` gate (speech-tag
|
||||
# console, references, last-frame upload) actually lights up in the mock.
|
||||
"MiniMaxAI/MiniMax-H3",
|
||||
],
|
||||
"t2i": [
|
||||
"black-forest-labs/FLUX.1-schnell",
|
||||
@@ -98,16 +126,76 @@ _mock_media_cache: dict[str, str] = {}
|
||||
_mock_media_lock = threading.Lock()
|
||||
|
||||
|
||||
def _testsrc_frame(width: int, height: int, index: int):
|
||||
"""A cheap, dependency-light stand-in for ffmpeg's `testsrc` pattern:
|
||||
a gradient that shifts over `index` so an encoded clip visibly animates."""
|
||||
import numpy as np
|
||||
|
||||
frame = np.zeros((height, width, 3), dtype=np.uint8)
|
||||
frame[:, :, 0] = np.linspace(0, 255, width, dtype=np.uint8)[None, :]
|
||||
frame[:, :, 1] = np.linspace(0, 255, height, dtype=np.uint8)[:, None]
|
||||
frame[:, :, 2] = (index * 10) % 256
|
||||
return frame
|
||||
|
||||
|
||||
def _build_mock_media_with_pyav(kind: str, path: str) -> None:
|
||||
"""ffmpeg isn't installed on every dev box; PyAV (a real FastVideo
|
||||
dependency already, used for reference-video decoding) can encode the
|
||||
same tiny clip directly via its bundled libav, no external binary."""
|
||||
from PIL import Image
|
||||
|
||||
if kind == "png":
|
||||
Image.fromarray(_testsrc_frame(320, 240, 0)).save(path)
|
||||
return
|
||||
|
||||
import av as pyav
|
||||
|
||||
# h264 is what every browser's <video> element actually plays; mpeg4
|
||||
# (MPEG-4 Part 2) is a PyAV-always-has-it fallback that no mainstream
|
||||
# browser decodes, so gallery/job previews would just fail to load.
|
||||
try:
|
||||
pyav.codec.Codec("h264", "w")
|
||||
codec_name = "h264"
|
||||
except Exception:
|
||||
logger.warning("PyAV has no h264 encoder available; falling back to mpeg4, which most "
|
||||
"browsers cannot play in <video>. Install ffmpeg for real preview playback.")
|
||||
codec_name = "mpeg4"
|
||||
|
||||
container = pyav.open(path, mode="w")
|
||||
try:
|
||||
stream = container.add_stream(codec_name, rate=24)
|
||||
stream.width = 320
|
||||
stream.height = 240
|
||||
stream.pix_fmt = "yuv420p"
|
||||
for i in range(24): # 1s at 24fps, matching the ffmpeg path's clip length
|
||||
frame = pyav.VideoFrame.from_ndarray(_testsrc_frame(320, 240, i), format="rgb24")
|
||||
for packet in stream.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in stream.encode():
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
|
||||
def _build_mock_media(kind: str) -> str:
|
||||
"""Build (once) a tiny testsrc clip of the given ``kind`` with ffmpeg; cache the path."""
|
||||
"""Build (once) a tiny testsrc-like clip of the given ``kind``; cache the path."""
|
||||
with _mock_media_lock:
|
||||
cached = _mock_media_cache.get(kind)
|
||||
if cached and os.path.isfile(cached):
|
||||
return cached
|
||||
if not FFMPEG_BIN:
|
||||
raise HTTPException(status_code=500, detail="ffmpeg is required to build mock media")
|
||||
fd, path = tempfile.mkstemp(prefix="fvstudio_mock_", suffix=f".{kind}")
|
||||
os.close(fd)
|
||||
if not FFMPEG_BIN:
|
||||
try:
|
||||
_build_mock_media_with_pyav(kind, path)
|
||||
except Exception as exc:
|
||||
with contextlib.suppress(OSError):
|
||||
os.remove(path)
|
||||
raise HTTPException(status_code=500, detail=f"PyAV fallback failed to build mock {kind}: {exc}") from exc
|
||||
if not os.path.isfile(path) or os.path.getsize(path) == 0:
|
||||
raise HTTPException(status_code=500, detail=f"PyAV fallback produced no mock {kind} bytes")
|
||||
_mock_media_cache[kind] = path
|
||||
return path
|
||||
if kind == "png":
|
||||
command = [
|
||||
FFMPEG_BIN,
|
||||
@@ -166,33 +254,34 @@ def _build_mock_media(kind: str) -> str:
|
||||
|
||||
|
||||
def _new_job_dict(req: CreateJobRequest) -> dict[str, Any]:
|
||||
"""Build a job dict (mirrors job_runner.Job.to_dict()) in the pending state."""
|
||||
"""Build a job dict (mirrors job_runner.Job.to_dict()) in the pending state.
|
||||
|
||||
Starts from req.model_dump() rather than re-listing CreateJobRequest's
|
||||
fields by hand -- the previous version silently dropped name,
|
||||
last_image_path, references, and every offload/tuning flag because it
|
||||
was never updated as CreateJobRequest grew those fields. Spreading the
|
||||
request means new fields flow through automatically.
|
||||
"""
|
||||
job_id = str(uuid.uuid4())
|
||||
return {
|
||||
**req.model_dump(),
|
||||
"id": job_id,
|
||||
"model_id": req.model_id,
|
||||
"prompt": req.prompt,
|
||||
"workload_type": req.workload_type or "t2v",
|
||||
"job_type": req.job_type or "inference",
|
||||
"image_path": req.image_path or "",
|
||||
"negative_prompt": req.negative_prompt or "",
|
||||
"data_path": req.data_path or "",
|
||||
"status": "pending",
|
||||
"created_at": time.time(),
|
||||
"queued_at": None,
|
||||
"started_at": None,
|
||||
"finished_at": None,
|
||||
"error": None,
|
||||
"output_path": None,
|
||||
"log_file_path": None,
|
||||
"num_inference_steps": req.num_inference_steps,
|
||||
"num_frames": req.num_frames,
|
||||
"height": req.height,
|
||||
"width": req.width,
|
||||
"guidance_scale": req.guidance_scale,
|
||||
"guidance_rescale": req.guidance_rescale,
|
||||
"fps": req.fps,
|
||||
"seed": req.seed,
|
||||
"negative_prompt": req.negative_prompt or "",
|
||||
"num_gpus": req.num_gpus,
|
||||
"data_path": req.data_path or "",
|
||||
"edit_start_seconds": 0.0,
|
||||
"edit_end_seconds": None,
|
||||
"edit_segments": [],
|
||||
"progress": 0.0,
|
||||
"progress_msg": "",
|
||||
"phase": "pending",
|
||||
@@ -204,8 +293,13 @@ def _advance_job(job: dict[str, Any]) -> None:
|
||||
|
||||
Status is *computed on read* from the recorded start timestamp, so polling
|
||||
the job list / logs naturally shows pending -> running -> completed.
|
||||
|
||||
Jobs loaded from a persisted DB snapshot (``_FROM_SNAPSHOT_KEY``) are a
|
||||
frozen historical record, not something this process is simulating — skip
|
||||
them so a real "pending" or "running" row doesn't silently start ticking
|
||||
toward a fake "completed" just because someone polled it.
|
||||
"""
|
||||
if job["status"] != "running" or not job.get("started_at"):
|
||||
if job.get(_FROM_SNAPSHOT_KEY) or job["status"] != "running" or not job.get("started_at"):
|
||||
return
|
||||
elapsed = time.time() - job["started_at"]
|
||||
if elapsed >= COMPLETE_AFTER_SECONDS:
|
||||
@@ -219,9 +313,9 @@ def _advance_job(job: dict[str, Any]) -> None:
|
||||
|
||||
|
||||
def _public_job(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Advance + return a copy safe to serialize."""
|
||||
"""Advance + return a copy safe to serialize (internal-only keys stripped)."""
|
||||
_advance_job(job)
|
||||
return dict(job)
|
||||
return {k: v for k, v in job.items() if not k.startswith("_")}
|
||||
|
||||
|
||||
_LOG_TAIL = [
|
||||
@@ -252,8 +346,8 @@ def _compute_logs(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Return JobLogs-shaped data, growing the visible lines as time passes."""
|
||||
seq = _log_sequence(job)
|
||||
status = job["status"]
|
||||
if status == "pending":
|
||||
return {"lines": [], "progress": 0.0, "progress_msg": "", "phase": "pending"}
|
||||
if status in ("pending", "queued"):
|
||||
return {"lines": [], "progress": 0.0, "progress_msg": "", "phase": status}
|
||||
if status == "completed":
|
||||
return {"lines": seq, "progress": 100.0, "progress_msg": "50/50 steps", "phase": "done"}
|
||||
if status in ("failed", "stopped"):
|
||||
@@ -346,6 +440,89 @@ def _seed() -> None:
|
||||
}
|
||||
|
||||
|
||||
# --- DB snapshot (--data-dir) ------------------------------------------------
|
||||
|
||||
|
||||
def _decode_references(value: Any) -> list[dict[str, Any]]:
|
||||
"""Reference lists round-trip through the DB as JSON text."""
|
||||
if not value:
|
||||
return []
|
||||
if isinstance(value, list):
|
||||
return list(value)
|
||||
try:
|
||||
decoded = json.loads(value)
|
||||
except (TypeError, ValueError):
|
||||
return []
|
||||
return list(decoded) if isinstance(decoded, list) else []
|
||||
|
||||
|
||||
def _job_from_snapshot(row: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Adapt a persisted job row into the mock's job dict shape.
|
||||
|
||||
Frozen as a read-only historical record (see _FROM_SNAPSHOT_KEY): the mock
|
||||
never actually ran this job, so it doesn't get to simulate progress for
|
||||
one. A row still marked "running" is orphaned -- the real JobRunner
|
||||
reclassifies those as failed on its own restart, so mirror that here.
|
||||
"""
|
||||
job = dict(row)
|
||||
status = job["status"]
|
||||
if status == "running":
|
||||
status = "failed"
|
||||
job["error"] = job.get("error") or "Server restarted (job was running)"
|
||||
job["finished_at"] = job.get("finished_at") or time.time()
|
||||
elif status == "queued":
|
||||
# The real server resumes its queue on restart; the mock never runs
|
||||
# snapshot jobs, so leave it as something the user can queue again.
|
||||
status = "pending"
|
||||
job["queued_at"] = None
|
||||
job["status"] = status
|
||||
job["references"] = _decode_references(row.get("references"))
|
||||
job["progress"] = 100.0 if status == "completed" else 0.0
|
||||
job["progress_msg"] = "50/50 steps" if status == "completed" else ""
|
||||
job["phase"] = "done" if status == "completed" else status
|
||||
job[_FROM_SNAPSHOT_KEY] = True
|
||||
return job
|
||||
|
||||
|
||||
def _load_snapshot_from_db(data_dir: Path) -> bool:
|
||||
"""Seed jobs/datasets/settings from a real server's persisted SQLite db.
|
||||
|
||||
Read-only: opens the db (Database.__init__ runs the same idempotent
|
||||
schema migrations a real server startup would) and copies rows into the
|
||||
mock's in-memory state; nothing the mock does afterward is written back.
|
||||
Returns False (leaving the caller to fall back to _seed()) if no db file
|
||||
is found at that path, rather than silently creating an empty one.
|
||||
"""
|
||||
db_path = data_dir / "fastvideo_ui.db"
|
||||
if not db_path.is_file():
|
||||
logger.warning("--data-dir %s has no fastvideo_ui.db; falling back to fake demo data.", data_dir)
|
||||
return False
|
||||
|
||||
db = Database(db_path)
|
||||
settings = db.get_settings()
|
||||
jobs = [_job_from_snapshot(row) for row in db.get_all_jobs()]
|
||||
datasets = db.get_all_datasets()
|
||||
|
||||
with _state_lock:
|
||||
_settings.update({k: v for k, v in settings.items() if v is not None})
|
||||
_jobs.clear()
|
||||
_jobs.update({j["id"]: j for j in jobs})
|
||||
_datasets.clear()
|
||||
_datasets.update({d["id"]: d for d in datasets})
|
||||
_dataset_files.clear()
|
||||
for d in datasets:
|
||||
captions = db.get_dataset_captions(d["id"])
|
||||
_dataset_files[d["id"]] = {"file_names": list(captions.keys()), "captions": captions}
|
||||
|
||||
logger.info(
|
||||
"Loaded read-only snapshot from %s: %d job(s), %d dataset(s).",
|
||||
db_path,
|
||||
len(jobs),
|
||||
len(datasets),
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
# --- App --------------------------------------------------------------------
|
||||
|
||||
app = FastAPI(title="FastVideo Studio Mock API", version="0.1.0")
|
||||
@@ -357,8 +534,6 @@ app.add_middleware(
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
_seed()
|
||||
|
||||
|
||||
@app.get("/api/__mock__")
|
||||
def mock_sentinel() -> dict[str, bool]:
|
||||
@@ -423,6 +598,32 @@ async def upload_image(file: Annotated[UploadFile, File()]) -> dict[str, str]:
|
||||
return {"path": f"/mock/uploads/{uuid.uuid4().hex}_{os.path.basename(name)}"}
|
||||
|
||||
|
||||
ALLOWED_REFERENCE_VIDEO_EXTENSIONS = {".mp4", ".mov", ".mkv", ".webm", ".avi"}
|
||||
ALLOWED_REFERENCE_AUDIO_EXTENSIONS = {".wav", ".mp3", ".flac", ".m4a", ".ogg"}
|
||||
ALLOWED_REFERENCE_IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".bmp"}
|
||||
|
||||
|
||||
@app.post("/api/upload-media")
|
||||
async def upload_media(file: Annotated[UploadFile, File()]) -> dict[str, str]:
|
||||
"""Ref2VA reference upload (image/video/audio) -- the real server's
|
||||
counterpart to upload_image, which only ever handles images."""
|
||||
name = file.filename or "reference"
|
||||
ext = os.path.splitext(name)[1].lower()
|
||||
allowed = ALLOWED_REFERENCE_IMAGE_EXTENSIONS | ALLOWED_REFERENCE_VIDEO_EXTENSIONS | ALLOWED_REFERENCE_AUDIO_EXTENSIONS
|
||||
if ext not in allowed:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid file type. Allowed: {', '.join(sorted(allowed))}")
|
||||
if ext in ALLOWED_REFERENCE_VIDEO_EXTENSIONS:
|
||||
media_type = "video"
|
||||
elif ext in ALLOWED_REFERENCE_AUDIO_EXTENSIONS:
|
||||
media_type = "audio"
|
||||
else:
|
||||
media_type = "image"
|
||||
return {
|
||||
"path": f"/mock/uploads/{uuid.uuid4().hex}_{os.path.basename(name)}",
|
||||
"media_type": media_type,
|
||||
}
|
||||
|
||||
|
||||
@app.post("/api/upload-raw-dataset")
|
||||
async def upload_raw_dataset(files: Annotated[list[UploadFile], File()]) -> dict[str, Any]:
|
||||
video_exts = {".mp4", ".webm", ".avi", ".mov", ".mkv"}
|
||||
@@ -446,6 +647,7 @@ def list_jobs(job_type: str | None = None) -> list[dict[str, Any]]:
|
||||
# _public_job mutates the shared job dict via _advance_job, so it must run
|
||||
# under the lock (matching every other job route) to avoid racing start/stop.
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
jobs = list(_jobs.values())
|
||||
if job_type:
|
||||
jobs = [j for j in jobs if j.get("job_type") == job_type]
|
||||
@@ -456,6 +658,7 @@ def list_jobs(job_type: str | None = None) -> list[dict[str, Any]]:
|
||||
@app.get("/api/jobs/{job_id}")
|
||||
def get_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
@@ -466,15 +669,20 @@ def get_job(job_id: str) -> dict[str, Any]:
|
||||
def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
job = _new_job_dict(req)
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
_jobs[job["id"]] = job
|
||||
if _settings.get("autoStartJob"):
|
||||
_start(job)
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
def _start(job: dict[str, Any]) -> None:
|
||||
def _start(job: dict[str, Any], at: float | None = None) -> None:
|
||||
# Once the mock is actually driving this job, it's no longer a frozen
|
||||
# historical record -- let _advance_job simulate it like any other.
|
||||
job.pop(_FROM_SNAPSHOT_KEY, None)
|
||||
job["status"] = "running"
|
||||
job["started_at"] = time.time()
|
||||
job["queued_at"] = None
|
||||
job["started_at"] = at if at is not None else time.time()
|
||||
job["finished_at"] = None
|
||||
job["error"] = None
|
||||
job["output_path"] = None
|
||||
@@ -486,17 +694,281 @@ def _start(job: dict[str, Any]) -> None:
|
||||
job["phase"] = "starting"
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/start")
|
||||
def start_job(job_id: str) -> dict[str, Any]:
|
||||
class _QueueView:
|
||||
"""The scheduler's view of a mock job dict (it reads attributes, not keys)."""
|
||||
|
||||
def __init__(self, job: dict[str, Any]):
|
||||
self.id = job["id"]
|
||||
self.name = job.get("name", "")
|
||||
self.status = job["status"]
|
||||
self.references = job.get("references")
|
||||
self.queued_at = job.get("queued_at")
|
||||
|
||||
|
||||
def _drain_queue() -> None:
|
||||
"""Run the queue up to now. Call with ``_state_lock`` held.
|
||||
|
||||
Like everything else in the mock this is computed on read: each pass
|
||||
finishes jobs whose time is up, then starts whatever the queue allows.
|
||||
A job takes over the slot at the moment the previous job *finished* (not at
|
||||
the moment someone happened to poll), so a queue that has been sitting for
|
||||
a minute shows every clip already done rather than advancing one per poll.
|
||||
Same rules as the real runner: one at a time, oldest first, and a clip that
|
||||
takes another's last frame waits for it -- see job_queue.plan_dispatch.
|
||||
"""
|
||||
while True:
|
||||
for job in _jobs.values():
|
||||
_advance_job(job)
|
||||
plan = plan_dispatch({jid: _QueueView(j) for jid, j in _jobs.items()}, max_concurrent=1)
|
||||
now = time.time()
|
||||
for job_id, reason in plan.fail.items():
|
||||
job = _jobs[job_id]
|
||||
job["status"] = "failed"
|
||||
job["error"] = reason
|
||||
job["finished_at"] = now
|
||||
job["queued_at"] = None
|
||||
job["phase"] = "failed"
|
||||
if not plan.start:
|
||||
if plan.fail:
|
||||
continue # a failure can free a slot or fail more jobs
|
||||
return
|
||||
for job_id in plan.start:
|
||||
job = _jobs[job_id]
|
||||
finished = [j["finished_at"] for j in _jobs.values() if j.get("finished_at")]
|
||||
# The slot came free when the last job finished, if that was after the job was queued.
|
||||
slot_free = max([job.get("queued_at") or 0.0, *[f for f in finished if f <= now]])
|
||||
_start(job, at=min(now, slot_free))
|
||||
|
||||
|
||||
def _check_dependencies_finished(job: dict[str, Any]) -> None:
|
||||
for dep_id in dependencies(job.get("references")):
|
||||
dep = _jobs.get(dep_id)
|
||||
if dep is None:
|
||||
raise HTTPException(status_code=409,
|
||||
detail="This job starts from the ending of a job that no longer exists.")
|
||||
if dep["status"] != "completed":
|
||||
label = f"'{dep['name']}'" if dep.get("name") else dep["id"]
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=(f"This job starts from the ending of {label}, which hasn't finished "
|
||||
f"(it is {dep['status']}). Queue it to run after that job."),
|
||||
)
|
||||
|
||||
|
||||
# Mirrors JobRunner.EDITABLE_STATUSES: exactly the jobs that can still be started.
|
||||
_EDITABLE_STATUSES = ("pending", "failed", "stopped")
|
||||
|
||||
|
||||
@app.patch("/api/jobs/{job_id}")
|
||||
def update_job(job_id: str, updates: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Edit a job's configuration; only jobs that have not produced a result."""
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
if job["status"] == "running":
|
||||
raise HTTPException(status_code=409, detail="Job is already running")
|
||||
if job["status"] == "completed":
|
||||
raise HTTPException(status_code=409, detail="Job already completed. Delete and re-create to run again.")
|
||||
if job["status"] not in _EDITABLE_STATUSES:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=(f"Job is {job['status']}; only {', '.join(_EDITABLE_STATUSES)} jobs can be edited. "
|
||||
"Duplicate it instead."),
|
||||
)
|
||||
unknown = set(updates) - set(CreateJobRequest.model_fields)
|
||||
if unknown:
|
||||
raise HTTPException(status_code=400, detail=f"Not editable: {', '.join(sorted(unknown))}")
|
||||
job.update(updates)
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/last-frame")
|
||||
def extract_last_frame(job_id: str) -> dict[str, str]:
|
||||
"""Stand-in for the real route: there is no video to read, so hand back a made-up path."""
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_advance_job(job)
|
||||
if job["status"] != "completed" or not job.get("output_path"):
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
return {"path": f"/mock/last_frames/last_frame_{job_id}.png", "media_type": "image"}
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/last-clip")
|
||||
def extract_last_clip(job_id: str) -> dict[str, str]:
|
||||
"""Stand-in for the real route: there is no video to read, so hand back a made-up path."""
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_advance_job(job)
|
||||
if job["status"] != "completed" or not job.get("output_path"):
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
return {"path": f"/mock/last_clips/last_clip_{job_id}.mp4", "media_type": "video"}
|
||||
|
||||
|
||||
#: Key holding a trimmed job's frame count before any trim, so it can be restored.
|
||||
_ORIGINAL_NUM_FRAMES_KEY = "_original_num_frames"
|
||||
|
||||
|
||||
def _trimmable_job(job_id: str) -> dict[str, Any]:
|
||||
"""Look up a completed job with an output, or raise. Call with _state_lock held."""
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_advance_job(job)
|
||||
if job["status"] != "completed" or not job.get("output_path"):
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
return job
|
||||
|
||||
|
||||
def _check_grade_range(name: str, value: float, bounds: tuple[float, float]) -> None:
|
||||
low, high = bounds
|
||||
if not (low <= value <= high):
|
||||
raise HTTPException(status_code=400, detail=f"{name} must be between {low:g} and {high:g}, got {value:g}.")
|
||||
|
||||
|
||||
def _check_segments(segments: list[GradeSegment]) -> None:
|
||||
"""Structural checks only (ordering, open-endedness, color bounds) -- the mock has no
|
||||
frames to count, so it doesn't reproduce trim.py's frame-rounding/coverage checks.
|
||||
|
||||
An empty list isn't an error here either, matching trim.py: it defaults to
|
||||
one neutral section covering everything (Pydantic's own default_factory
|
||||
already supplies that when the field is omitted, but an explicit "segments":
|
||||
[] reaches here as a real empty list).
|
||||
"""
|
||||
prev_end = 0.0
|
||||
for i, s in enumerate(segments):
|
||||
_check_grade_range("brightness", s.brightness, BRIGHTNESS_RANGE)
|
||||
_check_grade_range("contrast", s.contrast, CONTRAST_RANGE)
|
||||
_check_grade_range("saturation", s.saturation, SATURATION_RANGE)
|
||||
if s.end_seconds is None:
|
||||
if i != len(segments) - 1:
|
||||
raise HTTPException(status_code=400, detail="Only the last section can be open-ended.")
|
||||
else:
|
||||
if s.end_seconds <= prev_end:
|
||||
raise HTTPException(status_code=400, detail="Sections must be in increasing order.")
|
||||
prev_end = s.end_seconds
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/trim")
|
||||
def trim_job(job_id: str, req: TrimRequest) -> dict[str, Any]:
|
||||
"""Stand-in for the real route: no video to cut or grade, so just resize num_frames to match."""
|
||||
with _state_lock:
|
||||
job = _trimmable_job(job_id)
|
||||
if req.start_seconds < 0:
|
||||
raise HTTPException(status_code=400, detail="The start of the range can't be negative.")
|
||||
_check_segments(req.segments)
|
||||
fps = job.get("fps") or 24
|
||||
original_frames = job.setdefault(_ORIGINAL_NUM_FRAMES_KEY, job["num_frames"])
|
||||
duration = original_frames / fps
|
||||
end = duration if req.end_seconds is None else min(req.end_seconds, duration)
|
||||
if end <= req.start_seconds:
|
||||
raise HTTPException(status_code=400, detail="The end of the range must come after the start.")
|
||||
job["num_frames"] = max(1, round((end - req.start_seconds) * fps))
|
||||
# Recorded so the dialog can reopen showing what's actually applied,
|
||||
# like the real server's edit_* job fields.
|
||||
job["edit_start_seconds"] = req.start_seconds
|
||||
job["edit_end_seconds"] = req.end_seconds
|
||||
job["edit_segments"] = [s.model_dump() for s in req.segments]
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/restore-video")
|
||||
def restore_job_video(job_id: str) -> dict[str, Any]:
|
||||
"""Stand-in for the real route: undo the simulated edit by restoring num_frames."""
|
||||
with _state_lock:
|
||||
job = _trimmable_job(job_id)
|
||||
if _ORIGINAL_NUM_FRAMES_KEY not in job:
|
||||
raise HTTPException(status_code=404, detail="This job has not been edited.")
|
||||
job["num_frames"] = job.pop(_ORIGINAL_NUM_FRAMES_KEY)
|
||||
job["edit_start_seconds"] = 0.0
|
||||
job["edit_end_seconds"] = None
|
||||
job["edit_segments"] = []
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
def _check_startable(job: dict[str, Any]) -> None:
|
||||
if job["status"] == "running":
|
||||
raise HTTPException(status_code=409, detail="Job is already running")
|
||||
if job["status"] == "queued":
|
||||
raise HTTPException(status_code=409, detail="Job is already queued")
|
||||
if job["status"] == "completed":
|
||||
raise HTTPException(status_code=409, detail="Job already completed. Delete and re-create to run again.")
|
||||
|
||||
|
||||
def _enqueue(job_ids: list[str]) -> list[dict[str, Any]]:
|
||||
"""Queue jobs in order, all-or-nothing (mirrors JobRunner.enqueue_jobs). Lock held."""
|
||||
if len(set(job_ids)) != len(job_ids):
|
||||
raise HTTPException(status_code=409, detail="A job was listed more than once.")
|
||||
jobs = []
|
||||
for job_id in job_ids:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_check_startable(job)
|
||||
jobs.append(job)
|
||||
views = {jid: _QueueView(j) for jid, j in _jobs.items()}
|
||||
for job in jobs:
|
||||
loop = circular_dependency(views, job["id"])
|
||||
if loop:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="Jobs would wait on each other's last frames in a loop: " + " -> ".join(loop),
|
||||
)
|
||||
for job in jobs:
|
||||
job.pop(_FROM_SNAPSHOT_KEY, None)
|
||||
job.update(status="queued", queued_at=time.time(), error=None, output_path=None, log_file_path=None,
|
||||
started_at=None, finished_at=None, progress=0.0, progress_msg="", phase="queued")
|
||||
# Strictly increasing, so jobs queued together keep their order.
|
||||
others = [j["queued_at"] for j in _jobs.values() if j.get("queued_at") and j is not job]
|
||||
job["queued_at"] = max([job["queued_at"], *[o + 1e-6 for o in others]])
|
||||
_drain_queue()
|
||||
return [_public_job(j) for j in jobs]
|
||||
|
||||
|
||||
@app.post("/api/jobs/queue")
|
||||
def queue_jobs(req: QueueJobsRequest) -> list[dict[str, Any]]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
return _enqueue(req.job_ids)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/queue")
|
||||
def queue_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
return _enqueue([job_id])[0]
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/dequeue")
|
||||
def dequeue_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
if job["status"] != "queued":
|
||||
raise HTTPException(status_code=409, detail=f"Job is not queued (status={job['status']})")
|
||||
job["status"] = "pending"
|
||||
job["queued_at"] = None
|
||||
_drain_queue()
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/start")
|
||||
def start_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
_check_startable(job)
|
||||
_check_dependencies_finished(job)
|
||||
_start(job)
|
||||
return _public_job(job)
|
||||
|
||||
@@ -504,10 +976,16 @@ def start_job(job_id: str) -> dict[str, Any]:
|
||||
@app.post("/api/jobs/{job_id}/stop")
|
||||
def stop_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
if job["status"] == "queued":
|
||||
job["status"] = "pending"
|
||||
job["queued_at"] = None
|
||||
_drain_queue()
|
||||
return _public_job(job)
|
||||
if job["status"] != "running":
|
||||
raise HTTPException(status_code=409, detail=f"Job is not running (status={job['status']})")
|
||||
job["status"] = "stopped"
|
||||
@@ -519,14 +997,17 @@ def stop_job(job_id: str) -> dict[str, Any]:
|
||||
@app.delete("/api/jobs/{job_id}")
|
||||
def delete_job(job_id: str) -> dict[str, str]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
if _jobs.pop(job_id, None) is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_drain_queue() # clips waiting on this one's last frame can no longer run
|
||||
return {"detail": f"Job {job_id} deleted"}
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/logs")
|
||||
def get_job_logs(job_id: str, after: int = 0) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
@@ -545,6 +1026,7 @@ def get_job_logs(job_id: str, after: int = 0) -> dict[str, Any]:
|
||||
@app.get("/api/jobs/{job_id}/video")
|
||||
def get_video(job_id: str) -> FileResponse:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
@@ -552,6 +1034,12 @@ def get_video(job_id: str) -> FileResponse:
|
||||
output_path = job.get("output_path")
|
||||
if job["status"] != "completed" or not output_path:
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
# A job loaded from a --data-dir snapshot may point at a real file this
|
||||
# process never wrote, only reads -- serve it if it's actually there,
|
||||
# otherwise fall back to the same synthetic clip every other job gets.
|
||||
if os.path.isfile(output_path):
|
||||
media_type = "video/mp4" if output_path.endswith(".mp4") else "image/png"
|
||||
return FileResponse(output_path, media_type=media_type, filename=os.path.basename(output_path))
|
||||
# Mirror the real server: image workloads (t2i) output a .png served as an
|
||||
# image; everything else is a video.
|
||||
if output_path.endswith(".png"):
|
||||
@@ -559,9 +1047,59 @@ def get_video(job_id: str) -> FileResponse:
|
||||
return FileResponse(_build_mock_media("mp4"), media_type="video/mp4", filename=f"job_{job_id}.mp4")
|
||||
|
||||
|
||||
_MERGED_DIR = tempfile.mkdtemp(prefix="fvstudio_merged_")
|
||||
|
||||
|
||||
@app.post("/api/scenes/merge")
|
||||
def merge_scene(req: MergeSceneRequest) -> dict[str, Any]:
|
||||
"""Real merge (same code as the real server) of whatever videos the jobs have.
|
||||
|
||||
Jobs whose video only exists in the mock get the stand-in clip, so the
|
||||
flow can be tried end to end without generating anything.
|
||||
"""
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
picked = []
|
||||
for job_id in req.job_ids:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
picked.append(dict(job))
|
||||
all_real = all(j.get("output_path") and os.path.isfile(j["output_path"]) for j in picked)
|
||||
stand_in = None if all_real else _build_mock_media("mp4")
|
||||
jobs = [
|
||||
SimpleNamespace(
|
||||
id=j["id"],
|
||||
name=j.get("name", ""),
|
||||
status=j["status"],
|
||||
output_path=(j["output_path"] if all_real else (stand_in if j["status"] == "completed" else None)),
|
||||
) for j in picked
|
||||
]
|
||||
try:
|
||||
result = merge_jobs(jobs, _MERGED_DIR, req.name)
|
||||
except MergeError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return {
|
||||
"filename": result.filename,
|
||||
"url": f"/api/merged/{result.filename}",
|
||||
"clips": result.clips,
|
||||
"seconds": result.seconds,
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/merged/{filename}")
|
||||
def get_merged_video(filename: str) -> FileResponse:
|
||||
try:
|
||||
path = merged_path(_MERGED_DIR, filename)
|
||||
except MergeError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return FileResponse(path, media_type="video/mp4", filename=filename)
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/download_log")
|
||||
def download_log(job_id: str) -> PlainTextResponse:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
@@ -651,8 +1189,18 @@ def main() -> None:
|
||||
parser.add_argument("--host", default="127.0.0.1", help="Bind address (default: 127.0.0.1)")
|
||||
# Default off the real server's 8189 so the mock never shadows a real API.
|
||||
parser.add_argument("--port", type=int, default=8190, help="Port number (default: 8190)")
|
||||
parser.add_argument(
|
||||
"--data-dir",
|
||||
default=None,
|
||||
help=("Read-only: seed jobs/datasets/settings from a real server's "
|
||||
"fastvideo_ui.db under this directory (its --data-dir) instead "
|
||||
"of fake demo data. Nothing the mock does is written back."),
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.data_dir is None or not _load_snapshot_from_db(Path(args.data_dir).resolve()):
|
||||
_seed()
|
||||
|
||||
uvicorn.run(app, host=args.host, port=args.port, log_level="info")
|
||||
|
||||
|
||||
|
||||
@@ -6,6 +6,9 @@ from fastvideo_studio.models.create_job_request import CreateJobRequest
|
||||
from fastvideo_studio.models.settings_update import SettingsUpdate
|
||||
from fastvideo_studio.models.create_dataset_request import CreateDatasetRequest
|
||||
from fastvideo_studio.models.update_caption_request import UpdateCaptionRequest
|
||||
from fastvideo_studio.models.queue_jobs_request import QueueJobsRequest
|
||||
from fastvideo_studio.models.merge_scene_request import MergeSceneRequest
|
||||
from fastvideo_studio.models.trim_request import GradeSegment, TrimRequest
|
||||
|
||||
|
||||
def model_label(model_path: str) -> str:
|
||||
@@ -18,5 +21,9 @@ __all__ = [
|
||||
"SettingsUpdate",
|
||||
"CreateDatasetRequest",
|
||||
"UpdateCaptionRequest",
|
||||
"QueueJobsRequest",
|
||||
"MergeSceneRequest",
|
||||
"GradeSegment",
|
||||
"TrimRequest",
|
||||
"model_label",
|
||||
]
|
||||
|
||||
@@ -1,15 +1,20 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Request model for creating a job."""
|
||||
|
||||
from typing import Any
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class CreateJobRequest(BaseModel):
|
||||
model_id: str
|
||||
name: str = ""
|
||||
prompt: str
|
||||
workload_type: str = "t2v"
|
||||
job_type: str = "inference"
|
||||
image_path: str = ""
|
||||
last_image_path: str = ""
|
||||
references: list[dict[str, Any]] | None = None
|
||||
data_path: str = ""
|
||||
max_train_steps: int = 1000
|
||||
train_batch_size: int = 1
|
||||
@@ -28,6 +33,7 @@ class CreateJobRequest(BaseModel):
|
||||
seed: int = 1024
|
||||
num_gpus: int = 1
|
||||
dit_cpu_offload: bool = False
|
||||
dit_layerwise_offload: bool = False
|
||||
text_encoder_cpu_offload: bool = False
|
||||
vae_cpu_offload: bool = False
|
||||
image_encoder_cpu_offload: bool = False
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Request model for merging a scene's clips into one video."""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class MergeSceneRequest(BaseModel):
|
||||
#: Completed jobs, in the order their videos should play.
|
||||
job_ids: list[str]
|
||||
#: Used to name the merged file.
|
||||
name: str = ""
|
||||
@@ -0,0 +1,8 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Request model for queueing jobs."""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class QueueJobsRequest(BaseModel):
|
||||
job_ids: list[str]
|
||||
@@ -0,0 +1,30 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Request model for cutting a job's video and color-grading it section by section."""
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class GradeSegment(BaseModel):
|
||||
"""One razor-cut section of the kept range, and its own color grade.
|
||||
|
||||
``end_seconds`` is seconds into the *kept* (already start/end-trimmed)
|
||||
range -- the same timeline the preview video plays -- not the original
|
||||
file's. Only the last section in a list may leave it ``None``, meaning
|
||||
"to the end"; sections must be given in increasing order and cover the
|
||||
whole kept range, with no gaps or reordering.
|
||||
"""
|
||||
end_seconds: float | None = None
|
||||
#: Added directly to 0-255 pixel values. 0 = unchanged.
|
||||
brightness: float = 0.0
|
||||
#: 1.0 = unchanged, 0 = flat gray, >1 = more contrast.
|
||||
contrast: float = 1.0
|
||||
#: 1.0 = unchanged, 0 = grayscale, >1 = more saturated.
|
||||
saturation: float = 1.0
|
||||
|
||||
|
||||
class TrimRequest(BaseModel):
|
||||
start_seconds: float = 0.0
|
||||
#: None keeps to the end of the video.
|
||||
end_seconds: float | None = None
|
||||
#: Defaults to one neutral section covering the whole kept range, i.e. a plain trim with no grading.
|
||||
segments: list[GradeSegment] = Field(default_factory=lambda: [GradeSegment()])
|
||||
@@ -18,6 +18,7 @@ import argparse
|
||||
import contextlib
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import signal
|
||||
import time
|
||||
@@ -32,10 +33,12 @@ from fastapi.responses import FileResponse
|
||||
|
||||
from fastvideo.registry import (get_registered_model_paths, get_registered_models_with_workloads)
|
||||
from fastvideo_studio.database import Database, _get_db_path
|
||||
from fastvideo_studio.frames import FrameError, last_clip_for_job, last_frame_for_job
|
||||
from fastvideo_studio.gpu import get_gpu_snapshot
|
||||
from fastvideo_studio.job_runner import JobRunner, JobStatus
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, SettingsUpdate, UpdateCaptionRequest,
|
||||
model_label)
|
||||
from fastvideo_studio.merge import MergeError, merge_jobs, merged_path
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, MergeSceneRequest, QueueJobsRequest,
|
||||
SettingsUpdate, TrimRequest, UpdateCaptionRequest, model_label)
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
@@ -116,7 +119,30 @@ def list_models(workload_type: str | None = None) -> list[dict[str, Any]]:
|
||||
return _available_models
|
||||
|
||||
|
||||
def _safe_upload_name(filename: str | None, ext: str) -> str:
|
||||
"""A filesystem-safe version of the client's filename, keeping it readable.
|
||||
|
||||
Uploads live under a per-file uuid directory, so the basename does not have
|
||||
to be unique -- only safe. Keeping the original name means the path stays
|
||||
self-describing wherever it travels: the database, job logs, and payloads
|
||||
copied back out to the API.
|
||||
"""
|
||||
stem = os.path.basename(filename or "").rsplit(".", 1)[0]
|
||||
stem = re.sub(r"[^A-Za-z0-9._-]+", "_", stem).strip("._-")
|
||||
return f"{stem[:80] or 'upload'}{ext}"
|
||||
|
||||
|
||||
def _upload_destination(ext: str, filename: str | None) -> str:
|
||||
"""<upload_dir>/<uuid4>/<safe original name><ext>"""
|
||||
directory = os.path.join(upload_dir, uuid.uuid4().hex)
|
||||
os.makedirs(directory, exist_ok=True)
|
||||
return os.path.join(directory, _safe_upload_name(filename, ext))
|
||||
|
||||
|
||||
ALLOWED_IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".bmp"}
|
||||
ALLOWED_VIDEO_EXTENSIONS = {".mp4", ".mov", ".mkv", ".webm", ".avi"}
|
||||
ALLOWED_AUDIO_EXTENSIONS = {".wav", ".mp3", ".flac", ".m4a", ".ogg"}
|
||||
ALLOWED_MEDIA_EXTENSIONS = (ALLOWED_IMAGE_EXTENSIONS | ALLOWED_VIDEO_EXTENSIONS | ALLOWED_AUDIO_EXTENSIONS)
|
||||
|
||||
|
||||
@app.post("/api/upload-image")
|
||||
@@ -136,8 +162,7 @@ async def upload_image(file: Annotated[UploadFile, File()], ) -> dict[str, str]:
|
||||
f"{', '.join(ALLOWED_IMAGE_EXTENSIONS)}"),
|
||||
)
|
||||
os.makedirs(upload_dir, exist_ok=True)
|
||||
unique_name = f"{uuid.uuid4().hex}{ext}"
|
||||
dest_path = os.path.join(upload_dir, unique_name)
|
||||
dest_path = _upload_destination(ext, file.filename)
|
||||
try:
|
||||
contents = await file.read()
|
||||
with open(dest_path, "wb") as f:
|
||||
@@ -150,6 +175,47 @@ async def upload_image(file: Annotated[UploadFile, File()], ) -> dict[str, str]:
|
||||
return {"path": os.path.abspath(dest_path)}
|
||||
|
||||
|
||||
@app.post("/api/upload-media")
|
||||
async def upload_media(file: Annotated[UploadFile, File()], ) -> dict[str, str]:
|
||||
"""Upload an image, video or audio file for Ref2VA references.
|
||||
|
||||
Returns the absolute path plus the media_type MiniMax-H3 expects, so the
|
||||
caller does not have to re-derive it from the extension.
|
||||
"""
|
||||
global upload_dir # noqa: PLW0603
|
||||
if not upload_dir:
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail="Upload directory not configured",
|
||||
)
|
||||
ext = Path(file.filename or "").suffix.lower()
|
||||
if ext not in ALLOWED_MEDIA_EXTENSIONS:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=(f"Invalid file type. Allowed: "
|
||||
f"{', '.join(sorted(ALLOWED_MEDIA_EXTENSIONS))}"),
|
||||
)
|
||||
if ext in ALLOWED_VIDEO_EXTENSIONS:
|
||||
media_type = "video"
|
||||
elif ext in ALLOWED_AUDIO_EXTENSIONS:
|
||||
media_type = "audio"
|
||||
else:
|
||||
media_type = "image"
|
||||
|
||||
os.makedirs(upload_dir, exist_ok=True)
|
||||
dest_path = _upload_destination(ext, file.filename)
|
||||
try:
|
||||
contents = await file.read()
|
||||
with open(dest_path, "wb") as f:
|
||||
f.write(contents)
|
||||
except OSError as e:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Failed to save upload: {e}",
|
||||
) from e
|
||||
return {"path": os.path.abspath(dest_path), "media_type": media_type}
|
||||
|
||||
|
||||
ALLOWED_VIDEO_EXTENSIONS = {".mp4", ".webm", ".avi", ".mov", ".mkv"}
|
||||
|
||||
|
||||
@@ -282,10 +348,13 @@ def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
job = job_runner.create_job(
|
||||
job_id=str(uuid.uuid4()),
|
||||
model_id=req.model_id,
|
||||
name=req.name or "",
|
||||
prompt=req.prompt,
|
||||
workload_type=req.workload_type or "t2v",
|
||||
job_type=job_type,
|
||||
image_path=req.image_path or "",
|
||||
last_image_path=req.last_image_path or "",
|
||||
references=req.references or [],
|
||||
data_path=data_path,
|
||||
max_train_steps=req.max_train_steps,
|
||||
train_batch_size=req.train_batch_size,
|
||||
@@ -304,6 +373,7 @@ def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
seed=req.seed,
|
||||
num_gpus=req.num_gpus,
|
||||
dit_cpu_offload=req.dit_cpu_offload,
|
||||
dit_layerwise_offload=req.dit_layerwise_offload,
|
||||
text_encoder_cpu_offload=req.text_encoder_cpu_offload,
|
||||
vae_cpu_offload=req.vae_cpu_offload,
|
||||
image_encoder_cpu_offload=req.image_encoder_cpu_offload,
|
||||
@@ -336,6 +406,28 @@ def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
return job.to_dict()
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/duplicate", status_code=201)
|
||||
def duplicate_job(job_id: str) -> dict[str, Any]:
|
||||
"""Create a new pending job with the same configuration as an existing one."""
|
||||
try:
|
||||
job = job_runner.duplicate_job(job_id, str(uuid.uuid4()))
|
||||
except ValueError as e:
|
||||
raise HTTPException(status_code=404, detail=str(e)) from e
|
||||
return job.to_dict()
|
||||
|
||||
|
||||
@app.patch("/api/jobs/{job_id}")
|
||||
def update_job(job_id: str, updates: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Edit a pending job's configuration. Started jobs cannot be edited."""
|
||||
try:
|
||||
job = job_runner.update_job_config(job_id, updates)
|
||||
except ValueError as e:
|
||||
detail = str(e)
|
||||
status = 404 if "not found" in detail else 400
|
||||
raise HTTPException(status_code=status, detail=detail) from e
|
||||
return job.to_dict()
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/start")
|
||||
def start_job(job_id: str) -> dict[str, Any]:
|
||||
"""Start (or restart) a pending / stopped / failed job."""
|
||||
@@ -349,6 +441,41 @@ def start_job(job_id: str) -> dict[str, Any]:
|
||||
) from e
|
||||
|
||||
|
||||
def _queue_error(e: ValueError) -> HTTPException:
|
||||
return HTTPException(status_code=404 if "not found" in str(e) else 409, detail=str(e))
|
||||
|
||||
|
||||
@app.post("/api/jobs/queue")
|
||||
def queue_jobs(req: QueueJobsRequest) -> list[dict[str, Any]]:
|
||||
"""Queue jobs, in the order given, to run as GPU slots free up.
|
||||
|
||||
A job that starts from another job's last frame waits for that job to
|
||||
complete. All-or-nothing: if any job can't be queued, none are.
|
||||
"""
|
||||
try:
|
||||
return [j.to_dict() for j in job_runner.enqueue_jobs(req.job_ids)]
|
||||
except ValueError as e:
|
||||
raise _queue_error(e) from e
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/queue")
|
||||
def queue_job(job_id: str) -> dict[str, Any]:
|
||||
"""Queue one job to run when a slot is free."""
|
||||
try:
|
||||
return job_runner.enqueue_job(job_id).to_dict()
|
||||
except ValueError as e:
|
||||
raise _queue_error(e) from e
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/dequeue")
|
||||
def dequeue_job(job_id: str) -> dict[str, Any]:
|
||||
"""Take a queued job out of the queue (back to pending)."""
|
||||
try:
|
||||
return job_runner.dequeue_job(job_id).to_dict()
|
||||
except ValueError as e:
|
||||
raise _queue_error(e) from e
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/stop")
|
||||
def stop_job(job_id: str) -> dict[str, Any]:
|
||||
"""Request a running job to stop.
|
||||
@@ -575,6 +702,113 @@ def get_video(job_id: str) -> FileResponse:
|
||||
return FileResponse(job.output_path, media_type=media_type)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/last-frame")
|
||||
def extract_last_frame(job_id: str) -> dict[str, str]:
|
||||
"""Save the last frame of a completed job's video and return its path.
|
||||
|
||||
The path is meant to be attached to another job as an image reference, so
|
||||
the next clip of a scene can start from where this one ended.
|
||||
"""
|
||||
job = job_runner.get_job(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
try:
|
||||
path = last_frame_for_job(job, upload_dir)
|
||||
except FrameError as e:
|
||||
if e.status_code >= 500:
|
||||
logger.warning("Last-frame extraction failed for job %s: %s", job_id, e.detail)
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return {"path": path, "media_type": "image"}
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/last-clip")
|
||||
def extract_last_clip(job_id: str) -> dict[str, str]:
|
||||
"""Save the trailing second of a completed job's video and return its path.
|
||||
|
||||
The path is meant to be attached to another job as a video reference, so
|
||||
the next clip continues with real motion instead of guessing it from a
|
||||
single still frame.
|
||||
"""
|
||||
job = job_runner.get_job(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
try:
|
||||
path = last_clip_for_job(job, upload_dir)
|
||||
except FrameError as e:
|
||||
if e.status_code >= 500:
|
||||
logger.warning("Last-clip extraction failed for job %s: %s", job_id, e.detail)
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return {"path": path, "media_type": "video"}
|
||||
|
||||
|
||||
def _trim_error(e: Exception) -> HTTPException:
|
||||
if isinstance(e, FrameError):
|
||||
return HTTPException(status_code=e.status_code, detail=e.detail)
|
||||
return HTTPException(status_code=404 if "not found" in str(e) else 400, detail=str(e))
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/trim")
|
||||
def trim_job(job_id: str, req: TrimRequest) -> dict[str, Any]:
|
||||
"""Cut a completed job's video to [start_seconds, end_seconds) and grade it section by section, in place.
|
||||
|
||||
The untouched original is kept, so this can be called again with a
|
||||
different range and/or sections, or undone with POST .../restore-video.
|
||||
"""
|
||||
try:
|
||||
segments = [s.model_dump() for s in req.segments]
|
||||
job = job_runner.trim_job(job_id, req.start_seconds, req.end_seconds, segments)
|
||||
except (ValueError, FrameError) as e:
|
||||
raise _trim_error(e) from e
|
||||
return job.to_dict()
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/restore-video")
|
||||
def restore_job_video(job_id: str) -> dict[str, Any]:
|
||||
"""Undo every trim on a job's video, back to what it originally generated."""
|
||||
try:
|
||||
job = job_runner.restore_job_video(job_id)
|
||||
except (ValueError, FrameError) as e:
|
||||
raise _trim_error(e) from e
|
||||
return job.to_dict()
|
||||
|
||||
|
||||
@app.post("/api/scenes/merge")
|
||||
def merge_scene(req: MergeSceneRequest) -> dict[str, Any]:
|
||||
"""Join the finished clips of a scene, in the order given, into one video.
|
||||
|
||||
The clips are copied, not re-encoded, so this takes about a second.
|
||||
"""
|
||||
jobs = []
|
||||
for job_id in req.job_ids:
|
||||
job = job_runner.get_job(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
jobs.append(job)
|
||||
try:
|
||||
result = merge_jobs(jobs, job_runner.output_dir, req.name)
|
||||
except MergeError as e:
|
||||
if e.status_code >= 500:
|
||||
logger.warning("Scene merge failed: %s", e.detail)
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
logger.info("Merged %d clips into %s", result.clips, result.path)
|
||||
return {
|
||||
"filename": result.filename,
|
||||
"url": f"/api/merged/{result.filename}",
|
||||
"clips": result.clips,
|
||||
"seconds": result.seconds,
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/merged/{filename}")
|
||||
def get_merged_video(filename: str) -> FileResponse:
|
||||
"""Stream a merged scene video."""
|
||||
try:
|
||||
path = merged_path(job_runner.output_dir, filename)
|
||||
except MergeError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return FileResponse(path, media_type="video/mp4", filename=filename)
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/download_log")
|
||||
def get_job_log_file(job_id: str) -> FileResponse:
|
||||
"""Download the log file for a job."""
|
||||
@@ -660,6 +894,14 @@ def main() -> None:
|
||||
help=("Directory for SQLite database (jobs + settings persistence) "
|
||||
f"(default: {default_data_dir})"),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-concurrent-jobs",
|
||||
type=int,
|
||||
default=1,
|
||||
help=("How many inference jobs may run at once (default: 1). Each gets its own "
|
||||
"long-lived worker that keeps its model loaded between jobs, so raising "
|
||||
"this loads one copy of the model per worker."),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose",
|
||||
action="store_true",
|
||||
@@ -685,6 +927,8 @@ def main() -> None:
|
||||
log_dir=log_dir,
|
||||
verbose=args.verbose,
|
||||
database=database,
|
||||
max_concurrent_jobs=args.max_concurrent_jobs,
|
||||
upload_dir=upload_dir,
|
||||
)
|
||||
|
||||
logger.info("Output directory: %s", output_dir)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,276 @@
|
||||
'use client';
|
||||
|
||||
import { AlertTriangle, Loader2, RefreshCw } from 'lucide-react';
|
||||
import * as React from 'react';
|
||||
|
||||
import CreateJobModal from '@/components/jobs/CreateJobModal';
|
||||
import { EDITABLE_STATUSES, SceneBoard } from '@/components/scenes/SceneBoard';
|
||||
import { MergedSceneDialog } from '@/components/scenes/MergedSceneDialog';
|
||||
import { SceneQueueDialog } from '@/components/scenes/SceneQueueDialog';
|
||||
import { TrimClipDialog } from '@/components/scenes/TrimClipDialog';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { Card } from '@/components/ui/card';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { toast } from 'sonner';
|
||||
import { dequeueJob, getJobsList, mergeScene, queueJobs, updateJob, type MergedScene } from '@/lib/api';
|
||||
import { attachedLastFrames, attachLastFrame, deferredLastFrameSource, detachLastFrame } from '@/lib/continuation';
|
||||
import type { SceneQueuePlan } from '@/lib/sceneQueue';
|
||||
import { buildScenes, type Scene, type SceneClip } from '@/lib/scenes';
|
||||
import type { Job } from '@/lib/types';
|
||||
|
||||
const POLL_MS = 3000;
|
||||
|
||||
export default function ScenesPage() {
|
||||
const [jobs, setJobs] = React.useState<Job[]>([]);
|
||||
const [isLoading, setIsLoading] = React.useState(true);
|
||||
const [error, setError] = React.useState<string | null>(null);
|
||||
const [reloadKey, setReloadKey] = React.useState(0);
|
||||
const [selectedKey, setSelectedKey] = React.useState<string | null>(null);
|
||||
const [openJob, setOpenJob] = React.useState<Job | null>(null);
|
||||
// which re-run of a clip number the user picked, by clip
|
||||
const [choices, setChoices] = React.useState<Record<string, string>>({});
|
||||
const [queueTarget, setQueueTarget] = React.useState<Scene | null>(null);
|
||||
const [merged, setMerged] = React.useState<(MergedScene & { title: string }) | null>(null);
|
||||
const [trimTarget, setTrimTarget] = React.useState<Job | null>(null);
|
||||
// Bumped per job id whenever its video is edited in place, to cache-bust every
|
||||
// <video> for that job (the dialog's own preview and the scene board's). Owned
|
||||
// here, not by the dialog, so it survives the dialog re-rendering with a fresh
|
||||
// job object after each edit -- a per-dialog counter got reset by that same
|
||||
// re-render before the browser ever saw the new URL.
|
||||
const [videoVersions, setVideoVersions] = React.useState<Record<string, number>>({});
|
||||
const bumpVideoVersion = (jobId: string) =>
|
||||
setVideoVersions((v) => ({ ...v, [jobId]: (v[jobId] ?? 0) + 1 }));
|
||||
|
||||
React.useEffect(() => {
|
||||
let cancelled = false;
|
||||
async function load() {
|
||||
try {
|
||||
const list = await getJobsList('inference');
|
||||
if (cancelled) return;
|
||||
setJobs(list);
|
||||
setError(null);
|
||||
} catch (e) {
|
||||
if (!cancelled) setError(e instanceof Error ? e.message : 'Failed to load jobs');
|
||||
} finally {
|
||||
if (!cancelled) setIsLoading(false);
|
||||
}
|
||||
}
|
||||
void load();
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [reloadKey]);
|
||||
|
||||
function reload() {
|
||||
setIsLoading(true);
|
||||
setReloadKey((k) => k + 1);
|
||||
}
|
||||
|
||||
// While anything is queued or running, keep the board current without the
|
||||
// loading state (which would blank or flicker it).
|
||||
const hasActive = jobs.some((j) => j.status === 'queued' || j.status === 'running');
|
||||
React.useEffect(() => {
|
||||
if (!hasActive) return;
|
||||
const interval = setInterval(() => {
|
||||
getJobsList('inference')
|
||||
.then(setJobs)
|
||||
.catch(() => {});
|
||||
}, POLL_MS);
|
||||
return () => clearInterval(interval);
|
||||
}, [hasActive]);
|
||||
|
||||
// Attach the previous clip's last frame to `clip` and rewrite its prompt to open on
|
||||
// it. The frame is a reference the server resolves when the clip runs, so the
|
||||
// previous clip doesn't have to be finished yet. Returns the picture label.
|
||||
async function attachPreviousFrame(clip: SceneClip, previous: SceneClip): Promise<string> {
|
||||
const result = attachLastFrame(
|
||||
clip.job.prompt,
|
||||
clip.job.references ?? [],
|
||||
deferredLastFrameSource(previous.job.id),
|
||||
);
|
||||
if (!result.ok) throw new Error(result.error);
|
||||
await updateJob(clip.job.id, { references: result.references, prompt: result.prompt });
|
||||
return result.label;
|
||||
}
|
||||
|
||||
async function useLastFrame(clip: SceneClip, previous: SceneClip) {
|
||||
try {
|
||||
const label = await attachPreviousFrame(clip, previous);
|
||||
toast.success(`Clip ${clip.position} now starts from the last frame of clip ${previous.position} (${label}).`);
|
||||
reload();
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't use the last frame.");
|
||||
}
|
||||
}
|
||||
|
||||
// Undo the above: take every attached last frame back off the clip.
|
||||
async function stopUsingLastFrame(clip: SceneClip) {
|
||||
try {
|
||||
let prompt = clip.job.prompt;
|
||||
let references = clip.job.references ?? [];
|
||||
// Newest first: only the most recently added image can be removed cleanly.
|
||||
for (const { source } of attachedLastFrames(references).reverse()) {
|
||||
const result = detachLastFrame(prompt, references, source);
|
||||
if (!result.ok) throw new Error(result.error);
|
||||
prompt = result.prompt;
|
||||
references = result.references;
|
||||
}
|
||||
await updateJob(clip.job.id, { references, prompt });
|
||||
toast.success(`Clip ${clip.position} no longer starts from a previous clip's last frame.`);
|
||||
reload();
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't remove the last frame.");
|
||||
}
|
||||
}
|
||||
|
||||
// Join the scene's clips, in the order shown (the take chosen for each), into one video.
|
||||
async function mergeSceneClips(scene: Scene) {
|
||||
try {
|
||||
const result = await mergeScene(scene.clips.map((c) => c.job.id), scene.title);
|
||||
setMerged({ ...result, title: scene.title });
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't merge the scene.");
|
||||
}
|
||||
}
|
||||
|
||||
async function queueClip(clip: SceneClip) {
|
||||
try {
|
||||
await queueJobs([clip.job.id]);
|
||||
toast.success(`Queued clip ${clip.position}.`);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't queue the clip.");
|
||||
}
|
||||
reload();
|
||||
}
|
||||
|
||||
async function dequeueClip(clip: SceneClip) {
|
||||
try {
|
||||
await dequeueJob(clip.job.id);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't remove the clip from the queue.");
|
||||
}
|
||||
reload();
|
||||
}
|
||||
|
||||
// Link clips to the frames before them (if asked), then queue them all in scene order.
|
||||
async function queueScene(plan: SceneQueuePlan) {
|
||||
try {
|
||||
for (const { clip, previous } of plan.chain) {
|
||||
await attachPreviousFrame(clip, previous).catch((e) => {
|
||||
throw new Error(`Clip ${clip.position}: ${e instanceof Error ? e.message : e}. Nothing was queued.`);
|
||||
});
|
||||
}
|
||||
await queueJobs(plan.queue.map((c) => c.job.id));
|
||||
const n = plan.queue.length;
|
||||
toast.success(`Queued ${n} clip${n === 1 ? '' : 's'}.`);
|
||||
setQueueTarget(null);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't queue the scene.");
|
||||
}
|
||||
reload();
|
||||
}
|
||||
|
||||
const scenes = React.useMemo(() => buildScenes(jobs, choices), [jobs, choices]);
|
||||
const scene = scenes.find((s) => s.key === selectedKey) ?? scenes[0];
|
||||
|
||||
return (
|
||||
<div className="mx-auto w-full max-w-[1200px] px-4 pb-12 pt-4">
|
||||
<Card className="p-6">
|
||||
<div className="mb-1 flex flex-wrap items-center gap-3">
|
||||
<h2 className="text-2xl font-semibold text-foreground">Scenes</h2>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="ml-auto gap-1.5"
|
||||
onClick={reload}
|
||||
disabled={isLoading}
|
||||
>
|
||||
<RefreshCw className={isLoading ? 'size-3.5 animate-spin' : 'size-3.5'} aria-hidden />
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
<p className="mb-6 text-sm text-muted-foreground">
|
||||
Every shot across your jobs, in order. Jobs named with a shared stem and a number, like{' '}
|
||||
<code className="font-mono text-xs">wolf-lunch-clip-01</code>,{' '}
|
||||
<code className="font-mono text-xs">wolf-lunch-clip-02</code>, are grouped into one scene.
|
||||
</p>
|
||||
|
||||
{isLoading && jobs.length === 0 ? (
|
||||
<div className="flex items-center gap-3 p-8 text-muted-foreground">
|
||||
<Loader2 className="h-6 w-6 animate-spin text-primary" />
|
||||
<span>Loading scenes…</span>
|
||||
</div>
|
||||
) : error && jobs.length === 0 ? (
|
||||
<div role="alert" className="flex flex-col items-center gap-3 py-8 text-center">
|
||||
<AlertTriangle className="size-6 text-destructive" aria-hidden />
|
||||
<p className="max-w-md text-sm text-muted-foreground">{error}</p>
|
||||
<Button type="button" variant="outline" onClick={reload}>
|
||||
Try Again
|
||||
</Button>
|
||||
</div>
|
||||
) : !scene ? (
|
||||
<p className="py-8 text-center text-muted-foreground">
|
||||
No inference jobs yet. Create some and they'll show up here.
|
||||
</p>
|
||||
) : (
|
||||
<div className="flex flex-col gap-6">
|
||||
{scenes.length > 1 && (
|
||||
<label className="flex max-w-md flex-col gap-1 text-xs text-muted-foreground">
|
||||
Scene
|
||||
<NativeSelect value={scene.key} onChange={(e) => setSelectedKey(e.target.value)}>
|
||||
{scenes.map((s) => (
|
||||
<option key={s.key} value={s.key}>
|
||||
{s.title} ({s.clips.length} clip{s.clips.length === 1 ? '' : 's'})
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</label>
|
||||
)}
|
||||
<SceneBoard
|
||||
scene={scene}
|
||||
onOpenJob={setOpenJob}
|
||||
onUseLastFrame={useLastFrame}
|
||||
onDetachLastFrame={stopUsingLastFrame}
|
||||
onQueueScene={setQueueTarget}
|
||||
onMergeScene={mergeSceneClips}
|
||||
onQueueClip={queueClip}
|
||||
onDequeueClip={dequeueClip}
|
||||
onTrimClip={(clip) => setTrimTarget(clip.job)}
|
||||
onChooseTake={(takeKey, jobId) => setChoices((c) => ({ ...c, [takeKey]: jobId }))}
|
||||
videoVersionFor={(jobId) => videoVersions[jobId] ?? 0}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</Card>
|
||||
|
||||
<MergedSceneDialog merged={merged} onClose={() => setMerged(null)} />
|
||||
<SceneQueueDialog scene={queueTarget} onConfirm={queueScene} onClose={() => setQueueTarget(null)} />
|
||||
<TrimClipDialog
|
||||
clip={trimTarget}
|
||||
videoVersion={trimTarget ? (videoVersions[trimTarget.id] ?? 0) : 0}
|
||||
onChanged={(job) => {
|
||||
setTrimTarget(job);
|
||||
bumpVideoVersion(job.id);
|
||||
getJobsList('inference').then(setJobs).catch(() => {});
|
||||
}}
|
||||
onClose={() => setTrimTarget(null)}
|
||||
/>
|
||||
|
||||
{openJob && (
|
||||
<CreateJobModal
|
||||
isOpen
|
||||
readOnly={!EDITABLE_STATUSES.includes(openJob.status)}
|
||||
editingJob={openJob}
|
||||
jobType={(openJob.job_type ?? 'inference') as never}
|
||||
workloadType={openJob.workload_type ?? 't2v'}
|
||||
onClose={() => setOpenJob(null)}
|
||||
onSuccess={() => {
|
||||
setOpenJob(null);
|
||||
reload();
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -1,19 +1,29 @@
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { toast } from 'sonner';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import CreateJobButton from './CreateJobButton';
|
||||
|
||||
vi.mock('sonner', () => ({ toast: { error: vi.fn(), warning: vi.fn() } }));
|
||||
|
||||
vi.mock('./CreateJobModal', () => ({
|
||||
default: ({
|
||||
isOpen,
|
||||
workloadType,
|
||||
prefillJob,
|
||||
}: {
|
||||
isOpen: boolean;
|
||||
workloadType: string;
|
||||
prefillJob?: { name?: string; prompt: string } | null;
|
||||
}) =>
|
||||
isOpen ? (
|
||||
<div role="dialog" data-workload-type={workloadType}>
|
||||
<div
|
||||
role="dialog"
|
||||
data-workload-type={workloadType}
|
||||
data-prefill-name={prefillJob?.name ?? ''}
|
||||
data-prefill-prompt={prefillJob?.prompt ?? ''}
|
||||
>
|
||||
Create job form
|
||||
</div>
|
||||
) : null,
|
||||
@@ -27,7 +37,11 @@ describe('CreateJobButton', () => {
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(screen.getByRole('menuitem', { name: /I2V/i }));
|
||||
|
||||
expect(screen.getByRole('dialog')).toHaveAttribute(
|
||||
// The dialog opens a frame after the menu selection (see CreateJobButton's
|
||||
// openModal: deferred via requestAnimationFrame to dodge a Radix
|
||||
// DropdownMenu + Dialog pointer-events race), so wait for it rather than
|
||||
// asserting synchronously.
|
||||
expect(await screen.findByRole('dialog')).toHaveAttribute(
|
||||
'data-workload-type',
|
||||
'i2v',
|
||||
);
|
||||
@@ -45,9 +59,75 @@ describe('CreateJobButton', () => {
|
||||
expect(firstItem).toHaveFocus();
|
||||
await user.keyboard('{Enter}');
|
||||
|
||||
expect(screen.getByRole('dialog')).toHaveAttribute(
|
||||
expect(await screen.findByRole('dialog')).toHaveAttribute(
|
||||
'data-workload-type',
|
||||
't2v',
|
||||
);
|
||||
});
|
||||
|
||||
describe('importing a job file', () => {
|
||||
const file = (contents: string, name = 'job.json') =>
|
||||
new File([contents], name, { type: 'application/json' });
|
||||
|
||||
it('opens the form for the file\'s workload with the job pre-filled', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<CreateJobButton jobType="inference" />);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(screen.getByRole('menuitem', { name: /Import job JSON/ }));
|
||||
await user.upload(
|
||||
screen.getByLabelText('Import job JSON file'),
|
||||
file(
|
||||
JSON.stringify({
|
||||
model_id: 'MiniMaxAI/MiniMax-H3',
|
||||
name: 'from-file',
|
||||
workload_type: 'i2v',
|
||||
prompt: ['summary:', 'Hi.'],
|
||||
}),
|
||||
),
|
||||
);
|
||||
|
||||
const dialog = await screen.findByRole('dialog');
|
||||
expect(dialog).toHaveAttribute('data-workload-type', 'i2v');
|
||||
expect(dialog).toHaveAttribute('data-prefill-name', 'from-file');
|
||||
expect(dialog).toHaveAttribute('data-prefill-prompt', 'summary:\nHi.');
|
||||
});
|
||||
|
||||
it('reports an unreadable file instead of opening the form', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<CreateJobButton jobType="inference" />);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(screen.getByRole('menuitem', { name: /Import job JSON/ }));
|
||||
await user.upload(screen.getByLabelText('Import job JSON file'), file('not json', 'bad.json'));
|
||||
|
||||
await vi.waitFor(() => expect(toast.error).toHaveBeenCalled());
|
||||
expect(vi.mocked(toast.error).mock.calls.at(-1)?.[0]).toContain("bad.json");
|
||||
expect(screen.queryByRole('dialog')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('a normal open after an import starts clean', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<CreateJobButton jobType="inference" />);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(screen.getByRole('menuitem', { name: /Import job JSON/ }));
|
||||
await user.upload(
|
||||
screen.getByLabelText('Import job JSON file'),
|
||||
file(JSON.stringify({ model_id: 'm', name: 'from-file', prompt: 'p' })),
|
||||
);
|
||||
expect(await screen.findByRole('dialog')).toHaveAttribute('data-prefill-name', 'from-file');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(await screen.findByRole('menuitem', { name: /T2V/i }));
|
||||
await vi.waitFor(() => expect(screen.getByRole('dialog')).toHaveAttribute('data-prefill-name', ''));
|
||||
});
|
||||
|
||||
it('is not offered on the training pages', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<CreateJobButton jobType="finetuning" />);
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
expect(screen.queryByRole('menuitem', { name: /Import job JSON/ })).not.toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,7 +6,10 @@ import * as DropdownMenu from '@radix-ui/react-dropdown-menu';
|
||||
|
||||
import CreateJobModal from '@/components/jobs/CreateJobModal';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { toast } from 'sonner';
|
||||
import { WORKLOAD_OPTIONS } from '@/lib/jobConfig';
|
||||
import { parseJobJson } from '@/lib/jobJson';
|
||||
import type { JobLike } from '@/lib/jobToFields';
|
||||
import type { JobType } from '@/lib/types';
|
||||
import { triggerRefresh } from '@/stores/jobsRefresh';
|
||||
|
||||
@@ -22,9 +25,32 @@ export default function CreateJobButton({ jobType }: CreateJobButtonProps) {
|
||||
options[0]?.type ?? 't2v',
|
||||
);
|
||||
|
||||
function openModal(type: string) {
|
||||
// An imported job file pre-fills the form; a normal open starts clean.
|
||||
const [prefillJob, setPrefillJob] = React.useState<JobLike | null>(null);
|
||||
const fileInputRef = React.useRef<HTMLInputElement>(null);
|
||||
|
||||
async function handleImportFile(e: React.ChangeEvent<HTMLInputElement>) {
|
||||
const file = e.target.files?.[0];
|
||||
e.target.value = ''; // allow re-picking the same file
|
||||
if (!file) return;
|
||||
const result = parseJobJson(await file.text());
|
||||
if (!result.ok) {
|
||||
toast.error(`Couldn't import ${file.name}: ${result.error}`);
|
||||
return;
|
||||
}
|
||||
openModal(result.workloadType, result.job);
|
||||
}
|
||||
|
||||
function openModal(type: string, prefill: JobLike | null = null) {
|
||||
setWorkloadType(type);
|
||||
setModalOpen(true);
|
||||
setPrefillJob(prefill);
|
||||
// Opening the Dialog synchronously inside the DropdownMenu.Item's
|
||||
// onSelect races the two Radix layers' pointer-events body lock/unlock:
|
||||
// the dropdown's own close cleanup can lose that race, leaving <body>
|
||||
// stuck with `pointer-events: none` after this dialog later closes --
|
||||
// silently blocking every click, including this same button. Deferring
|
||||
// to the next frame lets the dropdown finish closing first.
|
||||
requestAnimationFrame(() => setModalOpen(true));
|
||||
}
|
||||
|
||||
function handleSuccess() {
|
||||
@@ -60,15 +86,39 @@ export default function CreateJobButton({ jobType }: CreateJobButtonProps) {
|
||||
</span>
|
||||
</DropdownMenu.Item>
|
||||
))}
|
||||
{jobType === 'inference' && (
|
||||
<>
|
||||
<DropdownMenu.Separator className="my-1 h-px bg-border" />
|
||||
<DropdownMenu.Item
|
||||
onSelect={() => fileInputRef.current?.click()}
|
||||
className="flex min-h-11 cursor-pointer select-none flex-col justify-center px-4 py-2 text-left text-sm font-medium outline-none data-[highlighted]:bg-secondary"
|
||||
>
|
||||
Import job JSON…
|
||||
<span className="mt-0.5 block text-xs font-normal text-muted-foreground">
|
||||
Fill the form from a job file
|
||||
</span>
|
||||
</DropdownMenu.Item>
|
||||
</>
|
||||
)}
|
||||
</DropdownMenu.Content>
|
||||
</DropdownMenu.Portal>
|
||||
</DropdownMenu.Root>
|
||||
<input
|
||||
ref={fileInputRef}
|
||||
type="file"
|
||||
accept=".json,application/json"
|
||||
aria-label="Import job JSON file"
|
||||
tabIndex={-1}
|
||||
className="sr-only"
|
||||
onChange={handleImportFile}
|
||||
/>
|
||||
<CreateJobModal
|
||||
isOpen={modalOpen}
|
||||
onClose={() => setModalOpen(false)}
|
||||
onSuccess={handleSuccess}
|
||||
jobType={jobType}
|
||||
workloadType={workloadType}
|
||||
prefillJob={prefillJob}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
|
||||
@@ -4,10 +4,13 @@ import userEvent from '@testing-library/user-event';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import CreateJobModal from './CreateJobModal';
|
||||
import { toast } from 'sonner';
|
||||
import { createJob, getDatasets, getModels, uploadImage } from '@/lib/api';
|
||||
import { defaultOptionsStore } from '@/stores/defaultOptions';
|
||||
import { DEFAULT_OPTIONS } from '@/lib/defaultOptions';
|
||||
|
||||
vi.mock('sonner', () => ({ toast: { warning: vi.fn(), error: vi.fn(), success: vi.fn() } }));
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
createJob: vi.fn(),
|
||||
getModels: vi.fn(),
|
||||
@@ -206,4 +209,179 @@ describe('CreateJobModal', () => {
|
||||
|
||||
await waitFor(() => expect(onSuccess).toHaveBeenCalledTimes(1));
|
||||
});
|
||||
|
||||
it('shows the H3 speech-tag console only for an H3 model and inserts at the prompt cursor', async () => {
|
||||
vi.mocked(getModels).mockResolvedValue([
|
||||
...MODELS,
|
||||
{ id: 'minimax-h3/i2v', label: 'MiniMax H3', type: 'i2v' },
|
||||
]);
|
||||
const user = userEvent.setup();
|
||||
renderModal({ workloadType: 'i2v' });
|
||||
|
||||
await screen.findByRole('option', { name: 'Wan T2V (wan/t2v-1.3b)' });
|
||||
expect(screen.queryByText(/Speech tags/)).not.toBeInTheDocument();
|
||||
|
||||
await user.selectOptions(
|
||||
screen.getByLabelText('Model'),
|
||||
'minimax-h3/i2v',
|
||||
);
|
||||
expect(await screen.findByText(/Speech tags/)).toBeInTheDocument();
|
||||
|
||||
const promptField = screen.getByLabelText('Prompt') as HTMLTextAreaElement;
|
||||
await user.type(promptField, 'Hello there.');
|
||||
promptField.setSelectionRange(5, 5);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: '<pause>' }));
|
||||
|
||||
expect(promptField.value).toBe('Hello <pause> there.');
|
||||
// Focus and cursor return to the prompt field so tags can be chained.
|
||||
await waitFor(() => expect(promptField).toHaveFocus());
|
||||
});
|
||||
});
|
||||
|
||||
const H3 = { id: 'minimax-h3/i2v', label: 'MiniMax H3', type: 'i2v' };
|
||||
|
||||
const BASE_PROMPT = [
|
||||
'For the target video, <Picture 1> is fully referenced.',
|
||||
'',
|
||||
'integrated_multimodal_description: [Shot 1] A close-up. He (S1) says: <d>[English] Sorry.</d> <cutoff>',
|
||||
'',
|
||||
'overall_soundscape: Room tone.',
|
||||
'',
|
||||
'non_diegetic_music: N/A',
|
||||
].join('\n');
|
||||
|
||||
describe('CreateJobModal with a prefilled job', () => {
|
||||
beforeEach(() => {
|
||||
vi.mocked(getModels).mockResolvedValue([...MODELS, H3]);
|
||||
vi.mocked(toast.warning).mockClear();
|
||||
});
|
||||
|
||||
it('fills the form from the job and creates a new job on submit', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onSuccess } = renderModal({
|
||||
workloadType: 'i2v',
|
||||
prefillJob: {
|
||||
id: 'imported-1',
|
||||
model_id: H3.id,
|
||||
name: 'from-file',
|
||||
prompt: BASE_PROMPT,
|
||||
workload_type: 'i2v',
|
||||
references: [{ source: '/data/inputs/face.png', media_type: 'image' }],
|
||||
num_frames: 243,
|
||||
seed: 7,
|
||||
},
|
||||
});
|
||||
|
||||
await screen.findByRole('option', { name: /MiniMax H3/ });
|
||||
await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(H3.id));
|
||||
expect(screen.getByLabelText('Name (optional)')).toHaveValue('from-file');
|
||||
expect(screen.getByLabelText('Prompt')).toHaveValue(BASE_PROMPT);
|
||||
expect(screen.getByText('face.png')).toBeInTheDocument();
|
||||
expect(screen.getByText('New Inference Job (I2V)')).toBeInTheDocument();
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await waitFor(() => expect(onSuccess).toHaveBeenCalledTimes(1));
|
||||
const payload = vi.mocked(createJob).mock.calls[0][0] as unknown as Record<string, unknown>;
|
||||
expect(payload).toMatchObject({
|
||||
model_id: H3.id,
|
||||
name: 'from-file',
|
||||
prompt: BASE_PROMPT,
|
||||
num_frames: 243,
|
||||
seed: 7,
|
||||
references: [{ source: '/data/inputs/face.png', media_type: 'image' }],
|
||||
});
|
||||
});
|
||||
|
||||
it("warns when the file's model isn't offered for the workload", async () => {
|
||||
renderModal({
|
||||
workloadType: 't2v',
|
||||
prefillJob: { id: 'imported-2', model_id: 'not/a-real-model', prompt: 'p' },
|
||||
});
|
||||
await waitFor(() => expect(toast.warning).toHaveBeenCalled());
|
||||
expect(vi.mocked(toast.warning).mock.calls[0][0]).toContain('not/a-real-model');
|
||||
expect(screen.getByLabelText('Model')).toHaveValue(MODELS[0].id);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CreateJobModal shot list on a raw prompt', () => {
|
||||
beforeEach(() => {
|
||||
vi.mocked(getModels).mockResolvedValue([...MODELS, H3]);
|
||||
});
|
||||
|
||||
async function openShotList(user: ReturnType<typeof userEvent.setup>) {
|
||||
await user.click(screen.getByRole('button', { name: 'Open shot list' }));
|
||||
return screen.findByRole('dialog', { name: 'Shot list' });
|
||||
}
|
||||
|
||||
it('edits dialogue inside integrated_multimodal_description and leaves the rest of the prompt alone', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderModal({
|
||||
workloadType: 'i2v',
|
||||
prefillJob: {
|
||||
id: 'imported-3',
|
||||
model_id: H3.id,
|
||||
prompt: BASE_PROMPT,
|
||||
references: [{ source: '/data/face.png', media_type: 'image' }],
|
||||
},
|
||||
});
|
||||
await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(H3.id));
|
||||
|
||||
const dialog = await openShotList(user);
|
||||
expect(within(dialog).getAllByText(/integrated_multimodal_description/).length).toBeGreaterThan(0);
|
||||
expect(within(dialog).queryByText('Retention analysis')).not.toBeInTheDocument();
|
||||
|
||||
const line = within(dialog).getByPlaceholderText('What the speaker says') as HTMLTextAreaElement;
|
||||
expect(line.value).toBe('Sorry.');
|
||||
await user.type(line, ' Really.');
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Apply' }));
|
||||
|
||||
await waitFor(() =>
|
||||
expect(screen.getByLabelText('Prompt')).toHaveValue(BASE_PROMPT.replace('Sorry.', 'Sorry. Really.')),
|
||||
);
|
||||
});
|
||||
|
||||
it('treats a plain prompt as the shot text and writes shot markers back over it', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderModal({ workloadType: 't2v' });
|
||||
await screen.findByRole('option', { name: /MiniMax H3/ });
|
||||
await user.selectOptions(screen.getByLabelText('Model'), H3.id);
|
||||
await user.type(screen.getByLabelText('Prompt'), 'A dock at dawn.');
|
||||
|
||||
const dialog = await openShotList(user);
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Apply' }));
|
||||
await waitFor(() => expect(screen.getByLabelText('Prompt')).toHaveValue('[Shot 1] A dock at dawn.'));
|
||||
});
|
||||
|
||||
it('keeps retention analysis in step when the raw prompt has a detailed_description and retention_analysis', async () => {
|
||||
const user = userEvent.setup();
|
||||
const sections = [
|
||||
'subject_definitions:',
|
||||
'<Subject 1> is the courier.',
|
||||
'',
|
||||
'retention_analysis:',
|
||||
'<Subject 1> (appears in [Shot 1]): fully_preserved - jacket.',
|
||||
'',
|
||||
'detailed_description:',
|
||||
'[Shot 1] Wide shot. A rooftop with <Subject 1>.',
|
||||
].join('\n');
|
||||
renderModal({ workloadType: 't2v', prefillJob: { id: 'imported-4', model_id: H3.id, prompt: sections } });
|
||||
await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(H3.id));
|
||||
|
||||
const dialog = await openShotList(user);
|
||||
expect(within(dialog).getByText('Retention analysis')).toBeInTheDocument();
|
||||
await user.click(within(dialog).getAllByRole('button', { name: /Add shot/ })[0]);
|
||||
await user.click(within(dialog).getAllByRole('button', { name: '<Subject 1>' })[1]);
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Apply' }));
|
||||
|
||||
const out = (screen.getByLabelText('Prompt') as HTMLTextAreaElement).value;
|
||||
expect(out).toContain('<Subject 1> (appears in [Shot 1], [Shot 2]): fully_preserved - jacket.');
|
||||
expect(out).toContain('subject_definitions:\n<Subject 1> is the courier.');
|
||||
});
|
||||
|
||||
it('is not offered for a non-H3 model', async () => {
|
||||
renderModal({ workloadType: 't2v' });
|
||||
await screen.findByRole('option', { name: 'Wan T2V (wan/t2v-1.3b)' });
|
||||
expect(screen.queryByRole('button', { name: 'Open shot list' })).not.toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -22,15 +22,45 @@ import { useStore } from '@/hooks/useStore';
|
||||
import { defaultOptionsStore } from '@/stores/defaultOptions';
|
||||
import {
|
||||
createJob,
|
||||
updateJob,
|
||||
getDatasets,
|
||||
getModels,
|
||||
uploadImage,
|
||||
uploadMedia,
|
||||
type CreateJobRequest,
|
||||
type Model,
|
||||
} from '@/lib/api';
|
||||
import { getDefaultModelForWorkload } from '@/lib/defaultOptions';
|
||||
import { WORKLOAD_OPTIONS } from '@/lib/jobConfig';
|
||||
import type { JobType } from '@/lib/types';
|
||||
import {
|
||||
H3_MAX_REFERENCES,
|
||||
labelReferences,
|
||||
referencePromptSeed,
|
||||
validateReferences,
|
||||
type H3Reference,
|
||||
} from '@/lib/h3References';
|
||||
import {
|
||||
EMPTY_H3_PROMPT_FIELDS,
|
||||
H3_PROMPT_SECTIONS,
|
||||
H3_SECTION_HINTS,
|
||||
H3_SECTION_LABELS,
|
||||
isEmptyPromptFields,
|
||||
parseH3Prompt,
|
||||
serializeH3Prompt,
|
||||
type H3PromptFields,
|
||||
type H3PromptSection,
|
||||
} from '@/lib/h3Prompt';
|
||||
import { applyH3Tag, type H3Tag } from '@/lib/h3Tags';
|
||||
import { H3TagConsole } from '@/components/jobs/H3TagConsole';
|
||||
import { H3DialogueEditor } from '@/components/jobs/H3DialogueEditor';
|
||||
import { H3ShotListEditor } from '@/components/jobs/H3ShotListEditor';
|
||||
import { jobToFormFields, type JobLike } from '@/lib/jobToFields';
|
||||
import { readField, shotTarget, writeField, writeShotTarget } from '@/lib/h3PromptFields';
|
||||
import { toast } from 'sonner';
|
||||
|
||||
/** Prompt fields the speech-tag console can insert into. */
|
||||
type TagTarget = 'prompt' | H3PromptSection;
|
||||
|
||||
export interface CreateJobModalProps {
|
||||
isOpen: boolean;
|
||||
@@ -38,6 +68,12 @@ export interface CreateJobModalProps {
|
||||
onSuccess: () => void;
|
||||
jobType: JobType;
|
||||
workloadType: string;
|
||||
/** When set, the modal edits this pending job instead of creating a new one. */
|
||||
editingJob?: JobLike | null;
|
||||
/** Seed the form from this job (an imported job file) but still create a new job on submit. */
|
||||
prefillJob?: JobLike | null;
|
||||
/** Show the configuration without allowing changes (started/finished jobs). */
|
||||
readOnly?: boolean;
|
||||
}
|
||||
|
||||
export default function CreateJobModal({
|
||||
@@ -46,6 +82,9 @@ export default function CreateJobModal({
|
||||
onSuccess,
|
||||
jobType,
|
||||
workloadType,
|
||||
editingJob,
|
||||
prefillJob,
|
||||
readOnly = false,
|
||||
}: CreateJobModalProps) {
|
||||
const { options } = useStore(defaultOptionsStore);
|
||||
|
||||
@@ -55,8 +94,22 @@ export default function CreateJobModal({
|
||||
|
||||
const [models, setModels] = React.useState<Model[]>([]);
|
||||
const [modelId, setModelId] = React.useState('');
|
||||
const [name, setName] = React.useState('');
|
||||
const [prompt, setPrompt] = React.useState('');
|
||||
const [imagePath, setImagePath] = React.useState('');
|
||||
const [lastImagePath, setLastImagePath] = React.useState('');
|
||||
const [references, setReferences] = React.useState<H3Reference[]>([]);
|
||||
const [isUploadingReference, setIsUploadingReference] = React.useState(false);
|
||||
const [referenceError, setReferenceError] = React.useState<string | null>(null);
|
||||
const [promptFields, setPromptFields] = React.useState<H3PromptFields>(
|
||||
EMPTY_H3_PROMPT_FIELDS,
|
||||
);
|
||||
const [useGuidedPrompt, setUseGuidedPrompt] = React.useState(true);
|
||||
const [lastImageFileName, setLastImageFileName] = React.useState('');
|
||||
const [isUploadingLastImage, setIsUploadingLastImage] = React.useState(false);
|
||||
const [lastImageUploadError, setLastImageUploadError] = React.useState<
|
||||
string | null
|
||||
>(null);
|
||||
const [imageFileName, setImageFileName] = React.useState('');
|
||||
const [isUploadingImage, setIsUploadingImage] = React.useState(false);
|
||||
const [negativePrompt, setNegativePrompt] = React.useState('');
|
||||
@@ -70,12 +123,124 @@ export default function CreateJobModal({
|
||||
const [seed, setSeed] = React.useState(1024);
|
||||
const [numGpus, setNumGpus] = React.useState(1);
|
||||
const [ditCpuOffload, setDitCpuOffload] = React.useState(false);
|
||||
const [ditLayerwiseOffload, setDitLayerwiseOffload] = React.useState(false);
|
||||
const [textEncoderCpuOffload, setTextEncoderCpuOffload] =
|
||||
React.useState(false);
|
||||
const [vaeCpuOffload, setVaeCpuOffload] = React.useState(false);
|
||||
const [imageEncoderCpuOffload, setImageEncoderCpuOffload] =
|
||||
React.useState(false);
|
||||
const [useFsdpInference, setUseFsdpInference] = React.useState(false);
|
||||
|
||||
// H3 is the only registered model with an end frame or references.
|
||||
const supportsLastImage = modelId.toLowerCase().includes('minimax-h3');
|
||||
const usingReferences = supportsLastImage && references.length > 0;
|
||||
// Same check, named for where it gates H3-only prompt features (speech tags)
|
||||
// rather than the end-frame/reference upload UI.
|
||||
const isH3Model = supportsLastImage;
|
||||
|
||||
// Which prompt field the speech-tag console inserts into: whichever one is
|
||||
// actually visible, tracked via each field's onFocus below.
|
||||
const [activeTagField, setActiveTagField] = React.useState<TagTarget>('prompt');
|
||||
const tagFieldRefs = React.useRef<Partial<Record<TagTarget, HTMLTextAreaElement | null>>>({});
|
||||
const showGuidedFields = usingReferences && useGuidedPrompt;
|
||||
React.useEffect(() => {
|
||||
setActiveTagField(showGuidedFields ? 'detailed_description' : 'prompt');
|
||||
}, [showGuidedFields]);
|
||||
|
||||
function getTagFieldValue(field: TagTarget): string {
|
||||
return field === 'prompt' ? prompt : promptFields[field];
|
||||
}
|
||||
function setTagFieldValue(field: TagTarget, value: string) {
|
||||
if (field === 'prompt') setPrompt(value);
|
||||
else setPromptField(field, value);
|
||||
}
|
||||
|
||||
function insertH3Tag(tag: H3Tag) {
|
||||
const field = activeTagField;
|
||||
const el = tagFieldRefs.current[field] ?? null;
|
||||
const currentValue = getTagFieldValue(field);
|
||||
const start = el?.selectionStart ?? currentValue.length;
|
||||
const end = el?.selectionEnd ?? currentValue.length;
|
||||
const result = applyH3Tag(currentValue, start, end, tag);
|
||||
setTagFieldValue(field, result.value);
|
||||
// The textarea re-renders with the new value before this runs; restore
|
||||
// focus and the cursor position so tags can be chained without reaching
|
||||
// for the mouse each time.
|
||||
requestAnimationFrame(() => {
|
||||
const node = tagFieldRefs.current[field];
|
||||
if (node) {
|
||||
node.focus();
|
||||
node.setSelectionRange(result.start, result.end);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Full-screen dialogue editor: opens with a copy of whichever field the
|
||||
// speech-tag console would currently insert into, and on Apply overwrites
|
||||
// that same field wholesale (not just an insertion at the cursor).
|
||||
const [dialogueEditorOpen, setDialogueEditorOpen] = React.useState(false);
|
||||
const dialogueEditorField = activeTagField;
|
||||
|
||||
// Shot list: writes into detailed_description AND retention_analysis
|
||||
// together, so unlike the speech-tag console/dialogue editor it isn't
|
||||
// scoped to "whichever field is focused" -- it's specific to those two.
|
||||
const [shotListOpen, setShotListOpen] = React.useState(false);
|
||||
// 'sections' edits the guided Detailed description; 'prompt' edits the shots
|
||||
// inside the raw prompt (base format, or a plain prompt).
|
||||
const [shotListMode, setShotListMode] = React.useState<'sections' | 'prompt'>('sections');
|
||||
function applyShotList(detailedDescription: string, retentionAnalysis: string) {
|
||||
setPromptFields((prev) => ({
|
||||
...prev,
|
||||
detailed_description: detailedDescription,
|
||||
retention_analysis: retentionAnalysis,
|
||||
}));
|
||||
}
|
||||
|
||||
const promptShots = React.useMemo(() => shotTarget(prompt), [prompt]);
|
||||
const promptSubjects = React.useMemo(() => readField(prompt, 'subject_definitions') ?? '', [prompt]);
|
||||
const promptRetention = React.useMemo(() => readField(prompt, 'retention_analysis'), [prompt]);
|
||||
const promptSyncsRetention = promptShots.field === 'detailed_description' && promptRetention !== null;
|
||||
|
||||
function applyShotListToPrompt(shotText: string, retentionAnalysis: string) {
|
||||
let next = writeShotTarget(prompt, promptShots, shotText);
|
||||
if (promptSyncsRetention) next = writeField(next, 'retention_analysis', retentionAnalysis);
|
||||
setPrompt(next);
|
||||
}
|
||||
|
||||
// JobCard re-renders on every job-list poll, so `editingJob` is a fresh
|
||||
// object each time. Effects must depend on these, never on the object.
|
||||
const seedJob = editingJob ?? prefillJob ?? null;
|
||||
const seedJobId = seedJob?.id ?? null;
|
||||
const seedJobModelId = seedJob?.model_id ?? null;
|
||||
const isPrefill = !editingJob && !!prefillJob;
|
||||
|
||||
// Layerwise offload and FSDP compete for the DiT weights and FastVideoArgs
|
||||
// silently picks a winner (fastvideo_args.py:859); resolve it visibly here.
|
||||
// dit_cpu_offload is deliberately not interlocked -- it is a modifier, not a
|
||||
// competing strategy.
|
||||
const handleDitLayerwiseOffloadChange = React.useCallback((next: boolean) => {
|
||||
setDitLayerwiseOffload(next);
|
||||
if (next) {
|
||||
setUseFsdpInference(false);
|
||||
}
|
||||
}, []);
|
||||
|
||||
const handleUseFsdpInferenceChange = React.useCallback((next: boolean) => {
|
||||
setUseFsdpInference(next);
|
||||
if (next) {
|
||||
setDitLayerwiseOffload(false);
|
||||
}
|
||||
}, []);
|
||||
|
||||
const handleNumGpusChange = React.useCallback((next: number) => {
|
||||
setNumGpus(next);
|
||||
if (next > 1) {
|
||||
// Dropping back to one GPU leaves FSDP alone: single-GPU FSDP is a
|
||||
// valid way to reach its CPU offload (docs/inference/offloading.md).
|
||||
setUseFsdpInference(true);
|
||||
setDitLayerwiseOffload(false);
|
||||
}
|
||||
}, []);
|
||||
const [enableTorchCompile, setEnableTorchCompile] = React.useState(false);
|
||||
const [vsaSparsity, setVsaSparsity] = React.useState(0);
|
||||
const [tpSize, setTpSize] = React.useState(-1);
|
||||
@@ -126,6 +291,47 @@ export default function CreateJobModal({
|
||||
const justOpened = isOpen && !justOpenedRef.current;
|
||||
justOpenedRef.current = isOpen;
|
||||
if (!justOpened) return;
|
||||
if (seedJob) {
|
||||
// Must not fall through to the defaults below: a partially-seeded
|
||||
// form silently edits values the user never saw.
|
||||
const f = jobToFormFields(seedJob);
|
||||
setModelId(f.modelId);
|
||||
setName(f.name);
|
||||
setPrompt(f.prompt);
|
||||
setNegativePrompt(f.negativePrompt);
|
||||
setImagePath(f.imagePath);
|
||||
setImageFileName(f.imagePath.split('/').pop() ?? '');
|
||||
setLastImagePath(f.lastImagePath);
|
||||
setLastImageFileName(f.lastImagePath.split('/').pop() ?? '');
|
||||
setReferences(f.references);
|
||||
setPromptFields(f.promptFields ?? EMPTY_H3_PROMPT_FIELDS);
|
||||
setUseGuidedPrompt(f.promptFields !== null);
|
||||
setNumInferenceSteps(f.numInferenceSteps);
|
||||
setNumFrames(f.numFrames);
|
||||
setHeight(f.height);
|
||||
setWidth(f.width);
|
||||
setGuidanceScale(f.guidanceScale);
|
||||
setGuidanceRescale(f.guidanceRescale);
|
||||
setFps(f.fps);
|
||||
setSeed(f.seed);
|
||||
setNumGpus(f.numGpus);
|
||||
setDitCpuOffload(f.ditCpuOffload);
|
||||
setDitLayerwiseOffload(f.ditLayerwiseOffload);
|
||||
setTextEncoderCpuOffload(f.textEncoderCpuOffload);
|
||||
setVaeCpuOffload(f.vaeCpuOffload);
|
||||
setImageEncoderCpuOffload(f.imageEncoderCpuOffload);
|
||||
setUseFsdpInference(f.useFsdpInference);
|
||||
setEnableTorchCompile(f.enableTorchCompile);
|
||||
setVsaSparsity(f.vsaSparsity);
|
||||
setTpSize(f.tpSize);
|
||||
setSpSize(f.spSize);
|
||||
setReferenceError(null);
|
||||
setModelLoadError(null);
|
||||
setImageUploadError(null);
|
||||
setLastImageUploadError(null);
|
||||
setSubmitError(null);
|
||||
return;
|
||||
}
|
||||
const opts = options;
|
||||
setNumInferenceSteps(opts.numInferenceSteps);
|
||||
setNumFrames(workloadType === 't2i' ? 1 : opts.numFrames);
|
||||
@@ -137,6 +343,7 @@ export default function CreateJobModal({
|
||||
setSeed(opts.seed);
|
||||
setNumGpus(opts.numGpus);
|
||||
setDitCpuOffload(opts.ditCpuOffload);
|
||||
setDitLayerwiseOffload(opts.ditLayerwiseOffload ?? false);
|
||||
setTextEncoderCpuOffload(opts.textEncoderCpuOffload);
|
||||
setVaeCpuOffload(opts.vaeCpuOffload);
|
||||
setImageEncoderCpuOffload(opts.imageEncoderCpuOffload);
|
||||
@@ -151,8 +358,16 @@ export default function CreateJobModal({
|
||||
inferenceWorkload as 't2v' | 'i2v' | 't2i',
|
||||
),
|
||||
);
|
||||
setName('');
|
||||
setImagePath('');
|
||||
setImageFileName('');
|
||||
setLastImagePath('');
|
||||
setLastImageFileName('');
|
||||
setLastImageUploadError(null);
|
||||
setReferences([]);
|
||||
setReferenceError(null);
|
||||
setPromptFields(EMPTY_H3_PROMPT_FIELDS);
|
||||
setUseGuidedPrompt(true);
|
||||
setSelectedDatasetId('');
|
||||
setSelectedValidationDatasetId('');
|
||||
setModelLoadError(null);
|
||||
@@ -168,7 +383,7 @@ export default function CreateJobModal({
|
||||
setRealScoreModelPath('');
|
||||
setFakeScoreModelPath('');
|
||||
}
|
||||
}, [isOpen, workloadType, inferenceWorkload, options]);
|
||||
}, [isOpen, workloadType, inferenceWorkload, options, seedJobId]);
|
||||
|
||||
// Load the models available for this workload.
|
||||
React.useEffect(() => {
|
||||
@@ -188,8 +403,22 @@ export default function CreateJobModal({
|
||||
opts,
|
||||
inferenceWorkload as 't2v' | 'i2v' | 't2i',
|
||||
);
|
||||
const chosen = ids.includes(defaultId) ? defaultId : (list[0]?.id ?? '');
|
||||
// When editing, the job's own model wins over the workload default --
|
||||
// this resolves after the seeding effect, so choosing a default here
|
||||
// would silently swap the model out from under the user.
|
||||
const editedId = seedJobModelId;
|
||||
const chosen =
|
||||
editedId && ids.includes(editedId)
|
||||
? editedId
|
||||
: ids.includes(defaultId)
|
||||
? defaultId
|
||||
: (list[0]?.id ?? '');
|
||||
setModelId(chosen);
|
||||
if (isPrefill && editedId && chosen !== editedId) {
|
||||
toast.warning(
|
||||
`${editedId} isn't available for ${inferenceWorkload.toUpperCase()}${chosen ? `, so ${chosen} is selected instead` : ''}.`,
|
||||
);
|
||||
}
|
||||
if (workloadType === 'dmd_t2v') {
|
||||
setRealScoreModelPath(chosen);
|
||||
setFakeScoreModelPath(chosen);
|
||||
@@ -210,7 +439,7 @@ export default function CreateJobModal({
|
||||
return () => {
|
||||
stale = true;
|
||||
};
|
||||
}, [isOpen, inferenceWorkload, workloadType]);
|
||||
}, [isOpen, inferenceWorkload, workloadType, seedJobModelId, isPrefill]);
|
||||
|
||||
// Training jobs need a dataset; load the ready datasets when relevant.
|
||||
React.useEffect(() => {
|
||||
@@ -262,6 +491,106 @@ export default function CreateJobModal({
|
||||
}
|
||||
}
|
||||
|
||||
async function handleLastImageChange(
|
||||
e: React.ChangeEvent<HTMLInputElement>,
|
||||
) {
|
||||
const file = e.target.files?.[0];
|
||||
if (!file) {
|
||||
setLastImagePath('');
|
||||
setLastImageFileName('');
|
||||
setLastImageUploadError(null);
|
||||
return;
|
||||
}
|
||||
setIsUploadingLastImage(true);
|
||||
setLastImageFileName(file.name);
|
||||
setLastImageUploadError(null);
|
||||
try {
|
||||
const { path } = await uploadImage(file);
|
||||
setLastImagePath(path);
|
||||
} catch (error) {
|
||||
console.error('Failed to upload end image:', error);
|
||||
setLastImagePath('');
|
||||
setLastImageFileName('');
|
||||
setLastImageUploadError(
|
||||
error instanceof Error
|
||||
? `${error.message}. Choose the image again to retry.`
|
||||
: 'The image could not be uploaded. Choose it again to retry.',
|
||||
);
|
||||
} finally {
|
||||
setIsUploadingLastImage(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleAddReference(
|
||||
e: React.ChangeEvent<HTMLInputElement>,
|
||||
) {
|
||||
const file = e.target.files?.[0];
|
||||
e.target.value = ''; // allow re-picking the same file
|
||||
if (!file) return;
|
||||
setIsUploadingReference(true);
|
||||
setReferenceError(null);
|
||||
try {
|
||||
const { path, media_type } = await uploadMedia(file);
|
||||
const next: H3Reference[] = [
|
||||
...references,
|
||||
{
|
||||
id: `${Date.now()}-${file.name}`,
|
||||
source: path,
|
||||
media_type,
|
||||
fileName: file.name,
|
||||
},
|
||||
];
|
||||
setReferences(next);
|
||||
setReferenceError(validateReferences(next));
|
||||
} catch (error) {
|
||||
console.error('Failed to upload reference:', error);
|
||||
setReferenceError(
|
||||
error instanceof Error ? error.message : 'The file could not be uploaded.',
|
||||
);
|
||||
} finally {
|
||||
setIsUploadingReference(false);
|
||||
}
|
||||
}
|
||||
|
||||
function removeReference(id: string) {
|
||||
const next = references.filter((r) => r.id !== id);
|
||||
setReferences(next);
|
||||
setReferenceError(validateReferences(next));
|
||||
}
|
||||
|
||||
function seedPromptFields() {
|
||||
setPromptFields({
|
||||
...EMPTY_H3_PROMPT_FIELDS,
|
||||
...referencePromptSeed(references),
|
||||
});
|
||||
setUseGuidedPrompt(true);
|
||||
}
|
||||
|
||||
function setPromptField(section: string, value: string) {
|
||||
setPromptFields((prev) => ({ ...prev, [section]: value }));
|
||||
}
|
||||
|
||||
// Switching between the guided fields and the raw editor keeps whatever was
|
||||
// typed: serialize on the way out, parse back on the way in.
|
||||
function toggleGuidedPrompt() {
|
||||
if (useGuidedPrompt) {
|
||||
if (!isEmptyPromptFields(promptFields)) {
|
||||
setPrompt(serializeH3Prompt(promptFields));
|
||||
}
|
||||
setUseGuidedPrompt(false);
|
||||
} else {
|
||||
const parsed = parseH3Prompt(prompt);
|
||||
if (parsed) setPromptFields(parsed);
|
||||
setUseGuidedPrompt(true);
|
||||
}
|
||||
}
|
||||
|
||||
function clearLastImage() {
|
||||
setLastImagePath('');
|
||||
setLastImageFileName('');
|
||||
setLastImageUploadError(null);
|
||||
}
|
||||
|
||||
function clearImage() {
|
||||
setImagePath('');
|
||||
setImageFileName('');
|
||||
@@ -271,7 +600,16 @@ export default function CreateJobModal({
|
||||
|
||||
async function handleSubmit(e: React.FormEvent<HTMLFormElement>) {
|
||||
e.preventDefault();
|
||||
if (isInference && workloadType === 'i2v' && !imagePath) return;
|
||||
if (isInference && workloadType === 'i2v' && !imagePath && !usingReferences)
|
||||
return;
|
||||
if (usingReferences && validateReferences(references)) return;
|
||||
if (
|
||||
usingReferences &&
|
||||
useGuidedPrompt &&
|
||||
isEmptyPromptFields(promptFields) &&
|
||||
!prompt.trim()
|
||||
)
|
||||
return;
|
||||
// Send the dataset id; the backend resolves it to the on-disk media dir.
|
||||
const effectiveDataPath = selectedDatasetId ?? '';
|
||||
if (!isInference && !selectedDatasetId) return;
|
||||
@@ -285,14 +623,35 @@ export default function CreateJobModal({
|
||||
try {
|
||||
const payload: CreateJobRequest = {
|
||||
model_id: modelId,
|
||||
prompt,
|
||||
name: name.trim(),
|
||||
prompt:
|
||||
usingReferences && useGuidedPrompt && !isEmptyPromptFields(promptFields)
|
||||
? serializeH3Prompt(promptFields)
|
||||
: prompt,
|
||||
workload_type: workloadType,
|
||||
job_type: effectiveJobType,
|
||||
...(isInference
|
||||
? {
|
||||
...(workloadType === 'i2v' && imagePath
|
||||
// Ref2VA and the FL2VA keyframes are mutually exclusive:
|
||||
// _prepare_ref2va rejects image_path/last_image_path outright
|
||||
// when references are present.
|
||||
...(workloadType === 'i2v' && !usingReferences && imagePath
|
||||
? { image_path: imagePath }
|
||||
: {}),
|
||||
...(workloadType === 'i2v' &&
|
||||
supportsLastImage &&
|
||||
!usingReferences &&
|
||||
lastImagePath
|
||||
? { last_image_path: lastImagePath }
|
||||
: {}),
|
||||
...(workloadType === 'i2v' && supportsLastImage && references.length
|
||||
? {
|
||||
references: references.map((r) => ({
|
||||
source: r.source,
|
||||
media_type: r.media_type,
|
||||
})),
|
||||
}
|
||||
: {}),
|
||||
negative_prompt: negativePrompt,
|
||||
num_inference_steps: numInferenceSteps,
|
||||
num_frames: numFrames,
|
||||
@@ -304,6 +663,7 @@ export default function CreateJobModal({
|
||||
seed,
|
||||
num_gpus: numGpus,
|
||||
dit_cpu_offload: ditCpuOffload,
|
||||
dit_layerwise_offload: ditLayerwiseOffload,
|
||||
text_encoder_cpu_offload: textEncoderCpuOffload,
|
||||
vae_cpu_offload: vaeCpuOffload,
|
||||
image_encoder_cpu_offload: imageEncoderCpuOffload,
|
||||
@@ -334,7 +694,14 @@ export default function CreateJobModal({
|
||||
: {}),
|
||||
}),
|
||||
};
|
||||
await createJob(payload);
|
||||
if (editingJob) {
|
||||
await updateJob(
|
||||
editingJob.id,
|
||||
payload as unknown as Record<string, unknown>,
|
||||
);
|
||||
} else {
|
||||
await createJob(payload);
|
||||
}
|
||||
onSuccess();
|
||||
onClose();
|
||||
} catch (err) {
|
||||
@@ -356,9 +723,9 @@ export default function CreateJobModal({
|
||||
|
||||
const workloadLabel =
|
||||
WORKLOAD_OPTIONS[jobType]?.find((o) => o.type === workloadType)?.label ?? '';
|
||||
const title = `New ${jobType.charAt(0).toUpperCase() + jobType.slice(1)} Job${
|
||||
workloadLabel ? ` (${workloadLabel})` : ''
|
||||
}`;
|
||||
const title = `${readOnly ? 'View' : editingJob ? 'Edit' : 'New'} ${
|
||||
jobType.charAt(0).toUpperCase() + jobType.slice(1)
|
||||
} Job${workloadLabel ? ` (${workloadLabel})` : ''}`;
|
||||
|
||||
return (
|
||||
<Dialog
|
||||
@@ -385,6 +752,23 @@ export default function CreateJobModal({
|
||||
autoComplete="off"
|
||||
className="flex flex-col gap-3.5"
|
||||
>
|
||||
{/* disabled cascades to every control inside; display:contents
|
||||
keeps the parent's flex layout. */}
|
||||
<fieldset
|
||||
disabled={readOnly}
|
||||
style={{ display: 'contents' }}
|
||||
className="contents"
|
||||
>
|
||||
<FieldRow htmlFor="modal-name" label="Name (optional)">
|
||||
<Input
|
||||
id="modal-name"
|
||||
value={name}
|
||||
onChange={(e) => setName(e.target.value)}
|
||||
placeholder="Shown on the job card and used for the output filename"
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</FieldRow>
|
||||
|
||||
<FieldRow htmlFor="modal-modelId" label="Model">
|
||||
<NativeSelect
|
||||
id="modal-modelId"
|
||||
@@ -429,12 +813,12 @@ export default function CreateJobModal({
|
||||
type="file"
|
||||
accept=".png,.jpg,.jpeg,.webp,.bmp"
|
||||
onChange={handleImageChange}
|
||||
disabled={isSubmitting || isUploadingImage}
|
||||
disabled={isSubmitting || isUploadingImage || usingReferences}
|
||||
aria-describedby={
|
||||
imageUploadError ? 'modal-image-error' : undefined
|
||||
}
|
||||
aria-invalid={imageUploadError ? true : undefined}
|
||||
required
|
||||
required={!usingReferences}
|
||||
className="h-auto py-2 file:mr-3 file:cursor-pointer file:rounded-md file:border-0 file:bg-secondary file:px-2 file:py-1 file:text-sm file:text-secondary-foreground"
|
||||
/>
|
||||
{imageFileName && (
|
||||
@@ -462,24 +846,223 @@ export default function CreateJobModal({
|
||||
</FieldRow>
|
||||
)}
|
||||
|
||||
<FieldRow
|
||||
htmlFor="modal-prompt"
|
||||
label={isInference ? 'Prompt' : 'Description'}
|
||||
>
|
||||
<Textarea
|
||||
id="modal-prompt"
|
||||
value={prompt}
|
||||
onChange={(e) => setPrompt(e.target.value)}
|
||||
rows={isInference ? 3 : 2}
|
||||
placeholder={
|
||||
isInference
|
||||
? 'A curious raccoon peers through a vibrant field of yellow sunflowers…'
|
||||
: 'Brief description of this training job…'
|
||||
}
|
||||
required
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
</FieldRow>
|
||||
{isInference && workloadType === 'i2v' && supportsLastImage && (
|
||||
<FieldRow htmlFor="modal-last-image" label="End Frame (optional)">
|
||||
<Input
|
||||
id="modal-last-image"
|
||||
type="file"
|
||||
accept=".png,.jpg,.jpeg,.webp,.bmp"
|
||||
onChange={handleLastImageChange}
|
||||
disabled={isSubmitting || isUploadingLastImage}
|
||||
aria-describedby={
|
||||
lastImageUploadError ? 'modal-last-image-error' : undefined
|
||||
}
|
||||
aria-invalid={lastImageUploadError ? true : undefined}
|
||||
className="h-auto py-2 file:mr-3 file:cursor-pointer file:rounded-md file:border-0 file:bg-secondary file:px-2 file:py-1 file:text-sm file:text-secondary-foreground"
|
||||
/>
|
||||
{lastImageFileName && (
|
||||
<span className="mt-0.5 text-xs text-muted-foreground">
|
||||
{isUploadingLastImage ? 'Uploading…' : lastImageFileName} ·{' '}
|
||||
<button
|
||||
type="button"
|
||||
onClick={clearLastImage}
|
||||
disabled={isSubmitting || isUploadingLastImage}
|
||||
className="text-accent-blue underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
Clear
|
||||
</button>
|
||||
</span>
|
||||
)}
|
||||
{lastImageUploadError && (
|
||||
<p
|
||||
id="modal-last-image-error"
|
||||
role="alert"
|
||||
className="text-sm text-destructive"
|
||||
>
|
||||
{lastImageUploadError}
|
||||
</p>
|
||||
)}
|
||||
</FieldRow>
|
||||
)}
|
||||
|
||||
{isInference && workloadType === 'i2v' && supportsLastImage && (
|
||||
<FieldRow htmlFor="modal-reference" label="References (Ref2VA)">
|
||||
<Input
|
||||
id="modal-reference"
|
||||
type="file"
|
||||
accept=".png,.jpg,.jpeg,.webp,.bmp,.mp4,.mov,.mkv,.webm,.avi,.wav,.mp3,.flac,.m4a,.ogg"
|
||||
onChange={handleAddReference}
|
||||
disabled={
|
||||
isSubmitting ||
|
||||
isUploadingReference ||
|
||||
references.length >= H3_MAX_REFERENCES
|
||||
}
|
||||
className="h-auto py-2 file:mr-3 file:cursor-pointer file:rounded-md file:border-0 file:bg-secondary file:px-2 file:py-1 file:text-sm file:text-secondary-foreground"
|
||||
/>
|
||||
{isUploadingReference && (
|
||||
<span className="mt-0.5 text-xs text-muted-foreground">
|
||||
Uploading…
|
||||
</span>
|
||||
)}
|
||||
{references.length > 0 && (
|
||||
<ul className="mt-1 flex list-none flex-col gap-1 p-0">
|
||||
{references.map((reference, index) => (
|
||||
<li
|
||||
key={reference.id}
|
||||
className="flex items-center gap-2 text-xs text-muted-foreground"
|
||||
>
|
||||
<code className="font-mono text-accent-blue">
|
||||
{labelReferences(references)[index]}
|
||||
</code>
|
||||
<span className="truncate">{reference.fileName}</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => removeReference(reference.id)}
|
||||
disabled={isSubmitting}
|
||||
className="ml-auto text-accent-blue underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
Remove
|
||||
</button>
|
||||
</li>
|
||||
))}
|
||||
</ul>
|
||||
)}
|
||||
{references.length > 0 && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={seedPromptFields}
|
||||
disabled={isSubmitting}
|
||||
className="mt-1 self-start text-xs text-accent-blue underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
Fill prompt sections from references
|
||||
</button>
|
||||
)}
|
||||
{referenceError && (
|
||||
<p role="alert" className="mt-0.5 text-xs text-destructive">
|
||||
{referenceError}
|
||||
</p>
|
||||
)}
|
||||
<span className="mt-0.5 text-xs text-muted-foreground">
|
||||
Ref2VA replaces the keyframes: up to 9 images, 3 videos, 3 audio
|
||||
(12 total). Audio needs at least one image or video.
|
||||
</span>
|
||||
</FieldRow>
|
||||
)}
|
||||
|
||||
{isH3Model && isInference && workloadType !== 't2i' && (
|
||||
<div className="flex flex-col gap-2">
|
||||
<H3TagConsole onInsert={insertH3Tag} disabled={isSubmitting} />
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="self-start"
|
||||
onClick={() => setDialogueEditorOpen(true)}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
Open dialogue editor
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{usingReferences && useGuidedPrompt ? (
|
||||
/* Six-section format from the model's reference prompt guide. */
|
||||
<>
|
||||
{H3_PROMPT_SECTIONS.map((section) => (
|
||||
<FieldRow
|
||||
key={section}
|
||||
htmlFor={`modal-prompt-${section}`}
|
||||
label={H3_SECTION_LABELS[section]}
|
||||
>
|
||||
<Textarea
|
||||
id={`modal-prompt-${section}`}
|
||||
ref={(el) => {
|
||||
tagFieldRefs.current[section] = el;
|
||||
}}
|
||||
value={promptFields[section]}
|
||||
onChange={(e) => setPromptField(section, e.target.value)}
|
||||
onFocus={() => setActiveTagField(section)}
|
||||
rows={section === 'detailed_description' ? 5 : 2}
|
||||
placeholder={H3_SECTION_HINTS[section]}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
{section === 'detailed_description' && (
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="self-start"
|
||||
onClick={() => {
|
||||
setShotListMode('sections');
|
||||
setShotListOpen(true);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
Open shot list
|
||||
</Button>
|
||||
)}
|
||||
</FieldRow>
|
||||
))}
|
||||
<button
|
||||
type="button"
|
||||
onClick={toggleGuidedPrompt}
|
||||
disabled={isSubmitting}
|
||||
className="self-start text-xs text-accent-blue underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
Edit as raw prompt
|
||||
</button>
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<FieldRow
|
||||
htmlFor="modal-prompt"
|
||||
label={isInference ? 'Prompt' : 'Description'}
|
||||
>
|
||||
<Textarea
|
||||
id="modal-prompt"
|
||||
ref={(el) => {
|
||||
tagFieldRefs.current.prompt = el;
|
||||
}}
|
||||
value={prompt}
|
||||
onChange={(e) => setPrompt(e.target.value)}
|
||||
onFocus={() => setActiveTagField('prompt')}
|
||||
rows={isInference ? 3 : 2}
|
||||
placeholder={
|
||||
isInference
|
||||
? 'A curious raccoon peers through a vibrant field of yellow sunflowers…'
|
||||
: 'Brief description of this training job…'
|
||||
}
|
||||
required={!(usingReferences && useGuidedPrompt)}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
{isH3Model && isInference && workloadType !== 't2i' && (
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="self-start"
|
||||
onClick={() => {
|
||||
setShotListMode('prompt');
|
||||
setShotListOpen(true);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
Open shot list
|
||||
</Button>
|
||||
)}
|
||||
</FieldRow>
|
||||
{usingReferences && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={toggleGuidedPrompt}
|
||||
disabled={isSubmitting}
|
||||
className="self-start text-xs text-accent-blue underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
Edit as prompt sections
|
||||
</button>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
|
||||
{isInference && (
|
||||
<FieldRow htmlFor="modal-negative-prompt" label="Negative Prompt">
|
||||
@@ -849,6 +1432,13 @@ export default function CreateJobModal({
|
||||
onChange={setDitCpuOffload}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="modal-dit-layerwise-offload"
|
||||
label="DiT Layerwise Offload"
|
||||
checked={ditLayerwiseOffload}
|
||||
onChange={handleDitLayerwiseOffloadChange}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<ToggleRow
|
||||
id="modal-text-encoder-cpu-offload"
|
||||
label="Text Encoder CPU Offload"
|
||||
@@ -860,7 +1450,7 @@ export default function CreateJobModal({
|
||||
id="modal-use-fsdp-inference"
|
||||
label="Use FSDP Inference"
|
||||
checked={useFsdpInference}
|
||||
onChange={setUseFsdpInference}
|
||||
onChange={handleUseFsdpInferenceChange}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<ToggleRow
|
||||
@@ -891,7 +1481,7 @@ export default function CreateJobModal({
|
||||
max={8}
|
||||
step={1}
|
||||
value={numGpus}
|
||||
onChange={setNumGpus}
|
||||
onChange={handleNumGpusChange}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
<NumberRow
|
||||
@@ -906,6 +1496,8 @@ export default function CreateJobModal({
|
||||
</details>
|
||||
)}
|
||||
|
||||
</fieldset>
|
||||
|
||||
<div className="flex flex-col items-start gap-2">
|
||||
{submitError && (
|
||||
<p role="alert" className="text-sm text-destructive">
|
||||
@@ -914,17 +1506,57 @@ export default function CreateJobModal({
|
||||
)}
|
||||
<Button
|
||||
type="submit"
|
||||
hidden={readOnly}
|
||||
disabled={
|
||||
readOnly ||
|
||||
isSubmitting ||
|
||||
isUploadingImage ||
|
||||
!!modelLoadError ||
|
||||
!!datasetLoadError
|
||||
}
|
||||
>
|
||||
{isSubmitting ? 'Creating…' : 'Create Job'}
|
||||
{isSubmitting
|
||||
? editingJob
|
||||
? 'Saving…'
|
||||
: 'Creating…'
|
||||
: editingJob
|
||||
? 'Save Changes'
|
||||
: 'Create Job'}
|
||||
</Button>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
{/* Nested in the React tree (still portaled to <body>) so Radix treats
|
||||
clicks and focus inside these as inside this dialog. As siblings
|
||||
they counted as "outside", and closing one could close this too. */}
|
||||
<H3DialogueEditor
|
||||
open={dialogueEditorOpen}
|
||||
initialValue={getTagFieldValue(dialogueEditorField)}
|
||||
onApply={(value) => setTagFieldValue(dialogueEditorField, value)}
|
||||
onClose={() => setDialogueEditorOpen(false)}
|
||||
/>
|
||||
<H3ShotListEditor
|
||||
open={shotListOpen}
|
||||
subjectDefinitions={shotListMode === 'sections' ? promptFields.subject_definitions : promptSubjects}
|
||||
initialDetailedDescription={
|
||||
shotListMode === 'sections' ? promptFields.detailed_description : promptShots.body
|
||||
}
|
||||
initialRetentionAnalysis={
|
||||
shotListMode === 'sections' ? promptFields.retention_analysis : (promptRetention ?? '')
|
||||
}
|
||||
targetLabel={
|
||||
shotListMode === 'sections'
|
||||
? undefined
|
||||
: promptShots.field === 'integrated_multimodal_description'
|
||||
? 'integrated_multimodal_description'
|
||||
: promptShots.field === 'detailed_description'
|
||||
? 'Detailed description'
|
||||
: 'the prompt'
|
||||
}
|
||||
withRetention={shotListMode === 'sections' || promptSyncsRetention}
|
||||
onApply={shotListMode === 'sections' ? applyShotList : applyShotListToPrompt}
|
||||
onClose={() => setShotListOpen(false)}
|
||||
/>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
|
||||
@@ -0,0 +1,346 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { H3DialoguePreview } from '@/components/jobs/H3DialoguePreview';
|
||||
import { H3TagPalette } from '@/components/jobs/H3TagPalette';
|
||||
import { useUndoableText } from '@/hooks/useUndoableText';
|
||||
import { applyH3Preset, applyH3Tag, type H3Preset, type H3Tag } from '@/lib/h3Tags';
|
||||
import {
|
||||
SPEECH_RATES,
|
||||
dialogueTree,
|
||||
estimateDuration,
|
||||
type DialogueNode,
|
||||
type SpeechRate,
|
||||
} from '@/lib/h3DialogueEstimate';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface H3DialogueEditorProps {
|
||||
open: boolean;
|
||||
/** Seeded from whichever prompt field the editor was opened for. */
|
||||
initialValue: string;
|
||||
onApply: (value: string) => void;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
/** Every `<d>...</d>` block's inner text -- what's actually spoken, ignoring surrounding shot/camera prose. */
|
||||
function extractDialogueBlocks(text: string): string[] {
|
||||
const blocks: string[] = [];
|
||||
const re = /<d>([\s\S]*?)<\/d>/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(text))) blocks.push(m[1].replace(/^\s*\[[^\]]*\]\s*/, '')); // drop a leading [Language] marker
|
||||
return blocks;
|
||||
}
|
||||
|
||||
/**
|
||||
* Full-screen dialogue editor for MiniMax-H3 speech tags: the same tag
|
||||
* registry as `H3TagConsole`, plus a duration estimate (scoped to `<d>`
|
||||
* blocks so scene-description prose doesn't inflate it), a styled
|
||||
* performance preview, and emotion presets. Opens with a copy of the target
|
||||
* field's text; `onApply` hands the edited text back, `onClose` discards it.
|
||||
* The shot list editor reuses the same palette, preview, and undo history.
|
||||
*/
|
||||
export function H3DialogueEditor({ open, initialValue, onApply, onClose }: H3DialogueEditorProps) {
|
||||
const { text, type, commit, undo, redo, reset, canUndo, canRedo } = useUndoableText(initialValue);
|
||||
const [pace, setPace] = React.useState<SpeechRate>('Natural');
|
||||
const [calib, setCalib] = React.useState(1);
|
||||
const [actualSeconds, setActualSeconds] = React.useState('');
|
||||
const taRef = React.useRef<HTMLTextAreaElement>(null);
|
||||
const pending = React.useRef<[number, number] | null>(null);
|
||||
|
||||
function restoreCaretAtEnd(restored: string | null) {
|
||||
if (restored != null) pending.current = [restored.length, restored.length];
|
||||
}
|
||||
|
||||
// Re-seed from the field's current value each time the editor opens, the
|
||||
// same "seed on the open transition only" guard CreateJobModal itself
|
||||
// uses -- otherwise an in-progress edit here could be clobbered by a
|
||||
// stray parent re-render while open. History resets with it: undo
|
||||
// shouldn't reach back into a previous time this editor was opened.
|
||||
const wasOpenRef = React.useRef(false);
|
||||
React.useEffect(() => {
|
||||
if (open && !wasOpenRef.current) {
|
||||
reset(initialValue);
|
||||
setActualSeconds('');
|
||||
setCalib(1);
|
||||
}
|
||||
wasOpenRef.current = open;
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [open]);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (pending.current && taRef.current) {
|
||||
const [s, e] = pending.current;
|
||||
taRef.current.focus();
|
||||
taRef.current.setSelectionRange(s, e);
|
||||
pending.current = null;
|
||||
}
|
||||
});
|
||||
|
||||
function handleKeyDown(e: React.KeyboardEvent<HTMLTextAreaElement>) {
|
||||
if (!(e.metaKey || e.ctrlKey) || e.key.toLowerCase() !== 'z') return;
|
||||
e.preventDefault();
|
||||
restoreCaretAtEnd(e.shiftKey ? redo() : undo());
|
||||
}
|
||||
|
||||
function insertTag(tag: H3Tag) {
|
||||
const ta = taRef.current;
|
||||
const result = applyH3Tag(text, ta?.selectionStart ?? text.length, ta?.selectionEnd ?? text.length, tag);
|
||||
commit(result.value);
|
||||
pending.current = [result.start, result.end];
|
||||
}
|
||||
|
||||
function applyPreset(p: H3Preset) {
|
||||
const ta = taRef.current;
|
||||
const result = applyH3Preset(text, ta?.selectionStart ?? text.length, ta?.selectionEnd ?? text.length, p);
|
||||
commit(result.value);
|
||||
pending.current = [result.start, result.end];
|
||||
}
|
||||
|
||||
const dialogueBlocks = React.useMemo(() => extractDialogueBlocks(text), [text]);
|
||||
// Falls back to the whole field only when it contains no <d> blocks at
|
||||
// all yet -- e.g. a fresh field the user is about to write dialogue into.
|
||||
const segments = dialogueBlocks.length > 0 ? dialogueBlocks : [text];
|
||||
const estimates = React.useMemo(
|
||||
() => segments.map((s) => estimateDuration(s, SPEECH_RATES[pace], calib)),
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
[segments.join('\n'), pace, calib],
|
||||
);
|
||||
const totalSeconds = estimates.reduce((sum, e) => sum + e.total, 0);
|
||||
|
||||
function calibrate() {
|
||||
const actual = parseFloat(actualSeconds);
|
||||
if (!actual || actual <= 0 || segments.length === 0) return;
|
||||
const base = segments.reduce((sum, s) => sum + estimateDuration(s, SPEECH_RATES[pace], 1).total, 0);
|
||||
if (base > 0) setCalib(actual / base);
|
||||
}
|
||||
|
||||
/* ---- rough browser-voice preview, for rhythm/pacing only ---- */
|
||||
const synth = typeof window !== 'undefined' ? window.speechSynthesis : null;
|
||||
const [playing, setPlaying] = React.useState(false);
|
||||
const playTokenRef = React.useRef({ cancelled: true });
|
||||
|
||||
function stopPreview() {
|
||||
playTokenRef.current.cancelled = true;
|
||||
synth?.cancel();
|
||||
setPlaying(false);
|
||||
}
|
||||
|
||||
function playPreview() {
|
||||
if (!synth) return;
|
||||
stopPreview();
|
||||
const token = { cancelled: false };
|
||||
playTokenRef.current = token;
|
||||
setPlaying(true);
|
||||
const VOX: Record<string, string> = {
|
||||
laughs: 'ha ha ha', chuckle: 'heh', sighs: 'haah', gasp: 'ah', uh: 'uh', mhm: 'mhm', phew: 'phew',
|
||||
};
|
||||
const atoms: { say?: string; gap?: number }[] = [];
|
||||
for (const [i, seg] of segments.entries()) {
|
||||
if (i > 0) atoms.push({ gap: 0.6 });
|
||||
const walk = (nodes: DialogueNode[]) => {
|
||||
for (const n of nodes) {
|
||||
if (n.t === 'text') {
|
||||
const t = n.v.replace(/\s+/g, ' ').trim();
|
||||
if (t) atoms.push({ say: t });
|
||||
} else if (n.t === 'point') {
|
||||
atoms.push(VOX[n.name] ? { say: VOX[n.name] } : { gap: 0.25 });
|
||||
} else {
|
||||
if (n.name === 'humming') atoms.push({ say: 'hmm hmm' });
|
||||
else walk(n.children);
|
||||
}
|
||||
}
|
||||
};
|
||||
walk(dialogueTree(seg));
|
||||
}
|
||||
let i = 0;
|
||||
const step = () => {
|
||||
if (token.cancelled) return;
|
||||
if (i >= atoms.length) {
|
||||
setPlaying(false);
|
||||
return;
|
||||
}
|
||||
const a = atoms[i++];
|
||||
if (a.gap != null) {
|
||||
setTimeout(() => { if (!token.cancelled) step(); }, a.gap * 1000);
|
||||
return;
|
||||
}
|
||||
const u = new SpeechSynthesisUtterance(a.say);
|
||||
u.rate = 0.95;
|
||||
u.onend = () => { if (!token.cancelled) step(); };
|
||||
u.onerror = () => { if (!token.cancelled) step(); };
|
||||
synth.speak(u);
|
||||
};
|
||||
step();
|
||||
}
|
||||
|
||||
React.useEffect(() => () => stopPreview(), []); // eslint-disable-line react-hooks/exhaustive-deps
|
||||
|
||||
function handleApply() {
|
||||
onApply(text);
|
||||
onClose();
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(o) => { if (!o) onClose(); }}>
|
||||
<DialogContent className="flex h-[92vh] max-h-[920px] w-[96vw] max-w-[1180px] flex-col gap-0 overflow-hidden p-0">
|
||||
<DialogHeader className="border-b border-border px-5 py-4">
|
||||
<DialogTitle>Dialogue editor</DialogTitle>
|
||||
<DialogDescription>
|
||||
Click a tag to insert it at the cursor, or select text first to wrap it. Applying writes the edited text
|
||||
back into the field you opened this from.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
<div className="flex min-h-0 flex-1">
|
||||
{/* palette */}
|
||||
<H3TagPalette
|
||||
onInsertTag={insertTag}
|
||||
onApplyPreset={applyPreset}
|
||||
className="w-64 shrink-0 overflow-y-auto border-r border-border p-3"
|
||||
/>
|
||||
|
||||
{/* editor + preview */}
|
||||
<div className="flex min-w-0 flex-1 flex-col gap-4 overflow-y-auto p-4">
|
||||
{/* duration estimate */}
|
||||
<div className="rounded-xl border border-border bg-secondary/30 p-4">
|
||||
<div className="flex flex-wrap items-baseline justify-between gap-3">
|
||||
<div className="flex items-baseline gap-3">
|
||||
<span className="text-2xl font-semibold tracking-tight text-foreground">
|
||||
{totalSeconds.toFixed(1)}<span className="ml-1 text-sm font-normal text-muted-foreground">s spoken</span>
|
||||
</span>
|
||||
<span className="font-mono text-xs text-muted-foreground">
|
||||
{dialogueBlocks.length > 0
|
||||
? `across ${dialogueBlocks.length} <d> block${dialogueBlocks.length === 1 ? '' : 's'}`
|
||||
: 'no <d> blocks yet — estimating the whole field'}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex items-center gap-1 rounded-lg border border-border bg-card p-1">
|
||||
{(Object.keys(SPEECH_RATES) as SpeechRate[]).map((p) => (
|
||||
<button
|
||||
key={p}
|
||||
type="button"
|
||||
onClick={() => setPace(p)}
|
||||
className={cn(
|
||||
'rounded-md px-2.5 py-1 text-xs transition-colors',
|
||||
pace === p ? 'bg-secondary text-foreground' : 'text-muted-foreground hover:bg-accent',
|
||||
)}
|
||||
>
|
||||
{p}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{segments.length > 1 && (
|
||||
<div className="mt-3 flex flex-wrap gap-1.5">
|
||||
{estimates.map((e, i) => (
|
||||
<span key={i} className="rounded-md border border-border bg-card px-2 py-0.5 font-mono text-[11px] text-muted-foreground">
|
||||
block {i + 1}: {e.total.toFixed(1)}s
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="mt-3 flex flex-wrap items-center gap-2 text-xs text-muted-foreground">
|
||||
<span title="Enter the real length of a clip you already generated, to tune the estimate to your setup">
|
||||
calibrate
|
||||
</span>
|
||||
<input
|
||||
value={actualSeconds}
|
||||
onChange={(e) => setActualSeconds(e.target.value)}
|
||||
placeholder="actual s"
|
||||
inputMode="decimal"
|
||||
className="w-16 rounded-md border border-input bg-card px-2 py-1 text-foreground"
|
||||
/>
|
||||
<Button type="button" variant="outline" size="sm" onClick={calibrate}>Set</Button>
|
||||
{Math.abs(calib - 1) > 0.001 && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => { setCalib(1); setActualSeconds(''); }}
|
||||
className="font-mono text-muted-foreground/80 hover:text-foreground"
|
||||
title="Reset calibration"
|
||||
>
|
||||
×{calib.toFixed(2)}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* audio preview */}
|
||||
{synth && (
|
||||
<div className="flex items-center gap-3 rounded-xl border border-border bg-secondary/30 p-3">
|
||||
<Button type="button" size="sm" onClick={playing ? stopPreview : playPreview}>
|
||||
{playing ? '■ Stop' : '▶ Play preview'}
|
||||
</Button>
|
||||
<span className="text-xs text-muted-foreground">
|
||||
A stand-in browser voice for rhythm only — it won't match MiniMax's voice.
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* the field itself */}
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-xs font-semibold text-foreground">Text</span>
|
||||
<div className="flex items-center gap-1">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => restoreCaretAtEnd(undo())}
|
||||
disabled={!canUndo}
|
||||
title="Undo (⌘/Ctrl+Z)"
|
||||
>
|
||||
↶ Undo
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => restoreCaretAtEnd(redo())}
|
||||
disabled={!canRedo}
|
||||
title="Redo (⌘/Ctrl+Shift+Z)"
|
||||
>
|
||||
↷ Redo
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
<Textarea
|
||||
ref={taRef}
|
||||
value={text}
|
||||
onChange={(e) => type(e.target.value)}
|
||||
onKeyDown={handleKeyDown}
|
||||
spellCheck={false}
|
||||
rows={8}
|
||||
className="flex-1 font-mono text-sm"
|
||||
/>
|
||||
|
||||
{/* performance preview */}
|
||||
<div>
|
||||
<div className="mb-2 text-xs font-semibold text-foreground">Performance preview</div>
|
||||
<H3DialoguePreview
|
||||
text={text}
|
||||
className="rounded-xl border border-border bg-card p-4 text-sm leading-7 text-foreground"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<DialogFooter className="border-t border-border px-5 py-3">
|
||||
<Button type="button" variant="outline" onClick={onClose}>Cancel</Button>
|
||||
<Button type="button" onClick={handleApply}>Apply to field</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { CATEGORY_STYLE } from '@/components/jobs/H3TagPalette';
|
||||
import { dialogueTree, type DialogueNode } from '@/lib/h3DialogueEstimate';
|
||||
import { H3_TAG_CATEGORIES } from '@/lib/h3Tags';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const CATEGORY_BY_TAG_NAME: Record<string, string> = Object.fromEntries(
|
||||
H3_TAG_CATEGORIES.flatMap((c) => c.tags.map((t) => [t.open.replace(/[<>]/g, ''), c.id])),
|
||||
);
|
||||
|
||||
function renderPreviewNodes(nodes: DialogueNode[], keyPrefix = ''): React.ReactNode[] {
|
||||
return nodes.map((n, i) => {
|
||||
const key = `${keyPrefix}${i}`;
|
||||
if (n.t === 'text') return <span key={key}>{n.v}</span>;
|
||||
if (n.t === 'point') {
|
||||
const cat = CATEGORY_BY_TAG_NAME[n.name] ?? 'body';
|
||||
const style = CATEGORY_STYLE[cat] ?? CATEGORY_STYLE.body;
|
||||
return (
|
||||
<span
|
||||
key={key}
|
||||
className={cn(
|
||||
'mx-0.5 inline-block rounded border px-1.5 py-0.5 font-mono text-[11px] leading-none',
|
||||
style.chip,
|
||||
)}
|
||||
>
|
||||
{n.name}
|
||||
</span>
|
||||
);
|
||||
}
|
||||
const inner = renderPreviewNodes(n.children, `${key}-`);
|
||||
if (n.name === 'i') return <strong key={key} className="italic">{inner}</strong>;
|
||||
if (n.name === 'whisper') return <span key={key} className="text-[0.92em] tracking-wide opacity-60">{inner}</span>;
|
||||
if (n.name === 'humming') return <em key={key} className="underline decoration-wavy decoration-emerald-500 underline-offset-4">{inner}</em>;
|
||||
return <span key={key}>{inner}</span>;
|
||||
});
|
||||
}
|
||||
|
||||
/** Styled "performance" rendering of speech tags: chips for point tags, styling for wrapping ones. */
|
||||
export function H3DialoguePreview({ text, className }: { text: string; className?: string }) {
|
||||
return <div className={className}>{renderPreviewNodes(dialogueTree(text))}</div>;
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
import { render, screen, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { H3DialogueEditor } from './H3DialogueEditor';
|
||||
import { H3ShotListEditor } from './H3ShotListEditor';
|
||||
|
||||
const SUBJECTS = '<Subject 1> is the courier.\n<Subject 2> is the rival.';
|
||||
const DETAILED = [
|
||||
'[Shot 1] Wide shot. A rooftop at night.',
|
||||
'[Shot 2] Close-up shot, handheld. He turns. (S2) <d>[English] I had no choice.</d>',
|
||||
].join('\n\n');
|
||||
|
||||
describe('H3DialogueEditor (shared palette + undo)', () => {
|
||||
it('inserts a tag at the cursor, undoes it, and applies the edited text', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = vi.fn();
|
||||
render(<H3DialogueEditor open initialValue="Hello there" onApply={onApply} onClose={() => {}} />);
|
||||
|
||||
const textarea = screen.getByDisplayValue('Hello there') as HTMLTextAreaElement;
|
||||
textarea.focus();
|
||||
textarea.setSelectionRange(5, 5);
|
||||
await user.click(screen.getByRole('button', { name: /<pause>/ }));
|
||||
expect(textarea.value).toBe('Hello <pause> there');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /Undo/ }));
|
||||
expect(textarea.value).toBe('Hello there');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Nervous' }));
|
||||
expect(textarea.value).toContain('<uh> <stutter>');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Apply to field' }));
|
||||
expect(onApply).toHaveBeenCalledWith(expect.stringContaining('<uh> <stutter>'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('H3ShotListEditor dialogue tooling', () => {
|
||||
function renderEditor(onApply = vi.fn()) {
|
||||
render(
|
||||
<H3ShotListEditor
|
||||
open
|
||||
subjectDefinitions={SUBJECTS}
|
||||
initialDetailedDescription={DETAILED}
|
||||
initialRetentionAnalysis=""
|
||||
onApply={onApply}
|
||||
onClose={() => {}}
|
||||
/>,
|
||||
);
|
||||
return onApply;
|
||||
}
|
||||
|
||||
it('disables the palette until a shot dialogue is focused', () => {
|
||||
renderEditor();
|
||||
expect(screen.getByText(/Click into a shot's dialogue/)).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: /<pause>/ })).toBeDisabled();
|
||||
});
|
||||
|
||||
it('inserts a palette tag into the focused shot dialogue and writes it on apply', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderEditor();
|
||||
|
||||
const dialogueBoxes = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
expect(dialogueBoxes).toHaveLength(1);
|
||||
expect(dialogueBoxes[0].value).toBe('I had no choice.');
|
||||
|
||||
await user.click(dialogueBoxes[0]);
|
||||
dialogueBoxes[0].setSelectionRange(1, 1);
|
||||
expect(screen.getByText('Tags insert into Shot 2 dialogue')).toBeInTheDocument();
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /<pause>/ }));
|
||||
expect(dialogueBoxes[0].value).toBe('I <pause> had no choice.');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply).toHaveBeenCalledTimes(1);
|
||||
expect(onApply.mock.calls[0][0]).toContain('<d>[English] I <pause> had no choice.</d>');
|
||||
});
|
||||
|
||||
it('adds and removes dialogue lines, writing every line on apply', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderEditor();
|
||||
|
||||
await user.click(screen.getAllByRole('button', { name: '+ Add line' })[1]);
|
||||
const boxes = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
await user.type(boxes[1], 'Then go.');
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toContain(
|
||||
'(S2) <d>[English] I had no choice.</d> (S2) <d>[English] Then go.</d>',
|
||||
);
|
||||
});
|
||||
|
||||
it('shows the styled preview once a shot dialogue contains tags', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
const boxes = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
await user.click(boxes[0]);
|
||||
await user.click(screen.getByRole('button', { name: /<long pause>/ }));
|
||||
|
||||
const shot2 = boxes[0].closest('[id^="shot-card-"]') as HTMLElement;
|
||||
expect(within(shot2).getByText('long pause')).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
describe('H3ShotListEditor ordered body', () => {
|
||||
function renderWith(detailed: string) {
|
||||
const onApply = vi.fn();
|
||||
render(
|
||||
<H3ShotListEditor
|
||||
open
|
||||
subjectDefinitions={SUBJECTS}
|
||||
initialDetailedDescription={detailed}
|
||||
initialRetentionAnalysis=""
|
||||
onApply={onApply}
|
||||
onClose={() => {}}
|
||||
/>,
|
||||
);
|
||||
return onApply;
|
||||
}
|
||||
|
||||
it('shows hand-written dialogue as an editable line between its prose and applies the text unchanged', async () => {
|
||||
const user = userEvent.setup();
|
||||
const text = "[Shot 1] He (S1) says: <d>[English] I think you're sorry.</d> <cutoff>";
|
||||
const onApply = renderWith(text);
|
||||
|
||||
expect(screen.getByText(/keeps its original wording/)).toBeInTheDocument();
|
||||
const lines = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
expect(lines).toHaveLength(1);
|
||||
expect(lines[0].value).toBe("I think you're sorry.");
|
||||
const prose = screen.getAllByPlaceholderText("What's happening in this shot") as HTMLTextAreaElement[];
|
||||
expect(prose.map((p) => p.value)).toEqual(['He (S1) says: ', ' <cutoff>']);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe(text);
|
||||
});
|
||||
|
||||
it('editing a hand-written line keeps the tag after it where it was', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith("[Shot 1] He (S1) says: <d>[English] Sorry.</d> <cutoff>");
|
||||
const [line] = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
await user.type(line, ' Really.');
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe("[Shot 1] He (S1) says: <d>[English] Sorry. Really.</d> <cutoff>");
|
||||
});
|
||||
|
||||
it('reorders dialogue lines with the move buttons', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith('[Shot 1] Wide shot. (S1) <d>[English] One.</d> (S2) <d>[English] Two.</d>');
|
||||
await user.click(screen.getByRole('button', { name: 'Move line 2 up' }));
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe(
|
||||
'[Shot 1] Wide shot. (S2) <d>[English] Two.</d> (S1) <d>[English] One.</d>',
|
||||
);
|
||||
});
|
||||
|
||||
it('adds a text block after the existing content', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith('[Shot 1] Wide shot. A dock.');
|
||||
await user.click(screen.getByRole('button', { name: '+ Add text' }));
|
||||
const prose = screen.getAllByPlaceholderText("What's happening in this shot") as HTMLTextAreaElement[];
|
||||
expect(prose).toHaveLength(2);
|
||||
await user.type(prose[1], 'Rain falls.');
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe('[Shot 1] Wide shot. A dock. Rain falls.');
|
||||
});
|
||||
|
||||
it('removes a text block', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith('[Shot 1] Wide shot. A dock. (S1) <d>[English] Hi.</d>');
|
||||
await user.click(screen.getByRole('button', { name: 'Remove text 1' }));
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe('[Shot 1] Wide shot. (S1) <d>[English] Hi.</d>');
|
||||
});
|
||||
|
||||
it('adds a new line at the end of the shot, after any closing prose', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith('[Shot 1] Wide shot. He speaks. (S1) <d>[English] First.</d> He pauses.');
|
||||
await user.click(screen.getByRole('button', { name: '+ Add line' }));
|
||||
const lines = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
await user.type(lines[1], 'Later.');
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe(
|
||||
'[Shot 1] Wide shot. He speaks. (S1) <d>[English] First.</d> He pauses. (S1) <d>[English] Later.</d>',
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,123 @@
|
||||
import { render, screen, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { H3ShotListEditor } from './H3ShotListEditor';
|
||||
|
||||
// Original test material -- not from any real screenplay.
|
||||
const SCRIPT = [
|
||||
'CAPTAIN: Sit down. (EXHALES) We need to talk about the ledger.',
|
||||
'',
|
||||
'CLERK: Yes, sir.',
|
||||
'',
|
||||
'CAPTAIN: Three crates are missing and nobody on this dock saw a thing.',
|
||||
].join('\n');
|
||||
|
||||
const SUBJECTS = '<Subject 1> is the captain.\n<Subject 2> is the clerk.';
|
||||
|
||||
function renderEditor(detailed = '', onApply = vi.fn()) {
|
||||
render(
|
||||
<H3ShotListEditor
|
||||
open
|
||||
subjectDefinitions={SUBJECTS}
|
||||
initialDetailedDescription={detailed}
|
||||
initialRetentionAnalysis=""
|
||||
onApply={onApply}
|
||||
onClose={() => {}}
|
||||
/>,
|
||||
);
|
||||
return onApply;
|
||||
}
|
||||
|
||||
async function importScript(user: ReturnType<typeof userEvent.setup>) {
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
const dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
await user.click(within(dialog).getByLabelText('Script'));
|
||||
await user.paste(SCRIPT);
|
||||
return dialog;
|
||||
}
|
||||
|
||||
describe('script import', () => {
|
||||
it('maps speakers to subjects by default and previews the clip', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
const dialog = await importScript(user);
|
||||
|
||||
expect(within(dialog).getByLabelText('CAPTAIN subject')).toHaveValue('<Subject 1>');
|
||||
expect(within(dialog).getByLabelText('CLERK subject')).toHaveValue('<Subject 2>');
|
||||
expect(within(dialog).getByLabelText('CAPTAIN speaker tag')).toHaveValue('S1');
|
||||
expect(within(dialog).getByLabelText('CLERK speaker tag')).toHaveValue('S2');
|
||||
expect(within(dialog).getByRole('option', { name: /Clip 1 of \d+/ })).toBeInTheDocument();
|
||||
expect(within(dialog).getByText('Establishing')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('loads the chosen clip into the shot list and writes it on apply', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderEditor();
|
||||
const dialog = await importScript(user);
|
||||
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
|
||||
const boxes = (await screen.findAllByPlaceholderText('What the speaker says')) as HTMLTextAreaElement[];
|
||||
expect(boxes[0].value).toBe('Sit down. <exhale> We need to talk about the ledger.');
|
||||
expect(boxes[1].value).toBe('Yes, sir.');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
const [detailed, retention] = onApply.mock.calls[0];
|
||||
expect(detailed).toContain('[Shot 1] Establishing shot.');
|
||||
expect(detailed).toContain('(S1) <d>[English] Sit down. <exhale> We need to talk about the ledger.</d> (S2) <d>[English] Yes, sir.</d>');
|
||||
expect(retention).toContain('<Subject 1> (appears in');
|
||||
expect(retention).toContain('<Subject 2> (appears in');
|
||||
});
|
||||
|
||||
it('warns that loading replaces the shots already in the list', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor('[Shot 1] Wide shot. A dock.\n\n[Shot 2] Close-up shot. A face.');
|
||||
const dialog = await importScript(user);
|
||||
expect(within(dialog).getByText('Replaces the 2 shots currently in the list.')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('reloading the same clip after edits restores the imported text', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
let dialog = await importScript(user);
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
|
||||
let boxes = (await screen.findAllByPlaceholderText('What the speaker says')) as HTMLTextAreaElement[];
|
||||
await user.type(boxes[1], ' EDITED');
|
||||
expect(boxes[1].value).toBe('Yes, sir. EDITED');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
// Loading a clip advances the picker to the next one; choose clip 1 again.
|
||||
await user.selectOptions(within(dialog).getByLabelText('Clip to load'), '0');
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
|
||||
boxes = (await screen.findAllByPlaceholderText('What the speaker says')) as HTMLTextAreaElement[];
|
||||
expect(boxes[1].value).toBe('Yes, sir.');
|
||||
});
|
||||
|
||||
it('advances to the next clip after loading one', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
let dialog = await importScript(user);
|
||||
const picker = within(dialog).getByLabelText('Clip to load') as HTMLSelectElement;
|
||||
expect(picker.options.length).toBeGreaterThan(1);
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
expect((within(dialog).getByLabelText('Clip to load') as HTMLSelectElement).value).toBe('1');
|
||||
});
|
||||
|
||||
it('explains when the text has no speaker turns', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
const dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
await user.click(within(dialog).getByLabelText('Script'));
|
||||
await user.paste('just some prose');
|
||||
expect(within(dialog).getByRole('alert')).toHaveTextContent(/No "NAME: line" turns found/);
|
||||
expect(within(dialog).getByRole('button', { name: 'Load clip' })).toBeDisabled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,247 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { buildClips, parseScript, scriptSpeakers, type ImportSpeaker } from '@/lib/h3ScriptImport';
|
||||
import { SHOT_TYPES, shotLines, shotProse, type Shot } from '@/lib/h3Shots';
|
||||
|
||||
const TYPE_LABEL = Object.fromEntries(SHOT_TYPES.map((t) => [t.id, t.label]));
|
||||
const DEFAULT_CLIP_SECONDS = 11;
|
||||
|
||||
export interface H3ScriptImportDialogProps {
|
||||
open: boolean;
|
||||
subjectLabels: string[];
|
||||
/** shots currently in the list -- loading a clip replaces them */
|
||||
existingShotCount: number;
|
||||
onLoad: (shots: Shot[]) => void;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
function preview(shot: Shot): string {
|
||||
const line = shotLines(shot).find((l) => l.text.trim());
|
||||
const text = line ? `${line.speaker ? `${line.speaker}: ` : ''}“${line.text.trim()}”` : shotProse(shot);
|
||||
return text.length > 90 ? `${text.slice(0, 89)}…` : text;
|
||||
}
|
||||
|
||||
/**
|
||||
* Paste (or load) a "NAME: line" script and pick one clip-sized chunk of it
|
||||
* to load into the shot list. The script and speaker mapping persist between
|
||||
* openings, so working through a long scene is: load clip, apply, next clip.
|
||||
*/
|
||||
export function H3ScriptImportDialog({
|
||||
open,
|
||||
subjectLabels,
|
||||
existingShotCount,
|
||||
onLoad,
|
||||
onClose,
|
||||
}: H3ScriptImportDialogProps) {
|
||||
const [script, setScript] = React.useState('');
|
||||
const [clipSecondsInput, setClipSecondsInput] = React.useState(String(DEFAULT_CLIP_SECONDS));
|
||||
const [overrides, setOverrides] = React.useState<Record<string, Partial<ImportSpeaker>>>({});
|
||||
const [clipIndex, setClipIndex] = React.useState(0);
|
||||
|
||||
const turns = React.useMemo(() => parseScript(script), [script]);
|
||||
const names = React.useMemo(() => scriptSpeakers(turns), [turns]);
|
||||
|
||||
const speakers = React.useMemo(() => {
|
||||
const map: Record<string, ImportSpeaker> = {};
|
||||
names.forEach((name, i) => {
|
||||
const o = overrides[name] ?? {};
|
||||
map[name] = {
|
||||
subject: 'subject' in o ? (o.subject ?? null) : (subjectLabels[i] ?? null),
|
||||
tag: o.tag ?? `S${i + 1}`,
|
||||
};
|
||||
});
|
||||
return map;
|
||||
}, [names, overrides, subjectLabels]);
|
||||
|
||||
const parsedSeconds = Number(clipSecondsInput);
|
||||
const targetSeconds = Number.isFinite(parsedSeconds) ? Math.min(Math.max(parsedSeconds, 4), 12) : DEFAULT_CLIP_SECONDS;
|
||||
|
||||
const clips = React.useMemo(
|
||||
() => (turns.length ? buildClips(turns, { speakers, targetSeconds }) : []),
|
||||
[turns, speakers, targetSeconds],
|
||||
);
|
||||
const selected = Math.min(clipIndex, Math.max(clips.length - 1, 0));
|
||||
const clip = clips[selected];
|
||||
|
||||
function setOverride(name: string, patch: Partial<ImportSpeaker>) {
|
||||
setOverrides((prev) => ({ ...prev, [name]: { ...prev[name], ...patch } }));
|
||||
}
|
||||
|
||||
async function handleFile(e: React.ChangeEvent<HTMLInputElement>) {
|
||||
const file = e.target.files?.[0];
|
||||
e.target.value = '';
|
||||
if (!file) return;
|
||||
setScript(await file.text());
|
||||
setClipIndex(0);
|
||||
}
|
||||
|
||||
function handleLoad() {
|
||||
if (!clip) return;
|
||||
onLoad(clip.shots);
|
||||
setClipIndex(Math.min(selected + 1, clips.length - 1));
|
||||
onClose();
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(o) => { if (!o) onClose(); }}>
|
||||
<DialogContent className="flex max-h-[90vh] w-[94vw] max-w-[760px] flex-col gap-0 overflow-hidden p-0">
|
||||
<DialogHeader className="border-b border-border px-5 py-4">
|
||||
<DialogTitle>Import script</DialogTitle>
|
||||
<DialogDescription>
|
||||
Paste a script of NAME: line turns. It's split into clip-sized chunks; load one at a time into the
|
||||
shot list, then set the camera and descriptions.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
<div className="flex min-h-0 flex-1 flex-col gap-4 overflow-y-auto p-5">
|
||||
<div className="flex flex-col gap-2">
|
||||
<label htmlFor="script-import-text" className="text-xs font-semibold text-foreground">
|
||||
Script
|
||||
</label>
|
||||
<Textarea
|
||||
id="script-import-text"
|
||||
value={script}
|
||||
onChange={(e) => {
|
||||
setScript(e.target.value);
|
||||
setClipIndex(0);
|
||||
}}
|
||||
rows={8}
|
||||
spellCheck={false}
|
||||
placeholder={'MARK: First line...\n\nJORDAN: Reply...'}
|
||||
className="font-mono text-xs"
|
||||
/>
|
||||
<Input
|
||||
type="file"
|
||||
accept=".md,.txt,.text"
|
||||
aria-label="Load script file"
|
||||
onChange={handleFile}
|
||||
className="h-auto py-2 file:mr-3 file:cursor-pointer file:rounded-md file:border-0 file:bg-secondary file:px-2 file:py-1 file:text-sm file:text-secondary-foreground"
|
||||
/>
|
||||
</div>
|
||||
|
||||
{script.trim() && turns.length === 0 && (
|
||||
<p role="alert" className="text-sm text-destructive">
|
||||
No "NAME: line" turns found. Each turn should start with a speaker name and a colon.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{names.length > 0 && (
|
||||
<div className="flex flex-col gap-2">
|
||||
<span className="text-xs font-semibold text-foreground">Speakers</span>
|
||||
{names.map((name) => (
|
||||
<div key={name} className="grid grid-cols-[1fr_1fr_5rem] items-center gap-2">
|
||||
<span className="truncate text-sm text-foreground">{name}</span>
|
||||
<NativeSelect
|
||||
aria-label={`${name} subject`}
|
||||
value={speakers[name].subject ?? ''}
|
||||
onChange={(e) => setOverride(name, { subject: e.target.value || null })}
|
||||
>
|
||||
<option value="">Not a tracked subject</option>
|
||||
{subjectLabels.map((label) => (
|
||||
<option key={label} value={label}>{label}</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
<Input
|
||||
aria-label={`${name} speaker tag`}
|
||||
value={speakers[name].tag}
|
||||
onChange={(e) => setOverride(name, { tag: e.target.value })}
|
||||
className="font-mono"
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
{subjectLabels.length === 0 && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
No subjects are defined yet, so speakers can't be linked. Add them under Subject definitions
|
||||
to track who appears in each shot.
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{turns.length > 0 && (
|
||||
<>
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Clip length target (seconds)
|
||||
<Input
|
||||
type="number"
|
||||
min={4}
|
||||
max={12}
|
||||
step={1}
|
||||
value={clipSecondsInput}
|
||||
onChange={(e) => {
|
||||
setClipSecondsInput(e.target.value);
|
||||
setClipIndex(0);
|
||||
}}
|
||||
className="w-28"
|
||||
/>
|
||||
<span>One generation runs about 11 s (277 frames at 24 fps), so each clip becomes its own job.</span>
|
||||
</label>
|
||||
|
||||
<div className="flex flex-col gap-2">
|
||||
<label htmlFor="script-import-clip" className="text-xs font-semibold text-foreground">
|
||||
Clip to load
|
||||
</label>
|
||||
<NativeSelect
|
||||
id="script-import-clip"
|
||||
value={selected}
|
||||
onChange={(e) => setClipIndex(Number(e.target.value))}
|
||||
>
|
||||
{clips.map((c, i) => (
|
||||
<option key={i} value={i}>
|
||||
Clip {i + 1} of {clips.length} · {c.shots.length} shot{c.shots.length === 1 ? '' : 's'}
|
||||
{c.continuation ? ' · continues previous shot' : ''}
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
|
||||
{clip?.continuation && (
|
||||
<p className="rounded-lg border border-border bg-secondary/30 px-3 py-2 text-xs text-muted-foreground">
|
||||
This clip continues the previous clip's last shot, cut short because that speech didn't
|
||||
fit one generation. Generate it from the previous clip's last frame (as the start Image)
|
||||
with the same references.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{clip && (
|
||||
<ol className="flex flex-col gap-1 rounded-xl border border-border bg-secondary/20 p-3">
|
||||
{clip.shots.map((shot, i) => (
|
||||
<li key={shot.id} className="flex gap-2 text-xs">
|
||||
<span className="w-6 shrink-0 font-mono text-muted-foreground">{i + 1}</span>
|
||||
<span className="w-24 shrink-0 font-medium text-foreground">{TYPE_LABEL[shot.type]}</span>
|
||||
<span className="min-w-0 text-muted-foreground">{preview(shot)}</span>
|
||||
</li>
|
||||
))}
|
||||
</ol>
|
||||
)}
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<DialogFooter className="items-center border-t border-border px-5 py-3">
|
||||
{existingShotCount > 0 && clip && (
|
||||
<span className="mr-auto text-xs text-muted-foreground">
|
||||
Replaces the {existingShotCount} shot{existingShotCount === 1 ? '' : 's'} currently in the list.
|
||||
</span>
|
||||
)}
|
||||
<Button type="button" variant="outline" onClick={onClose}>Cancel</Button>
|
||||
<Button type="button" onClick={handleLoad} disabled={!clip}>Load clip</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { H3DialoguePreview } from '@/components/jobs/H3DialoguePreview';
|
||||
import { applyH3Preset, applyH3Tag, type H3Preset, type H3Tag } from '@/lib/h3Tags';
|
||||
|
||||
/** What the shared tag palette drives: the last-focused shot dialogue field. */
|
||||
export interface H3ShotDialogueHandle {
|
||||
insertTag: (tag: H3Tag) => void;
|
||||
applyPreset: (preset: H3Preset) => void;
|
||||
}
|
||||
|
||||
export interface H3ShotDialogueFieldProps {
|
||||
/** The line's text. The parent owns it (and the undo history), so this is fully controlled. */
|
||||
value: string;
|
||||
/** Typing. */
|
||||
onChange: (value: string) => void;
|
||||
/** A tag or preset inserted from the palette -- a discrete edit, unlike typing. */
|
||||
onInsert: (value: string) => void;
|
||||
onFocus: () => void;
|
||||
active?: boolean;
|
||||
label?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* One spoken line: a textarea plus, once it contains tags, a styled preview.
|
||||
* The shared palette inserts at the cursor through the ref. Undo/redo isn't
|
||||
* here -- the shot list editor keeps one history for everything in it.
|
||||
*/
|
||||
export const H3ShotDialogueField = React.forwardRef<H3ShotDialogueHandle, H3ShotDialogueFieldProps>(
|
||||
function H3ShotDialogueField({ value, onChange, onInsert, onFocus, active, label = 'Dialogue' }, ref) {
|
||||
const taRef = React.useRef<HTMLTextAreaElement>(null);
|
||||
// Selection to restore once the inserted text has rendered.
|
||||
const pending = React.useRef<[number, number] | null>(null);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (pending.current && taRef.current) {
|
||||
const [s, e] = pending.current;
|
||||
taRef.current.focus();
|
||||
taRef.current.setSelectionRange(s, e);
|
||||
pending.current = null;
|
||||
}
|
||||
});
|
||||
|
||||
function selection(): [number, number] {
|
||||
const ta = taRef.current;
|
||||
return [ta?.selectionStart ?? value.length, ta?.selectionEnd ?? value.length];
|
||||
}
|
||||
|
||||
React.useImperativeHandle(ref, () => ({
|
||||
insertTag(tag) {
|
||||
const [s, e] = selection();
|
||||
const result = applyH3Tag(value, s, e, tag);
|
||||
pending.current = [result.start, result.end];
|
||||
onInsert(result.value);
|
||||
},
|
||||
applyPreset(preset) {
|
||||
const [s, e] = selection();
|
||||
const result = applyH3Preset(value, s, e, preset);
|
||||
pending.current = [result.start, result.end];
|
||||
onInsert(result.value);
|
||||
},
|
||||
}));
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-1">
|
||||
<div className="text-xs text-muted-foreground">
|
||||
{label}
|
||||
{active && <span className="ml-2 text-accent-blue">tags insert here</span>}
|
||||
</div>
|
||||
<Textarea
|
||||
ref={taRef}
|
||||
value={value}
|
||||
onChange={(e) => onChange(e.target.value)}
|
||||
onFocus={onFocus}
|
||||
rows={2}
|
||||
spellCheck={false}
|
||||
placeholder="What the speaker says"
|
||||
className="font-mono text-sm"
|
||||
/>
|
||||
{value.includes('<') && (
|
||||
<H3DialoguePreview
|
||||
text={value}
|
||||
className="rounded-lg border border-border bg-card px-3 py-2 text-sm leading-7 text-foreground"
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,690 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { H3ShotDialogueField, type H3ShotDialogueHandle } from '@/components/jobs/H3ShotDialogueField';
|
||||
import { H3ScriptImportDialog } from '@/components/jobs/H3ScriptImportDialog';
|
||||
import { H3ShotStoryboard } from '@/components/jobs/H3ShotStoryboard';
|
||||
import { H3TagPalette } from '@/components/jobs/H3TagPalette';
|
||||
import type { H3Preset, H3Tag } from '@/lib/h3Tags';
|
||||
import {
|
||||
CAMERA_MOVEMENTS,
|
||||
SHOT_TYPES,
|
||||
detailedDescriptionFromShots,
|
||||
formatCutTime,
|
||||
linePart,
|
||||
newLine,
|
||||
newShot,
|
||||
newTextPart,
|
||||
parseCutTime,
|
||||
parseDetailedDescription,
|
||||
parseSubjectLabels,
|
||||
resolveShotStarts,
|
||||
retentionAnalysisFromShots,
|
||||
shotLines,
|
||||
withFreshIds,
|
||||
type CameraMovement,
|
||||
type DialogueLine,
|
||||
type LinePart,
|
||||
type ShotPart,
|
||||
type Shot,
|
||||
type ShotType,
|
||||
} from '@/lib/h3Shots';
|
||||
import { useUndoableState } from '@/hooks/useUndoableState';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface H3ShotListEditorProps {
|
||||
open: boolean;
|
||||
/** Raw subject_definitions text -- parsed for <Subject N> labels to track per shot. */
|
||||
subjectDefinitions: string;
|
||||
initialDetailedDescription: string;
|
||||
initialRetentionAnalysis: string;
|
||||
onApply: (detailedDescription: string, retentionAnalysis: string) => void;
|
||||
onClose: () => void;
|
||||
/** Where the shots get written, for the description text. */
|
||||
targetLabel?: string;
|
||||
/** False when there is no retention analysis to keep in sync (base-format and plain prompts). */
|
||||
withRetention?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Structured camera-direction editor for MiniMax-H3: an ordered shot list
|
||||
* (type, movement, which subjects appear, action, optional dialogue) that
|
||||
* writes into detailed_description's [Shot N] blocks and keeps
|
||||
* retention_analysis's "appears in [...]" lines in sync automatically,
|
||||
* instead of hand-maintaining both fields' free text in agreement.
|
||||
*/
|
||||
/** Cut-in time field: blank means "automatic" (shown as the placeholder); Enter or blur commits. */
|
||||
function CutTimeInput({
|
||||
value,
|
||||
auto,
|
||||
disabled,
|
||||
onCommit,
|
||||
}: {
|
||||
value: number | null;
|
||||
auto: number;
|
||||
disabled?: boolean;
|
||||
onCommit: (seconds: number | null) => void;
|
||||
}) {
|
||||
const show = (v: number | null) => (v == null ? '' : formatCutTime(v));
|
||||
const [text, setText] = React.useState(show(value));
|
||||
React.useEffect(() => setText(show(value)), [value]);
|
||||
|
||||
function commit() {
|
||||
const trimmed = text.trim();
|
||||
if (!trimmed) return onCommit(null);
|
||||
const parsed = parseCutTime(trimmed);
|
||||
if (parsed == null) return setText(show(value));
|
||||
onCommit(parsed);
|
||||
}
|
||||
|
||||
return (
|
||||
<Input
|
||||
value={text}
|
||||
onChange={(e) => setText(e.target.value)}
|
||||
onBlur={commit}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter') {
|
||||
e.preventDefault();
|
||||
commit();
|
||||
}
|
||||
}}
|
||||
placeholder={disabled ? 'Opening shot' : formatCutTime(auto)}
|
||||
disabled={disabled}
|
||||
inputMode="decimal"
|
||||
className="font-mono"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
interface EditorState {
|
||||
shots: Shot[];
|
||||
preamble: string;
|
||||
}
|
||||
|
||||
export function H3ShotListEditor({
|
||||
open,
|
||||
subjectDefinitions,
|
||||
initialDetailedDescription,
|
||||
initialRetentionAnalysis,
|
||||
onApply,
|
||||
onClose,
|
||||
targetLabel = 'Detailed description',
|
||||
withRetention = true,
|
||||
}: H3ShotListEditorProps) {
|
||||
// One history covers everything editable here: the shots (structure, fields and
|
||||
// every line of text) and the scene notes. Typing carries a key so a burst
|
||||
// merges into one step; everything else is a step of its own.
|
||||
const history = useUndoableState<EditorState>({ shots: [], preamble: '' });
|
||||
const { shots, preamble } = history.state;
|
||||
// Returning the same list (or text) hands the history back the same state object,
|
||||
// which it treats as "nothing changed" and does not record.
|
||||
const editShots = (fn: (shots: Shot[]) => Shot[], key: string | null = null) =>
|
||||
history.set((s) => {
|
||||
const next = fn(s.shots);
|
||||
return next === s.shots ? s : { ...s, shots: next };
|
||||
}, key);
|
||||
const setPreamble = (value: string) =>
|
||||
history.set((s) => (s.preamble === value ? s : { ...s, preamble: value }), 'preamble');
|
||||
const [selectedId, setSelectedId] = React.useState<string | null>(null);
|
||||
// The shared tag palette inserts into whichever shot's dialogue was last focused.
|
||||
const [dialogueTargetId, setDialogueTargetId] = React.useState<string | null>(null);
|
||||
const dialogueFields = React.useRef(new Map<string, H3ShotDialogueHandle>());
|
||||
const [importOpen, setImportOpen] = React.useState(false);
|
||||
const subjectLabels = React.useMemo(() => parseSubjectLabels(subjectDefinitions), [subjectDefinitions]);
|
||||
|
||||
// Re-seed from the field's current text each time the editor opens, the
|
||||
// same "seed on the open transition only" guard H3DialogueEditor uses --
|
||||
// otherwise a stray parent re-render while open could clobber in-progress edits.
|
||||
const wasOpenRef = React.useRef(false);
|
||||
React.useEffect(() => {
|
||||
if (open && !wasOpenRef.current) {
|
||||
const { preamble: seededPreamble, shots: seeded } = parseDetailedDescription(
|
||||
initialDetailedDescription,
|
||||
subjectLabels,
|
||||
);
|
||||
history.reset({ preamble: seededPreamble, shots: seeded });
|
||||
setSelectedId(seeded[0]?.id ?? null);
|
||||
setDialogueTargetId(null);
|
||||
}
|
||||
wasOpenRef.current = open;
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [open]);
|
||||
|
||||
function addShot() {
|
||||
const shot = newShot();
|
||||
editShots((prev) => [...prev, shot]);
|
||||
selectShot(shot.id);
|
||||
}
|
||||
|
||||
// Storyboard panel -> jump to that shot's editor card.
|
||||
function selectShot(id: string) {
|
||||
setSelectedId(id);
|
||||
requestAnimationFrame(() =>
|
||||
document.getElementById(`shot-card-${id}`)?.scrollIntoView({ behavior: 'smooth', block: 'nearest' }),
|
||||
);
|
||||
}
|
||||
|
||||
function removeShot(id: string) {
|
||||
editShots((prev) => prev.filter((s) => s.id !== id));
|
||||
setDialogueTargetId((cur) => (cur === id ? null : cur));
|
||||
}
|
||||
|
||||
function insertTag(tag: H3Tag) {
|
||||
if (dialogueTargetId) dialogueFields.current.get(dialogueTargetId)?.insertTag(tag);
|
||||
}
|
||||
|
||||
function applyPreset(preset: H3Preset) {
|
||||
if (dialogueTargetId) dialogueFields.current.get(dialogueTargetId)?.applyPreset(preset);
|
||||
}
|
||||
|
||||
const partKey = (p: ShotPart) => (p.kind === 'text' ? p.id : p.line.id);
|
||||
|
||||
// An edit that changes nothing returns the same list and so records no undo step
|
||||
// (a blur that re-commits the same value, for instance).
|
||||
function updateBody(shotId: string, fn: (body: ShotPart[]) => ShotPart[], editKey: string | null = null) {
|
||||
editShots((prev) => {
|
||||
let changed = false;
|
||||
const next = prev.map((s) => {
|
||||
if (s.id !== shotId) return s;
|
||||
const body = fn(s.body);
|
||||
if (body === s.body) return s;
|
||||
changed = true;
|
||||
return { ...s, body };
|
||||
});
|
||||
return changed ? next : prev;
|
||||
}, editKey);
|
||||
}
|
||||
|
||||
function addLine(shotId: string) {
|
||||
updateBody(shotId, (body) => {
|
||||
const lastSpeaker = [...body].reverse().find((p): p is LinePart => p.kind === 'line')?.line.speaker;
|
||||
return [...body, linePart(newLine({ speaker: lastSpeaker ?? 'S1' }))];
|
||||
});
|
||||
}
|
||||
|
||||
function addText(shotId: string) {
|
||||
updateBody(shotId, (body) => [...body, newTextPart()]);
|
||||
}
|
||||
|
||||
function removePart(shotId: string, key: string) {
|
||||
updateBody(shotId, (body) => body.filter((p) => partKey(p) !== key));
|
||||
setDialogueTargetId((cur) => (cur === key ? null : cur));
|
||||
}
|
||||
|
||||
function movePart(shotId: string, key: string, dir: -1 | 1) {
|
||||
updateBody(shotId, (body) => {
|
||||
const i = body.findIndex((p) => partKey(p) === key);
|
||||
const j = i + dir;
|
||||
if (i < 0 || j < 0 || j >= body.length) return body;
|
||||
const next = [...body];
|
||||
[next[i], next[j]] = [next[j], next[i]];
|
||||
return next;
|
||||
});
|
||||
}
|
||||
|
||||
function updateText(shotId: string, key: string, text: string) {
|
||||
updateBody(
|
||||
shotId,
|
||||
(body) => {
|
||||
const part = body.find((p) => p.kind === 'text' && p.id === key);
|
||||
if (!part || (part.kind === 'text' && part.text === text)) return body;
|
||||
return body.map((p) => (p === part ? { ...part, text } : p));
|
||||
},
|
||||
`text:${key}`,
|
||||
);
|
||||
}
|
||||
|
||||
/** `editKey` groups typing into one undo step; omit it for a discrete edit like a tag insertion. */
|
||||
function updateLine(shotId: string, lineId: string, patch: Partial<DialogueLine>, editKey?: string) {
|
||||
updateBody(
|
||||
shotId,
|
||||
(body) => {
|
||||
const part = body.find((p) => p.kind === 'line' && p.line.id === lineId);
|
||||
if (!part || part.kind !== 'line') return body;
|
||||
const unchanged = (Object.keys(patch) as (keyof DialogueLine)[]).every((k) => part.line[k] === patch[k]);
|
||||
return unchanged ? body : body.map((p) => (p === part ? { ...part, line: { ...part.line, ...patch } } : p));
|
||||
},
|
||||
editKey ?? null,
|
||||
);
|
||||
}
|
||||
|
||||
const undo = () => void history.undo();
|
||||
const redo = () => void history.redo();
|
||||
|
||||
// Undo/redo for the whole list, from anywhere inside it -- including from a textbox,
|
||||
// where the browser's own undo would only know about that one box. Not while the
|
||||
// script import (nested inside this dialog) is open: its box keeps native undo.
|
||||
function handleKeyDown(e: React.KeyboardEvent) {
|
||||
if (importOpen || e.altKey || !(e.metaKey || e.ctrlKey)) return;
|
||||
const k = e.key.toLowerCase();
|
||||
if (k === 'z') {
|
||||
e.preventDefault();
|
||||
if (e.shiftKey) redo();
|
||||
else undo();
|
||||
} else if (k === 'y' && !e.shiftKey) {
|
||||
e.preventDefault();
|
||||
redo();
|
||||
}
|
||||
}
|
||||
|
||||
// An imported clip replaces the list wholesale, as one undoable step.
|
||||
function loadImportedShots(imported: Shot[]) {
|
||||
const next = withFreshIds(imported);
|
||||
editShots(() => next);
|
||||
setSelectedId(next[0]?.id ?? null);
|
||||
setDialogueTargetId(null);
|
||||
}
|
||||
|
||||
const dialogueTarget = (() => {
|
||||
for (let i = 0; i < shots.length; i += 1) {
|
||||
const lines = shotLines(shots[i]);
|
||||
const li = lines.findIndex((l) => l.id === dialogueTargetId);
|
||||
if (li >= 0) return { shot: i + 1, line: li + 1, multi: lines.length > 1 };
|
||||
}
|
||||
return null;
|
||||
})();
|
||||
|
||||
function moveShot(id: string, dir: -1 | 1) {
|
||||
editShots((prev) => {
|
||||
const i = prev.findIndex((s) => s.id === id);
|
||||
const j = i + dir;
|
||||
if (i < 0 || j < 0 || j >= prev.length) return prev;
|
||||
const next = [...prev];
|
||||
[next[i], next[j]] = [next[j], next[i]];
|
||||
return next;
|
||||
});
|
||||
}
|
||||
|
||||
function updateShot(id: string, patch: Partial<Shot>) {
|
||||
editShots((prev) => {
|
||||
const shot = prev.find((s) => s.id === id);
|
||||
const unchanged = shot && (Object.keys(patch) as (keyof Shot)[]).every((k) => shot[k] === patch[k]);
|
||||
return !shot || unchanged ? prev : prev.map((s) => (s === shot ? { ...s, ...patch } : s));
|
||||
});
|
||||
}
|
||||
|
||||
function toggleSubject(id: string, label: string) {
|
||||
editShots((prev) =>
|
||||
prev.map((s) =>
|
||||
s.id === id
|
||||
? {
|
||||
...s,
|
||||
subjectIds: s.subjectIds.includes(label)
|
||||
? s.subjectIds.filter((l) => l !== label)
|
||||
: [...s.subjectIds, label],
|
||||
}
|
||||
: s,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
const starts = React.useMemo(() => resolveShotStarts(shots), [shots]);
|
||||
const previewDetailedDescription = React.useMemo(
|
||||
() => detailedDescriptionFromShots(shots, preamble),
|
||||
[shots, preamble],
|
||||
);
|
||||
const previewRetentionAnalysis = React.useMemo(
|
||||
() => (withRetention ? retentionAnalysisFromShots(shots, subjectLabels, initialRetentionAnalysis) : ''),
|
||||
[withRetention, shots, subjectLabels, initialRetentionAnalysis],
|
||||
);
|
||||
|
||||
function handleApply() {
|
||||
onApply(previewDetailedDescription, previewRetentionAnalysis);
|
||||
onClose();
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(o) => { if (!o) onClose(); }}>
|
||||
<DialogContent
|
||||
onKeyDown={handleKeyDown}
|
||||
className="flex h-[92vh] max-h-[920px] w-[96vw] max-w-[1100px] flex-col gap-0 overflow-hidden p-0"
|
||||
>
|
||||
<DialogHeader className="border-b border-border px-5 py-4">
|
||||
<DialogTitle>Shot list</DialogTitle>
|
||||
<DialogDescription>
|
||||
Build the scene shot by shot. Applying writes [Shot N] blocks into {targetLabel}
|
||||
{withRetention ? ' and keeps Retention analysis\u2019s \u201cappears in\u201d lines in sync' : ''}.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
<div className="flex shrink-0 flex-col gap-2 border-b border-border bg-secondary/10 px-4 py-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-xs font-semibold text-foreground">Storyboard</span>
|
||||
<span className="flex items-center gap-1.5">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={undo}
|
||||
disabled={!history.canUndo}
|
||||
title="Undo (Ctrl/Cmd+Z)"
|
||||
>
|
||||
↶ Undo
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={redo}
|
||||
disabled={!history.canRedo}
|
||||
title="Redo (Ctrl/Cmd+Shift+Z)"
|
||||
>
|
||||
↷ Redo
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => setImportOpen(true)}>
|
||||
Import script…
|
||||
</Button>
|
||||
</span>
|
||||
</div>
|
||||
<H3ShotStoryboard shots={shots} selectedId={selectedId} onSelect={selectShot} onAdd={addShot} />
|
||||
</div>
|
||||
|
||||
<div className="flex min-h-0 flex-1 flex-col md:flex-row">
|
||||
<div className="flex min-h-0 min-w-0 flex-1 flex-col gap-4 overflow-y-auto p-4">
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Scene notes (optional, written before Shot 1 -- style, lighting, anything true of the whole clip)
|
||||
<Textarea
|
||||
value={preamble}
|
||||
onChange={(e) => setPreamble(e.target.value)}
|
||||
rows={2}
|
||||
placeholder="e.g. Live-action, cinematic, shallow depth of field."
|
||||
/>
|
||||
</label>
|
||||
|
||||
{shots.length === 0 && (
|
||||
<p className="rounded-xl border border-dashed border-border p-4 text-sm text-muted-foreground">
|
||||
No shots yet.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{shots.map((shot, i) => (
|
||||
<div
|
||||
key={shot.id}
|
||||
id={`shot-card-${shot.id}`}
|
||||
onFocusCapture={() => setSelectedId(shot.id)}
|
||||
className={cn(
|
||||
'flex flex-col gap-3 rounded-xl border bg-secondary/20 p-3',
|
||||
shot.id === selectedId ? 'border-accent-blue' : 'border-border',
|
||||
)}
|
||||
>
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm font-semibold text-foreground">
|
||||
Shot {i + 1}
|
||||
{i > 0 && (
|
||||
<span className="ml-2 font-mono text-xs font-normal text-muted-foreground">
|
||||
cuts in at {formatCutTime(starts[i])}
|
||||
</span>
|
||||
)}
|
||||
</span>
|
||||
<div className="flex items-center gap-1">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => moveShot(shot.id, -1)}
|
||||
disabled={i === 0}
|
||||
title="Move up"
|
||||
>
|
||||
↑
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => moveShot(shot.id, 1)}
|
||||
disabled={i === shots.length - 1}
|
||||
title="Move down"
|
||||
>
|
||||
↓
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => removeShot(shot.id)}>
|
||||
Remove
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 gap-2 sm:grid-cols-3">
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Shot type
|
||||
<NativeSelect
|
||||
value={shot.type}
|
||||
onChange={(e) => updateShot(shot.id, { type: e.target.value as ShotType, freeLead: false })}
|
||||
>
|
||||
{SHOT_TYPES.map((t) => (
|
||||
<option key={t.id} value={t.id}>{t.label}</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</label>
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Camera movement
|
||||
<NativeSelect
|
||||
value={shot.movement}
|
||||
onChange={(e) => updateShot(shot.id, { movement: e.target.value as CameraMovement, freeLead: false })}
|
||||
>
|
||||
{CAMERA_MOVEMENTS.map((m) => (
|
||||
<option key={m.id} value={m.id}>{m.label}</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</label>
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Cut time (blank = automatic)
|
||||
<CutTimeInput
|
||||
value={shot.startSeconds}
|
||||
auto={starts[i]}
|
||||
disabled={i === 0}
|
||||
onCommit={(startSeconds) => updateShot(shot.id, { startSeconds })}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
{shot.freeLead && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
This shot keeps its original wording, so type and movement aren't written. Changing either
|
||||
adds a generated lead-in in front of the text below.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{subjectLabels.length > 0 && (
|
||||
<div className="flex flex-col gap-1">
|
||||
<span className="text-xs text-muted-foreground">Subjects in this shot</span>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{subjectLabels.map((label) => {
|
||||
const active = shot.subjectIds.includes(label);
|
||||
return (
|
||||
<button
|
||||
key={label}
|
||||
type="button"
|
||||
aria-pressed={active}
|
||||
onClick={() => toggleSubject(shot.id, label)}
|
||||
className={cn(
|
||||
'rounded-full border px-2 py-0.5 font-mono text-[11px] transition-colors',
|
||||
active
|
||||
? 'border-accent-blue bg-accent-blue/15 text-accent-blue'
|
||||
: 'border-input bg-card text-muted-foreground hover:bg-accent',
|
||||
)}
|
||||
>
|
||||
{label}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex flex-col gap-2">
|
||||
<span className="text-xs text-muted-foreground">
|
||||
Description and dialogue, in the order they play. Dialogue is wrapped in <d> on apply.
|
||||
</span>
|
||||
{(() => {
|
||||
let lineNo = 0;
|
||||
let textNo = 0;
|
||||
return shot.body.map((part, bi) => {
|
||||
const key = partKey(part);
|
||||
const noun = part.kind === 'line' ? `line ${(lineNo += 1)}` : `text ${(textNo += 1)}`;
|
||||
const controls = (
|
||||
<div className="mt-6 flex shrink-0 gap-1">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => movePart(shot.id, key, -1)}
|
||||
disabled={bi === 0}
|
||||
aria-label={`Move ${noun} up`}
|
||||
title="Move up"
|
||||
>
|
||||
↑
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => movePart(shot.id, key, 1)}
|
||||
disabled={bi === shot.body.length - 1}
|
||||
aria-label={`Move ${noun} down`}
|
||||
title="Move down"
|
||||
>
|
||||
↓
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => removePart(shot.id, key)}
|
||||
aria-label={`Remove ${noun}`}
|
||||
title="Remove"
|
||||
>
|
||||
×
|
||||
</Button>
|
||||
</div>
|
||||
);
|
||||
|
||||
if (part.kind === 'text') {
|
||||
return (
|
||||
<div key={key} className="flex items-start gap-2">
|
||||
<label className="flex min-w-0 flex-1 flex-col gap-1 text-xs text-muted-foreground">
|
||||
{textNo === 1 ? 'Description (composition, action, environment)' : `Text ${textNo}`}
|
||||
<Textarea
|
||||
value={part.text}
|
||||
onChange={(e) => updateText(shot.id, key, e.target.value)}
|
||||
rows={2}
|
||||
placeholder="What's happening in this shot"
|
||||
/>
|
||||
</label>
|
||||
{controls}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const line = part.line;
|
||||
return (
|
||||
<div key={key} className="flex items-start gap-2">
|
||||
<Input
|
||||
value={line.speaker}
|
||||
onChange={(e) => updateLine(shot.id, line.id, { speaker: e.target.value }, `speaker:${line.id}`)}
|
||||
placeholder="S1"
|
||||
aria-label={`Line ${lineNo} speaker`}
|
||||
className="mt-7 w-16 shrink-0 font-mono"
|
||||
/>
|
||||
<div className="min-w-0 flex-1">
|
||||
<H3ShotDialogueField
|
||||
ref={(handle) => {
|
||||
if (handle) dialogueFields.current.set(line.id, handle);
|
||||
else dialogueFields.current.delete(line.id);
|
||||
}}
|
||||
value={line.text}
|
||||
onChange={(text) => updateLine(shot.id, line.id, { text }, `line:${line.id}`)}
|
||||
onInsert={(text) => updateLine(shot.id, line.id, { text })}
|
||||
onFocus={() => setDialogueTargetId(line.id)}
|
||||
active={line.id === dialogueTargetId}
|
||||
label={`Line ${lineNo}`}
|
||||
/>
|
||||
</div>
|
||||
{controls}
|
||||
</div>
|
||||
);
|
||||
});
|
||||
})()}
|
||||
<div className="flex gap-2">
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => addText(shot.id)}>
|
||||
+ Add text
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => addLine(shot.id)}>
|
||||
+ Add line
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
|
||||
<Button type="button" variant="outline" onClick={addShot} className="self-start">
|
||||
+ Add shot
|
||||
</Button>
|
||||
|
||||
<div className="flex flex-col gap-2 rounded-xl border border-border bg-card p-3">
|
||||
<span className="text-xs font-semibold text-foreground">Preview</span>
|
||||
<div className="flex flex-col gap-1">
|
||||
<span className="text-[11px] font-medium uppercase tracking-wide text-muted-foreground">
|
||||
{targetLabel}
|
||||
</span>
|
||||
<pre className="whitespace-pre-wrap rounded-lg bg-secondary/30 p-2 font-mono text-[11px] text-foreground">
|
||||
{previewDetailedDescription || '—'}
|
||||
</pre>
|
||||
</div>
|
||||
{withRetention && (
|
||||
<div className="flex flex-col gap-1">
|
||||
<span className="text-[11px] font-medium uppercase tracking-wide text-muted-foreground">
|
||||
Retention analysis
|
||||
</span>
|
||||
<pre className="whitespace-pre-wrap rounded-lg bg-secondary/30 p-2 font-mono text-[11px] text-foreground">
|
||||
{previewRetentionAnalysis || '—'}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<aside className="flex max-h-56 shrink-0 flex-col border-t border-border md:max-h-none md:w-64 md:border-l md:border-t-0">
|
||||
<p className="border-b border-border px-3 py-2 text-xs text-muted-foreground">
|
||||
{dialogueTarget
|
||||
? `Tags insert into Shot ${dialogueTarget.shot}${dialogueTarget.multi ? `, line ${dialogueTarget.line}` : ''} dialogue`
|
||||
: 'Click into a shot\'s dialogue to insert tags'}
|
||||
</p>
|
||||
<H3TagPalette
|
||||
onInsertTag={insertTag}
|
||||
onApplyPreset={applyPreset}
|
||||
disabled={!dialogueTarget}
|
||||
className="overflow-y-auto p-3"
|
||||
/>
|
||||
</aside>
|
||||
</div>
|
||||
|
||||
<DialogFooter className="border-t border-border px-5 py-3">
|
||||
<Button type="button" variant="outline" onClick={onClose}>Cancel</Button>
|
||||
<Button type="button" onClick={handleApply}>Apply</Button>
|
||||
</DialogFooter>
|
||||
|
||||
{/* Nested so Radix treats it as part of this dialog (see CreateJobModal). */}
|
||||
<H3ScriptImportDialog
|
||||
open={importOpen}
|
||||
subjectLabels={subjectLabels}
|
||||
existingShotCount={shots.length}
|
||||
onLoad={loadImportedShots}
|
||||
onClose={() => setImportOpen(false)}
|
||||
/>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,214 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import {
|
||||
CAMERA_MOVEMENTS,
|
||||
SHOT_TYPES,
|
||||
shotLines,
|
||||
shotProse,
|
||||
type CameraMovement,
|
||||
type Shot,
|
||||
type ShotType,
|
||||
} from '@/lib/h3Shots';
|
||||
|
||||
const TYPE_LABEL = Object.fromEntries(SHOT_TYPES.map((t) => [t.id, t.label])) as Record<ShotType, string>;
|
||||
const MOVEMENT_LABEL = Object.fromEntries(CAMERA_MOVEMENTS.map((m) => [m.id, m.label])) as Record<
|
||||
CameraMovement,
|
||||
string
|
||||
>;
|
||||
|
||||
// Schematic framing per shot type: `scale` sizes the figures and `baseY` is
|
||||
// where their shoulders bottom out in the 160x90 frame, so a wide shot reads
|
||||
// as small distant figures and a close-up as a head filling the frame.
|
||||
const FRAMING: Partial<Record<ShotType, { scale: number; baseY: number }>> = {
|
||||
establishing: { scale: 0.45, baseY: 72 },
|
||||
wide: { scale: 0.8, baseY: 78 },
|
||||
medium: { scale: 1.5, baseY: 92 },
|
||||
'close-up': { scale: 2.6, baseY: 118 },
|
||||
};
|
||||
|
||||
function figureXs(count: number): number[] {
|
||||
if (count <= 1) return [80];
|
||||
if (count === 2) return [55, 105];
|
||||
return [38, 80, 122];
|
||||
}
|
||||
|
||||
function Figure({ x, baseY, scale, opacity }: { x: number; baseY: number; scale: number; opacity: number }) {
|
||||
return (
|
||||
<g fill="currentColor" opacity={opacity}>
|
||||
<circle cx={x} cy={baseY - 30 * scale} r={8 * scale} />
|
||||
<ellipse cx={x} cy={baseY - 12 * scale} rx={14 * scale} ry={12 * scale} />
|
||||
</g>
|
||||
);
|
||||
}
|
||||
|
||||
export function ShotFrame({ shot }: { shot: Shot }) {
|
||||
const count = Math.min(Math.max(shot.subjectIds.length, 1), 3);
|
||||
const empty = shot.subjectIds.length === 0;
|
||||
const opacity = empty ? 0.25 : 0.6;
|
||||
const framing = FRAMING[shot.type];
|
||||
|
||||
let scene: React.ReactNode;
|
||||
if (shot.type === 'extreme-close-up') {
|
||||
scene = (
|
||||
<g fill="currentColor" opacity={opacity}>
|
||||
<circle cx={80} cy={52} r={62} />
|
||||
<ellipse cx={60} cy={42} rx={7} ry={4} className="fill-background" />
|
||||
<ellipse cx={100} cy={42} rx={7} ry={4} className="fill-background" />
|
||||
</g>
|
||||
);
|
||||
} else if (shot.type === 'over-the-shoulder') {
|
||||
scene = (
|
||||
<>
|
||||
<Figure x={22} baseY={135} scale={3} opacity={0.85} />
|
||||
<Figure x={116} baseY={80} scale={1} opacity={opacity} />
|
||||
</>
|
||||
);
|
||||
} else if (shot.type === 'pov') {
|
||||
scene = (
|
||||
<>
|
||||
<Figure x={80} baseY={72} scale={0.9} opacity={opacity} />
|
||||
<path
|
||||
d="M0 0H160V90H0Z M80 45m-52 0a52 40 0 1 0 104 0a52 40 0 1 0 -104 0"
|
||||
fill="currentColor"
|
||||
fillRule="evenodd"
|
||||
opacity={0.3}
|
||||
/>
|
||||
<path d="M74 45H86M80 39V51" stroke="currentColor" strokeWidth={1.5} opacity={0.6} />
|
||||
</>
|
||||
);
|
||||
} else if (shot.type === 'insert') {
|
||||
scene = (
|
||||
<g opacity={0.6} fill="none" stroke="currentColor" strokeWidth={2.5}>
|
||||
<rect x={52} y={16} width={56} height={58} rx={8} />
|
||||
<path d="M64 34H96M64 46H96M64 58H84" strokeLinecap="round" />
|
||||
</g>
|
||||
);
|
||||
} else if (framing) {
|
||||
scene = (
|
||||
<>
|
||||
{shot.type === 'establishing' && (
|
||||
<path d="M0 66H160" stroke="currentColor" strokeWidth={1} opacity={0.35} />
|
||||
)}
|
||||
{figureXs(count).map((x, i) => (
|
||||
<Figure
|
||||
key={x}
|
||||
x={shot.type === 'close-up' ? 80 : x}
|
||||
baseY={framing.baseY}
|
||||
scale={framing.scale}
|
||||
opacity={opacity * (i === 0 ? 1 : 0.75)}
|
||||
/>
|
||||
))}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<svg viewBox="0 0 160 90" className="block h-full w-full text-foreground" aria-hidden>
|
||||
{scene}
|
||||
<MovementOverlay movement={shot.movement} />
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
|
||||
function MovementOverlay({ movement }: { movement: CameraMovement }) {
|
||||
const g = (children: React.ReactNode) => (
|
||||
<g
|
||||
className="text-accent-blue"
|
||||
stroke="currentColor"
|
||||
fill="none"
|
||||
strokeWidth={2.5}
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
>
|
||||
{children}
|
||||
</g>
|
||||
);
|
||||
switch (movement) {
|
||||
case 'pan':
|
||||
return g(<path d="M28 80H132M28 80l6-4M28 80l6 4M132 80l-6-4M132 80l-6 4" />);
|
||||
case 'tilt':
|
||||
return g(<path d="M146 18V72M146 18l-4 6M146 18l4 6M146 72l-4-6M146 72l4-6" />);
|
||||
case 'dolly':
|
||||
return g(<path d="M10 86L54 56M150 86L106 56M10 86H150" strokeDasharray="4 4" />);
|
||||
case 'zoom':
|
||||
return g(<path d="M16 26V14H28M132 14H144V26M16 64V76H28M132 76H144V64" />);
|
||||
case 'handheld':
|
||||
return g(
|
||||
<rect x={7} y={7} width={146} height={76} rx={4} strokeDasharray="3 4" transform="rotate(-1.6 80 45)" />,
|
||||
);
|
||||
case 'tracking':
|
||||
return g(
|
||||
<>
|
||||
<path d="M22 82H134M134 82l-6-4M134 82l-6 4" />
|
||||
<circle cx={30} cy={82} r={3} fill="currentColor" />
|
||||
</>,
|
||||
);
|
||||
case 'crane':
|
||||
return g(<path d="M24 80L134 20M134 20h-9M134 20l-3 8" />);
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function subjectShortLabel(label: string): string {
|
||||
const m = label.match(/(\d+)/);
|
||||
return m ? `Subj ${m[1]}` : label;
|
||||
}
|
||||
|
||||
export interface H3ShotPanelProps {
|
||||
shot: Shot;
|
||||
/** the number shown in the badge */
|
||||
number: number | string;
|
||||
/** time printed on the frame, e.g. "00:03.500" */
|
||||
timeLabel: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The contents of one storyboard panel: badge, shot type and camera move, a
|
||||
* schematic frame, who is in it, what is said, and the shot's prose. Callers
|
||||
* supply the surrounding card (a button in the editor, plain in the scene view).
|
||||
*/
|
||||
export function H3ShotPanel({ shot, number, timeLabel }: H3ShotPanelProps) {
|
||||
const spoken = shotLines(shot).filter((l) => l.text.trim());
|
||||
return (
|
||||
<>
|
||||
<div className="flex items-center gap-1.5">
|
||||
<span className="flex size-5 shrink-0 items-center justify-center rounded-full bg-secondary text-[11px] font-semibold text-foreground">
|
||||
{number}
|
||||
</span>
|
||||
<span className="truncate text-xs font-medium text-foreground">{TYPE_LABEL[shot.type]}</span>
|
||||
<span className="ml-auto shrink-0 text-[10px] text-muted-foreground">{MOVEMENT_LABEL[shot.movement]}</span>
|
||||
</div>
|
||||
|
||||
<div className="relative aspect-video overflow-hidden rounded-md border border-border bg-secondary/40">
|
||||
<ShotFrame shot={shot} />
|
||||
<span className="absolute bottom-1 left-1 rounded bg-background/80 px-1 font-mono text-[9px] text-muted-foreground">
|
||||
{timeLabel}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{shot.subjectIds.length > 0 && (
|
||||
<div className="flex flex-wrap gap-1">
|
||||
{shot.subjectIds.map((label) => (
|
||||
<span
|
||||
key={label}
|
||||
className="rounded-full border border-input bg-secondary/50 px-1.5 py-px font-mono text-[10px] text-muted-foreground"
|
||||
>
|
||||
{subjectShortLabel(label)}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{spoken.length > 0 && (
|
||||
<p className="line-clamp-3 rounded-md bg-accent-blue/10 px-1.5 py-1 text-[11px] italic text-foreground">
|
||||
{spoken.map((l) => `${l.speaker.trim() ? `${l.speaker.trim()}: ` : ''}“${l.text.trim()}”`).join(' ')}
|
||||
</p>
|
||||
)}
|
||||
|
||||
<p className="line-clamp-2 text-[11px] text-muted-foreground">{shotProse(shot) || 'No description yet'}</p>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { H3ShotStoryboard } from './H3ShotStoryboard';
|
||||
import { CAMERA_MOVEMENTS, SHOT_TYPES, makeShot, newLine, type Shot, type ShotInit } from '@/lib/h3Shots';
|
||||
|
||||
function shot(init: ShotInit): Shot {
|
||||
return makeShot(init);
|
||||
}
|
||||
|
||||
describe('H3ShotStoryboard', () => {
|
||||
it('lays shots out in order with type, movement, subjects, and dialogue', () => {
|
||||
const shots = [
|
||||
shot({ type: 'establishing', description: 'Wide rooftop.', subjectIds: ['<Subject 1>', '<Subject 2>'] }),
|
||||
shot({ type: 'close-up', movement: 'handheld', lines: [newLine({ speaker: 'S2', text: 'I had no choice.' }), newLine({ speaker: 'S1', text: 'Then go.' })], subjectIds: ['<Subject 2>'] }),
|
||||
];
|
||||
render(<H3ShotStoryboard shots={shots} selectedId={shots[1].id} onSelect={() => {}} onAdd={() => {}} />);
|
||||
|
||||
expect(screen.getByText('Establishing')).toBeInTheDocument();
|
||||
expect(screen.getByText('Close-up')).toBeInTheDocument();
|
||||
expect(screen.getByText('Handheld')).toBeInTheDocument();
|
||||
expect(screen.getByText(/S2: “I had no choice.” S1: “Then go.”/)).toBeInTheDocument();
|
||||
expect(screen.getAllByText('Subj 2')).toHaveLength(2);
|
||||
|
||||
const panels = screen.getAllByRole('button').filter((b) => b.hasAttribute('aria-current') || b.textContent?.match(/^\d/));
|
||||
expect(panels[0]).toHaveTextContent(/^1Establishing/);
|
||||
expect(screen.getByRole('button', { name: /Close-up/ })).toHaveAttribute('aria-current', 'true');
|
||||
});
|
||||
|
||||
it('selects a shot on click and adds via the trailing tile', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onSelect = vi.fn();
|
||||
const onAdd = vi.fn();
|
||||
const shots = [shot({ description: 'First.' }), shot({ description: 'Second.' })];
|
||||
render(<H3ShotStoryboard shots={shots} selectedId={null} onSelect={onSelect} onAdd={onAdd} />);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /Second\./ }));
|
||||
expect(onSelect).toHaveBeenCalledWith(shots[1].id);
|
||||
await user.click(screen.getByRole('button', { name: /Add shot/ }));
|
||||
expect(onAdd).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('renders every shot type and camera movement without crashing', () => {
|
||||
const shots = SHOT_TYPES.flatMap((t) =>
|
||||
CAMERA_MOVEMENTS.map((m) => shot({ type: t.id, movement: m.id, subjectIds: ['<Subject 1>'] })),
|
||||
);
|
||||
const { container } = render(<H3ShotStoryboard shots={shots} selectedId={null} onSelect={() => {}} onAdd={() => {}} />);
|
||||
expect(container.querySelectorAll('svg')).toHaveLength(SHOT_TYPES.length * CAMERA_MOVEMENTS.length);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,61 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { H3ShotPanel } from '@/components/jobs/H3ShotPanel';
|
||||
import { formatCutTime, resolveShotStarts, type Shot } from '@/lib/h3Shots';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface H3ShotStoryboardProps {
|
||||
shots: Shot[];
|
||||
selectedId: string | null;
|
||||
onSelect: (id: string) => void;
|
||||
onAdd: () => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* The shot list as a storyboard: one panel per shot, left to right in
|
||||
* playback order. Panels are schematic (no generated imagery exists yet) --
|
||||
* subject size reflects shot type and an overlay marks camera movement --
|
||||
* so the sequence, who's in frame, and where the dialogue falls read at a
|
||||
* glance without scrolling the per-shot editors.
|
||||
*/
|
||||
export function H3ShotStoryboard({ shots, selectedId, onSelect, onAdd }: H3ShotStoryboardProps) {
|
||||
const starts = React.useMemo(() => resolveShotStarts(shots), [shots]);
|
||||
return (
|
||||
<div className="flex items-stretch gap-2 overflow-x-auto pb-1" aria-label="Storyboard">
|
||||
{shots.map((shot, i) => {
|
||||
const selected = shot.id === selectedId;
|
||||
return (
|
||||
<React.Fragment key={shot.id}>
|
||||
{i > 0 && (
|
||||
<span className="flex shrink-0 items-center text-muted-foreground/60" aria-hidden>
|
||||
›
|
||||
</span>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => onSelect(shot.id)}
|
||||
aria-current={selected ? 'true' : undefined}
|
||||
className={cn(
|
||||
'flex w-52 shrink-0 flex-col gap-1.5 rounded-xl border bg-card p-2 text-left transition-colors hover:bg-accent/40',
|
||||
selected ? 'border-accent-blue ring-2 ring-accent-blue/50' : 'border-border',
|
||||
)}
|
||||
>
|
||||
<H3ShotPanel shot={shot} number={i + 1} timeLabel={formatCutTime(starts[i])} />
|
||||
</button>
|
||||
</React.Fragment>
|
||||
);
|
||||
})}
|
||||
|
||||
<button
|
||||
type="button"
|
||||
onClick={onAdd}
|
||||
className="flex min-h-32 w-28 shrink-0 flex-col items-center justify-center gap-1 rounded-xl border border-dashed border-border text-xs text-muted-foreground transition-colors hover:bg-accent/40 hover:text-foreground"
|
||||
>
|
||||
<span className="text-lg leading-none">+</span>
|
||||
Add shot
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { H3_TAG_CATEGORIES, type H3Tag } from '@/lib/h3Tags';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface H3TagConsoleProps {
|
||||
/** Called with the tag to insert into whichever field is currently focused. */
|
||||
onInsert: (tag: H3Tag) => void;
|
||||
disabled?: boolean;
|
||||
className?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* A microexpression "console" for MiniMax-H3 speech tags: click a tag to
|
||||
* insert it at the cursor of whichever prompt field was last focused. Kept
|
||||
* as plain buttons (not a menu) so every tag stays one click away.
|
||||
*/
|
||||
export function H3TagConsole({ onInsert, disabled, className }: H3TagConsoleProps) {
|
||||
return (
|
||||
<div
|
||||
className={cn(
|
||||
'flex flex-col gap-1.5 rounded-xl border border-input bg-secondary/40 p-2',
|
||||
className,
|
||||
)}
|
||||
>
|
||||
<span className="px-0.5 text-[11px] font-medium uppercase tracking-wide text-muted-foreground">
|
||||
Speech tags — click to insert at the cursor
|
||||
</span>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{H3_TAG_CATEGORIES.map((category) => (
|
||||
<div key={category.id} className="flex flex-wrap items-center gap-1">
|
||||
<span className="mr-0.5 w-full shrink-0 text-[10px] text-muted-foreground/80 sm:w-auto">
|
||||
{category.label}
|
||||
</span>
|
||||
{category.tags.map((tag) => (
|
||||
<button
|
||||
key={tag.id}
|
||||
type="button"
|
||||
title={`${tag.description} · e.g. "${tag.example}"`}
|
||||
// Keep the target textarea focused (and its selection intact)
|
||||
// so the click lands where the user was typing, not nowhere.
|
||||
onMouseDown={(e) => e.preventDefault()}
|
||||
onClick={() => onInsert(tag)}
|
||||
disabled={disabled}
|
||||
className="rounded-full border border-input bg-card px-2 py-0.5 font-mono text-[11px] text-foreground transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
{tag.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { H3_PRESETS, H3_TAG_CATEGORIES, type H3Preset, type H3Tag } from '@/lib/h3Tags';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
// Reuses the app's existing status hues as a purely visual category key --
|
||||
// no status meaning implied, just four distinguishable accents that already
|
||||
// work in both themes.
|
||||
export const CATEGORY_STYLE: Record<string, { chip: string; dot: string }> = {
|
||||
pace: { chip: 'border-blue-400/35 bg-blue-500/10 text-blue-900 dark:text-blue-100', dot: 'bg-blue-500' },
|
||||
delivery: { chip: 'border-amber-400/35 bg-amber-500/10 text-amber-900 dark:text-amber-100', dot: 'bg-amber-500' },
|
||||
vocal: { chip: 'border-emerald-400/35 bg-emerald-500/10 text-emerald-900 dark:text-emerald-100', dot: 'bg-emerald-500' },
|
||||
body: { chip: 'border-rose-400/35 bg-rose-500/10 text-rose-900 dark:text-rose-100', dot: 'bg-rose-500' },
|
||||
};
|
||||
|
||||
export interface H3TagPaletteProps {
|
||||
onInsertTag: (tag: H3Tag) => void;
|
||||
onApplyPreset: (preset: H3Preset) => void;
|
||||
disabled?: boolean;
|
||||
className?: string;
|
||||
}
|
||||
|
||||
/** Full speech-tag palette + emotion presets. Buttons keep the target textarea focused so the click lands at its cursor. */
|
||||
export function H3TagPalette({ onInsertTag, onApplyPreset, disabled, className }: H3TagPaletteProps) {
|
||||
return (
|
||||
<div className={className}>
|
||||
{H3_TAG_CATEGORIES.map((cat) => {
|
||||
const style = CATEGORY_STYLE[cat.id] ?? CATEGORY_STYLE.body;
|
||||
return (
|
||||
<div key={cat.id} className="mb-4">
|
||||
<div className="mb-1.5 flex items-center gap-2 px-0.5">
|
||||
<span className={cn('size-2 rounded-full', style.dot)} />
|
||||
<span className="text-xs font-semibold text-foreground">{cat.label}</span>
|
||||
</div>
|
||||
<div className="flex flex-col gap-1">
|
||||
{cat.tags.map((tag) => (
|
||||
<button
|
||||
key={tag.id}
|
||||
type="button"
|
||||
title={tag.example}
|
||||
disabled={disabled}
|
||||
onMouseDown={(e) => e.preventDefault()}
|
||||
onClick={() => onInsertTag(tag)}
|
||||
className="flex items-baseline justify-between gap-2 rounded-lg border border-input bg-card px-2.5 py-1.5 text-left transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
<span className="font-mono text-[11.5px] text-foreground">{tag.label}</span>
|
||||
<span className="truncate text-[10.5px] text-muted-foreground">{tag.description}</span>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
|
||||
<div className="mb-1.5 mt-5 px-0.5 text-xs font-semibold text-foreground">Emotion presets</div>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{H3_PRESETS.map((p) => (
|
||||
<button
|
||||
key={p.label}
|
||||
type="button"
|
||||
disabled={disabled}
|
||||
onMouseDown={(e) => e.preventDefault()}
|
||||
onClick={() => onApplyPreset(p)}
|
||||
className="rounded-full border border-input bg-card px-2.5 py-1 text-[11.5px] text-foreground transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
{p.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -5,7 +5,9 @@ import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import JobCard from '@/components/jobs/JobCard';
|
||||
import {
|
||||
deleteJob,
|
||||
dequeueJob,
|
||||
downloadJobVideo,
|
||||
queueJobs,
|
||||
startJob,
|
||||
stopJob,
|
||||
} from '@/lib/api';
|
||||
@@ -18,6 +20,8 @@ vi.mock('@/lib/api', () => ({
|
||||
stopJob: vi.fn(),
|
||||
deleteJob: vi.fn(),
|
||||
downloadJobVideo: vi.fn(),
|
||||
queueJobs: vi.fn(),
|
||||
dequeueJob: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/utils', async (importOriginal) => {
|
||||
@@ -47,6 +51,8 @@ beforeEach(() => {
|
||||
setActiveJobId(null);
|
||||
vi.mocked(startJob).mockResolvedValue({} as Job);
|
||||
vi.mocked(stopJob).mockResolvedValue({} as Job);
|
||||
vi.mocked(queueJobs).mockResolvedValue([]);
|
||||
vi.mocked(dequeueJob).mockResolvedValue({} as Job);
|
||||
vi.mocked(deleteJob).mockResolvedValue(undefined);
|
||||
vi.mocked(downloadJobVideo).mockResolvedValue(new Blob());
|
||||
vi.spyOn(window, 'confirm').mockReturnValue(true);
|
||||
@@ -91,6 +97,46 @@ describe('JobCard', () => {
|
||||
expect(onJobUpdated).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(['pending', 'failed', 'stopped'])('queues a %s job and notifies the parent', async (status) => {
|
||||
const onJobUpdated = vi.fn();
|
||||
render(<JobCard job={makeJob({ status })} onJobUpdated={onJobUpdated} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Queue' }));
|
||||
await waitFor(() => expect(queueJobs).toHaveBeenCalledWith(['job-1']));
|
||||
expect(onJobUpdated).toHaveBeenCalled();
|
||||
expect(startJob).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('reports a job that cannot be queued', async () => {
|
||||
vi.mocked(queueJobs).mockRejectedValue(new Error('Jobs would wait on each other'));
|
||||
render(<JobCard job={makeJob({ status: 'pending' })} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Queue' }));
|
||||
await waitFor(() => expect(window.alert).toHaveBeenCalledWith('Jobs would wait on each other'));
|
||||
});
|
||||
|
||||
it('shows a queued job as queued, and takes it back out of the queue', async () => {
|
||||
const onJobUpdated = vi.fn();
|
||||
render(<JobCard job={makeJob({ status: 'queued' })} onJobUpdated={onJobUpdated} />);
|
||||
|
||||
expect(screen.getByText('queued')).toBeInTheDocument();
|
||||
// nothing to start or queue again; it can't be edited either
|
||||
for (const name of ['Start', 'Restart', 'Queue', 'Stop', 'Edit']) {
|
||||
expect(screen.queryByRole('button', { name })).not.toBeInTheDocument();
|
||||
}
|
||||
expect(screen.getByRole('button', { name: 'View' })).toBeInTheDocument();
|
||||
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Remove from queue' }));
|
||||
await waitFor(() => expect(dequeueJob).toHaveBeenCalledWith('job-1'));
|
||||
expect(onJobUpdated).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not offer to queue a running or completed job', () => {
|
||||
for (const status of ['running', 'completed']) {
|
||||
const { unmount } = render(<JobCard job={makeJob({ status, started_at: Date.now() })} />);
|
||||
expect(screen.queryByRole('button', { name: 'Queue' })).not.toBeInTheDocument();
|
||||
unmount();
|
||||
}
|
||||
});
|
||||
|
||||
it('stops a running job', async () => {
|
||||
render(<JobCard job={makeJob({ status: 'running', started_at: Date.now() })} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Stop' }));
|
||||
|
||||
@@ -3,12 +3,16 @@
|
||||
import * as React from 'react';
|
||||
import { Timer } from 'lucide-react';
|
||||
|
||||
import CreateJobModal from '@/components/jobs/CreateJobModal';
|
||||
import { Badge, type BadgeProps } from '@/components/ui/badge';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { useStore } from '@/hooks/useStore';
|
||||
import {
|
||||
deleteJob,
|
||||
duplicateJob,
|
||||
dequeueJob,
|
||||
downloadJobVideo,
|
||||
queueJobs,
|
||||
startJob,
|
||||
stopJob,
|
||||
} from '@/lib/api';
|
||||
@@ -49,6 +53,7 @@ function computeElapsed(job: Job, currentTime: number): string | null {
|
||||
|
||||
const BADGE_VARIANTS: Record<string, BadgeProps['variant']> = {
|
||||
pending: 'secondary',
|
||||
queued: 'default',
|
||||
running: 'warning',
|
||||
completed: 'success',
|
||||
ready: 'success',
|
||||
@@ -62,6 +67,8 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
const isSelected = activeJobId === job.id;
|
||||
|
||||
const [isLoading, setIsLoading] = React.useState(false);
|
||||
const [isEditing, setIsEditing] = React.useState(false);
|
||||
const [isViewing, setIsViewing] = React.useState(false);
|
||||
const [currentTime, setCurrentTime] = React.useState(() => Date.now());
|
||||
|
||||
const elapsedTime = computeElapsed(job, currentTime);
|
||||
@@ -88,6 +95,36 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
}
|
||||
}
|
||||
|
||||
async function handleQueue(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await queueJobs([job.id]);
|
||||
onJobUpdated?.();
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to queue job');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDequeue(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await dequeueJob(job.id);
|
||||
onJobUpdated?.();
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to remove job from the queue');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleStop(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
@@ -103,6 +140,21 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDuplicate(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await duplicateJob(job.id);
|
||||
onJobUpdated?.();
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to duplicate job');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDelete(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
@@ -156,16 +208,23 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
>
|
||||
<span className="flex flex-wrap items-center justify-between gap-2">
|
||||
<span className="text-[0.95rem] font-semibold text-foreground">
|
||||
{job.model_id}
|
||||
{job.name?.trim() || job.model_id}
|
||||
</span>
|
||||
<Badge variant={BADGE_VARIANTS[job.status] ?? 'secondary'}>
|
||||
{job.status}
|
||||
</Badge>
|
||||
</span>
|
||||
<span className="max-w-full overflow-hidden text-ellipsis whitespace-nowrap text-sm text-muted-foreground">
|
||||
{job.prompt}
|
||||
{job.name?.trim() ? `${job.model_id} · ${job.prompt}` : job.prompt}
|
||||
</span>
|
||||
<span className="flex flex-wrap items-center gap-4 text-xs text-muted-foreground">
|
||||
{/* Short job id; logs and output dirs are keyed on the full UUID. */}
|
||||
<span
|
||||
className="font-mono text-muted-foreground/80"
|
||||
title={job.id}
|
||||
>
|
||||
{job.id.slice(0, 8)}
|
||||
</span>
|
||||
{job.job_type === 'inference' ? (
|
||||
<>
|
||||
<span>{job.num_frames} frames</span>
|
||||
@@ -212,7 +271,30 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
>
|
||||
Start
|
||||
</Button>
|
||||
) : job.status === 'queued' ? (
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={handleDequeue}
|
||||
disabled={isLoading}
|
||||
title="Take this job out of the queue"
|
||||
>
|
||||
Remove from queue
|
||||
</Button>
|
||||
) : null}
|
||||
{(job.status === 'pending' ||
|
||||
job.status === 'failed' ||
|
||||
job.status === 'stopped') && (
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={handleQueue}
|
||||
disabled={isLoading}
|
||||
title="Run this job when the ones queued before it have finished"
|
||||
>
|
||||
Queue
|
||||
</Button>
|
||||
)}
|
||||
{job.status === 'completed' &&
|
||||
job.output_path &&
|
||||
job.job_type === 'inference' && (
|
||||
@@ -226,6 +308,51 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
Download Video
|
||||
</Button>
|
||||
)}
|
||||
{!(
|
||||
job.status === 'pending' ||
|
||||
job.status === 'failed' ||
|
||||
job.status === 'stopped'
|
||||
) && (
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={(e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
setIsViewing(true);
|
||||
}}
|
||||
disabled={isLoading}
|
||||
title="View this job's configuration"
|
||||
>
|
||||
View
|
||||
</Button>
|
||||
)}
|
||||
{(job.status === 'pending' ||
|
||||
job.status === 'failed' ||
|
||||
job.status === 'stopped') && (
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={(e) => {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
setIsEditing(true);
|
||||
}}
|
||||
disabled={isLoading}
|
||||
title="Edit this job's configuration"
|
||||
>
|
||||
Edit
|
||||
</Button>
|
||||
)}
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={handleDuplicate}
|
||||
disabled={isLoading}
|
||||
title="Create a new pending job with this configuration"
|
||||
>
|
||||
Duplicate
|
||||
</Button>
|
||||
<Button
|
||||
size="sm"
|
||||
variant="destructive"
|
||||
@@ -235,6 +362,30 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
Delete
|
||||
</Button>
|
||||
</div>
|
||||
{isViewing && (
|
||||
<CreateJobModal
|
||||
isOpen
|
||||
readOnly
|
||||
editingJob={job}
|
||||
jobType={(job.job_type ?? 'inference') as never}
|
||||
workloadType={job.workload_type ?? 't2v'}
|
||||
onClose={() => setIsViewing(false)}
|
||||
onSuccess={() => setIsViewing(false)}
|
||||
/>
|
||||
)}
|
||||
{isEditing && (
|
||||
<CreateJobModal
|
||||
isOpen
|
||||
editingJob={job}
|
||||
jobType={(job.job_type ?? 'inference') as never}
|
||||
workloadType={job.workload_type ?? 't2v'}
|
||||
onClose={() => setIsEditing(false)}
|
||||
onSuccess={() => {
|
||||
setIsEditing(false);
|
||||
onJobUpdated?.();
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</article>
|
||||
);
|
||||
}
|
||||
|
||||
@@ -77,7 +77,7 @@ export default function JobDetailsSidebar({
|
||||
const isRestarting =
|
||||
previousJobId.current === job.id &&
|
||||
wasTerminal &&
|
||||
(job.status === 'pending' || job.status === 'running');
|
||||
(job.status === 'pending' || job.status === 'queued' || job.status === 'running');
|
||||
|
||||
if (previousJobId.current !== job.id || isRestarting) {
|
||||
stateRef.current.text = '';
|
||||
@@ -89,7 +89,10 @@ export default function JobDetailsSidebar({
|
||||
}, [job.id, job.status]);
|
||||
|
||||
React.useEffect(() => {
|
||||
const shouldPoll = job.status === 'running' || job.status === 'pending';
|
||||
const shouldPoll =
|
||||
job.status === 'running' ||
|
||||
job.status === 'pending' ||
|
||||
job.status === 'queued';
|
||||
let pollInterval: ReturnType<typeof setInterval> | null = null;
|
||||
let mounted = true;
|
||||
// Effect-local lock so this job's first fetch is never blocked by a
|
||||
@@ -153,9 +156,16 @@ export default function JobDetailsSidebar({
|
||||
}}
|
||||
>
|
||||
<div className="flex items-center justify-between border-b border-border px-5 py-4">
|
||||
<h2 className="m-0 text-base font-semibold text-foreground">
|
||||
Job Details
|
||||
</h2>
|
||||
<div className="min-w-0">
|
||||
<h2 className="m-0 text-base font-semibold text-foreground">
|
||||
Job Details
|
||||
</h2>
|
||||
{/* Full job id: keys ~/h3_studio_logs/<id>.log, the output directory
|
||||
and every API route, so make it selectable for copy/paste. */}
|
||||
<code className="mt-0.5 block select-all truncate font-mono text-xs text-muted-foreground">
|
||||
{job.id}
|
||||
</code>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<Button
|
||||
type="button"
|
||||
|
||||
@@ -108,7 +108,8 @@ export default function JobQueue({ jobType, jobTypesForList }: JobQueueProps) {
|
||||
|
||||
// Poll every second while any job is running/pending; stop otherwise.
|
||||
const hasActive = jobs.some(
|
||||
(j) => j.status === 'running' || j.status === 'pending',
|
||||
(j) =>
|
||||
j.status === 'running' || j.status === 'pending' || j.status === 'queued',
|
||||
);
|
||||
React.useEffect(() => {
|
||||
if (!hasActive) return;
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
import * as React from 'react';
|
||||
import { flushSync } from 'react-dom';
|
||||
import { render, screen, waitFor, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import CreateJobModal from './CreateJobModal';
|
||||
import { createJob, getDatasets, getModels } from '@/lib/api';
|
||||
import { defaultOptionsStore } from '@/stores/defaultOptions';
|
||||
import { DEFAULT_OPTIONS } from '@/lib/defaultOptions';
|
||||
|
||||
vi.mock('sonner', () => ({ toast: { warning: vi.fn(), error: vi.fn(), success: vi.fn() } }));
|
||||
vi.mock('@/lib/api', () => ({
|
||||
createJob: vi.fn(),
|
||||
getModels: vi.fn(),
|
||||
getDatasets: vi.fn(),
|
||||
uploadImage: vi.fn(),
|
||||
getSettings: vi.fn(),
|
||||
updateSettings: vi.fn(),
|
||||
}));
|
||||
|
||||
const H3 = { id: 'minimax-h3/i2v', label: 'MiniMax H3', type: 'i2v' };
|
||||
|
||||
// A browser runs a microtask checkpoint -- so React flushes -- between the
|
||||
// listeners of a single native event; jsdom's synchronous dispatch never does.
|
||||
// Radix decides whether a parent dialog should dismiss from the layer stack at
|
||||
// the moment its listener runs, so without this the bug can't show up here.
|
||||
const wrapped = new WeakMap<object, EventListener>();
|
||||
beforeEach(() => {
|
||||
const realAdd = Document.prototype.addEventListener;
|
||||
const realRemove = Document.prototype.removeEventListener;
|
||||
vi.spyOn(document, 'addEventListener').mockImplementation(function (
|
||||
this: Document,
|
||||
type: string,
|
||||
listener: unknown,
|
||||
options?: unknown,
|
||||
) {
|
||||
if (typeof listener === 'function' && ['keydown', 'pointerdown', 'click'].includes(type)) {
|
||||
const flushing: EventListener = (e) => {
|
||||
(listener as EventListener)(e);
|
||||
flushSync(() => {});
|
||||
};
|
||||
wrapped.set(listener, flushing);
|
||||
return realAdd.call(this, type, flushing, options as AddEventListenerOptions);
|
||||
}
|
||||
return realAdd.call(this, type, listener as EventListener, options as AddEventListenerOptions);
|
||||
} as typeof document.addEventListener);
|
||||
vi.spyOn(document, 'removeEventListener').mockImplementation(function (
|
||||
this: Document,
|
||||
type: string,
|
||||
listener: unknown,
|
||||
options?: unknown,
|
||||
) {
|
||||
return realRemove.call(
|
||||
this,
|
||||
type,
|
||||
(wrapped.get(listener as object) ?? listener) as EventListener,
|
||||
options as EventListenerOptions,
|
||||
);
|
||||
} as typeof document.removeEventListener);
|
||||
|
||||
defaultOptionsStore.set({ options: DEFAULT_OPTIONS });
|
||||
vi.mocked(getModels).mockResolvedValue([H3]);
|
||||
vi.mocked(getDatasets).mockResolvedValue([]);
|
||||
vi.mocked(createJob).mockResolvedValue({ id: 'j' } as never);
|
||||
});
|
||||
|
||||
function Harness() {
|
||||
const [open, setOpen] = React.useState(true);
|
||||
return (
|
||||
<>
|
||||
<div data-testid="state">{open ? 'open' : 'closed'}</div>
|
||||
<CreateJobModal
|
||||
isOpen={open}
|
||||
onClose={() => setOpen(false)}
|
||||
onSuccess={() => {}}
|
||||
jobType="inference"
|
||||
workloadType="i2v"
|
||||
/>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
type User = ReturnType<typeof userEvent.setup>;
|
||||
type Closer = (user: User, dialog: HTMLElement) => Promise<void>;
|
||||
|
||||
async function renderH3Modal(user: User) {
|
||||
render(<Harness />);
|
||||
await screen.findByRole('option', { name: /MiniMax H3/ });
|
||||
await user.selectOptions(screen.getByLabelText('Model'), H3.id);
|
||||
}
|
||||
|
||||
// Radix marks every dialog under the topmost one aria-hidden, which also hides its
|
||||
// accessible name, so the job dialog is identified by its title text instead.
|
||||
const expectParentStillOpen = () => {
|
||||
expect(screen.getByTestId('state')).toHaveTextContent('open');
|
||||
expect(screen.getByText(/New Inference Job/)).toBeInTheDocument();
|
||||
};
|
||||
// [0] is the job dialog's overlay; the last is the topmost dialog's.
|
||||
const topOverlay = () => [...document.querySelectorAll('.bg-black\\/70')].at(-1) as Element;
|
||||
|
||||
const CLOSERS: [string, Closer][] = [
|
||||
['Cancel', (u, d) => u.click(within(d).getByRole('button', { name: 'Cancel' }))],
|
||||
['the X button', (u, d) => u.click(within(d).getByRole('button', { name: 'Close' }))],
|
||||
['Escape', (u) => u.keyboard('{Escape}')],
|
||||
['a click on the overlay', (u) => u.click(topOverlay())],
|
||||
];
|
||||
|
||||
describe('closing a dialog opened from the job form', () => {
|
||||
describe.each([
|
||||
['shot list', 'Open shot list', 'Shot list', [...CLOSERS, ['Apply', (u: User, d: HTMLElement) => u.click(within(d).getByRole('button', { name: 'Apply' }))] as [string, Closer]]],
|
||||
['dialogue editor', 'Open dialogue editor', 'Dialogue editor', [...CLOSERS, ['Apply to field', (u: User, d: HTMLElement) => u.click(within(d).getByRole('button', { name: 'Apply to field' }))] as [string, Closer]]],
|
||||
] as const)('%s', (_label, openButton, title, closers) => {
|
||||
it.each(closers)('closing with %s leaves the job form open', async (_name, close) => {
|
||||
const user = userEvent.setup();
|
||||
await renderH3Modal(user);
|
||||
await user.click(await screen.findByRole('button', { name: openButton }));
|
||||
const child = await screen.findByRole('dialog', { name: title });
|
||||
|
||||
await close(user, child);
|
||||
|
||||
await waitFor(() => expect(screen.queryByRole('dialog', { name: title })).not.toBeInTheDocument());
|
||||
expectParentStillOpen();
|
||||
});
|
||||
});
|
||||
|
||||
it('the job form is still usable afterwards', async () => {
|
||||
const user = userEvent.setup();
|
||||
await renderH3Modal(user);
|
||||
await user.click(await screen.findByRole('button', { name: 'Open shot list' }));
|
||||
const child = await screen.findByRole('dialog', { name: 'Shot list' });
|
||||
await user.click(within(child).getByRole('button', { name: 'Cancel' }));
|
||||
await waitFor(() => expect(screen.queryByRole('dialog', { name: 'Shot list' })).not.toBeInTheDocument());
|
||||
|
||||
await user.type(screen.getByLabelText('Name (optional)'), 'still here');
|
||||
expect(screen.getByLabelText('Name (optional)')).toHaveValue('still here');
|
||||
// ...and the dialog can be opened again
|
||||
await user.click(screen.getByRole('button', { name: 'Open shot list' }));
|
||||
expect(await screen.findByRole('dialog', { name: 'Shot list' })).toBeInTheDocument();
|
||||
});
|
||||
|
||||
describe('the script import inside the shot list', () => {
|
||||
it.each(CLOSERS.slice(0, 4))('closing it with %s leaves the shot list and the job form open', async (_name, close) => {
|
||||
const user = userEvent.setup();
|
||||
await renderH3Modal(user);
|
||||
await user.click(await screen.findByRole('button', { name: 'Open shot list' }));
|
||||
const shotList = await screen.findByRole('dialog', { name: 'Shot list' });
|
||||
await user.click(within(shotList).getByRole('button', { name: 'Import script…' }));
|
||||
const importDialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
|
||||
await close(user, importDialog);
|
||||
|
||||
await waitFor(() => expect(screen.queryByRole('dialog', { name: 'Import script' })).not.toBeInTheDocument());
|
||||
expect(screen.getByRole('dialog', { name: 'Shot list' })).toBeInTheDocument();
|
||||
expectParentStillOpen();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,253 @@
|
||||
import { render, screen, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { H3ShotListEditor } from './H3ShotListEditor';
|
||||
|
||||
const TWO_SHOTS = '[Shot 1] Wide shot. A dock.\n[Shot 2] At 00:02.000, the shot cuts to a close-up shot. A face.';
|
||||
const WITH_LINE = '[Shot 1] Wide shot. (S1) <d>[English] Hi.</d>';
|
||||
|
||||
type User = ReturnType<typeof userEvent.setup>;
|
||||
|
||||
function setup(detailed: string, extra: { open?: boolean } = {}) {
|
||||
const onApply = vi.fn();
|
||||
const ui = (open: boolean) => (
|
||||
<H3ShotListEditor
|
||||
open={open}
|
||||
subjectDefinitions="<Subject 1> is a man."
|
||||
initialDetailedDescription={detailed}
|
||||
initialRetentionAnalysis=""
|
||||
onApply={onApply}
|
||||
onClose={() => {}}
|
||||
/>
|
||||
);
|
||||
const view = render(ui(extra.open ?? true));
|
||||
return { onApply, view, ui };
|
||||
}
|
||||
|
||||
const undoButton = () => screen.getByRole('button', { name: /Undo/ });
|
||||
const redoButton = () => screen.getByRole('button', { name: /Redo/ });
|
||||
const applied = (onApply: ReturnType<typeof vi.fn>) => onApply.mock.calls.at(-1)?.[0] as string;
|
||||
const apply = (user: User) => user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
const lineBox = () => screen.getByPlaceholderText('What the speaker says') as HTMLTextAreaElement;
|
||||
const ctrl = (key: string) => `{Control>}${key}{/Control}`;
|
||||
|
||||
describe('shot list undo/redo', () => {
|
||||
it('starts with nothing to undo or redo', () => {
|
||||
setup(TWO_SHOTS);
|
||||
expect(undoButton()).toBeDisabled();
|
||||
expect(redoButton()).toBeDisabled();
|
||||
});
|
||||
|
||||
it('undoes and redoes moving a shot', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
expect(undoButton()).toBeEnabled();
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(redoButton()).toBeEnabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
|
||||
it('redo reapplies what was undone', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
await user.click(undoButton());
|
||||
await user.click(redoButton());
|
||||
await apply(user);
|
||||
expect(applied(onApply).startsWith('[Shot 1] Close-up shot. A face.')).toBe(true);
|
||||
});
|
||||
|
||||
it('undoes removing a shot, dialogue and all', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(`${WITH_LINE}\n[Shot 2] At 00:02.500, the shot cuts to a wide shot. B.`);
|
||||
await user.click(screen.getAllByRole('button', { name: 'Remove' })[0]);
|
||||
expect(screen.queryByPlaceholderText('What the speaker says')).not.toBeInTheDocument();
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(lineBox().value).toBe('Hi.');
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toContain('(S1) <d>[English] Hi.</d>');
|
||||
});
|
||||
|
||||
it('undoes an add, a type change, a subject toggle, and a cut time in turn', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
|
||||
await user.click(screen.getAllByRole('button', { name: /Add shot/ })[0]);
|
||||
await user.selectOptions(screen.getAllByLabelText('Shot type')[0], 'pov');
|
||||
await user.click(screen.getAllByRole('button', { name: '<Subject 1>' })[0]);
|
||||
const cut = screen.getAllByPlaceholderText(/\d\d:\d\d\.\d{3}/)[0] as HTMLInputElement;
|
||||
await user.type(cut, '5{Enter}');
|
||||
|
||||
// four discrete steps, undone one at a time
|
||||
for (let i = 0; i < 4; i += 1) await user.click(undoButton());
|
||||
expect(undoButton()).toBeDisabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
|
||||
it('committing a cut time that has not changed is not a step', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(TWO_SHOTS);
|
||||
const cut = screen.getAllByPlaceholderText(/\d\d:\d\d\.\d{3}/)[0] as HTMLInputElement;
|
||||
await user.type(cut, '5{Enter}');
|
||||
await user.click(undoButton()); // the click blurs the box, which re-commits 00:05.000
|
||||
expect(undoButton()).toBeDisabled();
|
||||
});
|
||||
|
||||
it('a burst of typing is one step', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(WITH_LINE);
|
||||
await user.type(lineBox(), ' there my friend');
|
||||
expect(lineBox().value).toBe('Hi. there my friend');
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(lineBox().value).toBe('Hi.');
|
||||
expect(undoButton()).toBeDisabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toContain('<d>[English] Hi.</d>');
|
||||
});
|
||||
|
||||
it('a tag inserted from the palette is its own step, separate from typing', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(WITH_LINE);
|
||||
await user.type(lineBox(), ' there');
|
||||
lineBox().setSelectionRange(lineBox().value.length, lineBox().value.length);
|
||||
await user.click(screen.getByRole('button', { name: /<pause>/ }));
|
||||
expect(lineBox().value).toBe('Hi. there <pause>');
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(lineBox().value).toBe('Hi. there');
|
||||
await user.click(undoButton());
|
||||
expect(lineBox().value).toBe('Hi.');
|
||||
});
|
||||
|
||||
it('undoes edits to the description and the scene notes', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup('[Shot 1] Wide shot. A dock.');
|
||||
await user.type(screen.getByPlaceholderText(/Live-action, cinematic/), 'Noir.');
|
||||
await user.type(screen.getByPlaceholderText("What's happening in this shot"), ' At dawn.');
|
||||
|
||||
await user.click(undoButton());
|
||||
await user.click(undoButton());
|
||||
expect(undoButton()).toBeDisabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe('[Shot 1] Wide shot. A dock.');
|
||||
});
|
||||
|
||||
it('typing in different boxes makes separate steps', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup('[Shot 1] Wide shot. A dock.');
|
||||
await user.type(screen.getByPlaceholderText(/Live-action, cinematic/), 'Noir.');
|
||||
await user.type(screen.getByPlaceholderText("What's happening in this shot"), ' At dawn.');
|
||||
|
||||
await user.click(undoButton());
|
||||
expect((screen.getByPlaceholderText("What's happening in this shot") as HTMLTextAreaElement).value).toBe('A dock.');
|
||||
expect((screen.getByPlaceholderText(/Live-action, cinematic/) as HTMLTextAreaElement).value).toBe('Noir.');
|
||||
});
|
||||
|
||||
it('a new edit clears what could be redone', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
await user.click(undoButton());
|
||||
expect(redoButton()).toBeEnabled();
|
||||
await user.click(screen.getAllByRole('button', { name: /Add shot/ })[0]);
|
||||
expect(redoButton()).toBeDisabled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('shot list undo/redo shortcuts', () => {
|
||||
it('Ctrl+Z from inside a textbox undoes the whole list, not just that box', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]); // move a shot
|
||||
// focus a different textbox, then undo from there
|
||||
const box = screen.getAllByPlaceholderText("What's happening in this shot")[0] as HTMLTextAreaElement;
|
||||
await user.click(box);
|
||||
await user.keyboard(ctrl('z'));
|
||||
expect(undoButton()).toBeDisabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
|
||||
it('redoes with Ctrl+Shift+Z and with Ctrl+Y', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
const box = screen.getAllByPlaceholderText("What's happening in this shot")[0];
|
||||
await user.click(box);
|
||||
|
||||
await user.keyboard(ctrl('z'));
|
||||
expect(redoButton()).toBeEnabled();
|
||||
await user.keyboard('{Control>}{Shift>}z{/Shift}{/Control}');
|
||||
expect(redoButton()).toBeDisabled();
|
||||
|
||||
await user.keyboard(ctrl('z'));
|
||||
await user.keyboard(ctrl('y'));
|
||||
expect(redoButton()).toBeDisabled();
|
||||
expect(undoButton()).toBeEnabled();
|
||||
});
|
||||
|
||||
it('Cmd+Z works too', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
await user.click(screen.getAllByPlaceholderText("What's happening in this shot")[0]);
|
||||
await user.keyboard('{Meta>}z{/Meta}');
|
||||
expect(undoButton()).toBeDisabled();
|
||||
});
|
||||
|
||||
it('leaves the script import box to its own undo', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
expect(undoButton()).toBeEnabled();
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
const dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
const box = within(dialog).getByLabelText('Script');
|
||||
await user.click(box);
|
||||
await user.keyboard(ctrl('z'));
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Cancel' }));
|
||||
|
||||
// the shot list's history was untouched: the move is still there to undo
|
||||
await user.click(await screen.findByRole('button', { name: /Undo/ }));
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
});
|
||||
|
||||
describe('shot list history across imports and reopening', () => {
|
||||
it('loading a script clip is one undoable step', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
const dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
await user.click(within(dialog).getByLabelText('Script'));
|
||||
await user.paste('CAPTAIN: Sit down and hear me out.\n\nCLERK: Yes, sir.');
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
await screen.findAllByPlaceholderText('What the speaker says');
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(screen.queryByPlaceholderText('What the speaker says')).not.toBeInTheDocument();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
|
||||
it('opening the editor again starts a fresh history', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { view, ui } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
expect(undoButton()).toBeEnabled();
|
||||
|
||||
view.rerender(ui(false));
|
||||
view.rerender(ui(true));
|
||||
expect(await screen.findByRole('button', { name: /Undo/ })).toBeDisabled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,111 @@
|
||||
import { fireEvent, render, screen } from '@testing-library/react';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { BeforeAfterGrade } from './BeforeAfterGrade';
|
||||
|
||||
// jsdom has no real video decoder, canvas 2D context, or `ImageData` constructor,
|
||||
// so all three are stubbed: the video always "has" an 8x6 frame ready, the canvas
|
||||
// records what was drawn/read/written instead of actually rasterizing anything,
|
||||
// and ImageData is a plain data holder (its real job -- backing a canvas -- is
|
||||
// already covered by the getContext stub).
|
||||
if (typeof globalThis.ImageData === 'undefined') {
|
||||
class FakeImageData {
|
||||
data: Uint8ClampedArray;
|
||||
width: number;
|
||||
height: number;
|
||||
constructor(data: Uint8ClampedArray, width: number, height: number) {
|
||||
this.data = data;
|
||||
this.width = width;
|
||||
this.height = height;
|
||||
}
|
||||
}
|
||||
// @ts-expect-error -- test-only polyfill for a browser global jsdom doesn't provide
|
||||
globalThis.ImageData = FakeImageData;
|
||||
}
|
||||
|
||||
function stubVideoFrame() {
|
||||
Object.defineProperty(HTMLMediaElement.prototype, 'readyState', { configurable: true, get: () => 4 });
|
||||
Object.defineProperty(HTMLVideoElement.prototype, 'videoWidth', { configurable: true, get: () => 8 });
|
||||
Object.defineProperty(HTMLVideoElement.prototype, 'videoHeight', { configurable: true, get: () => 6 });
|
||||
}
|
||||
|
||||
function stubCanvasContext() {
|
||||
const data = new Uint8ClampedArray(8 * 6 * 4).fill(100);
|
||||
const putImageData = vi.fn();
|
||||
const drawImage = vi.fn();
|
||||
const getImageData = vi.fn(() => ({ width: 8, height: 6, data }));
|
||||
vi.spyOn(HTMLCanvasElement.prototype, 'getContext').mockReturnValue({
|
||||
drawImage,
|
||||
getImageData,
|
||||
putImageData,
|
||||
} as unknown as CanvasRenderingContext2D);
|
||||
return { drawImage, getImageData, putImageData, data };
|
||||
}
|
||||
|
||||
describe('BeforeAfterGrade', () => {
|
||||
beforeEach(() => {
|
||||
stubVideoFrame();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it('shows the "load a frame" hint until a video frame is captured', () => {
|
||||
const ref = { current: null };
|
||||
render(<BeforeAfterGrade videoRef={ref} brightness={0} contrast={1} saturation={1} />);
|
||||
expect(screen.getByText(/Play or seek the preview above/)).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('captures a frame once the video has data, and draws it into both canvases', () => {
|
||||
const { getImageData, putImageData } = stubCanvasContext();
|
||||
const video = document.createElement('video');
|
||||
document.body.appendChild(video);
|
||||
const ref = { current: video };
|
||||
|
||||
render(<BeforeAfterGrade videoRef={ref} brightness={0} contrast={1} saturation={1} />);
|
||||
fireEvent.loadedData(video);
|
||||
|
||||
expect(getImageData).toHaveBeenCalled();
|
||||
expect(putImageData).toHaveBeenCalled();
|
||||
expect(screen.queryByText(/Play or seek the preview above/)).not.toBeInTheDocument();
|
||||
document.body.removeChild(video);
|
||||
});
|
||||
|
||||
it('re-grades the "after" canvas when the color values change, without touching the video again', () => {
|
||||
const { putImageData } = stubCanvasContext();
|
||||
const video = document.createElement('video');
|
||||
document.body.appendChild(video);
|
||||
const ref = { current: video };
|
||||
|
||||
const { rerender } = render(<BeforeAfterGrade videoRef={ref} brightness={0} contrast={1} saturation={1} />);
|
||||
fireEvent.loadedData(video);
|
||||
const callsAfterCapture = putImageData.mock.calls.length;
|
||||
|
||||
rerender(<BeforeAfterGrade videoRef={ref} brightness={50} contrast={1} saturation={1} />);
|
||||
expect(putImageData.mock.calls.length).toBeGreaterThan(callsAfterCapture);
|
||||
|
||||
// Neutral values were drawn as-is; the brightened redraw's pixel buffer must differ.
|
||||
const neutralData = putImageData.mock.calls[0][0].data as Uint8ClampedArray;
|
||||
const brightenedData = putImageData.mock.calls[putImageData.mock.calls.length - 1][0].data as Uint8ClampedArray;
|
||||
expect(brightenedData[0]).toBeGreaterThan(neutralData[0]);
|
||||
|
||||
document.body.removeChild(video);
|
||||
});
|
||||
|
||||
it('"Use current frame" re-captures the video at its current position', () => {
|
||||
const { drawImage } = stubCanvasContext();
|
||||
const video = document.createElement('video');
|
||||
document.body.appendChild(video);
|
||||
const ref = { current: video };
|
||||
|
||||
render(<BeforeAfterGrade videoRef={ref} brightness={0} contrast={1} saturation={1} />);
|
||||
fireEvent.loadedData(video);
|
||||
const callsAfterAutoCapture = drawImage.mock.calls.length;
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: 'Use current frame' }));
|
||||
expect(drawImage.mock.calls.length).toBeGreaterThan(callsAfterAutoCapture);
|
||||
|
||||
document.body.removeChild(video);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,113 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { applyGradeInPlace, isNeutralGrade } from '@/lib/grading';
|
||||
|
||||
export interface BeforeAfterGradeProps {
|
||||
/** the video the snapshot is captured from -- must be `crossOrigin="anonymous"` for pixel access to work */
|
||||
videoRef: React.RefObject<HTMLVideoElement | null>;
|
||||
brightness: number;
|
||||
contrast: number;
|
||||
saturation: number;
|
||||
/** called with the video's currentTime whenever a frame is (re)captured, so a caller with multiple
|
||||
* sections can tell which one the previewed frame falls into and keep its own selection in sync */
|
||||
onFrameCaptured?: (timeSeconds: number) => void;
|
||||
}
|
||||
|
||||
// Downscaled so the per-pixel pass stays instant on every slider tick.
|
||||
const PREVIEW_WIDTH = 320;
|
||||
|
||||
/**
|
||||
* A still frame from `videoRef`, shown twice: untouched, and with the current
|
||||
* color values applied via `applyGradeInPlace` -- entirely in the browser, so
|
||||
* dragging a slider never waits on the server. Captures a frame automatically
|
||||
* once the video has one to show; "Use current frame" re-captures whatever the
|
||||
* video is currently paused/seeked to, for comparing a specific moment.
|
||||
*/
|
||||
export function BeforeAfterGrade({ videoRef, brightness, contrast, saturation, onFrameCaptured }: BeforeAfterGradeProps) {
|
||||
const beforeRef = React.useRef<HTMLCanvasElement>(null);
|
||||
const afterRef = React.useRef<HTMLCanvasElement>(null);
|
||||
const sourceRef = React.useRef<ImageData | null>(null);
|
||||
const [ready, setReady] = React.useState(false);
|
||||
// Kept in a ref, not a `capture` dependency, so an inline arrow function prop
|
||||
// doesn't force the video-listener effect below to tear down and re-add itself every render.
|
||||
const onFrameCapturedRef = React.useRef(onFrameCaptured);
|
||||
onFrameCapturedRef.current = onFrameCaptured;
|
||||
|
||||
const capture = React.useCallback(() => {
|
||||
const video = videoRef.current;
|
||||
const before = beforeRef.current;
|
||||
if (!video || !before || video.readyState < 2 || !video.videoWidth) return;
|
||||
|
||||
const scale = PREVIEW_WIDTH / video.videoWidth;
|
||||
const width = PREVIEW_WIDTH;
|
||||
const height = Math.round(video.videoHeight * scale);
|
||||
before.width = width;
|
||||
before.height = height;
|
||||
const ctx = before.getContext('2d');
|
||||
if (!ctx) return;
|
||||
ctx.drawImage(video, 0, 0, width, height);
|
||||
try {
|
||||
sourceRef.current = ctx.getImageData(0, 0, width, height);
|
||||
setReady(true);
|
||||
onFrameCapturedRef.current?.(video.currentTime);
|
||||
} catch {
|
||||
// A cross-origin frame the browser wouldn't expose pixels for -- comparison stays unavailable.
|
||||
sourceRef.current = null;
|
||||
setReady(false);
|
||||
}
|
||||
}, [videoRef]);
|
||||
|
||||
React.useEffect(() => {
|
||||
const video = videoRef.current;
|
||||
if (!video) return;
|
||||
if (video.readyState >= 2) capture();
|
||||
video.addEventListener('loadeddata', capture);
|
||||
video.addEventListener('seeked', capture);
|
||||
return () => {
|
||||
video.removeEventListener('loadeddata', capture);
|
||||
video.removeEventListener('seeked', capture);
|
||||
};
|
||||
}, [videoRef, capture]);
|
||||
|
||||
React.useEffect(() => {
|
||||
const source = sourceRef.current;
|
||||
const after = afterRef.current;
|
||||
if (!source || !after) return;
|
||||
after.width = source.width;
|
||||
after.height = source.height;
|
||||
const ctx = after.getContext('2d');
|
||||
if (!ctx) return;
|
||||
const copy = new ImageData(new Uint8ClampedArray(source.data), source.width, source.height);
|
||||
if (!isNeutralGrade(brightness, contrast, saturation)) {
|
||||
applyGradeInPlace(copy.data, brightness, contrast, saturation);
|
||||
}
|
||||
ctx.putImageData(copy, 0, 0);
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps -- redraws whenever a new frame is captured (`ready`) too
|
||||
}, [brightness, contrast, saturation, ready]);
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-2">
|
||||
<div className="flex items-center justify-between">
|
||||
<p className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Before / after</p>
|
||||
<button type="button" onClick={capture} className="text-xs text-accent-blue hover:underline">
|
||||
Use current frame
|
||||
</button>
|
||||
</div>
|
||||
<div className="grid grid-cols-2 gap-2">
|
||||
<div className="flex flex-col gap-1">
|
||||
<canvas ref={beforeRef} role="img" aria-label="Before color grading" className="w-full rounded-md border border-border bg-black" />
|
||||
<span className="text-center text-[11px] text-muted-foreground">Before</span>
|
||||
</div>
|
||||
<div className="flex flex-col gap-1">
|
||||
<canvas ref={afterRef} role="img" aria-label="After color grading" className="w-full rounded-md border border-border bg-black" />
|
||||
<span className="text-center text-[11px] text-muted-foreground">After</span>
|
||||
</div>
|
||||
</div>
|
||||
{!ready && (
|
||||
<p className="text-xs text-muted-foreground">Play or seek the preview above to load a frame to compare.</p>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { toast } from 'sonner';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { downloadMergedVideo, getMergedVideoUrl, type MergedScene } from '@/lib/api';
|
||||
import { formatClock } from '@/lib/scenes';
|
||||
import { downloadBlob } from '@/lib/utils';
|
||||
|
||||
export interface MergedSceneDialogProps {
|
||||
/** the merged video to show, or null when closed */
|
||||
merged: (MergedScene & { title: string }) | null;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
/** The finished, merged scene: play it, and save it. */
|
||||
export function MergedSceneDialog({ merged, onClose }: MergedSceneDialogProps) {
|
||||
const [downloading, setDownloading] = React.useState(false);
|
||||
|
||||
async function download() {
|
||||
if (!merged) return;
|
||||
setDownloading(true);
|
||||
try {
|
||||
downloadBlob(await downloadMergedVideo(merged.filename), merged.filename);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't download the video.");
|
||||
} finally {
|
||||
setDownloading(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={!!merged} onOpenChange={(open) => !open && onClose()}>
|
||||
<DialogContent className="max-w-3xl">
|
||||
<DialogHeader>
|
||||
<DialogTitle>{merged?.title} merged</DialogTitle>
|
||||
<DialogDescription>
|
||||
{merged &&
|
||||
`${merged.clips} clip${merged.clips === 1 ? '' : 's'} joined in order · ${formatClock(merged.seconds)}`}
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
{merged && (
|
||||
<video
|
||||
src={getMergedVideoUrl(merged.filename)}
|
||||
aria-label={`Merged video of ${merged.title}`}
|
||||
className="max-h-[60vh] w-full rounded-md border border-border bg-black"
|
||||
controls
|
||||
playsInline
|
||||
preload="metadata"
|
||||
/>
|
||||
)}
|
||||
<DialogFooter>
|
||||
<Button type="button" variant="outline" onClick={onClose}>
|
||||
Close
|
||||
</Button>
|
||||
<Button type="button" onClick={download} disabled={downloading}>
|
||||
{downloading ? 'Downloading…' : 'Download'}
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,453 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { H3ShotPanel } from '@/components/jobs/H3ShotPanel';
|
||||
import { Badge, type BadgeProps } from '@/components/ui/badge';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { getJobVideoUrl } from '@/lib/api';
|
||||
import { attachedLastFrames, whyCannotAttach } from '@/lib/continuation';
|
||||
import { formatCutTime } from '@/lib/h3Shots';
|
||||
import { QUEUEABLE_STATUSES } from '@/lib/sceneQueue';
|
||||
import { formatClock, type Scene, type SceneClip } from '@/lib/scenes';
|
||||
import type { Job } from '@/lib/types';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const BADGE_VARIANTS: Record<string, BadgeProps['variant']> = {
|
||||
pending: 'secondary',
|
||||
queued: 'default',
|
||||
running: 'warning',
|
||||
completed: 'success',
|
||||
failed: 'destructive',
|
||||
stopped: 'secondary',
|
||||
};
|
||||
|
||||
const SEGMENT_TONE: Record<string, string> = {
|
||||
completed: 'bg-emerald-500/70',
|
||||
running: 'bg-amber-500/70',
|
||||
queued: 'bg-sky-500/60',
|
||||
failed: 'bg-rose-500/70',
|
||||
};
|
||||
|
||||
/** Statuses a job can still be edited in (mirrors the Jobs page). */
|
||||
export const EDITABLE_STATUSES = ['pending', 'failed', 'stopped'];
|
||||
|
||||
function clipLabel(clip: SceneClip): string {
|
||||
return clip.job.name?.trim() || `Job ${clip.job.id.slice(0, 8)}`;
|
||||
}
|
||||
|
||||
function takeLabel(job: Job, index: number, total: number): string {
|
||||
const ms = job.created_at < 1e12 ? job.created_at * 1000 : job.created_at;
|
||||
const when = new Date(ms).toLocaleString(undefined, {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
});
|
||||
return `Take ${index + 1} of ${total} · ${job.status} · ${when}`;
|
||||
}
|
||||
|
||||
function scrollToClip(id: string) {
|
||||
document.getElementById(`clip-${id}`)?.scrollIntoView({ behavior: 'smooth', block: 'start' });
|
||||
}
|
||||
|
||||
function Timeline({ scene }: { scene: Scene }) {
|
||||
return (
|
||||
<div className="flex h-4 w-full gap-px overflow-hidden rounded-md" role="group" aria-label="Scene timeline">
|
||||
{scene.clips.map((clip) => (
|
||||
<button
|
||||
key={clip.job.id}
|
||||
type="button"
|
||||
onClick={() => scrollToClip(clip.job.id)}
|
||||
aria-label={`Go to clip ${clip.position}: ${clipLabel(clip)}`}
|
||||
title={`Clip ${clip.position} · ${clipLabel(clip)} · ${clip.job.status} · ${formatClock(clip.startSeconds)}–${formatClock(clip.startSeconds + clip.durationSeconds)}`}
|
||||
style={{ flexGrow: Math.max(clip.durationSeconds, 0.1), flexBasis: 0 }}
|
||||
className={cn(
|
||||
'min-w-[3px] transition-opacity hover:opacity-70',
|
||||
SEGMENT_TONE[clip.job.status] ?? 'bg-muted-foreground/40',
|
||||
)}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function ClipPreview({ job, videoVersion }: { job: Job; videoVersion: number }) {
|
||||
const isImage = job.output_path?.toLowerCase().endsWith('.png') ?? false;
|
||||
const src = getJobVideoUrl(job.id, videoVersion);
|
||||
// Sets the *initial* muted state only, imperatively, so it doesn't fight the
|
||||
// user's own unmute via the native controls: `muted` as a plain JSX prop is
|
||||
// controlled, and React re-applies it on every re-render of this component
|
||||
// (e.g. the scene board's poll while a job is queued/running), silently
|
||||
// re-muting a video the user just unmuted.
|
||||
const videoRef = React.useRef<HTMLVideoElement | null>(null);
|
||||
React.useEffect(() => {
|
||||
if (videoRef.current) videoRef.current.muted = true;
|
||||
}, []);
|
||||
return isImage ? (
|
||||
// eslint-disable-next-line @next/next/no-img-element
|
||||
<img src={src} alt={job.name ?? 'Generated output'} className="max-h-56 rounded-md border border-border" />
|
||||
) : (
|
||||
<video
|
||||
ref={videoRef}
|
||||
src={src}
|
||||
aria-label={`Generated video for ${job.name ?? job.id}`}
|
||||
className="max-h-56 rounded-md border border-border bg-black"
|
||||
controls
|
||||
loop
|
||||
playsInline
|
||||
preload="metadata"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function ClipRow({
|
||||
clip,
|
||||
previous,
|
||||
firstShotNumber,
|
||||
onOpenJob,
|
||||
onChooseTake,
|
||||
onUseLastFrame,
|
||||
onDetachLastFrame,
|
||||
onQueueClip,
|
||||
onDequeueClip,
|
||||
onTrimClip,
|
||||
videoVersionFor,
|
||||
}: {
|
||||
clip: SceneClip;
|
||||
/** the clip before this one in the scene, if any */
|
||||
previous: SceneClip | undefined;
|
||||
firstShotNumber: number;
|
||||
onOpenJob: (job: Job) => void;
|
||||
onChooseTake: (takeKey: string, jobId: string) => void;
|
||||
onUseLastFrame: (clip: SceneClip, previous: SceneClip) => Promise<void>;
|
||||
onDetachLastFrame: (clip: SceneClip) => Promise<void>;
|
||||
onQueueClip: (clip: SceneClip) => Promise<void>;
|
||||
onDequeueClip: (clip: SceneClip) => Promise<void>;
|
||||
onTrimClip: (clip: SceneClip) => void;
|
||||
videoVersionFor: (jobId: string) => number;
|
||||
}) {
|
||||
const [previewing, setPreviewing] = React.useState(false);
|
||||
const [attaching, setAttaching] = React.useState(false);
|
||||
const [queueing, setQueueing] = React.useState(false);
|
||||
const { job } = clip;
|
||||
const canPreview = job.status === 'completed' && !!job.output_path;
|
||||
const editable = EDITABLE_STATUSES.includes(job.status);
|
||||
|
||||
const queued = job.status === 'queued';
|
||||
const canQueue = QUEUEABLE_STATUSES.includes(job.status);
|
||||
|
||||
// Starting from the previous clip's last frame: offered on clips that can still be
|
||||
// edited. The frame is taken when this clip runs, so the previous clip needn't be done yet.
|
||||
const frames = attachedLastFrames(job.references);
|
||||
const fromPrevious = previous && frames.some((f) => f.jobId === previous.job.id);
|
||||
const fromAnotherTake = frames.some((f) => f.jobId !== previous?.job.id);
|
||||
const previousDone = !!previous && previous.job.status === 'completed';
|
||||
const cannotUseFrame = previous ? whyCannotAttach(job.prompt, job.references) : null;
|
||||
|
||||
const usingLastFrame = frames.length > 0;
|
||||
const lastFrameBlocked = attaching || (!usingLastFrame && !!cannotUseFrame);
|
||||
|
||||
async function toggleLastFrame() {
|
||||
if (!previous) return;
|
||||
setAttaching(true);
|
||||
try {
|
||||
await (usingLastFrame ? onDetachLastFrame(clip) : onUseLastFrame(clip, previous));
|
||||
} finally {
|
||||
setAttaching(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function queueOrDequeue(action: (clip: SceneClip) => Promise<void>) {
|
||||
setQueueing(true);
|
||||
try {
|
||||
await action(clip);
|
||||
} finally {
|
||||
setQueueing(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<li id={`clip-${job.id}`} className="scroll-mt-32 rounded-xl border border-border bg-card p-3">
|
||||
<div className="mb-2 flex flex-wrap items-center gap-2">
|
||||
<span className="flex size-6 shrink-0 items-center justify-center rounded-md bg-secondary text-xs font-semibold text-foreground">
|
||||
{clip.position}
|
||||
</span>
|
||||
<h3 className="min-w-0 truncate text-sm font-semibold text-foreground">{clipLabel(clip)}</h3>
|
||||
<Badge variant={BADGE_VARIANTS[job.status] ?? 'secondary'}>{job.status}</Badge>
|
||||
<span className="font-mono text-xs text-muted-foreground">
|
||||
{formatClock(clip.startSeconds)}–{formatClock(clip.startSeconds + clip.durationSeconds)}
|
||||
</span>
|
||||
{clip.continuation && (
|
||||
<Badge variant="outline" className="normal-case tracking-normal">
|
||||
continues previous shot
|
||||
</Badge>
|
||||
)}
|
||||
<span className="ml-auto flex flex-wrap items-center justify-end gap-2">
|
||||
{previous && queued && fromPrevious && (
|
||||
<Badge variant="outline" className="normal-case tracking-normal">
|
||||
starts from clip {previous.position}'s last frame
|
||||
</Badge>
|
||||
)}
|
||||
{previous && queued && fromPrevious && !previousDone && (
|
||||
<Badge variant="outline" className="normal-case tracking-normal">
|
||||
waiting for clip {previous.position}
|
||||
</Badge>
|
||||
)}
|
||||
{previous && editable && (
|
||||
<label
|
||||
className={cn(
|
||||
'flex items-center gap-1.5 text-xs text-foreground',
|
||||
lastFrameBlocked && 'text-muted-foreground',
|
||||
)}
|
||||
title={
|
||||
usingLastFrame
|
||||
? "Stop starting this clip from the previous clip's last frame"
|
||||
: (cannotUseFrame ??
|
||||
`Attach the last frame of clip ${previous.position} as this clip's opening frame${
|
||||
previousDone ? '' : ` (taken when this clip runs, after clip ${previous.position} has finished)`
|
||||
}`)
|
||||
}
|
||||
>
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={usingLastFrame}
|
||||
onChange={toggleLastFrame}
|
||||
disabled={lastFrameBlocked}
|
||||
className="size-4"
|
||||
/>
|
||||
{attaching
|
||||
? usingLastFrame
|
||||
? 'Removing…'
|
||||
: 'Attaching…'
|
||||
: fromPrevious || !usingLastFrame
|
||||
? `Use last frame of clip ${previous.position}`
|
||||
: "Starts from another take's last frame"}
|
||||
</label>
|
||||
)}
|
||||
{clip.takeKey && clip.takes.length > 1 && (
|
||||
<NativeSelect
|
||||
aria-label={`Take for clip ${clip.position}`}
|
||||
value={job.id}
|
||||
onChange={(e) => onChooseTake(clip.takeKey as string, e.target.value)}
|
||||
className="h-8 w-auto py-0 text-xs"
|
||||
>
|
||||
{clip.takes.map((take, i) => (
|
||||
<option key={take.id} value={take.id}>
|
||||
{takeLabel(take, i, clip.takes.length)}
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
)}
|
||||
{canPreview && (
|
||||
<Button type="button" size="sm" variant="outline" onClick={() => setPreviewing((p) => !p)}>
|
||||
{previewing ? 'Hide video' : 'Preview'}
|
||||
</Button>
|
||||
)}
|
||||
{canPreview && (
|
||||
<Button type="button" size="sm" variant="outline" onClick={() => onTrimClip(clip)}>
|
||||
Edit video
|
||||
</Button>
|
||||
)}
|
||||
{canQueue && (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={() => queueOrDequeue(onQueueClip)}
|
||||
disabled={queueing}
|
||||
title="Run this clip when the ones queued before it have finished"
|
||||
>
|
||||
Queue
|
||||
</Button>
|
||||
)}
|
||||
{queued && (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={() => queueOrDequeue(onDequeueClip)}
|
||||
disabled={queueing}
|
||||
>
|
||||
Remove from queue
|
||||
</Button>
|
||||
)}
|
||||
<Button type="button" size="sm" variant="outline" onClick={() => onOpenJob(job)}>
|
||||
{editable ? 'Edit' : 'View'}
|
||||
</Button>
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{previewing && canPreview && (
|
||||
<div className="mb-2">
|
||||
<ClipPreview job={job} videoVersion={videoVersionFor(job.id)} />
|
||||
</div>
|
||||
)}
|
||||
|
||||
{clip.shots.length === 0 ? (
|
||||
<p className="rounded-lg border border-dashed border-border p-3 text-sm text-muted-foreground">
|
||||
This job's prompt has no shots.
|
||||
</p>
|
||||
) : (
|
||||
<div className="flex items-stretch gap-2 overflow-x-auto pb-1" aria-label={`Shots in ${clipLabel(clip)}`}>
|
||||
{clip.shots.map((shot, i) => (
|
||||
<div
|
||||
key={shot.id}
|
||||
className="flex w-52 shrink-0 flex-col gap-1.5 rounded-xl border border-border bg-background p-2"
|
||||
>
|
||||
<H3ShotPanel
|
||||
shot={shot}
|
||||
number={firstShotNumber + i}
|
||||
timeLabel={formatCutTime(clip.shotStarts[i])}
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</li>
|
||||
);
|
||||
}
|
||||
|
||||
export interface SceneBoardProps {
|
||||
scene: Scene;
|
||||
onOpenJob: (job: Job) => void;
|
||||
/** start a clip from the last frame of the one before it; resolves when done (errors are reported by the caller) */
|
||||
onUseLastFrame: (clip: SceneClip, previous: SceneClip) => Promise<void>;
|
||||
/** pick which re-run of a clip number to show */
|
||||
onChooseTake: (takeKey: string, jobId: string) => void;
|
||||
/** open the confirmation for queueing every runnable clip of the scene */
|
||||
onQueueScene: (scene: Scene) => void;
|
||||
/** join the scene's finished clips into one video; resolves when done (errors are reported by the caller) */
|
||||
onMergeScene: (scene: Scene) => Promise<void>;
|
||||
/** stop starting a clip from a previous clip's last frame; resolves when done */
|
||||
onDetachLastFrame: (clip: SceneClip) => Promise<void>;
|
||||
/** queue / unqueue one clip; resolve when done (errors are reported by the caller) */
|
||||
onQueueClip: (clip: SceneClip) => Promise<void>;
|
||||
onDequeueClip: (clip: SceneClip) => Promise<void>;
|
||||
/** open the trim dialog for one clip's video */
|
||||
onTrimClip: (clip: SceneClip) => void;
|
||||
/** cache-busting version for a job's video preview, bumped after each edit */
|
||||
videoVersionFor: (jobId: string) => number;
|
||||
}
|
||||
|
||||
/** One scene: a timeline of its clips, then every clip's shots in order, numbered straight through. */
|
||||
export function SceneBoard({
|
||||
scene,
|
||||
onOpenJob,
|
||||
onChooseTake,
|
||||
onUseLastFrame,
|
||||
onDetachLastFrame,
|
||||
onQueueScene,
|
||||
onMergeScene,
|
||||
onQueueClip,
|
||||
onDequeueClip,
|
||||
onTrimClip,
|
||||
videoVersionFor,
|
||||
}: SceneBoardProps) {
|
||||
let shotNumber = 1;
|
||||
const statuses = Object.entries(scene.statusCounts);
|
||||
const anythingToQueue = scene.clips.some((c) => QUEUEABLE_STATUSES.includes(c.job.status));
|
||||
|
||||
// Merging needs every clip's video, so a scene missing one can't pass for the whole thing.
|
||||
const unfinished = scene.clips.filter((c) => c.job.status !== 'completed' || !c.job.output_path).length;
|
||||
const [merging, setMerging] = React.useState(false);
|
||||
async function merge() {
|
||||
setMerging(true);
|
||||
try {
|
||||
await onMergeScene(scene);
|
||||
} finally {
|
||||
setMerging(false);
|
||||
}
|
||||
}
|
||||
|
||||
// The header stays where it is until it reaches the top of the scrolling area,
|
||||
// then sticks there. A 1px sentinel sitting just above it tells us which: once
|
||||
// the sentinel has scrolled out of view, the header is stuck.
|
||||
const sentinelRef = React.useRef<HTMLDivElement>(null);
|
||||
const [stuck, setStuck] = React.useState(false);
|
||||
React.useEffect(() => {
|
||||
const el = sentinelRef.current;
|
||||
if (!el || typeof IntersectionObserver === 'undefined') return;
|
||||
const observer = new IntersectionObserver(([entry]) => setStuck(!entry.isIntersecting));
|
||||
observer.observe(el);
|
||||
return () => observer.disconnect();
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<section aria-label={scene.title} className="flex flex-col gap-4">
|
||||
<div ref={sentinelRef} aria-hidden className="-mb-4 h-px" />
|
||||
<div
|
||||
data-stuck={stuck}
|
||||
className={cn(
|
||||
'sticky top-0 z-30 -mx-3 flex flex-col gap-2 bg-card px-3 pb-3 pt-2 transition-shadow',
|
||||
stuck && 'border-b border-border shadow-md',
|
||||
)}
|
||||
>
|
||||
<div className="flex flex-wrap items-baseline gap-x-4 gap-y-1">
|
||||
<h2 className="text-xl font-semibold text-foreground">{scene.title}</h2>
|
||||
<p className="text-sm text-muted-foreground">
|
||||
{scene.clips.length} clip{scene.clips.length === 1 ? '' : 's'} · {scene.shotCount} shot
|
||||
{scene.shotCount === 1 ? '' : 's'} · about {formatClock(scene.totalSeconds)}
|
||||
</p>
|
||||
<span className="flex flex-wrap gap-1.5">
|
||||
{statuses.map(([status, count]) => (
|
||||
<Badge key={status} variant={BADGE_VARIANTS[status] ?? 'secondary'}>
|
||||
{count} {status}
|
||||
</Badge>
|
||||
))}
|
||||
</span>
|
||||
<span className="ml-auto flex flex-wrap gap-2">
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={merge}
|
||||
disabled={unfinished > 0 || merging}
|
||||
title={
|
||||
unfinished > 0
|
||||
? `${unfinished} clip${unfinished === 1 ? " hasn't" : "s haven't"} finished yet`
|
||||
: 'Join all the clips, in order, into one video'
|
||||
}
|
||||
>
|
||||
{merging ? 'Merging…' : 'Merge scene'}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
onClick={() => onQueueScene(scene)}
|
||||
disabled={!anythingToQueue}
|
||||
title={anythingToQueue ? 'Queue every clip that has not run yet' : 'Every clip has already run or is queued'}
|
||||
>
|
||||
Queue scene
|
||||
</Button>
|
||||
</span>
|
||||
</div>
|
||||
<Timeline scene={scene} />
|
||||
</div>
|
||||
|
||||
<ol className="flex list-none flex-col gap-3 p-0">
|
||||
{scene.clips.map((clip, index) => {
|
||||
const first = shotNumber;
|
||||
shotNumber += clip.shots.length;
|
||||
return (
|
||||
<ClipRow
|
||||
key={clip.takeKey ?? clip.job.id}
|
||||
clip={clip}
|
||||
previous={scene.clips[index - 1]}
|
||||
firstShotNumber={first}
|
||||
onOpenJob={onOpenJob}
|
||||
onChooseTake={onChooseTake}
|
||||
onUseLastFrame={onUseLastFrame}
|
||||
onDetachLastFrame={onDetachLastFrame}
|
||||
onQueueClip={onQueueClip}
|
||||
onDequeueClip={onDequeueClip}
|
||||
onTrimClip={onTrimClip}
|
||||
videoVersionFor={videoVersionFor}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</ol>
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { attachedLastFrames } from '@/lib/continuation';
|
||||
import { planSceneQueue, QUEUEABLE_STATUSES, type SceneQueuePlan } from '@/lib/sceneQueue';
|
||||
import type { Scene, SceneClip } from '@/lib/scenes';
|
||||
|
||||
const plural = (n: number, word: string) => `${n} ${word}${n === 1 ? '' : 's'}`;
|
||||
const clipNumbers = (clips: SceneClip[]) => clips.map((c) => c.position).join(', ');
|
||||
|
||||
export interface SceneQueueDialogProps {
|
||||
scene: Scene | null;
|
||||
onConfirm: (plan: SceneQueuePlan) => Promise<void>;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Confirm queueing a whole scene: which clips are queued, and whether each
|
||||
* starts from the previous clip's last frame. Nothing is sent until "Queue".
|
||||
*/
|
||||
export function SceneQueueDialog({ scene, onConfirm, onClose }: SceneQueueDialogProps) {
|
||||
// Off by default: the per-clip "Use last frame" boxes are the source of truth, and
|
||||
// this would add links to every clip that doesn't have one.
|
||||
const [chain, setChain] = React.useState(false);
|
||||
const [busy, setBusy] = React.useState(false);
|
||||
|
||||
// Each opening starts from the default choice.
|
||||
React.useEffect(() => {
|
||||
if (scene) setChain(false);
|
||||
}, [scene]);
|
||||
|
||||
const plan = React.useMemo(() => (scene ? planSceneQueue(scene.clips, { chain }) : null), [scene, chain]);
|
||||
const alreadyLinked = React.useMemo(
|
||||
() =>
|
||||
scene
|
||||
? scene.clips.filter(
|
||||
(c) => QUEUEABLE_STATUSES.includes(c.job.status) && attachedLastFrames(c.job.references).length > 0,
|
||||
)
|
||||
: [],
|
||||
[scene],
|
||||
);
|
||||
// Whether the option would do anything is independent of the checkbox.
|
||||
const linkable = React.useMemo(
|
||||
() => (scene ? planSceneQueue(scene.clips, { chain: true }) : null),
|
||||
[scene],
|
||||
);
|
||||
|
||||
async function confirm() {
|
||||
if (!plan) return;
|
||||
setBusy(true);
|
||||
try {
|
||||
await onConfirm(plan);
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={!!scene} onOpenChange={(open) => !open && !busy && onClose()}>
|
||||
<DialogContent>
|
||||
<DialogHeader>
|
||||
<DialogTitle>Queue {scene?.title}</DialogTitle>
|
||||
<DialogDescription>
|
||||
{plan && plan.queue.length > 0
|
||||
? `${plural(plan.queue.length, 'clip')} will be queued and run one at a time, in order.`
|
||||
: 'There is nothing in this scene to queue.'}
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
{plan && linkable && (
|
||||
<div className="flex flex-col gap-3 text-sm">
|
||||
{alreadyLinked.length > 0 && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Clip {clipNumbers(alreadyLinked)} already start{alreadyLinked.length === 1 ? 's' : ''} from a previous
|
||||
frame (as you set on the clip{alreadyLinked.length === 1 ? '' : 's'}). Other clips are queued as they are.
|
||||
</p>
|
||||
)}
|
||||
{linkable.chain.length > 0 && (
|
||||
<label className="flex items-start gap-2">
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={chain}
|
||||
onChange={(e) => setChain(e.target.checked)}
|
||||
className="mt-1 size-4"
|
||||
/>
|
||||
<span>
|
||||
Start each clip from the previous clip's last frame
|
||||
<span className="block text-xs text-muted-foreground">
|
||||
Also links {plural(linkable.chain.length, 'clip')} that don't have it (
|
||||
{clipNumbers(linkable.chain.map((c) => c.clip))}), rewriting their prompts. Each waits for the clip
|
||||
before it to finish.
|
||||
</span>
|
||||
</span>
|
||||
</label>
|
||||
)}
|
||||
{chain && plan.cannotChain.length > 0 && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Clip {clipNumbers(plan.cannotChain.map((c) => c.clip))} can't be linked automatically (its prompt
|
||||
isn't in the six-section format) and will start without the previous frame.
|
||||
</p>
|
||||
)}
|
||||
{plan.skipped.length > 0 && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Skipping{' '}
|
||||
{plan.skipped.map((s) => `clip ${s.clip.position} (${s.reason})`).join(', ')}.
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<DialogFooter>
|
||||
<Button type="button" variant="outline" onClick={onClose} disabled={busy}>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button type="button" onClick={confirm} disabled={busy || !plan || plan.queue.length === 0}>
|
||||
{busy ? 'Queueing…' : plan ? `Queue ${plural(plan.queue.length, 'clip')}` : 'Queue'}
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user