Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c59cf79ea4 | ||
|
|
fab9b3159c | ||
|
|
d377f463f1 | ||
|
|
01050277ea |
@@ -16,6 +16,9 @@ The UI currently supports:
|
||||
```
|
||||
apps/fastvideo_studio/
|
||||
├── server.py / job_runner.py / database.py # FastAPI backend + job lifecycle
|
||||
├── job_queue.py # Queue scheduling rules + last-frame references (GPU-free)
|
||||
├── merge.py # Join a scene's clips into one video (ffmpeg stream copy, GPU-free)
|
||||
├── trim.py # Cut a clip to an in/out range, non-destructively (GPU-free)
|
||||
├── mock_server.py # In-memory API mock for e2e tests
|
||||
├── models/ # Pydantic request models (shared with the mock)
|
||||
├── training_config.py # Studio workloads → fastvideo/train YAML configs
|
||||
@@ -88,6 +91,93 @@ Running it this way allows you to pass command line parameters.
|
||||
The API server defaults to port 8189. To configure this, you can edit `.env.local` file.
|
||||
Refer to `.env.example` for reference.
|
||||
|
||||
### Job queue
|
||||
|
||||
`POST /api/jobs/queue` puts jobs in a queue; the server runs them oldest-first,
|
||||
`--max-concurrent-jobs` at a time (default 1). Start on a job while that many
|
||||
are already running is refused; queue it instead. Queued jobs survive a server
|
||||
restart and carry on.
|
||||
|
||||
Inference jobs run on that many long-lived worker threads. The model's GPU
|
||||
processes are killed when the thread that started them exits, so a worker that
|
||||
never exits is what keeps the model loaded from one job to the next (it loads
|
||||
once, not per job). A worker holds one model at a time: a job with a different
|
||||
model or settings replaces it, and a failed or stopped job unloads it so the next
|
||||
one starts clean. Each extra worker loads its own copy of the model.
|
||||
|
||||
A job can start from how another job's video ended, two ways:
|
||||
|
||||
* **Last frame** -- a reference whose source is `job-last-frame:<job id>`. A
|
||||
hard, exact opening frame: the prompt is rewritten to say the shot begins
|
||||
from it.
|
||||
* **Last clip** -- a reference whose source is `job-last-clip:<job id>`, and
|
||||
`media_type: "video"`. The trailing ~1s of the source job's video instead of
|
||||
a single still, so the model has real motion to continue rather than a pose
|
||||
to guess it from. Costs one of H3's 3 video-reference slots instead of one
|
||||
of its 9 image slots.
|
||||
|
||||
Either is extracted when the job runs, so the reference can be added before
|
||||
that job has finished, and the queue holds the job until it has. If that job
|
||||
fails, is stopped, or is deleted, the waiting job fails with a message saying
|
||||
so (and so does anything waiting on *it*); fix the earlier job and queue them
|
||||
again. Starting such a job directly while its source is unfinished is refused
|
||||
with a 409.
|
||||
|
||||
### Merging a scene
|
||||
|
||||
**Merge scene** on the Scenes page joins a scene's clips, in the order shown (the
|
||||
take chosen for each), into one video under `<output-dir>/merged/`, then plays it
|
||||
and offers a download. It is enabled once every clip has finished, and refuses
|
||||
otherwise, so a scene missing a clip can't pass for the whole thing.
|
||||
|
||||
The clips are copied, not re-encoded (about a second for 15 clips, no quality
|
||||
loss). That needs them to share size, frame rate and audio, which clips from one
|
||||
scene do; if one differs, the merge says which. Each clip is cut at the end of its
|
||||
last video frame, because H3's audio runs a few milliseconds longer and would
|
||||
otherwise leave a small gap at every join. It uses `ffmpeg` from `PATH`,
|
||||
`$FASTVIDEO_FFMPEG_BIN`, or the copy bundled with `imageio-ffmpeg`.
|
||||
|
||||
### Editing a clip's video
|
||||
|
||||
**Edit video** on a finished clip in Scenes cuts it to a chosen in/out range and
|
||||
adjusts its brightness, contrast and saturation, in place -- other jobs'
|
||||
last-frame/last-clip references, and a later merge, all just read the job's
|
||||
`output_path`, so nothing else needs to know a clip was edited. Frame-accurate
|
||||
(re-encoded, not stream-copied), so the cut isn't limited to the source's
|
||||
keyframes. Audio is cut to the same range and carried over too (re-encoded to
|
||||
AAC regardless of the source's codec), unless the source has none.
|
||||
|
||||
Color grading is per **section**, not the whole clip at once, like a razor/blade
|
||||
tool: **Split at current time** cuts the kept range wherever the preview is
|
||||
paused, and each resulting section gets its own brightness, contrast and
|
||||
saturation sliders, next to a before/after preview of a still frame
|
||||
(**Preview** on a section seeks to its start and captures that frame; **Use
|
||||
current frame** captures wherever the preview is paused instead). That preview
|
||||
is computed entirely in the browser (`src/lib/grading.ts` mirrors `trim.py`'s
|
||||
`_apply_grade` pixel math) as the sliders move, so it never waits on the
|
||||
server -- only **Apply** does that, to actually re-render the file. **Previous
|
||||
frame**/**Next frame** step the preview by exactly one frame (at the clip's
|
||||
own fps), for lining up a cut precisely -- the native scrub bar alone isn't
|
||||
frame-accurate. **Merge with next** undoes a cut, keeping the earlier of the
|
||||
two sections' grade.
|
||||
Sections are always contiguous and cover the whole kept range -- there's no
|
||||
reordering or gaps, only where to cut and what to grade each side; a plain
|
||||
range trim with no cuts is just the one-section case. Audio is one continuous
|
||||
track regardless of how many sections there are.
|
||||
|
||||
The untouched original is always kept alongside it the first time a clip is
|
||||
edited, and **every edit always re-renders from that original** with the full
|
||||
set of values shown in the dialog -- range and sections together, in one pass.
|
||||
That's what lets adjusting either one leave the other in place instead of each
|
||||
one silently discarding the other. The job itself records the range/sections
|
||||
of its last edit (`edit_start_seconds`/`edit_end_seconds`/`edit_segments`,
|
||||
alongside `output_path`), so reopening the dialog -- even after closing it or
|
||||
reloading the page -- shows exactly what produced the current video, not a
|
||||
guess: a never-edited clip shows one neutral section covering its full length;
|
||||
an edited one shows every section that made it, cuts and start offset
|
||||
included. **Restore original** undoes every edit in one step and resets those
|
||||
recorded values back to one neutral section too.
|
||||
|
||||
### API Endpoints
|
||||
|
||||
| Method | Path | Description |
|
||||
@@ -98,6 +188,15 @@ Refer to `.env.example` for reference.
|
||||
| `POST` | `/api/jobs` | Create a new job |
|
||||
| `POST` | `/api/jobs/{id}/start` | Start a pending/stopped/failed job |
|
||||
| `POST` | `/api/jobs/{id}/stop` | Request a running job to stop |
|
||||
| `POST` | `/api/jobs/queue` | Queue jobs (`{"job_ids": [...]}`), in order |
|
||||
| `POST` | `/api/jobs/{id}/queue` | Queue one job |
|
||||
| `POST` | `/api/jobs/{id}/dequeue` | Take a queued job back out of the queue |
|
||||
| `POST` | `/api/jobs/{id}/last-frame` | Save the job's last frame, for another job to start from |
|
||||
| `POST` | `/api/jobs/{id}/last-clip` | Save the job's trailing ~1s, for another job to continue from |
|
||||
| `POST` | `/api/scenes/merge` | Join finished jobs' videos (`{"job_ids": [...], "name": ""}`) in order |
|
||||
| `POST` | `/api/jobs/{id}/trim` | Cut a job's video to a range and/or grade it section by section (`{"start_seconds": 0, "end_seconds": null, "segments": [{"end_seconds": null, "brightness": 0, "contrast": 1, "saturation": 1}]}`), in place |
|
||||
| `POST` | `/api/jobs/{id}/restore-video`| Undo every edit, back to the original video |
|
||||
| `GET` | `/api/merged/{filename}` | Stream a merged scene video |
|
||||
| `DELETE` | `/api/jobs/{id}` | Delete a job |
|
||||
| `GET` | `/api/jobs/{id}/video` | Stream the generated video/image |
|
||||
| `GET` | `/api/jobs/{id}/logs` | Get job logs (polling, supports `?after=`) |
|
||||
|
||||
@@ -110,6 +110,15 @@ def _migrate_db(conn: sqlite3.Connection) -> None:
|
||||
_add_column_if_missing(conn, "jobs", "generator_update_interval", "INTEGER", "5")
|
||||
_add_column_if_missing(conn, "jobs", "real_score_model_path", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "fake_score_model_path", "TEXT", "''")
|
||||
_add_column_if_missing(conn, "jobs", "queued_at", "REAL", "NULL")
|
||||
_add_column_if_missing(conn, "jobs", "edit_start_seconds", "REAL", "0.0")
|
||||
_add_column_if_missing(conn, "jobs", "edit_end_seconds", "REAL", "NULL")
|
||||
# Superseded by edit_segments_json (a list of razor-cut sections, each with
|
||||
# its own grade); left in place, unused, rather than risking a DROP COLUMN.
|
||||
_add_column_if_missing(conn, "jobs", "edit_brightness", "REAL", "0.0")
|
||||
_add_column_if_missing(conn, "jobs", "edit_contrast", "REAL", "1.0")
|
||||
_add_column_if_missing(conn, "jobs", "edit_saturation", "REAL", "1.0")
|
||||
_add_column_if_missing(conn, "jobs", "edit_segments_json", "TEXT", "'[]'")
|
||||
# Settings table
|
||||
_add_column_if_missing(conn, "settings", "vae_cpu_offload", "INTEGER", "0")
|
||||
_add_column_if_missing(conn, "settings", "image_encoder_cpu_offload", "INTEGER", "0")
|
||||
@@ -326,6 +335,7 @@ class Database:
|
||||
return
|
||||
allowed = {
|
||||
"status",
|
||||
"queued_at",
|
||||
"started_at",
|
||||
"finished_at",
|
||||
"error",
|
||||
@@ -345,6 +355,43 @@ class Database:
|
||||
self._execute(sql, tuple(vals))
|
||||
self._commit()
|
||||
|
||||
#: Job fields stored under a different column name, and how they are encoded.
|
||||
_CONFIG_COLUMNS = {"references": "references_json", "edit_segments": "edit_segments_json"}
|
||||
_JSON_LIST_CONFIG_FIELDS = {"references", "edit_segments"}
|
||||
_BOOL_CONFIG_FIELDS = {
|
||||
"dit_cpu_offload",
|
||||
"text_encoder_cpu_offload",
|
||||
"vae_cpu_offload",
|
||||
"image_encoder_cpu_offload",
|
||||
"use_fsdp_inference",
|
||||
"enable_torch_compile",
|
||||
"dmd_use_vsa",
|
||||
}
|
||||
|
||||
def update_job_config(self, job_id: str, updates: dict[str, Any]) -> None:
|
||||
"""Persist edits to a job's configuration (prompt, references, sizes...).
|
||||
|
||||
Fields the table has no column for are skipped, as they are on insert.
|
||||
"""
|
||||
columns = _get_table_columns(self._conn(), "jobs")
|
||||
cols = []
|
||||
vals: list[Any] = []
|
||||
for key, value in updates.items():
|
||||
column = self._CONFIG_COLUMNS.get(key, key)
|
||||
if column not in columns or column == "id":
|
||||
continue
|
||||
if key in self._JSON_LIST_CONFIG_FIELDS:
|
||||
value = json.dumps(value or [])
|
||||
elif key in self._BOOL_CONFIG_FIELDS:
|
||||
value = 1 if value else 0
|
||||
cols.append(f"{column} = ?")
|
||||
vals.append(value)
|
||||
if not cols:
|
||||
return
|
||||
vals.append(job_id)
|
||||
self._execute(f"UPDATE jobs SET {', '.join(cols)} WHERE id = ?", tuple(vals))
|
||||
self._commit()
|
||||
|
||||
def delete_job(self, job_id: str) -> bool:
|
||||
"""Delete a job. Returns True if a row was deleted."""
|
||||
cur = self._execute("DELETE FROM jobs WHERE id = ?", (job_id, ))
|
||||
@@ -537,8 +584,31 @@ def _row_to_dataset(row: sqlite3.Row) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
|
||||
def _row_to_job(row: sqlite3.Row) -> dict[str, Any]:
|
||||
class _ColumnRow:
|
||||
"""A sqlite3.Row whose ``in`` tests column names.
|
||||
|
||||
sqlite3.Row's own ``in`` tests *values*, so ``"fps" in row`` is always False
|
||||
and a job restored from the database silently lost fps, negative_prompt and
|
||||
the other optional columns.
|
||||
"""
|
||||
|
||||
def __init__(self, row: sqlite3.Row):
|
||||
self._row = row
|
||||
self._columns = set(row.keys())
|
||||
|
||||
def __getitem__(self, key: str) -> Any:
|
||||
return self._row[key]
|
||||
|
||||
def __contains__(self, key: object) -> bool:
|
||||
return key in self._columns
|
||||
|
||||
def keys(self) -> list[str]:
|
||||
return self._row.keys()
|
||||
|
||||
|
||||
def _row_to_job(sqlite_row: sqlite3.Row) -> dict[str, Any]:
|
||||
"""Convert a DB row to job dict (snake_case, matching Job.to_dict)."""
|
||||
row = _ColumnRow(sqlite_row)
|
||||
int_defaults = {
|
||||
"max_train_steps": 1000,
|
||||
"train_batch_size": 1,
|
||||
@@ -562,6 +632,7 @@ def _row_to_job(row: sqlite3.Row) -> dict[str, Any]:
|
||||
"job_type": _sqlite_row_get(row, "job_type", "inference"),
|
||||
"status": row["status"],
|
||||
"created_at": row["created_at"],
|
||||
"queued_at": row["queued_at"] if "queued_at" in row else None,
|
||||
"started_at": row["started_at"],
|
||||
"finished_at": row["finished_at"],
|
||||
"error": row["error"],
|
||||
@@ -614,6 +685,13 @@ def _row_to_job(row: sqlite3.Row) -> dict[str, Any]:
|
||||
for col in ("dmd_denoising_steps", "real_score_model_path", "fake_score_model_path"):
|
||||
if col in row:
|
||||
result[col] = (row[col] or "") or ""
|
||||
if "edit_start_seconds" in row:
|
||||
start_val: Any = row["edit_start_seconds"]
|
||||
result["edit_start_seconds"] = float(start_val) if start_val is not None else 0.0
|
||||
if "edit_end_seconds" in row:
|
||||
end_val: Any = row["edit_end_seconds"]
|
||||
result["edit_end_seconds"] = float(end_val) if end_val is not None else None
|
||||
result["edit_segments"] = _sqlite_row_get(row, "edit_segments_json", "") or ""
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,146 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Save the end of a finished clip, so the next clip can continue from it.
|
||||
|
||||
Two forms: the final frame alone (a still, for a hard "opens on this exact
|
||||
picture" continuation), or a short trailing clip (a few hundred milliseconds of
|
||||
video, attached as an H3 video reference so the model has actual motion to
|
||||
continue rather than guessing it from one still). Kept apart from ``server.py``
|
||||
because it needs nothing from FastVideo -- only PyAV -- and so can be exercised
|
||||
without a GPU.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from fractions import Fraction
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
VIDEO_EXTENSIONS = {".mp4", ".webm", ".avi", ".mov", ".mkv"}
|
||||
|
||||
|
||||
class FrameError(Exception):
|
||||
"""A request that can't be satisfied, carrying the HTTP status to report."""
|
||||
|
||||
def __init__(self, status_code: int, detail: str):
|
||||
super().__init__(detail)
|
||||
self.status_code = status_code
|
||||
self.detail = detail
|
||||
|
||||
|
||||
def save_last_frame(video_path: str, out_path: str) -> None:
|
||||
"""Decode ``video_path`` and write its final frame to ``out_path`` as a PNG.
|
||||
|
||||
Clips are a few hundred frames, so decoding through to the end is simpler
|
||||
and more reliable than seeking (which lands on keyframes, not the last frame).
|
||||
"""
|
||||
import av
|
||||
|
||||
tmp_path = f"{out_path}.tmp.png"
|
||||
with av.open(video_path) as container:
|
||||
last = None
|
||||
for frame in container.decode(container.streams.video[0]):
|
||||
last = frame
|
||||
if last is None:
|
||||
raise ValueError("The video has no frames.")
|
||||
last.to_image().save(tmp_path)
|
||||
os.replace(tmp_path, out_path) # atomic, so a reader never sees a half-written file
|
||||
|
||||
|
||||
def last_frame_for_job(job: Any, upload_dir: str) -> str:
|
||||
"""Path of a PNG of the last frame of ``job``'s video, extracting it if needed.
|
||||
|
||||
Raises :class:`FrameError` with the status the API should return.
|
||||
"""
|
||||
status = getattr(job.status, "value", job.status)
|
||||
if status != "completed" or not job.output_path:
|
||||
raise FrameError(404, "No output available for this job")
|
||||
if not os.path.isfile(job.output_path):
|
||||
raise FrameError(404, "Output file not found on disk")
|
||||
if Path(job.output_path).suffix.lower() not in VIDEO_EXTENSIONS:
|
||||
raise FrameError(400, "This job's output is not a video")
|
||||
if not upload_dir:
|
||||
raise FrameError(503, "Upload directory not configured")
|
||||
|
||||
out_dir = os.path.join(upload_dir, "last_frames")
|
||||
os.makedirs(out_dir, exist_ok=True)
|
||||
out_path = os.path.abspath(os.path.join(out_dir, f"last_frame_{job.id}.png"))
|
||||
# Reuse an earlier extraction unless the video has been rewritten since.
|
||||
if not (os.path.isfile(out_path) and os.path.getmtime(out_path) >= os.path.getmtime(job.output_path)):
|
||||
try:
|
||||
save_last_frame(job.output_path, out_path)
|
||||
except Exception as e:
|
||||
raise FrameError(500, f"Could not read the last frame: {e}") from e
|
||||
return out_path
|
||||
|
||||
|
||||
DEFAULT_LAST_CLIP_SECONDS = 1.0
|
||||
|
||||
|
||||
def save_last_clip(video_path: str, out_path: str, seconds: float = DEFAULT_LAST_CLIP_SECONDS) -> None:
|
||||
"""Decode the final ``seconds`` of ``video_path`` and re-encode them to ``out_path``.
|
||||
|
||||
Re-encoded rather than stream-copied: the source's keyframes rarely align
|
||||
with an arbitrary cut ``seconds`` from the end, and a copy can only start on
|
||||
one. Silent -- the trailing video is for motion continuity; voice continuity
|
||||
already comes from each job's own audio reference.
|
||||
"""
|
||||
import av
|
||||
|
||||
if seconds <= 0:
|
||||
raise ValueError(f"seconds must be positive, got {seconds}.")
|
||||
|
||||
tmp_path = f"{out_path}.tmp.mp4"
|
||||
with av.open(video_path) as src:
|
||||
video_stream = src.streams.video[0]
|
||||
fps = float(video_stream.average_rate or video_stream.guessed_rate or 24)
|
||||
frames = list(src.decode(video_stream))
|
||||
if not frames:
|
||||
raise ValueError("The video has no frames.")
|
||||
keep = max(1, round(seconds * fps))
|
||||
tail = frames[-keep:]
|
||||
|
||||
with av.open(tmp_path, mode="w") as out:
|
||||
out_stream = out.add_stream("h264", rate=round(fps))
|
||||
out_stream.width, out_stream.height = tail[0].width, tail[0].height
|
||||
out_stream.pix_fmt = "yuv420p"
|
||||
# Each frame needs its own increasing pts at a known time base, or PyAV
|
||||
# stamps every frame identically (pts=0), which corrupts the muxed
|
||||
# container's average-frame-rate metadata -- H3's reference pipeline
|
||||
# trusts that metadata to resample the clip, so a wrong rate there
|
||||
# silently collapses a short clip down to as little as one frame.
|
||||
frame_time_base = Fraction(1, round(fps))
|
||||
for i, frame in enumerate(tail):
|
||||
frame.pts = i
|
||||
frame.time_base = frame_time_base
|
||||
for packet in out_stream.encode(frame):
|
||||
out.mux(packet)
|
||||
for packet in out_stream.encode():
|
||||
out.mux(packet)
|
||||
os.replace(tmp_path, out_path) # atomic, so a reader never sees a half-written file
|
||||
|
||||
|
||||
def last_clip_for_job(job: Any, upload_dir: str, seconds: float = DEFAULT_LAST_CLIP_SECONDS) -> str:
|
||||
"""Path of a short video of the end of ``job``'s video, extracting it if needed.
|
||||
|
||||
Raises :class:`FrameError` with the status the API should return.
|
||||
"""
|
||||
status = getattr(job.status, "value", job.status)
|
||||
if status != "completed" or not job.output_path:
|
||||
raise FrameError(404, "No output available for this job")
|
||||
if not os.path.isfile(job.output_path):
|
||||
raise FrameError(404, "Output file not found on disk")
|
||||
if Path(job.output_path).suffix.lower() not in VIDEO_EXTENSIONS:
|
||||
raise FrameError(400, "This job's output is not a video")
|
||||
if not upload_dir:
|
||||
raise FrameError(503, "Upload directory not configured")
|
||||
|
||||
out_dir = os.path.join(upload_dir, "last_clips")
|
||||
os.makedirs(out_dir, exist_ok=True)
|
||||
out_path = os.path.abspath(os.path.join(out_dir, f"last_clip_{job.id}.mp4"))
|
||||
if not (os.path.isfile(out_path) and os.path.getmtime(out_path) >= os.path.getmtime(job.output_path)):
|
||||
try:
|
||||
save_last_clip(job.output_path, out_path, seconds)
|
||||
except Exception as e:
|
||||
raise FrameError(500, f"Could not read the end of the clip: {e}") from e
|
||||
return out_path
|
||||
@@ -0,0 +1,208 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Scheduling rules for queued jobs, and references to another job's ending.
|
||||
|
||||
Kept apart from ``job_runner.py`` because it is pure -- it works on plain
|
||||
snapshots and needs neither FastVideo nor a GPU -- so it can be tested anywhere.
|
||||
|
||||
A job can start from how another job's video ended -- either its last frame (a
|
||||
still, for a hard "opens on this exact picture" continuation) or its last
|
||||
second or so (a short video, so the model has actual motion to continue, not
|
||||
just a pose to guess motion from). Neither exists until the other job
|
||||
finishes, so the reference is *deferred*: its source is ``job-last-frame:<job
|
||||
id>`` or ``job-last-clip:<job id>``, resolved to a real file only when the job
|
||||
runs. That link is also the queue's dependency graph: a queued job waits until
|
||||
every job it takes an ending from has completed.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable, Iterable, Mapping
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Protocol
|
||||
|
||||
DEFERRED_LAST_FRAME_PREFIX = "job-last-frame:"
|
||||
DEFERRED_LAST_CLIP_PREFIX = "job-last-clip:"
|
||||
_DEFERRED_PREFIXES = (DEFERRED_LAST_FRAME_PREFIX, DEFERRED_LAST_CLIP_PREFIX)
|
||||
|
||||
QUEUED = "queued"
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
PENDING = "pending"
|
||||
FAILED = "failed"
|
||||
STOPPED = "stopped"
|
||||
|
||||
|
||||
def deferred_last_frame_source(job_id: str) -> str:
|
||||
return f"{DEFERRED_LAST_FRAME_PREFIX}{job_id}"
|
||||
|
||||
|
||||
def deferred_last_clip_source(job_id: str) -> str:
|
||||
return f"{DEFERRED_LAST_CLIP_PREFIX}{job_id}"
|
||||
|
||||
|
||||
def dependency_of(reference: Mapping[str, Any] | None) -> str | None:
|
||||
"""The job a deferred reference takes its ending from, or None for any other."""
|
||||
source = (reference or {}).get("source")
|
||||
if not isinstance(source, str):
|
||||
return None
|
||||
for prefix in _DEFERRED_PREFIXES:
|
||||
if source.startswith(prefix):
|
||||
return source[len(prefix):] or None
|
||||
return None
|
||||
|
||||
|
||||
def dependencies(references: Iterable[Mapping[str, Any]] | None) -> list[str]:
|
||||
"""Job ids a job needs finished first, in reference order, without repeats."""
|
||||
found: list[str] = []
|
||||
for ref in references or []:
|
||||
dep = dependency_of(ref)
|
||||
if dep and dep not in found:
|
||||
found.append(dep)
|
||||
return found
|
||||
|
||||
|
||||
class JobView(Protocol):
|
||||
"""The parts of a job the scheduler looks at."""
|
||||
|
||||
id: str
|
||||
references: Any
|
||||
queued_at: float | None
|
||||
|
||||
@property
|
||||
def status(self) -> Any: # a JobStatus (str enum) or plain str
|
||||
...
|
||||
|
||||
|
||||
def _status(job: Any) -> str:
|
||||
return str(getattr(job.status, "value", job.status))
|
||||
|
||||
|
||||
def circular_dependency(jobs: Mapping[str, JobView], job_id: str) -> list[str] | None:
|
||||
"""The chain of job ids if ``job_id`` (transitively) waits on itself, else None."""
|
||||
|
||||
def walk(current: str, path: list[str]) -> list[str] | None:
|
||||
job = jobs.get(current)
|
||||
if job is None:
|
||||
return None
|
||||
for dep in dependencies(job.references):
|
||||
if dep == job_id:
|
||||
return [*path, current, dep]
|
||||
if dep in path or dep == current:
|
||||
continue
|
||||
found = walk(dep, [*path, current])
|
||||
if found:
|
||||
return found
|
||||
return None
|
||||
|
||||
return walk(job_id, [])
|
||||
|
||||
|
||||
def _label(job: Any) -> str:
|
||||
name = getattr(job, "name", "") or ""
|
||||
return f"'{name}'" if name else job.id
|
||||
|
||||
|
||||
@dataclass
|
||||
class Plan:
|
||||
"""What to do now: jobs to launch, and jobs that can never run."""
|
||||
|
||||
start: list[str] = field(default_factory=list)
|
||||
fail: dict[str, str] = field(default_factory=dict) # job id -> reason
|
||||
|
||||
|
||||
def plan_dispatch(jobs: Mapping[str, JobView], max_concurrent: int) -> Plan:
|
||||
"""Decide which queued jobs launch now and which have become impossible.
|
||||
|
||||
* Queued jobs run oldest-queued first, at most ``max_concurrent`` at a time
|
||||
(counting jobs already running).
|
||||
* A job whose frame source hasn't completed *waits*, and doesn't hold up
|
||||
jobs behind it that don't need it.
|
||||
* A job whose frame source failed, was stopped, or was deleted can never
|
||||
run, so it fails -- and so does anything waiting on *it*.
|
||||
* A job whose frame source is idle (neither queued nor running) keeps
|
||||
waiting: the user may still queue it.
|
||||
"""
|
||||
plan = Plan()
|
||||
status = {jid: _status(job) for jid, job in jobs.items()}
|
||||
queued = sorted(
|
||||
(job for job in jobs.values() if status[job.id] == QUEUED),
|
||||
key=lambda job: (job.queued_at or 0.0, job.id),
|
||||
)
|
||||
|
||||
# Failures cascade down a chain, so repeat until nothing new fails.
|
||||
changed = True
|
||||
while changed:
|
||||
changed = False
|
||||
for job in queued:
|
||||
if job.id in plan.fail:
|
||||
continue
|
||||
for dep_id in dependencies(job.references):
|
||||
dep = jobs.get(dep_id)
|
||||
if dep is None:
|
||||
plan.fail[job.id] = "Waiting on the ending of a job that was deleted."
|
||||
elif dep_id in plan.fail or status[dep_id] in (FAILED, STOPPED):
|
||||
reason = "failed" if dep_id in plan.fail or status[dep_id] == FAILED else "was stopped"
|
||||
plan.fail[job.id] = f"Needs the ending of {_label(dep)}, which {reason}."
|
||||
else:
|
||||
continue
|
||||
status[job.id] = FAILED
|
||||
changed = True
|
||||
break
|
||||
|
||||
slots = max(max_concurrent, 1) - sum(1 for s in status.values() if s == RUNNING)
|
||||
for job in queued:
|
||||
if slots <= 0:
|
||||
break
|
||||
if job.id in plan.fail:
|
||||
continue
|
||||
if all(status.get(dep) == COMPLETED for dep in dependencies(job.references)):
|
||||
plan.start.append(job.id)
|
||||
slots -= 1
|
||||
return plan
|
||||
|
||||
|
||||
class UnresolvedReference(ValueError):
|
||||
"""A deferred reference can't be turned into a real file."""
|
||||
|
||||
|
||||
def resolve_references(
|
||||
references: list[dict[str, Any]] | None,
|
||||
get_job: Callable[[str], Any],
|
||||
last_frame: Callable[[Any], str],
|
||||
last_clip: Callable[[Any], str] | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Copy of ``references`` with each deferred reference made real.
|
||||
|
||||
``last_frame(job)`` and ``last_clip(job)`` each return the path of an
|
||||
extracted file and may raise (see ``frames.FrameError``); the message is
|
||||
passed on. ``last_clip`` is optional so callers that never attach a
|
||||
trailing-clip reference (e.g. the mock server) don't need to supply one --
|
||||
it is only required if a ``job-last-clip:`` reference is actually present.
|
||||
Other references are returned unchanged, and the stored list is never
|
||||
modified, so a re-run resolves again against the source job's current
|
||||
output.
|
||||
"""
|
||||
resolved: list[dict[str, Any]] = []
|
||||
for ref in references or []:
|
||||
source = (ref or {}).get("source")
|
||||
prefix = next((p for p in _DEFERRED_PREFIXES if isinstance(source, str) and source.startswith(p)), None)
|
||||
if prefix is None:
|
||||
resolved.append(ref)
|
||||
continue
|
||||
dep_id = source[len(prefix):] or None
|
||||
extract = last_frame if prefix == DEFERRED_LAST_FRAME_PREFIX else last_clip
|
||||
what = "the last frame of" if prefix == DEFERRED_LAST_FRAME_PREFIX else "the ending of"
|
||||
if extract is None:
|
||||
raise UnresolvedReference("This server can't resolve a trailing-clip reference yet.")
|
||||
dep = get_job(dep_id) if dep_id else None
|
||||
if dep is None:
|
||||
raise UnresolvedReference(f"A reference needs {what} a job that no longer exists.")
|
||||
if _status(dep) != COMPLETED:
|
||||
raise UnresolvedReference(f"A reference needs {what} {_label(dep)}, which hasn't finished "
|
||||
f"(it is {_status(dep)}).")
|
||||
try:
|
||||
path = extract(dep)
|
||||
except Exception as exc: # FrameError carries a readable .detail
|
||||
raise UnresolvedReference(str(getattr(exc, "detail", None) or exc)) from exc
|
||||
resolved.append({**ref, "source": path})
|
||||
return resolved
|
||||
@@ -17,6 +17,7 @@ import logging
|
||||
import logging.handlers
|
||||
import multiprocessing as mp
|
||||
import os
|
||||
import queue
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
@@ -29,6 +30,15 @@ import yaml
|
||||
|
||||
from fastvideo.utils import get_mp_context
|
||||
from fastvideo_studio.database import Database
|
||||
from fastvideo_studio.frames import last_clip_for_job, last_frame_for_job
|
||||
from fastvideo_studio.trim import restore_original, trim_video
|
||||
from fastvideo_studio.job_queue import (
|
||||
UnresolvedReference,
|
||||
circular_dependency,
|
||||
dependencies,
|
||||
plan_dispatch,
|
||||
resolve_references,
|
||||
)
|
||||
from fastvideo_studio.training_config import (
|
||||
build_training_config,
|
||||
get_training_env,
|
||||
@@ -46,6 +56,7 @@ _MAX_LOG_LINES = 2000 # ring-buffer cap per job
|
||||
|
||||
class JobStatus(str, enum.Enum):
|
||||
PENDING = "pending"
|
||||
QUEUED = "queued"
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
@@ -138,6 +149,16 @@ class Job:
|
||||
finished_at: float | None = None
|
||||
error: str | None = None
|
||||
output_path: str | None = None
|
||||
queued_at: float | None = None
|
||||
# The range/sections last applied via trim_job(), so the Edit video dialog
|
||||
# can reopen showing what actually produced the current output_path instead
|
||||
# of resetting to neutral. Reset to these defaults by restore_job_video().
|
||||
edit_start_seconds: float = 0.0
|
||||
edit_end_seconds: float | None = None
|
||||
#: Ordered, contiguous razor-cut sections, each a {end_seconds, brightness,
|
||||
#: contrast, saturation} dict; see trim.py's DEFAULT_SEGMENTS. Empty means
|
||||
#: never edited (or restored) -- one neutral section covering everything.
|
||||
edit_segments: list[dict[str, Any]] = field(default_factory=list)
|
||||
log_file_path: str | None = None # Path to the job's log file
|
||||
num_inference_steps: int = 50
|
||||
num_frames: int = 81
|
||||
@@ -195,11 +216,15 @@ class Job:
|
||||
"references": self.references,
|
||||
"status": self.status.value,
|
||||
"created_at": self.created_at,
|
||||
"queued_at": self.queued_at,
|
||||
"started_at": self.started_at,
|
||||
"finished_at": self.finished_at,
|
||||
"error": self.error,
|
||||
"output_path": self.output_path,
|
||||
"log_file_path": self.log_file_path,
|
||||
"edit_start_seconds": self.edit_start_seconds,
|
||||
"edit_end_seconds": self.edit_end_seconds,
|
||||
"edit_segments": self.edit_segments,
|
||||
"num_inference_steps": self.num_inference_steps,
|
||||
"num_frames": self.num_frames,
|
||||
"height": self.height,
|
||||
@@ -278,8 +303,8 @@ def _job_log_path(output_dir: str, job_id: str) -> str:
|
||||
return os.path.join(output_dir, job_id, JOB_LOG_FILENAME)
|
||||
|
||||
|
||||
def _decode_references(value: Any) -> list[dict[str, Any]]:
|
||||
"""Reference lists round-trip through the DB as JSON text."""
|
||||
def _decode_json_list(value: Any) -> list[dict[str, Any]]:
|
||||
"""References and edit_segments both round-trip through the DB as JSON text."""
|
||||
if not value:
|
||||
return []
|
||||
if isinstance(value, list):
|
||||
@@ -287,7 +312,7 @@ def _decode_references(value: Any) -> list[dict[str, Any]]:
|
||||
try:
|
||||
decoded = json.loads(value)
|
||||
except (TypeError, ValueError):
|
||||
logger.warning("Could not decode stored references: %r", value)
|
||||
logger.warning("Could not decode stored JSON list: %r", value)
|
||||
return []
|
||||
return list(decoded) if isinstance(decoded, list) else []
|
||||
|
||||
@@ -320,6 +345,8 @@ class JobRunner:
|
||||
log_dir: str,
|
||||
database: Database,
|
||||
verbose: bool = False,
|
||||
max_concurrent_jobs: int = 1,
|
||||
upload_dir: str = "",
|
||||
):
|
||||
"""Initialize the job runner.
|
||||
|
||||
@@ -328,19 +355,38 @@ class JobRunner:
|
||||
log_dir: Directory where job log files are saved
|
||||
verbose: Whether to print full tracebacks in error messages
|
||||
database: Optional SQLite database for job persistence
|
||||
max_concurrent_jobs: How many *queued* jobs may run at once. Jobs
|
||||
started directly with start_job() run immediately regardless
|
||||
(but count toward the limit, so the queue waits for them).
|
||||
upload_dir: Where extracted last frames are saved.
|
||||
"""
|
||||
self.output_dir = output_dir
|
||||
self.log_dir = log_dir
|
||||
self.verbose = verbose
|
||||
self._db = database
|
||||
self.max_concurrent_jobs = max(max_concurrent_jobs, 1)
|
||||
self.upload_dir = upload_dir or os.path.join(output_dir, "uploads")
|
||||
|
||||
self._jobs: dict[str, Job] = {}
|
||||
self._jobs_lock = threading.Lock()
|
||||
# Serializes queue decisions (enqueue / dequeue / dispatch) so two
|
||||
# finishing jobs can't both take the same free slot.
|
||||
self._queue_lock = threading.RLock()
|
||||
self._last_queued_at = 0.0
|
||||
self._load_jobs()
|
||||
|
||||
# Cache of loaded generators keyed by model config so that we only pay
|
||||
# the model-loading cost once per model configuration.
|
||||
self._generators: dict[tuple, Any] = {}
|
||||
# Cached generators are keyed by (owning thread, config). The model's GPU
|
||||
# worker processes are killed (PR_SET_PDEATHSIG) when the *thread* that
|
||||
# started them exits, so a generator is only safe for that thread to reuse
|
||||
# -- which is why inference jobs run on long-lived worker threads (below)
|
||||
# rather than a new thread each.
|
||||
self._generator_threads: dict[tuple, threading.Thread] = {}
|
||||
# Inference jobs are handed to these threads through one shared queue.
|
||||
self._work: queue.Queue[Job | None] = queue.Queue()
|
||||
self._workers: list[threading.Thread] = []
|
||||
self._generators_lock = threading.Lock()
|
||||
|
||||
# Shared Manager for log queues (avoids spawning a new process per job)
|
||||
@@ -351,6 +397,14 @@ class JobRunner:
|
||||
os.makedirs(self.output_dir, exist_ok=True)
|
||||
os.makedirs(self.log_dir, exist_ok=True)
|
||||
|
||||
for i in range(self.max_concurrent_jobs):
|
||||
worker = threading.Thread(target=self._worker_loop, name=f"inference-worker-{i}", daemon=True)
|
||||
worker.start()
|
||||
self._workers.append(worker)
|
||||
|
||||
# Jobs that were queued when the server last stopped carry on.
|
||||
self._dispatch()
|
||||
|
||||
def _load_logs(self, job: Job) -> None:
|
||||
"""Populate job's log buffer from its log file if it exists."""
|
||||
path = job.log_file_path
|
||||
@@ -399,7 +453,7 @@ class JobRunner:
|
||||
job_type=row.get("job_type", "inference"),
|
||||
image_path=row.get("image_path", "") or "",
|
||||
last_image_path=row.get("last_image_path", "") or "",
|
||||
references=_decode_references(row.get("references")),
|
||||
references=_decode_json_list(row.get("references")),
|
||||
data_path=row.get("data_path", "") or "",
|
||||
max_train_steps=row.get("max_train_steps", 1000),
|
||||
train_batch_size=row.get("train_batch_size", 1),
|
||||
@@ -416,11 +470,16 @@ class JobRunner:
|
||||
fake_score_model_path=row.get("fake_score_model_path", "") or "",
|
||||
status=JobStatus(status),
|
||||
created_at=row["created_at"],
|
||||
queued_at=row.get("queued_at"),
|
||||
started_at=row.get("started_at"),
|
||||
finished_at=row.get("finished_at"),
|
||||
error=row.get("error"),
|
||||
output_path=row.get("output_path"),
|
||||
log_file_path=row.get("log_file_path"),
|
||||
# `or <default>` would be wrong here: 0.0 is a real, intentional value, not just "unset".
|
||||
edit_start_seconds=(float(row["edit_start_seconds"]) if row.get("edit_start_seconds") is not None else 0.0),
|
||||
edit_end_seconds=(float(row["edit_end_seconds"]) if row.get("edit_end_seconds") is not None else None),
|
||||
edit_segments=_decode_json_list(row.get("edit_segments")),
|
||||
num_inference_steps=row.get("num_inference_steps", 50),
|
||||
num_frames=row.get("num_frames", 81),
|
||||
height=row.get("height", 480),
|
||||
@@ -456,6 +515,7 @@ class JobRunner:
|
||||
self._db.update_job(
|
||||
job.id, {
|
||||
"status": job.status.value,
|
||||
"queued_at": job.queued_at,
|
||||
"started_at": job.started_at,
|
||||
"finished_at": job.finished_at,
|
||||
"error": job.error,
|
||||
@@ -466,7 +526,9 @@ class JobRunner:
|
||||
logger.warning("Failed to persist job %s: %s", job.id, exc)
|
||||
|
||||
def _shutdown(self) -> None:
|
||||
"""Shutdown the shared multiprocessing manager on exit."""
|
||||
"""Stop the inference workers and the shared multiprocessing manager on exit."""
|
||||
for _ in self._workers:
|
||||
self._work.put(None)
|
||||
try:
|
||||
self._mp_manager.shutdown()
|
||||
except Exception as exc:
|
||||
@@ -610,6 +672,9 @@ class JobRunner:
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to delete job %s from database: %s", job_id, exc)
|
||||
logger.info("Deleted job %s", job.id)
|
||||
# Anything queued behind this job's last frame can no longer run, and a
|
||||
# freed queue slot may let the next job start.
|
||||
self._dispatch()
|
||||
return True
|
||||
|
||||
CONFIG_FIELDS: tuple[str, ...] = (
|
||||
@@ -688,24 +753,30 @@ class JobRunner:
|
||||
for field_name, value in updates.items():
|
||||
setattr(job, field_name, value)
|
||||
self._save_job(job)
|
||||
try:
|
||||
self._db.update_job_config(job.id, updates)
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to persist edits to job %s: %s", job.id, exc)
|
||||
return job
|
||||
|
||||
def start_job(self, job_id: str) -> Job:
|
||||
"""Start (or restart) a pending / stopped / failed job.
|
||||
|
||||
Raises:
|
||||
ValueError: If job not found or cannot be started
|
||||
"""
|
||||
#: Statuses a job can be started or queued from.
|
||||
STARTABLE_STATUSES = (JobStatus.PENDING, JobStatus.FAILED, JobStatus.STOPPED)
|
||||
|
||||
def _startable_job(self, job_id: str) -> Job:
|
||||
with self._jobs_lock:
|
||||
job = self._jobs.get(job_id)
|
||||
if job is None:
|
||||
raise ValueError(f"Job {job_id} not found")
|
||||
if job.status == JobStatus.RUNNING:
|
||||
raise ValueError("Job is already running")
|
||||
if job.status == JobStatus.QUEUED:
|
||||
raise ValueError("Job is already queued")
|
||||
if job.status == JobStatus.COMPLETED:
|
||||
raise ValueError("Job already completed. Delete and re-create to run again.")
|
||||
# Reset state for re-run
|
||||
job.status = JobStatus.PENDING
|
||||
return job
|
||||
|
||||
def _reset_for_run(self, job: Job) -> None:
|
||||
"""Clear the previous attempt's state so the job starts clean."""
|
||||
job.error = None
|
||||
job.output_path = None
|
||||
job.log_file_path = None # Will be set when job starts
|
||||
@@ -716,26 +787,189 @@ class JobRunner:
|
||||
job.log_file_handler = None # Reset file handler
|
||||
job._process = None # Reset subprocess handle
|
||||
|
||||
# Wrap _run_job in an additional safety layer to catch any exceptions
|
||||
# that might escape (though they shouldn't with our comprehensive handling)
|
||||
def safe_run_job(job: Job):
|
||||
"""Wrapper to ensure _run_job never raises an unhandled exception."""
|
||||
try:
|
||||
self._run_job(job)
|
||||
except BaseException as exc:
|
||||
# This should never happen, but if it does, we catch it here
|
||||
logger.critical("Unhandled exception escaped from _run_job for job %s: %s", job.id, exc, exc_info=True)
|
||||
with contextlib.suppress(Exception):
|
||||
job.status = JobStatus.FAILED
|
||||
job.error = (f"Unhandled exception: {type(exc).__name__}: {str(exc)}")
|
||||
job.finished_at = time.time()
|
||||
def _launch(self, job: Job) -> None:
|
||||
"""Start ``job``; when it ends, look for more queued work.
|
||||
|
||||
thread = threading.Thread(target=safe_run_job, args=(job, ), daemon=True)
|
||||
Inference jobs go to a long-lived worker thread so the loaded model
|
||||
survives from one job to the next. Training jobs only babysit a
|
||||
subprocess, so they get a thread of their own.
|
||||
"""
|
||||
# Marked running now, not when the job gets going, so the queue counts it
|
||||
# immediately and can't start one job too many.
|
||||
job.status = JobStatus.RUNNING
|
||||
job.started_at = time.time()
|
||||
self._save_job(job)
|
||||
|
||||
if job.job_type == "inference":
|
||||
self._work.put(job)
|
||||
logger.info("Sent job %s to an inference worker", job.id)
|
||||
return
|
||||
|
||||
def run_training_job(job: Job):
|
||||
self._run_job_safely(job)
|
||||
self._dispatch_after_exit(threading.current_thread())
|
||||
|
||||
thread = threading.Thread(target=run_training_job, args=(job, ), daemon=True)
|
||||
job._thread = thread
|
||||
thread.start()
|
||||
logger.info("Started job %s", job.id)
|
||||
|
||||
def _run_job_safely(self, job: Job) -> None:
|
||||
"""Run a job, turning anything that escapes into a failed job."""
|
||||
try:
|
||||
self._run_job(job)
|
||||
except BaseException as exc:
|
||||
# This should never happen, but if it does, we catch it here
|
||||
logger.critical("Unhandled exception escaped from _run_job for job %s: %s", job.id, exc, exc_info=True)
|
||||
with contextlib.suppress(Exception):
|
||||
job.status = JobStatus.FAILED
|
||||
job.error = (f"Unhandled exception: {type(exc).__name__}: {str(exc)}")
|
||||
job.finished_at = time.time()
|
||||
self._save_job(job)
|
||||
|
||||
def _worker_loop(self) -> None:
|
||||
"""Run inference jobs one after another on this thread, for the life of the server.
|
||||
|
||||
The thread never exits between jobs. That matters: it started the model's
|
||||
GPU processes, which are killed when it exits, so staying alive is what
|
||||
lets the next job find the model already loaded.
|
||||
"""
|
||||
while True:
|
||||
job = self._work.get()
|
||||
if job is None:
|
||||
return
|
||||
self._run_job_safely(job)
|
||||
# The slot is free; dependents may now be runnable (or impossible).
|
||||
with contextlib.suppress(Exception):
|
||||
self._dispatch()
|
||||
|
||||
def start_job(self, job_id: str) -> Job:
|
||||
"""Start (or restart) a pending / stopped / failed job right now.
|
||||
|
||||
Refused if every inference worker is already busy; use enqueue_job() to
|
||||
wait for a free one. A job that takes another job's last frame can only
|
||||
start once that job has completed.
|
||||
|
||||
Raises:
|
||||
ValueError: If job not found or cannot be started
|
||||
"""
|
||||
with self._queue_lock:
|
||||
job = self._startable_job(job_id)
|
||||
# Fail here, before touching the job, rather than as a failed run.
|
||||
self._check_dependencies_finished(job)
|
||||
if job.job_type == "inference":
|
||||
busy = self._running_inference_jobs()
|
||||
if busy >= self.max_concurrent_jobs:
|
||||
raise ValueError(f"{busy} inference job(s) already running and the model workers are busy. "
|
||||
"Queue this job to run next.")
|
||||
self._reset_for_run(job)
|
||||
job.queued_at = None
|
||||
self._launch(job)
|
||||
return job
|
||||
|
||||
def _running_inference_jobs(self) -> int:
|
||||
with self._jobs_lock:
|
||||
return sum(1 for j in self._jobs.values() if j.job_type == "inference" and j.status == JobStatus.RUNNING)
|
||||
|
||||
def _check_dependencies_finished(self, job: Job) -> None:
|
||||
for dep_id in dependencies(job.references):
|
||||
dep = self.get_job(dep_id)
|
||||
if dep is None:
|
||||
raise ValueError("This job starts from the ending of a job that no longer exists.")
|
||||
if dep.status != JobStatus.COMPLETED:
|
||||
label = f"'{dep.name}'" if dep.name else dep.id
|
||||
raise ValueError(f"This job starts from the ending of {label}, which hasn't finished "
|
||||
f"(it is {dep.status.value}). Queue it to run after that job.")
|
||||
|
||||
# -- queue ---------------------------------------------------------------
|
||||
|
||||
def _next_queued_at(self) -> float:
|
||||
"""A strictly increasing timestamp, so jobs queued together keep their order."""
|
||||
self._last_queued_at = max(time.time(), self._last_queued_at + 1e-6)
|
||||
return self._last_queued_at
|
||||
|
||||
def enqueue_jobs(self, job_ids: list[str]) -> list[Job]:
|
||||
"""Queue jobs, in the given order, to run as slots free up.
|
||||
|
||||
All-or-nothing: if any job can't be queued, none are.
|
||||
|
||||
Raises:
|
||||
ValueError: If a job is missing, not startable, or waits on itself.
|
||||
"""
|
||||
if len(set(job_ids)) != len(job_ids):
|
||||
raise ValueError("A job was listed more than once.")
|
||||
with self._queue_lock:
|
||||
jobs = [self._startable_job(job_id) for job_id in job_ids]
|
||||
with self._jobs_lock:
|
||||
snapshot = dict(self._jobs)
|
||||
for job in jobs:
|
||||
loop = circular_dependency(snapshot, job.id)
|
||||
if loop:
|
||||
raise ValueError("Jobs would wait on each other's last frames in a loop: " + " -> ".join(loop))
|
||||
for job in jobs:
|
||||
self._reset_for_run(job)
|
||||
job.status = JobStatus.QUEUED
|
||||
job.queued_at = self._next_queued_at()
|
||||
self._save_job(job)
|
||||
logger.info("Queued job %s", job.id)
|
||||
self._dispatch()
|
||||
return jobs
|
||||
|
||||
def enqueue_job(self, job_id: str) -> Job:
|
||||
return self.enqueue_jobs([job_id])[0]
|
||||
|
||||
def dequeue_job(self, job_id: str) -> Job:
|
||||
"""Take a queued job out of the queue, back to pending.
|
||||
|
||||
Raises:
|
||||
ValueError: If the job is missing or isn't queued.
|
||||
"""
|
||||
with self._queue_lock:
|
||||
job = self.get_job(job_id)
|
||||
if job is None:
|
||||
raise ValueError(f"Job {job_id} not found")
|
||||
if job.status != JobStatus.QUEUED:
|
||||
raise ValueError(f"Job is not queued (status={job.status.value})")
|
||||
job.status = JobStatus.PENDING
|
||||
job.queued_at = None
|
||||
self._save_job(job)
|
||||
logger.info("Dequeued job %s", job.id)
|
||||
# Anything queued behind this one's frame keeps waiting; a slot it held
|
||||
# in line may now go to another job.
|
||||
self._dispatch()
|
||||
return job
|
||||
|
||||
def _dispatch_after_exit(self, finished: threading.Thread) -> None:
|
||||
"""Wait for a job's thread to end, then start whatever is next in the queue."""
|
||||
finished.join()
|
||||
with contextlib.suppress(Exception):
|
||||
self._dispatch()
|
||||
|
||||
def _dispatch(self) -> None:
|
||||
"""Launch queued jobs that are ready, and fail those that can never be."""
|
||||
with self._queue_lock:
|
||||
with self._jobs_lock:
|
||||
snapshot = dict(self._jobs)
|
||||
plan = plan_dispatch(snapshot, self.max_concurrent_jobs)
|
||||
for job_id, reason in plan.fail.items():
|
||||
job = snapshot[job_id]
|
||||
job.status = JobStatus.FAILED
|
||||
job.error = reason
|
||||
job.finished_at = time.time()
|
||||
self._save_job(job)
|
||||
logger.warning("Queued job %s can't run: %s", job_id, reason)
|
||||
for job_id in plan.start:
|
||||
job = snapshot[job_id]
|
||||
job.queued_at = None
|
||||
try:
|
||||
self._launch(job)
|
||||
except Exception as exc: # e.g. the OS refused a thread
|
||||
logger.exception("Could not launch queued job %s", job_id)
|
||||
job.status = JobStatus.FAILED
|
||||
job.error = f"Could not start: {exc}"
|
||||
job.finished_at = time.time()
|
||||
self._save_job(job)
|
||||
|
||||
def stop_job(self, job_id: str) -> Job:
|
||||
"""Request a running job to stop.
|
||||
|
||||
@@ -749,6 +983,8 @@ class JobRunner:
|
||||
job = self._jobs.get(job_id)
|
||||
if job is None:
|
||||
raise ValueError(f"Job {job_id} not found")
|
||||
if job.status == JobStatus.QUEUED:
|
||||
return self.dequeue_job(job_id)
|
||||
if job.status != JobStatus.RUNNING:
|
||||
raise ValueError(f"Job is not running (status={job.status.value})")
|
||||
|
||||
@@ -759,6 +995,91 @@ class JobRunner:
|
||||
logger.info("Stop requested for job %s", job.id)
|
||||
return job
|
||||
|
||||
# -- trimming --------------------------------------------------------------
|
||||
|
||||
def _set_output(
|
||||
self,
|
||||
job: Job,
|
||||
output_path: str,
|
||||
num_frames: int,
|
||||
*,
|
||||
edit_start_seconds: float = 0.0,
|
||||
edit_end_seconds: float | None = None,
|
||||
edit_segments: list[dict[str, Any]] | None = None,
|
||||
) -> None:
|
||||
"""Point a completed job at a different output file, e.g. after trimming.
|
||||
|
||||
The edit_* keyword args record what produced that file (left at their
|
||||
neutral/full-range defaults by restore_job_video()), so the Edit video
|
||||
dialog can reopen showing the edit actually in effect instead of always
|
||||
resetting to neutral -- nothing else needs these; other jobs' last-frame/
|
||||
last-clip references and a merge only ever read output_path.
|
||||
|
||||
Bypasses update_job_config()'s pending/failed/stopped gate on purpose:
|
||||
a completed job's *generation* config stays fixed, but which cut of its
|
||||
own video is showing is not a generation setting.
|
||||
"""
|
||||
edit_segments = edit_segments or []
|
||||
job.output_path = output_path
|
||||
job.num_frames = num_frames
|
||||
job.edit_start_seconds = edit_start_seconds
|
||||
job.edit_end_seconds = edit_end_seconds
|
||||
job.edit_segments = edit_segments
|
||||
self._save_job(job)
|
||||
try:
|
||||
self._db.update_job_config(
|
||||
job.id, {
|
||||
"num_frames": num_frames,
|
||||
"edit_start_seconds": edit_start_seconds,
|
||||
"edit_end_seconds": edit_end_seconds,
|
||||
"edit_segments": edit_segments,
|
||||
})
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to persist edit state for job %s: %s", job.id, exc)
|
||||
|
||||
def trim_job(
|
||||
self,
|
||||
job_id: str,
|
||||
start_seconds: float = 0.0,
|
||||
end_seconds: float | None = None,
|
||||
segments: list[dict[str, Any]] | None = None,
|
||||
) -> Job:
|
||||
"""Cut a completed job's video to [start_seconds, end_seconds) and grade it section by section, in place.
|
||||
|
||||
The untouched original is kept, so this can be called again with a
|
||||
different range and/or sections -- it always renders from that original
|
||||
with the full set of parameters given, never compounding quality loss
|
||||
or discarding one edit by changing the other -- or undone with
|
||||
restore_job_video().
|
||||
"""
|
||||
with self._jobs_lock:
|
||||
job = self._jobs.get(job_id)
|
||||
if job is None:
|
||||
raise ValueError(f"Job {job_id} not found")
|
||||
path, kept = trim_video(job, start_seconds, end_seconds, segments)
|
||||
self._set_output(
|
||||
job,
|
||||
path,
|
||||
kept,
|
||||
edit_start_seconds=start_seconds,
|
||||
edit_end_seconds=end_seconds,
|
||||
edit_segments=segments,
|
||||
)
|
||||
logger.info("Edited job %s: %.2f-%s s, %d section(s) (%d frames)", job.id, start_seconds,
|
||||
f"{end_seconds:.2f}" if end_seconds is not None else "end", len(segments or [1]), kept)
|
||||
return job
|
||||
|
||||
def restore_job_video(self, job_id: str) -> Job:
|
||||
"""Undo every trim on a job's video, back to what it originally generated."""
|
||||
with self._jobs_lock:
|
||||
job = self._jobs.get(job_id)
|
||||
if job is None:
|
||||
raise ValueError(f"Job {job_id} not found")
|
||||
path, kept = restore_original(job)
|
||||
self._set_output(job, path, kept)
|
||||
logger.info("Restored job %s to its original video", job.id)
|
||||
return job
|
||||
|
||||
def get_job_logs(self, job_id: str, after: int = 0) -> dict[str, Any]:
|
||||
"""Return log lines for a job.
|
||||
|
||||
@@ -786,6 +1107,22 @@ class JobRunner:
|
||||
"phase": job._log_buf.phase,
|
||||
}
|
||||
|
||||
def _discard_generators(self) -> None:
|
||||
"""Unload this thread's models so the next job on it starts from a clean load."""
|
||||
with self._generators_lock:
|
||||
for key in [k for k in self._generators if k[0] == threading.current_thread().ident]:
|
||||
self._drop_generator(key)
|
||||
|
||||
def _drop_generator(self, key: tuple) -> None:
|
||||
"""Forget a cached generator and shut it down. Call with ``_generators_lock`` held."""
|
||||
gen = self._generators.pop(key, None)
|
||||
self._generator_threads.pop(key, None)
|
||||
if gen is not None:
|
||||
try:
|
||||
gen.shutdown()
|
||||
except Exception:
|
||||
logger.debug("Shutdown of a cached generator failed", exc_info=True)
|
||||
|
||||
def _get_or_create_generator(
|
||||
self,
|
||||
model_id: str,
|
||||
@@ -823,23 +1160,27 @@ class JobRunner:
|
||||
sp_size,
|
||||
)
|
||||
|
||||
# Generators are cached by model_id and configuration parameters
|
||||
# Generators are cached per thread and configuration. Only the thread that
|
||||
# built one may reuse it: its GPU processes die with that thread.
|
||||
thread = threading.current_thread()
|
||||
entry_key = (thread.ident, cache_key)
|
||||
with self._generators_lock:
|
||||
cached = self._generators.get(cache_key)
|
||||
# Generators whose thread has exited have dead workers already.
|
||||
for key, owner in list(self._generator_threads.items()):
|
||||
if not owner.is_alive():
|
||||
self._drop_generator(key)
|
||||
cached = self._generators.get(entry_key)
|
||||
if cached is not None:
|
||||
if _generator_is_alive(cached):
|
||||
return cached
|
||||
# Workers can exit while a generator sits idle in the cache;
|
||||
# reusing it fails every later job with the same config.
|
||||
logger.warning(
|
||||
"Cached generator for %s has dead workers; reloading.",
|
||||
model_id,
|
||||
)
|
||||
self._generators.pop(cache_key, None)
|
||||
try:
|
||||
cached.shutdown()
|
||||
except Exception:
|
||||
logger.debug("Shutdown of the dead generator failed", exc_info=True)
|
||||
# Workers can exit while a generator sits idle in the cache.
|
||||
logger.warning("Cached generator for %s has dead workers; reloading.", model_id)
|
||||
self._drop_generator(entry_key)
|
||||
# A worker keeps one model resident. A different configuration replaces
|
||||
# it, freeing its GPU memory first, rather than loading beside it.
|
||||
for key in [k for k in self._generators if k[0] == thread.ident]:
|
||||
logger.info("Unloading the previous model to load %s.", model_id)
|
||||
self._drop_generator(key)
|
||||
|
||||
# Import lazily so starting the server is fast even without a GPU.
|
||||
from fastvideo import VideoGenerator
|
||||
@@ -883,11 +1224,8 @@ class JobRunner:
|
||||
)
|
||||
|
||||
with self._generators_lock:
|
||||
if cache_key not in self._generators:
|
||||
self._generators[cache_key] = gen
|
||||
else: # Another thread may have created it while we were loading.
|
||||
gen.shutdown()
|
||||
gen = self._generators[cache_key]
|
||||
self._generators[entry_key] = gen
|
||||
self._generator_threads[entry_key] = thread
|
||||
return gen
|
||||
|
||||
def _run_job(self, job: Job):
|
||||
@@ -1064,6 +1402,15 @@ class JobRunner:
|
||||
logger.warning("Job stopped before execution started")
|
||||
return
|
||||
|
||||
# Frames and trailing clips borrowed from other jobs are extracted now,
|
||||
# when they exist. Done before loading the model so a missing one fails fast.
|
||||
resolved_references = resolve_references(
|
||||
job.references,
|
||||
self.get_job,
|
||||
last_frame=lambda source: last_frame_for_job(source, self.upload_dir),
|
||||
last_clip=lambda source: last_clip_for_job(source, self.upload_dir),
|
||||
)
|
||||
|
||||
buf.phase = "loading model"
|
||||
logger.info("Loading model...")
|
||||
|
||||
@@ -1121,7 +1468,7 @@ class JobRunner:
|
||||
if job.image_path:
|
||||
gen_kwargs["image_path"] = job.image_path
|
||||
if job.references:
|
||||
gen_kwargs["references"] = _build_h3_references(job.references)
|
||||
gen_kwargs["references"] = _build_h3_references(resolved_references)
|
||||
if job.last_image_path:
|
||||
# _prepare_fl2va requires a PIL image, not a path.
|
||||
from PIL import Image as _PILImage
|
||||
@@ -1148,6 +1495,7 @@ class JobRunner:
|
||||
if job._stop_event.is_set():
|
||||
job.status = JobStatus.STOPPED
|
||||
logger.warning("Job was stopped during execution")
|
||||
self._discard_generators()
|
||||
else:
|
||||
job.status = JobStatus.COMPLETED
|
||||
buf.progress = 100.0
|
||||
@@ -1156,9 +1504,19 @@ class JobRunner:
|
||||
self._save_job(job)
|
||||
buf.phase = "done"
|
||||
|
||||
except UnresolvedReference as exception:
|
||||
logger.error("Job %s can't get its reference frame: %s", job.id, exception)
|
||||
job.status = JobStatus.FAILED
|
||||
job.error = str(exception)
|
||||
job.finished_at = time.time()
|
||||
self._save_job(job)
|
||||
buf.phase = "failed"
|
||||
|
||||
except Exception as exception:
|
||||
error_msg = str(exception)
|
||||
logger.exception("Critical error in job thread: %s", error_msg)
|
||||
# The model may be mid-request. Reload rather than trust it for the next job.
|
||||
self._discard_generators()
|
||||
job.status = JobStatus.FAILED
|
||||
job.error = f"Critical error ({type(exception).__name__}): {error_msg}"
|
||||
job.finished_at = time.time()
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Join a scene's clips, in order, into one video.
|
||||
|
||||
Kept apart from ``server.py`` because it needs nothing from FastVideo -- only
|
||||
PyAV (to read each clip's format) and an ``ffmpeg`` binary -- so it can be
|
||||
exercised without a GPU.
|
||||
|
||||
Clips from one scene share codec, size and frame rate, so they are joined by
|
||||
copying the streams rather than re-encoding: it takes about a second, loses no
|
||||
quality, and keeps picture and sound in step. Each clip is cut off at the end of
|
||||
its last video frame, because the audio track runs a few milliseconds past it and
|
||||
would otherwise open a gap between clips.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
VIDEO_EXTENSIONS = {".mp4", ".mov", ".mkv", ".webm", ".avi"}
|
||||
MERGED_DIRNAME = "merged"
|
||||
_FILENAME_RE = re.compile(r"^[\w.-]+\.mp4$")
|
||||
|
||||
|
||||
class MergeError(Exception):
|
||||
"""A merge that can't be done, carrying the HTTP status to report."""
|
||||
|
||||
def __init__(self, status_code: int, detail: str):
|
||||
super().__init__(detail)
|
||||
self.status_code = status_code
|
||||
self.detail = detail
|
||||
|
||||
|
||||
def find_ffmpeg() -> str | None:
|
||||
"""An ``ffmpeg`` to run: $FASTVIDEO_FFMPEG_BIN, then PATH, then imageio-ffmpeg's bundled one."""
|
||||
configured = os.getenv("FASTVIDEO_FFMPEG_BIN")
|
||||
if configured and (shutil.which(configured) or os.path.isfile(configured)):
|
||||
return shutil.which(configured) or configured
|
||||
on_path = shutil.which("ffmpeg")
|
||||
if on_path:
|
||||
return on_path
|
||||
try:
|
||||
import imageio_ffmpeg
|
||||
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ClipInfo:
|
||||
path: str
|
||||
video_codec: str
|
||||
width: int
|
||||
height: int
|
||||
fps: float
|
||||
pix_fmt: str
|
||||
video_seconds: float
|
||||
audio: tuple[str, int, int] | None # codec, sample rate, channels
|
||||
|
||||
def describe(self) -> str:
|
||||
audio = f"{self.audio[0]} {self.audio[1]} Hz x{self.audio[2]}" if self.audio else "no audio"
|
||||
return f"{self.width}x{self.height} {self.fps:.2f} fps {self.video_codec}/{self.pix_fmt}, {audio}"
|
||||
|
||||
def signature(self) -> tuple[Any, ...]:
|
||||
return (self.video_codec, self.width, self.height, round(self.fps, 3), self.pix_fmt, self.audio)
|
||||
|
||||
|
||||
def probe_clip(path: str) -> ClipInfo:
|
||||
"""Read what ``check_compatible`` needs from a video file."""
|
||||
import av
|
||||
|
||||
try:
|
||||
with av.open(path) as container:
|
||||
video = next((s for s in container.streams if s.type == "video"), None)
|
||||
if video is None:
|
||||
raise MergeError(409, f"{os.path.basename(path)} has no video stream.")
|
||||
audio = next((s for s in container.streams if s.type == "audio"), None)
|
||||
fps = float(video.average_rate or video.guessed_rate or 0)
|
||||
if video.duration is not None:
|
||||
seconds = float(video.duration * video.time_base)
|
||||
elif container.duration is not None:
|
||||
seconds = container.duration / 1_000_000
|
||||
else:
|
||||
seconds = (video.frames or 0) / fps if fps else 0.0
|
||||
if seconds <= 0:
|
||||
raise MergeError(409, f"{os.path.basename(path)} has no playable video.")
|
||||
return ClipInfo(
|
||||
path=path,
|
||||
video_codec=video.codec_context.name,
|
||||
width=video.codec_context.width,
|
||||
height=video.codec_context.height,
|
||||
fps=fps,
|
||||
pix_fmt=video.codec_context.pix_fmt or "",
|
||||
video_seconds=seconds,
|
||||
audio=((audio.codec_context.name, audio.codec_context.sample_rate, audio.codec_context.channels)
|
||||
if audio else None),
|
||||
)
|
||||
except MergeError:
|
||||
raise
|
||||
except Exception as exc: # av.error.* on a corrupt or unreadable file
|
||||
raise MergeError(409, f"Could not read {os.path.basename(path)}: {exc}") from exc
|
||||
|
||||
|
||||
def check_compatible(infos: list[ClipInfo], labels: list[str]) -> None:
|
||||
"""Refuse clips that can't be joined by copying, naming the first that differs."""
|
||||
first = infos[0]
|
||||
for info, label in zip(infos[1:], labels[1:], strict=True):
|
||||
if info.signature() != first.signature():
|
||||
raise MergeError(
|
||||
409,
|
||||
f"{label} ({info.describe()}) doesn't match {labels[0]} ({first.describe()}). "
|
||||
"Clips need the same size, frame rate and audio to be joined.",
|
||||
)
|
||||
|
||||
|
||||
def _concat_list(infos: list[ClipInfo]) -> str:
|
||||
lines = ["ffconcat version 1.0"]
|
||||
for info in infos:
|
||||
escaped = info.path.replace("'", "'\\''")
|
||||
lines += [f"file '{escaped}'", f"outpoint {info.video_seconds:.6f}"]
|
||||
return "\n".join(lines) + "\n"
|
||||
|
||||
|
||||
def merge_clips(paths: list[str], out_path: str, labels: list[str] | None = None, ffmpeg: str | None = None) -> float:
|
||||
"""Join ``paths`` in order into ``out_path``. Returns the merged length in seconds."""
|
||||
if not paths:
|
||||
raise MergeError(400, "No clips to merge.")
|
||||
labels = labels or [os.path.basename(p) for p in paths]
|
||||
ffmpeg = ffmpeg or find_ffmpeg()
|
||||
if not ffmpeg:
|
||||
raise MergeError(503, "ffmpeg isn't available on the server. Install it or set FASTVIDEO_FFMPEG_BIN.")
|
||||
|
||||
infos = [probe_clip(p) for p in paths]
|
||||
check_compatible(infos, labels)
|
||||
|
||||
os.makedirs(os.path.dirname(out_path) or ".", exist_ok=True)
|
||||
tmp_out = f"{out_path}.tmp.mp4"
|
||||
list_fd, list_path = tempfile.mkstemp(suffix=".ffconcat", text=True)
|
||||
try:
|
||||
with os.fdopen(list_fd, "w", encoding="utf-8") as f:
|
||||
f.write(_concat_list(infos))
|
||||
result = subprocess.run(
|
||||
[ffmpeg, "-y", "-loglevel", "error", "-f", "concat", "-safe", "0", "-i", list_path, "-c", "copy",
|
||||
"-movflags", "+faststart", tmp_out],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=600,
|
||||
check=False,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
tail = " ".join(result.stderr.strip().splitlines()[-3:])
|
||||
raise MergeError(500, f"ffmpeg failed: {tail or 'unknown error'}")
|
||||
os.replace(tmp_out, out_path) # atomic, so a reader never sees a half-written file
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
raise MergeError(500, "ffmpeg took too long.") from exc
|
||||
finally:
|
||||
for leftover in (list_path, tmp_out):
|
||||
try:
|
||||
os.remove(leftover)
|
||||
except OSError:
|
||||
pass
|
||||
return sum(i.video_seconds for i in infos)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MergeResult:
|
||||
filename: str
|
||||
path: str
|
||||
clips: int
|
||||
seconds: float
|
||||
|
||||
|
||||
def _safe_stem(name: str) -> str:
|
||||
return re.sub(r"[^\w.-]+", "-", name).strip("-.")[:60] or "scene"
|
||||
|
||||
|
||||
def _label(job: Any) -> str:
|
||||
name = (getattr(job, "name", "") or "").strip()
|
||||
return f"'{name}'" if name else str(job.id)
|
||||
|
||||
|
||||
def merge_jobs(jobs: list[Any], out_dir: str, name: str = "") -> MergeResult:
|
||||
"""Join the finished videos of ``jobs`` (in the order given) into ``<out_dir>/merged``.
|
||||
|
||||
Every job must be completed with a video on disk; otherwise nothing is made,
|
||||
so a scene missing a clip can't be mistaken for the whole thing.
|
||||
"""
|
||||
if not jobs:
|
||||
raise MergeError(400, "No clips to merge.")
|
||||
paths: list[str] = []
|
||||
for job in jobs:
|
||||
status = getattr(job.status, "value", job.status)
|
||||
if status != "completed" or not job.output_path:
|
||||
raise MergeError(409, f"{_label(job)} hasn't finished ({status}), so the scene can't be merged yet.")
|
||||
if not os.path.isfile(job.output_path):
|
||||
raise MergeError(409, f"{_label(job)}'s video is missing from disk.")
|
||||
if Path(job.output_path).suffix.lower() not in VIDEO_EXTENSIONS:
|
||||
raise MergeError(409, f"{_label(job)}'s output isn't a video.")
|
||||
paths.append(job.output_path)
|
||||
|
||||
filename = f"{_safe_stem(name)}-{time.strftime('%Y%m%d-%H%M%S')}.mp4"
|
||||
merged_dir = os.path.join(out_dir, MERGED_DIRNAME)
|
||||
out_path = os.path.join(merged_dir, filename)
|
||||
seconds = merge_clips(paths, out_path, labels=[_label(j) for j in jobs])
|
||||
return MergeResult(filename=filename, path=out_path, clips=len(paths), seconds=seconds)
|
||||
|
||||
|
||||
def merged_path(out_dir: str, filename: str) -> str:
|
||||
"""Path of a merged video by filename, refusing anything that isn't one."""
|
||||
if not _FILENAME_RE.match(filename):
|
||||
raise MergeError(404, "Merged video not found.")
|
||||
path = os.path.join(out_dir, MERGED_DIRNAME, filename)
|
||||
if not os.path.isfile(path):
|
||||
raise MergeError(404, "Merged video not found.")
|
||||
return path
|
||||
@@ -4,8 +4,8 @@ In-memory mock of the FastVideo Studio API (``server.py``).
|
||||
|
||||
This mock implements the same ``/api`` routes and response shapes as the real
|
||||
FastAPI server but keeps everything in memory and never touches the FastVideo
|
||||
library, a GPU, or a database. It is meant to back the Playwright e2e suite so
|
||||
the Next.js frontend can be exercised end-to-end without a real backend.
|
||||
library or a GPU. It is meant to back the Playwright e2e suite so the Next.js
|
||||
frontend can be exercised end-to-end without a real backend.
|
||||
|
||||
Job lifecycle is simulated by recording a start timestamp and *computing* the
|
||||
status on read: a started job reports ``running`` for a few seconds and then
|
||||
@@ -13,14 +13,24 @@ flips to ``completed`` with an ``output_path``, so polling the job list / logs
|
||||
shows progression. Generated media is a tiny 1s ``testsrc`` MP4 built lazily
|
||||
with ffmpeg and cached.
|
||||
|
||||
Pass ``--data-dir`` to seed jobs/datasets/settings from a real server's
|
||||
persisted SQLite database instead of the built-in fake demo data, so you can
|
||||
browse real job history through the mock without a GPU. This is a read-only
|
||||
snapshot taken once at startup: nothing created or changed while the mock
|
||||
runs is ever written back to that database, so it's safe to point at the
|
||||
same ``--data-dir`` a real ``server.py`` is using.
|
||||
|
||||
Usage (from the ``apps/`` directory)::
|
||||
|
||||
PYTHONPATH=.. python -m fastvideo_studio.mock_server --port 8190
|
||||
PYTHONPATH=.. python -m fastvideo_studio.mock_server --port 8190 --data-dir ../outputs/ui_data
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import random
|
||||
import shutil
|
||||
@@ -29,22 +39,36 @@ import tempfile
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
from typing import Annotated, Any
|
||||
|
||||
from fastapi import FastAPI, File, HTTPException, UploadFile
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, PlainTextResponse
|
||||
|
||||
from fastvideo_studio.database import default_settings_dict
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, SettingsUpdate, UpdateCaptionRequest,
|
||||
from fastvideo_studio.database import Database, default_settings_dict
|
||||
from fastvideo_studio.job_queue import circular_dependency, dependencies, plan_dispatch
|
||||
from fastvideo_studio.merge import MergeError, merge_jobs, merged_path
|
||||
from fastvideo_studio.trim import BRIGHTNESS_RANGE, CONTRAST_RANGE, SATURATION_RANGE
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, GradeSegment, MergeSceneRequest,
|
||||
QueueJobsRequest, SettingsUpdate, TrimRequest, UpdateCaptionRequest,
|
||||
model_label)
|
||||
|
||||
logging.basicConfig(level=logging.INFO, format="%(asctime)s [%(levelname)s] %(name)s: %(message)s")
|
||||
logger = logging.getLogger("fastvideo.studio.mock")
|
||||
|
||||
# --- Config -----------------------------------------------------------------
|
||||
|
||||
# How long a started job stays "running" before it flips to "completed".
|
||||
COMPLETE_AFTER_SECONDS = 3.0
|
||||
FFMPEG_BIN = shutil.which(os.getenv("FASTVIDEO_FFMPEG_BIN", "ffmpeg"))
|
||||
|
||||
# Marks a job dict as a frozen row loaded from a real --data-dir snapshot
|
||||
# (see _load_snapshot_from_db) rather than something this process is
|
||||
# simulating. Stripped from API responses by _public_job.
|
||||
_FROM_SNAPSHOT_KEY = "_from_db_snapshot"
|
||||
|
||||
# A small catalogue of fake models keyed by workload type. Mirrors the real
|
||||
# server's {id, label} shape (label derived from the HF-style path).
|
||||
_MODELS_BY_WORKLOAD: dict[str, list[str]] = {
|
||||
@@ -54,6 +78,10 @@ _MODELS_BY_WORKLOAD: dict[str, list[str]] = {
|
||||
],
|
||||
"i2v": [
|
||||
"Wan-AI/Wan2.1-I2V-14B-480P-Diffusers",
|
||||
# Real registered id (fastvideo/registry.py) -- needed so the frontend's
|
||||
# `modelId.toLowerCase().includes('minimax-h3')` gate (speech-tag
|
||||
# console, references, last-frame upload) actually lights up in the mock.
|
||||
"MiniMaxAI/MiniMax-H3",
|
||||
],
|
||||
"t2i": [
|
||||
"black-forest-labs/FLUX.1-schnell",
|
||||
@@ -98,16 +126,76 @@ _mock_media_cache: dict[str, str] = {}
|
||||
_mock_media_lock = threading.Lock()
|
||||
|
||||
|
||||
def _testsrc_frame(width: int, height: int, index: int):
|
||||
"""A cheap, dependency-light stand-in for ffmpeg's `testsrc` pattern:
|
||||
a gradient that shifts over `index` so an encoded clip visibly animates."""
|
||||
import numpy as np
|
||||
|
||||
frame = np.zeros((height, width, 3), dtype=np.uint8)
|
||||
frame[:, :, 0] = np.linspace(0, 255, width, dtype=np.uint8)[None, :]
|
||||
frame[:, :, 1] = np.linspace(0, 255, height, dtype=np.uint8)[:, None]
|
||||
frame[:, :, 2] = (index * 10) % 256
|
||||
return frame
|
||||
|
||||
|
||||
def _build_mock_media_with_pyav(kind: str, path: str) -> None:
|
||||
"""ffmpeg isn't installed on every dev box; PyAV (a real FastVideo
|
||||
dependency already, used for reference-video decoding) can encode the
|
||||
same tiny clip directly via its bundled libav, no external binary."""
|
||||
from PIL import Image
|
||||
|
||||
if kind == "png":
|
||||
Image.fromarray(_testsrc_frame(320, 240, 0)).save(path)
|
||||
return
|
||||
|
||||
import av as pyav
|
||||
|
||||
# h264 is what every browser's <video> element actually plays; mpeg4
|
||||
# (MPEG-4 Part 2) is a PyAV-always-has-it fallback that no mainstream
|
||||
# browser decodes, so gallery/job previews would just fail to load.
|
||||
try:
|
||||
pyav.codec.Codec("h264", "w")
|
||||
codec_name = "h264"
|
||||
except Exception:
|
||||
logger.warning("PyAV has no h264 encoder available; falling back to mpeg4, which most "
|
||||
"browsers cannot play in <video>. Install ffmpeg for real preview playback.")
|
||||
codec_name = "mpeg4"
|
||||
|
||||
container = pyav.open(path, mode="w")
|
||||
try:
|
||||
stream = container.add_stream(codec_name, rate=24)
|
||||
stream.width = 320
|
||||
stream.height = 240
|
||||
stream.pix_fmt = "yuv420p"
|
||||
for i in range(24): # 1s at 24fps, matching the ffmpeg path's clip length
|
||||
frame = pyav.VideoFrame.from_ndarray(_testsrc_frame(320, 240, i), format="rgb24")
|
||||
for packet in stream.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in stream.encode():
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
|
||||
def _build_mock_media(kind: str) -> str:
|
||||
"""Build (once) a tiny testsrc clip of the given ``kind`` with ffmpeg; cache the path."""
|
||||
"""Build (once) a tiny testsrc-like clip of the given ``kind``; cache the path."""
|
||||
with _mock_media_lock:
|
||||
cached = _mock_media_cache.get(kind)
|
||||
if cached and os.path.isfile(cached):
|
||||
return cached
|
||||
if not FFMPEG_BIN:
|
||||
raise HTTPException(status_code=500, detail="ffmpeg is required to build mock media")
|
||||
fd, path = tempfile.mkstemp(prefix="fvstudio_mock_", suffix=f".{kind}")
|
||||
os.close(fd)
|
||||
if not FFMPEG_BIN:
|
||||
try:
|
||||
_build_mock_media_with_pyav(kind, path)
|
||||
except Exception as exc:
|
||||
with contextlib.suppress(OSError):
|
||||
os.remove(path)
|
||||
raise HTTPException(status_code=500, detail=f"PyAV fallback failed to build mock {kind}: {exc}") from exc
|
||||
if not os.path.isfile(path) or os.path.getsize(path) == 0:
|
||||
raise HTTPException(status_code=500, detail=f"PyAV fallback produced no mock {kind} bytes")
|
||||
_mock_media_cache[kind] = path
|
||||
return path
|
||||
if kind == "png":
|
||||
command = [
|
||||
FFMPEG_BIN,
|
||||
@@ -166,33 +254,34 @@ def _build_mock_media(kind: str) -> str:
|
||||
|
||||
|
||||
def _new_job_dict(req: CreateJobRequest) -> dict[str, Any]:
|
||||
"""Build a job dict (mirrors job_runner.Job.to_dict()) in the pending state."""
|
||||
"""Build a job dict (mirrors job_runner.Job.to_dict()) in the pending state.
|
||||
|
||||
Starts from req.model_dump() rather than re-listing CreateJobRequest's
|
||||
fields by hand -- the previous version silently dropped name,
|
||||
last_image_path, references, and every offload/tuning flag because it
|
||||
was never updated as CreateJobRequest grew those fields. Spreading the
|
||||
request means new fields flow through automatically.
|
||||
"""
|
||||
job_id = str(uuid.uuid4())
|
||||
return {
|
||||
**req.model_dump(),
|
||||
"id": job_id,
|
||||
"model_id": req.model_id,
|
||||
"prompt": req.prompt,
|
||||
"workload_type": req.workload_type or "t2v",
|
||||
"job_type": req.job_type or "inference",
|
||||
"image_path": req.image_path or "",
|
||||
"negative_prompt": req.negative_prompt or "",
|
||||
"data_path": req.data_path or "",
|
||||
"status": "pending",
|
||||
"created_at": time.time(),
|
||||
"queued_at": None,
|
||||
"started_at": None,
|
||||
"finished_at": None,
|
||||
"error": None,
|
||||
"output_path": None,
|
||||
"log_file_path": None,
|
||||
"num_inference_steps": req.num_inference_steps,
|
||||
"num_frames": req.num_frames,
|
||||
"height": req.height,
|
||||
"width": req.width,
|
||||
"guidance_scale": req.guidance_scale,
|
||||
"guidance_rescale": req.guidance_rescale,
|
||||
"fps": req.fps,
|
||||
"seed": req.seed,
|
||||
"negative_prompt": req.negative_prompt or "",
|
||||
"num_gpus": req.num_gpus,
|
||||
"data_path": req.data_path or "",
|
||||
"edit_start_seconds": 0.0,
|
||||
"edit_end_seconds": None,
|
||||
"edit_segments": [],
|
||||
"progress": 0.0,
|
||||
"progress_msg": "",
|
||||
"phase": "pending",
|
||||
@@ -204,8 +293,13 @@ def _advance_job(job: dict[str, Any]) -> None:
|
||||
|
||||
Status is *computed on read* from the recorded start timestamp, so polling
|
||||
the job list / logs naturally shows pending -> running -> completed.
|
||||
|
||||
Jobs loaded from a persisted DB snapshot (``_FROM_SNAPSHOT_KEY``) are a
|
||||
frozen historical record, not something this process is simulating — skip
|
||||
them so a real "pending" or "running" row doesn't silently start ticking
|
||||
toward a fake "completed" just because someone polled it.
|
||||
"""
|
||||
if job["status"] != "running" or not job.get("started_at"):
|
||||
if job.get(_FROM_SNAPSHOT_KEY) or job["status"] != "running" or not job.get("started_at"):
|
||||
return
|
||||
elapsed = time.time() - job["started_at"]
|
||||
if elapsed >= COMPLETE_AFTER_SECONDS:
|
||||
@@ -219,9 +313,9 @@ def _advance_job(job: dict[str, Any]) -> None:
|
||||
|
||||
|
||||
def _public_job(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Advance + return a copy safe to serialize."""
|
||||
"""Advance + return a copy safe to serialize (internal-only keys stripped)."""
|
||||
_advance_job(job)
|
||||
return dict(job)
|
||||
return {k: v for k, v in job.items() if not k.startswith("_")}
|
||||
|
||||
|
||||
_LOG_TAIL = [
|
||||
@@ -252,8 +346,8 @@ def _compute_logs(job: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Return JobLogs-shaped data, growing the visible lines as time passes."""
|
||||
seq = _log_sequence(job)
|
||||
status = job["status"]
|
||||
if status == "pending":
|
||||
return {"lines": [], "progress": 0.0, "progress_msg": "", "phase": "pending"}
|
||||
if status in ("pending", "queued"):
|
||||
return {"lines": [], "progress": 0.0, "progress_msg": "", "phase": status}
|
||||
if status == "completed":
|
||||
return {"lines": seq, "progress": 100.0, "progress_msg": "50/50 steps", "phase": "done"}
|
||||
if status in ("failed", "stopped"):
|
||||
@@ -346,6 +440,89 @@ def _seed() -> None:
|
||||
}
|
||||
|
||||
|
||||
# --- DB snapshot (--data-dir) ------------------------------------------------
|
||||
|
||||
|
||||
def _decode_references(value: Any) -> list[dict[str, Any]]:
|
||||
"""Reference lists round-trip through the DB as JSON text."""
|
||||
if not value:
|
||||
return []
|
||||
if isinstance(value, list):
|
||||
return list(value)
|
||||
try:
|
||||
decoded = json.loads(value)
|
||||
except (TypeError, ValueError):
|
||||
return []
|
||||
return list(decoded) if isinstance(decoded, list) else []
|
||||
|
||||
|
||||
def _job_from_snapshot(row: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Adapt a persisted job row into the mock's job dict shape.
|
||||
|
||||
Frozen as a read-only historical record (see _FROM_SNAPSHOT_KEY): the mock
|
||||
never actually ran this job, so it doesn't get to simulate progress for
|
||||
one. A row still marked "running" is orphaned -- the real JobRunner
|
||||
reclassifies those as failed on its own restart, so mirror that here.
|
||||
"""
|
||||
job = dict(row)
|
||||
status = job["status"]
|
||||
if status == "running":
|
||||
status = "failed"
|
||||
job["error"] = job.get("error") or "Server restarted (job was running)"
|
||||
job["finished_at"] = job.get("finished_at") or time.time()
|
||||
elif status == "queued":
|
||||
# The real server resumes its queue on restart; the mock never runs
|
||||
# snapshot jobs, so leave it as something the user can queue again.
|
||||
status = "pending"
|
||||
job["queued_at"] = None
|
||||
job["status"] = status
|
||||
job["references"] = _decode_references(row.get("references"))
|
||||
job["progress"] = 100.0 if status == "completed" else 0.0
|
||||
job["progress_msg"] = "50/50 steps" if status == "completed" else ""
|
||||
job["phase"] = "done" if status == "completed" else status
|
||||
job[_FROM_SNAPSHOT_KEY] = True
|
||||
return job
|
||||
|
||||
|
||||
def _load_snapshot_from_db(data_dir: Path) -> bool:
|
||||
"""Seed jobs/datasets/settings from a real server's persisted SQLite db.
|
||||
|
||||
Read-only: opens the db (Database.__init__ runs the same idempotent
|
||||
schema migrations a real server startup would) and copies rows into the
|
||||
mock's in-memory state; nothing the mock does afterward is written back.
|
||||
Returns False (leaving the caller to fall back to _seed()) if no db file
|
||||
is found at that path, rather than silently creating an empty one.
|
||||
"""
|
||||
db_path = data_dir / "fastvideo_ui.db"
|
||||
if not db_path.is_file():
|
||||
logger.warning("--data-dir %s has no fastvideo_ui.db; falling back to fake demo data.", data_dir)
|
||||
return False
|
||||
|
||||
db = Database(db_path)
|
||||
settings = db.get_settings()
|
||||
jobs = [_job_from_snapshot(row) for row in db.get_all_jobs()]
|
||||
datasets = db.get_all_datasets()
|
||||
|
||||
with _state_lock:
|
||||
_settings.update({k: v for k, v in settings.items() if v is not None})
|
||||
_jobs.clear()
|
||||
_jobs.update({j["id"]: j for j in jobs})
|
||||
_datasets.clear()
|
||||
_datasets.update({d["id"]: d for d in datasets})
|
||||
_dataset_files.clear()
|
||||
for d in datasets:
|
||||
captions = db.get_dataset_captions(d["id"])
|
||||
_dataset_files[d["id"]] = {"file_names": list(captions.keys()), "captions": captions}
|
||||
|
||||
logger.info(
|
||||
"Loaded read-only snapshot from %s: %d job(s), %d dataset(s).",
|
||||
db_path,
|
||||
len(jobs),
|
||||
len(datasets),
|
||||
)
|
||||
return True
|
||||
|
||||
|
||||
# --- App --------------------------------------------------------------------
|
||||
|
||||
app = FastAPI(title="FastVideo Studio Mock API", version="0.1.0")
|
||||
@@ -357,8 +534,6 @@ app.add_middleware(
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
_seed()
|
||||
|
||||
|
||||
@app.get("/api/__mock__")
|
||||
def mock_sentinel() -> dict[str, bool]:
|
||||
@@ -423,6 +598,32 @@ async def upload_image(file: Annotated[UploadFile, File()]) -> dict[str, str]:
|
||||
return {"path": f"/mock/uploads/{uuid.uuid4().hex}_{os.path.basename(name)}"}
|
||||
|
||||
|
||||
ALLOWED_REFERENCE_VIDEO_EXTENSIONS = {".mp4", ".mov", ".mkv", ".webm", ".avi"}
|
||||
ALLOWED_REFERENCE_AUDIO_EXTENSIONS = {".wav", ".mp3", ".flac", ".m4a", ".ogg"}
|
||||
ALLOWED_REFERENCE_IMAGE_EXTENSIONS = {".png", ".jpg", ".jpeg", ".webp", ".bmp"}
|
||||
|
||||
|
||||
@app.post("/api/upload-media")
|
||||
async def upload_media(file: Annotated[UploadFile, File()]) -> dict[str, str]:
|
||||
"""Ref2VA reference upload (image/video/audio) -- the real server's
|
||||
counterpart to upload_image, which only ever handles images."""
|
||||
name = file.filename or "reference"
|
||||
ext = os.path.splitext(name)[1].lower()
|
||||
allowed = ALLOWED_REFERENCE_IMAGE_EXTENSIONS | ALLOWED_REFERENCE_VIDEO_EXTENSIONS | ALLOWED_REFERENCE_AUDIO_EXTENSIONS
|
||||
if ext not in allowed:
|
||||
raise HTTPException(status_code=400, detail=f"Invalid file type. Allowed: {', '.join(sorted(allowed))}")
|
||||
if ext in ALLOWED_REFERENCE_VIDEO_EXTENSIONS:
|
||||
media_type = "video"
|
||||
elif ext in ALLOWED_REFERENCE_AUDIO_EXTENSIONS:
|
||||
media_type = "audio"
|
||||
else:
|
||||
media_type = "image"
|
||||
return {
|
||||
"path": f"/mock/uploads/{uuid.uuid4().hex}_{os.path.basename(name)}",
|
||||
"media_type": media_type,
|
||||
}
|
||||
|
||||
|
||||
@app.post("/api/upload-raw-dataset")
|
||||
async def upload_raw_dataset(files: Annotated[list[UploadFile], File()]) -> dict[str, Any]:
|
||||
video_exts = {".mp4", ".webm", ".avi", ".mov", ".mkv"}
|
||||
@@ -446,6 +647,7 @@ def list_jobs(job_type: str | None = None) -> list[dict[str, Any]]:
|
||||
# _public_job mutates the shared job dict via _advance_job, so it must run
|
||||
# under the lock (matching every other job route) to avoid racing start/stop.
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
jobs = list(_jobs.values())
|
||||
if job_type:
|
||||
jobs = [j for j in jobs if j.get("job_type") == job_type]
|
||||
@@ -456,6 +658,7 @@ def list_jobs(job_type: str | None = None) -> list[dict[str, Any]]:
|
||||
@app.get("/api/jobs/{job_id}")
|
||||
def get_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
@@ -466,15 +669,20 @@ def get_job(job_id: str) -> dict[str, Any]:
|
||||
def create_job(req: CreateJobRequest) -> dict[str, Any]:
|
||||
job = _new_job_dict(req)
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
_jobs[job["id"]] = job
|
||||
if _settings.get("autoStartJob"):
|
||||
_start(job)
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
def _start(job: dict[str, Any]) -> None:
|
||||
def _start(job: dict[str, Any], at: float | None = None) -> None:
|
||||
# Once the mock is actually driving this job, it's no longer a frozen
|
||||
# historical record -- let _advance_job simulate it like any other.
|
||||
job.pop(_FROM_SNAPSHOT_KEY, None)
|
||||
job["status"] = "running"
|
||||
job["started_at"] = time.time()
|
||||
job["queued_at"] = None
|
||||
job["started_at"] = at if at is not None else time.time()
|
||||
job["finished_at"] = None
|
||||
job["error"] = None
|
||||
job["output_path"] = None
|
||||
@@ -486,17 +694,281 @@ def _start(job: dict[str, Any]) -> None:
|
||||
job["phase"] = "starting"
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/start")
|
||||
def start_job(job_id: str) -> dict[str, Any]:
|
||||
class _QueueView:
|
||||
"""The scheduler's view of a mock job dict (it reads attributes, not keys)."""
|
||||
|
||||
def __init__(self, job: dict[str, Any]):
|
||||
self.id = job["id"]
|
||||
self.name = job.get("name", "")
|
||||
self.status = job["status"]
|
||||
self.references = job.get("references")
|
||||
self.queued_at = job.get("queued_at")
|
||||
|
||||
|
||||
def _drain_queue() -> None:
|
||||
"""Run the queue up to now. Call with ``_state_lock`` held.
|
||||
|
||||
Like everything else in the mock this is computed on read: each pass
|
||||
finishes jobs whose time is up, then starts whatever the queue allows.
|
||||
A job takes over the slot at the moment the previous job *finished* (not at
|
||||
the moment someone happened to poll), so a queue that has been sitting for
|
||||
a minute shows every clip already done rather than advancing one per poll.
|
||||
Same rules as the real runner: one at a time, oldest first, and a clip that
|
||||
takes another's last frame waits for it -- see job_queue.plan_dispatch.
|
||||
"""
|
||||
while True:
|
||||
for job in _jobs.values():
|
||||
_advance_job(job)
|
||||
plan = plan_dispatch({jid: _QueueView(j) for jid, j in _jobs.items()}, max_concurrent=1)
|
||||
now = time.time()
|
||||
for job_id, reason in plan.fail.items():
|
||||
job = _jobs[job_id]
|
||||
job["status"] = "failed"
|
||||
job["error"] = reason
|
||||
job["finished_at"] = now
|
||||
job["queued_at"] = None
|
||||
job["phase"] = "failed"
|
||||
if not plan.start:
|
||||
if plan.fail:
|
||||
continue # a failure can free a slot or fail more jobs
|
||||
return
|
||||
for job_id in plan.start:
|
||||
job = _jobs[job_id]
|
||||
finished = [j["finished_at"] for j in _jobs.values() if j.get("finished_at")]
|
||||
# The slot came free when the last job finished, if that was after the job was queued.
|
||||
slot_free = max([job.get("queued_at") or 0.0, *[f for f in finished if f <= now]])
|
||||
_start(job, at=min(now, slot_free))
|
||||
|
||||
|
||||
def _check_dependencies_finished(job: dict[str, Any]) -> None:
|
||||
for dep_id in dependencies(job.get("references")):
|
||||
dep = _jobs.get(dep_id)
|
||||
if dep is None:
|
||||
raise HTTPException(status_code=409,
|
||||
detail="This job starts from the ending of a job that no longer exists.")
|
||||
if dep["status"] != "completed":
|
||||
label = f"'{dep['name']}'" if dep.get("name") else dep["id"]
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=(f"This job starts from the ending of {label}, which hasn't finished "
|
||||
f"(it is {dep['status']}). Queue it to run after that job."),
|
||||
)
|
||||
|
||||
|
||||
# Mirrors JobRunner.EDITABLE_STATUSES: exactly the jobs that can still be started.
|
||||
_EDITABLE_STATUSES = ("pending", "failed", "stopped")
|
||||
|
||||
|
||||
@app.patch("/api/jobs/{job_id}")
|
||||
def update_job(job_id: str, updates: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Edit a job's configuration; only jobs that have not produced a result."""
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
if job["status"] == "running":
|
||||
raise HTTPException(status_code=409, detail="Job is already running")
|
||||
if job["status"] == "completed":
|
||||
raise HTTPException(status_code=409, detail="Job already completed. Delete and re-create to run again.")
|
||||
if job["status"] not in _EDITABLE_STATUSES:
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=(f"Job is {job['status']}; only {', '.join(_EDITABLE_STATUSES)} jobs can be edited. "
|
||||
"Duplicate it instead."),
|
||||
)
|
||||
unknown = set(updates) - set(CreateJobRequest.model_fields)
|
||||
if unknown:
|
||||
raise HTTPException(status_code=400, detail=f"Not editable: {', '.join(sorted(unknown))}")
|
||||
job.update(updates)
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/last-frame")
|
||||
def extract_last_frame(job_id: str) -> dict[str, str]:
|
||||
"""Stand-in for the real route: there is no video to read, so hand back a made-up path."""
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_advance_job(job)
|
||||
if job["status"] != "completed" or not job.get("output_path"):
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
return {"path": f"/mock/last_frames/last_frame_{job_id}.png", "media_type": "image"}
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/last-clip")
|
||||
def extract_last_clip(job_id: str) -> dict[str, str]:
|
||||
"""Stand-in for the real route: there is no video to read, so hand back a made-up path."""
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_advance_job(job)
|
||||
if job["status"] != "completed" or not job.get("output_path"):
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
return {"path": f"/mock/last_clips/last_clip_{job_id}.mp4", "media_type": "video"}
|
||||
|
||||
|
||||
#: Key holding a trimmed job's frame count before any trim, so it can be restored.
|
||||
_ORIGINAL_NUM_FRAMES_KEY = "_original_num_frames"
|
||||
|
||||
|
||||
def _trimmable_job(job_id: str) -> dict[str, Any]:
|
||||
"""Look up a completed job with an output, or raise. Call with _state_lock held."""
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_advance_job(job)
|
||||
if job["status"] != "completed" or not job.get("output_path"):
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
return job
|
||||
|
||||
|
||||
def _check_grade_range(name: str, value: float, bounds: tuple[float, float]) -> None:
|
||||
low, high = bounds
|
||||
if not (low <= value <= high):
|
||||
raise HTTPException(status_code=400, detail=f"{name} must be between {low:g} and {high:g}, got {value:g}.")
|
||||
|
||||
|
||||
def _check_segments(segments: list[GradeSegment]) -> None:
|
||||
"""Structural checks only (ordering, open-endedness, color bounds) -- the mock has no
|
||||
frames to count, so it doesn't reproduce trim.py's frame-rounding/coverage checks.
|
||||
|
||||
An empty list isn't an error here either, matching trim.py: it defaults to
|
||||
one neutral section covering everything (Pydantic's own default_factory
|
||||
already supplies that when the field is omitted, but an explicit "segments":
|
||||
[] reaches here as a real empty list).
|
||||
"""
|
||||
prev_end = 0.0
|
||||
for i, s in enumerate(segments):
|
||||
_check_grade_range("brightness", s.brightness, BRIGHTNESS_RANGE)
|
||||
_check_grade_range("contrast", s.contrast, CONTRAST_RANGE)
|
||||
_check_grade_range("saturation", s.saturation, SATURATION_RANGE)
|
||||
if s.end_seconds is None:
|
||||
if i != len(segments) - 1:
|
||||
raise HTTPException(status_code=400, detail="Only the last section can be open-ended.")
|
||||
else:
|
||||
if s.end_seconds <= prev_end:
|
||||
raise HTTPException(status_code=400, detail="Sections must be in increasing order.")
|
||||
prev_end = s.end_seconds
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/trim")
|
||||
def trim_job(job_id: str, req: TrimRequest) -> dict[str, Any]:
|
||||
"""Stand-in for the real route: no video to cut or grade, so just resize num_frames to match."""
|
||||
with _state_lock:
|
||||
job = _trimmable_job(job_id)
|
||||
if req.start_seconds < 0:
|
||||
raise HTTPException(status_code=400, detail="The start of the range can't be negative.")
|
||||
_check_segments(req.segments)
|
||||
fps = job.get("fps") or 24
|
||||
original_frames = job.setdefault(_ORIGINAL_NUM_FRAMES_KEY, job["num_frames"])
|
||||
duration = original_frames / fps
|
||||
end = duration if req.end_seconds is None else min(req.end_seconds, duration)
|
||||
if end <= req.start_seconds:
|
||||
raise HTTPException(status_code=400, detail="The end of the range must come after the start.")
|
||||
job["num_frames"] = max(1, round((end - req.start_seconds) * fps))
|
||||
# Recorded so the dialog can reopen showing what's actually applied,
|
||||
# like the real server's edit_* job fields.
|
||||
job["edit_start_seconds"] = req.start_seconds
|
||||
job["edit_end_seconds"] = req.end_seconds
|
||||
job["edit_segments"] = [s.model_dump() for s in req.segments]
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/restore-video")
|
||||
def restore_job_video(job_id: str) -> dict[str, Any]:
|
||||
"""Stand-in for the real route: undo the simulated edit by restoring num_frames."""
|
||||
with _state_lock:
|
||||
job = _trimmable_job(job_id)
|
||||
if _ORIGINAL_NUM_FRAMES_KEY not in job:
|
||||
raise HTTPException(status_code=404, detail="This job has not been edited.")
|
||||
job["num_frames"] = job.pop(_ORIGINAL_NUM_FRAMES_KEY)
|
||||
job["edit_start_seconds"] = 0.0
|
||||
job["edit_end_seconds"] = None
|
||||
job["edit_segments"] = []
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
def _check_startable(job: dict[str, Any]) -> None:
|
||||
if job["status"] == "running":
|
||||
raise HTTPException(status_code=409, detail="Job is already running")
|
||||
if job["status"] == "queued":
|
||||
raise HTTPException(status_code=409, detail="Job is already queued")
|
||||
if job["status"] == "completed":
|
||||
raise HTTPException(status_code=409, detail="Job already completed. Delete and re-create to run again.")
|
||||
|
||||
|
||||
def _enqueue(job_ids: list[str]) -> list[dict[str, Any]]:
|
||||
"""Queue jobs in order, all-or-nothing (mirrors JobRunner.enqueue_jobs). Lock held."""
|
||||
if len(set(job_ids)) != len(job_ids):
|
||||
raise HTTPException(status_code=409, detail="A job was listed more than once.")
|
||||
jobs = []
|
||||
for job_id in job_ids:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_check_startable(job)
|
||||
jobs.append(job)
|
||||
views = {jid: _QueueView(j) for jid, j in _jobs.items()}
|
||||
for job in jobs:
|
||||
loop = circular_dependency(views, job["id"])
|
||||
if loop:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail="Jobs would wait on each other's last frames in a loop: " + " -> ".join(loop),
|
||||
)
|
||||
for job in jobs:
|
||||
job.pop(_FROM_SNAPSHOT_KEY, None)
|
||||
job.update(status="queued", queued_at=time.time(), error=None, output_path=None, log_file_path=None,
|
||||
started_at=None, finished_at=None, progress=0.0, progress_msg="", phase="queued")
|
||||
# Strictly increasing, so jobs queued together keep their order.
|
||||
others = [j["queued_at"] for j in _jobs.values() if j.get("queued_at") and j is not job]
|
||||
job["queued_at"] = max([job["queued_at"], *[o + 1e-6 for o in others]])
|
||||
_drain_queue()
|
||||
return [_public_job(j) for j in jobs]
|
||||
|
||||
|
||||
@app.post("/api/jobs/queue")
|
||||
def queue_jobs(req: QueueJobsRequest) -> list[dict[str, Any]]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
return _enqueue(req.job_ids)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/queue")
|
||||
def queue_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
return _enqueue([job_id])[0]
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/dequeue")
|
||||
def dequeue_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
if job["status"] != "queued":
|
||||
raise HTTPException(status_code=409, detail=f"Job is not queued (status={job['status']})")
|
||||
job["status"] = "pending"
|
||||
job["queued_at"] = None
|
||||
_drain_queue()
|
||||
return _public_job(job)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/start")
|
||||
def start_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
_check_startable(job)
|
||||
_check_dependencies_finished(job)
|
||||
_start(job)
|
||||
return _public_job(job)
|
||||
|
||||
@@ -504,10 +976,16 @@ def start_job(job_id: str) -> dict[str, Any]:
|
||||
@app.post("/api/jobs/{job_id}/stop")
|
||||
def stop_job(job_id: str) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
_advance_job(job)
|
||||
if job["status"] == "queued":
|
||||
job["status"] = "pending"
|
||||
job["queued_at"] = None
|
||||
_drain_queue()
|
||||
return _public_job(job)
|
||||
if job["status"] != "running":
|
||||
raise HTTPException(status_code=409, detail=f"Job is not running (status={job['status']})")
|
||||
job["status"] = "stopped"
|
||||
@@ -519,14 +997,17 @@ def stop_job(job_id: str) -> dict[str, Any]:
|
||||
@app.delete("/api/jobs/{job_id}")
|
||||
def delete_job(job_id: str) -> dict[str, str]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
if _jobs.pop(job_id, None) is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
_drain_queue() # clips waiting on this one's last frame can no longer run
|
||||
return {"detail": f"Job {job_id} deleted"}
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/logs")
|
||||
def get_job_logs(job_id: str, after: int = 0) -> dict[str, Any]:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
@@ -545,6 +1026,7 @@ def get_job_logs(job_id: str, after: int = 0) -> dict[str, Any]:
|
||||
@app.get("/api/jobs/{job_id}/video")
|
||||
def get_video(job_id: str) -> FileResponse:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
@@ -552,6 +1034,12 @@ def get_video(job_id: str) -> FileResponse:
|
||||
output_path = job.get("output_path")
|
||||
if job["status"] != "completed" or not output_path:
|
||||
raise HTTPException(status_code=404, detail="No output available for this job")
|
||||
# A job loaded from a --data-dir snapshot may point at a real file this
|
||||
# process never wrote, only reads -- serve it if it's actually there,
|
||||
# otherwise fall back to the same synthetic clip every other job gets.
|
||||
if os.path.isfile(output_path):
|
||||
media_type = "video/mp4" if output_path.endswith(".mp4") else "image/png"
|
||||
return FileResponse(output_path, media_type=media_type, filename=os.path.basename(output_path))
|
||||
# Mirror the real server: image workloads (t2i) output a .png served as an
|
||||
# image; everything else is a video.
|
||||
if output_path.endswith(".png"):
|
||||
@@ -559,9 +1047,59 @@ def get_video(job_id: str) -> FileResponse:
|
||||
return FileResponse(_build_mock_media("mp4"), media_type="video/mp4", filename=f"job_{job_id}.mp4")
|
||||
|
||||
|
||||
_MERGED_DIR = tempfile.mkdtemp(prefix="fvstudio_merged_")
|
||||
|
||||
|
||||
@app.post("/api/scenes/merge")
|
||||
def merge_scene(req: MergeSceneRequest) -> dict[str, Any]:
|
||||
"""Real merge (same code as the real server) of whatever videos the jobs have.
|
||||
|
||||
Jobs whose video only exists in the mock get the stand-in clip, so the
|
||||
flow can be tried end to end without generating anything.
|
||||
"""
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
picked = []
|
||||
for job_id in req.job_ids:
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
picked.append(dict(job))
|
||||
all_real = all(j.get("output_path") and os.path.isfile(j["output_path"]) for j in picked)
|
||||
stand_in = None if all_real else _build_mock_media("mp4")
|
||||
jobs = [
|
||||
SimpleNamespace(
|
||||
id=j["id"],
|
||||
name=j.get("name", ""),
|
||||
status=j["status"],
|
||||
output_path=(j["output_path"] if all_real else (stand_in if j["status"] == "completed" else None)),
|
||||
) for j in picked
|
||||
]
|
||||
try:
|
||||
result = merge_jobs(jobs, _MERGED_DIR, req.name)
|
||||
except MergeError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return {
|
||||
"filename": result.filename,
|
||||
"url": f"/api/merged/{result.filename}",
|
||||
"clips": result.clips,
|
||||
"seconds": result.seconds,
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/merged/{filename}")
|
||||
def get_merged_video(filename: str) -> FileResponse:
|
||||
try:
|
||||
path = merged_path(_MERGED_DIR, filename)
|
||||
except MergeError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return FileResponse(path, media_type="video/mp4", filename=filename)
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/download_log")
|
||||
def download_log(job_id: str) -> PlainTextResponse:
|
||||
with _state_lock:
|
||||
_drain_queue()
|
||||
job = _jobs.get(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
@@ -651,8 +1189,18 @@ def main() -> None:
|
||||
parser.add_argument("--host", default="127.0.0.1", help="Bind address (default: 127.0.0.1)")
|
||||
# Default off the real server's 8189 so the mock never shadows a real API.
|
||||
parser.add_argument("--port", type=int, default=8190, help="Port number (default: 8190)")
|
||||
parser.add_argument(
|
||||
"--data-dir",
|
||||
default=None,
|
||||
help=("Read-only: seed jobs/datasets/settings from a real server's "
|
||||
"fastvideo_ui.db under this directory (its --data-dir) instead "
|
||||
"of fake demo data. Nothing the mock does is written back."),
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.data_dir is None or not _load_snapshot_from_db(Path(args.data_dir).resolve()):
|
||||
_seed()
|
||||
|
||||
uvicorn.run(app, host=args.host, port=args.port, log_level="info")
|
||||
|
||||
|
||||
|
||||
@@ -6,6 +6,9 @@ from fastvideo_studio.models.create_job_request import CreateJobRequest
|
||||
from fastvideo_studio.models.settings_update import SettingsUpdate
|
||||
from fastvideo_studio.models.create_dataset_request import CreateDatasetRequest
|
||||
from fastvideo_studio.models.update_caption_request import UpdateCaptionRequest
|
||||
from fastvideo_studio.models.queue_jobs_request import QueueJobsRequest
|
||||
from fastvideo_studio.models.merge_scene_request import MergeSceneRequest
|
||||
from fastvideo_studio.models.trim_request import GradeSegment, TrimRequest
|
||||
|
||||
|
||||
def model_label(model_path: str) -> str:
|
||||
@@ -18,5 +21,9 @@ __all__ = [
|
||||
"SettingsUpdate",
|
||||
"CreateDatasetRequest",
|
||||
"UpdateCaptionRequest",
|
||||
"QueueJobsRequest",
|
||||
"MergeSceneRequest",
|
||||
"GradeSegment",
|
||||
"TrimRequest",
|
||||
"model_label",
|
||||
]
|
||||
|
||||
@@ -0,0 +1,11 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Request model for merging a scene's clips into one video."""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class MergeSceneRequest(BaseModel):
|
||||
#: Completed jobs, in the order their videos should play.
|
||||
job_ids: list[str]
|
||||
#: Used to name the merged file.
|
||||
name: str = ""
|
||||
@@ -0,0 +1,8 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Request model for queueing jobs."""
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class QueueJobsRequest(BaseModel):
|
||||
job_ids: list[str]
|
||||
@@ -0,0 +1,30 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Request model for cutting a job's video and color-grading it section by section."""
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class GradeSegment(BaseModel):
|
||||
"""One razor-cut section of the kept range, and its own color grade.
|
||||
|
||||
``end_seconds`` is seconds into the *kept* (already start/end-trimmed)
|
||||
range -- the same timeline the preview video plays -- not the original
|
||||
file's. Only the last section in a list may leave it ``None``, meaning
|
||||
"to the end"; sections must be given in increasing order and cover the
|
||||
whole kept range, with no gaps or reordering.
|
||||
"""
|
||||
end_seconds: float | None = None
|
||||
#: Added directly to 0-255 pixel values. 0 = unchanged.
|
||||
brightness: float = 0.0
|
||||
#: 1.0 = unchanged, 0 = flat gray, >1 = more contrast.
|
||||
contrast: float = 1.0
|
||||
#: 1.0 = unchanged, 0 = grayscale, >1 = more saturated.
|
||||
saturation: float = 1.0
|
||||
|
||||
|
||||
class TrimRequest(BaseModel):
|
||||
start_seconds: float = 0.0
|
||||
#: None keeps to the end of the video.
|
||||
end_seconds: float | None = None
|
||||
#: Defaults to one neutral section covering the whole kept range, i.e. a plain trim with no grading.
|
||||
segments: list[GradeSegment] = Field(default_factory=lambda: [GradeSegment()])
|
||||
@@ -33,10 +33,12 @@ from fastapi.responses import FileResponse
|
||||
|
||||
from fastvideo.registry import (get_registered_model_paths, get_registered_models_with_workloads)
|
||||
from fastvideo_studio.database import Database, _get_db_path
|
||||
from fastvideo_studio.frames import FrameError, last_clip_for_job, last_frame_for_job
|
||||
from fastvideo_studio.gpu import get_gpu_snapshot
|
||||
from fastvideo_studio.job_runner import JobRunner, JobStatus
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, SettingsUpdate, UpdateCaptionRequest,
|
||||
model_label)
|
||||
from fastvideo_studio.merge import MergeError, merge_jobs, merged_path
|
||||
from fastvideo_studio.models import (CreateDatasetRequest, CreateJobRequest, MergeSceneRequest, QueueJobsRequest,
|
||||
SettingsUpdate, TrimRequest, UpdateCaptionRequest, model_label)
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
@@ -439,6 +441,41 @@ def start_job(job_id: str) -> dict[str, Any]:
|
||||
) from e
|
||||
|
||||
|
||||
def _queue_error(e: ValueError) -> HTTPException:
|
||||
return HTTPException(status_code=404 if "not found" in str(e) else 409, detail=str(e))
|
||||
|
||||
|
||||
@app.post("/api/jobs/queue")
|
||||
def queue_jobs(req: QueueJobsRequest) -> list[dict[str, Any]]:
|
||||
"""Queue jobs, in the order given, to run as GPU slots free up.
|
||||
|
||||
A job that starts from another job's last frame waits for that job to
|
||||
complete. All-or-nothing: if any job can't be queued, none are.
|
||||
"""
|
||||
try:
|
||||
return [j.to_dict() for j in job_runner.enqueue_jobs(req.job_ids)]
|
||||
except ValueError as e:
|
||||
raise _queue_error(e) from e
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/queue")
|
||||
def queue_job(job_id: str) -> dict[str, Any]:
|
||||
"""Queue one job to run when a slot is free."""
|
||||
try:
|
||||
return job_runner.enqueue_job(job_id).to_dict()
|
||||
except ValueError as e:
|
||||
raise _queue_error(e) from e
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/dequeue")
|
||||
def dequeue_job(job_id: str) -> dict[str, Any]:
|
||||
"""Take a queued job out of the queue (back to pending)."""
|
||||
try:
|
||||
return job_runner.dequeue_job(job_id).to_dict()
|
||||
except ValueError as e:
|
||||
raise _queue_error(e) from e
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/stop")
|
||||
def stop_job(job_id: str) -> dict[str, Any]:
|
||||
"""Request a running job to stop.
|
||||
@@ -665,6 +702,113 @@ def get_video(job_id: str) -> FileResponse:
|
||||
return FileResponse(job.output_path, media_type=media_type)
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/last-frame")
|
||||
def extract_last_frame(job_id: str) -> dict[str, str]:
|
||||
"""Save the last frame of a completed job's video and return its path.
|
||||
|
||||
The path is meant to be attached to another job as an image reference, so
|
||||
the next clip of a scene can start from where this one ended.
|
||||
"""
|
||||
job = job_runner.get_job(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
try:
|
||||
path = last_frame_for_job(job, upload_dir)
|
||||
except FrameError as e:
|
||||
if e.status_code >= 500:
|
||||
logger.warning("Last-frame extraction failed for job %s: %s", job_id, e.detail)
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return {"path": path, "media_type": "image"}
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/last-clip")
|
||||
def extract_last_clip(job_id: str) -> dict[str, str]:
|
||||
"""Save the trailing second of a completed job's video and return its path.
|
||||
|
||||
The path is meant to be attached to another job as a video reference, so
|
||||
the next clip continues with real motion instead of guessing it from a
|
||||
single still frame.
|
||||
"""
|
||||
job = job_runner.get_job(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
try:
|
||||
path = last_clip_for_job(job, upload_dir)
|
||||
except FrameError as e:
|
||||
if e.status_code >= 500:
|
||||
logger.warning("Last-clip extraction failed for job %s: %s", job_id, e.detail)
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return {"path": path, "media_type": "video"}
|
||||
|
||||
|
||||
def _trim_error(e: Exception) -> HTTPException:
|
||||
if isinstance(e, FrameError):
|
||||
return HTTPException(status_code=e.status_code, detail=e.detail)
|
||||
return HTTPException(status_code=404 if "not found" in str(e) else 400, detail=str(e))
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/trim")
|
||||
def trim_job(job_id: str, req: TrimRequest) -> dict[str, Any]:
|
||||
"""Cut a completed job's video to [start_seconds, end_seconds) and grade it section by section, in place.
|
||||
|
||||
The untouched original is kept, so this can be called again with a
|
||||
different range and/or sections, or undone with POST .../restore-video.
|
||||
"""
|
||||
try:
|
||||
segments = [s.model_dump() for s in req.segments]
|
||||
job = job_runner.trim_job(job_id, req.start_seconds, req.end_seconds, segments)
|
||||
except (ValueError, FrameError) as e:
|
||||
raise _trim_error(e) from e
|
||||
return job.to_dict()
|
||||
|
||||
|
||||
@app.post("/api/jobs/{job_id}/restore-video")
|
||||
def restore_job_video(job_id: str) -> dict[str, Any]:
|
||||
"""Undo every trim on a job's video, back to what it originally generated."""
|
||||
try:
|
||||
job = job_runner.restore_job_video(job_id)
|
||||
except (ValueError, FrameError) as e:
|
||||
raise _trim_error(e) from e
|
||||
return job.to_dict()
|
||||
|
||||
|
||||
@app.post("/api/scenes/merge")
|
||||
def merge_scene(req: MergeSceneRequest) -> dict[str, Any]:
|
||||
"""Join the finished clips of a scene, in the order given, into one video.
|
||||
|
||||
The clips are copied, not re-encoded, so this takes about a second.
|
||||
"""
|
||||
jobs = []
|
||||
for job_id in req.job_ids:
|
||||
job = job_runner.get_job(job_id)
|
||||
if job is None:
|
||||
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
|
||||
jobs.append(job)
|
||||
try:
|
||||
result = merge_jobs(jobs, job_runner.output_dir, req.name)
|
||||
except MergeError as e:
|
||||
if e.status_code >= 500:
|
||||
logger.warning("Scene merge failed: %s", e.detail)
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
logger.info("Merged %d clips into %s", result.clips, result.path)
|
||||
return {
|
||||
"filename": result.filename,
|
||||
"url": f"/api/merged/{result.filename}",
|
||||
"clips": result.clips,
|
||||
"seconds": result.seconds,
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/merged/{filename}")
|
||||
def get_merged_video(filename: str) -> FileResponse:
|
||||
"""Stream a merged scene video."""
|
||||
try:
|
||||
path = merged_path(job_runner.output_dir, filename)
|
||||
except MergeError as e:
|
||||
raise HTTPException(status_code=e.status_code, detail=e.detail) from e
|
||||
return FileResponse(path, media_type="video/mp4", filename=filename)
|
||||
|
||||
|
||||
@app.get("/api/jobs/{job_id}/download_log")
|
||||
def get_job_log_file(job_id: str) -> FileResponse:
|
||||
"""Download the log file for a job."""
|
||||
@@ -750,6 +894,14 @@ def main() -> None:
|
||||
help=("Directory for SQLite database (jobs + settings persistence) "
|
||||
f"(default: {default_data_dir})"),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--max-concurrent-jobs",
|
||||
type=int,
|
||||
default=1,
|
||||
help=("How many inference jobs may run at once (default: 1). Each gets its own "
|
||||
"long-lived worker that keeps its model loaded between jobs, so raising "
|
||||
"this loads one copy of the model per worker."),
|
||||
)
|
||||
parser.add_argument(
|
||||
"--verbose",
|
||||
action="store_true",
|
||||
@@ -775,6 +927,8 @@ def main() -> None:
|
||||
log_dir=log_dir,
|
||||
verbose=args.verbose,
|
||||
database=database,
|
||||
max_concurrent_jobs=args.max_concurrent_jobs,
|
||||
upload_dir=upload_dir,
|
||||
)
|
||||
|
||||
logger.info("Output directory: %s", output_dir)
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,276 @@
|
||||
'use client';
|
||||
|
||||
import { AlertTriangle, Loader2, RefreshCw } from 'lucide-react';
|
||||
import * as React from 'react';
|
||||
|
||||
import CreateJobModal from '@/components/jobs/CreateJobModal';
|
||||
import { EDITABLE_STATUSES, SceneBoard } from '@/components/scenes/SceneBoard';
|
||||
import { MergedSceneDialog } from '@/components/scenes/MergedSceneDialog';
|
||||
import { SceneQueueDialog } from '@/components/scenes/SceneQueueDialog';
|
||||
import { TrimClipDialog } from '@/components/scenes/TrimClipDialog';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { Card } from '@/components/ui/card';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { toast } from 'sonner';
|
||||
import { dequeueJob, getJobsList, mergeScene, queueJobs, updateJob, type MergedScene } from '@/lib/api';
|
||||
import { attachedLastFrames, attachLastFrame, deferredLastFrameSource, detachLastFrame } from '@/lib/continuation';
|
||||
import type { SceneQueuePlan } from '@/lib/sceneQueue';
|
||||
import { buildScenes, type Scene, type SceneClip } from '@/lib/scenes';
|
||||
import type { Job } from '@/lib/types';
|
||||
|
||||
const POLL_MS = 3000;
|
||||
|
||||
export default function ScenesPage() {
|
||||
const [jobs, setJobs] = React.useState<Job[]>([]);
|
||||
const [isLoading, setIsLoading] = React.useState(true);
|
||||
const [error, setError] = React.useState<string | null>(null);
|
||||
const [reloadKey, setReloadKey] = React.useState(0);
|
||||
const [selectedKey, setSelectedKey] = React.useState<string | null>(null);
|
||||
const [openJob, setOpenJob] = React.useState<Job | null>(null);
|
||||
// which re-run of a clip number the user picked, by clip
|
||||
const [choices, setChoices] = React.useState<Record<string, string>>({});
|
||||
const [queueTarget, setQueueTarget] = React.useState<Scene | null>(null);
|
||||
const [merged, setMerged] = React.useState<(MergedScene & { title: string }) | null>(null);
|
||||
const [trimTarget, setTrimTarget] = React.useState<Job | null>(null);
|
||||
// Bumped per job id whenever its video is edited in place, to cache-bust every
|
||||
// <video> for that job (the dialog's own preview and the scene board's). Owned
|
||||
// here, not by the dialog, so it survives the dialog re-rendering with a fresh
|
||||
// job object after each edit -- a per-dialog counter got reset by that same
|
||||
// re-render before the browser ever saw the new URL.
|
||||
const [videoVersions, setVideoVersions] = React.useState<Record<string, number>>({});
|
||||
const bumpVideoVersion = (jobId: string) =>
|
||||
setVideoVersions((v) => ({ ...v, [jobId]: (v[jobId] ?? 0) + 1 }));
|
||||
|
||||
React.useEffect(() => {
|
||||
let cancelled = false;
|
||||
async function load() {
|
||||
try {
|
||||
const list = await getJobsList('inference');
|
||||
if (cancelled) return;
|
||||
setJobs(list);
|
||||
setError(null);
|
||||
} catch (e) {
|
||||
if (!cancelled) setError(e instanceof Error ? e.message : 'Failed to load jobs');
|
||||
} finally {
|
||||
if (!cancelled) setIsLoading(false);
|
||||
}
|
||||
}
|
||||
void load();
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [reloadKey]);
|
||||
|
||||
function reload() {
|
||||
setIsLoading(true);
|
||||
setReloadKey((k) => k + 1);
|
||||
}
|
||||
|
||||
// While anything is queued or running, keep the board current without the
|
||||
// loading state (which would blank or flicker it).
|
||||
const hasActive = jobs.some((j) => j.status === 'queued' || j.status === 'running');
|
||||
React.useEffect(() => {
|
||||
if (!hasActive) return;
|
||||
const interval = setInterval(() => {
|
||||
getJobsList('inference')
|
||||
.then(setJobs)
|
||||
.catch(() => {});
|
||||
}, POLL_MS);
|
||||
return () => clearInterval(interval);
|
||||
}, [hasActive]);
|
||||
|
||||
// Attach the previous clip's last frame to `clip` and rewrite its prompt to open on
|
||||
// it. The frame is a reference the server resolves when the clip runs, so the
|
||||
// previous clip doesn't have to be finished yet. Returns the picture label.
|
||||
async function attachPreviousFrame(clip: SceneClip, previous: SceneClip): Promise<string> {
|
||||
const result = attachLastFrame(
|
||||
clip.job.prompt,
|
||||
clip.job.references ?? [],
|
||||
deferredLastFrameSource(previous.job.id),
|
||||
);
|
||||
if (!result.ok) throw new Error(result.error);
|
||||
await updateJob(clip.job.id, { references: result.references, prompt: result.prompt });
|
||||
return result.label;
|
||||
}
|
||||
|
||||
async function useLastFrame(clip: SceneClip, previous: SceneClip) {
|
||||
try {
|
||||
const label = await attachPreviousFrame(clip, previous);
|
||||
toast.success(`Clip ${clip.position} now starts from the last frame of clip ${previous.position} (${label}).`);
|
||||
reload();
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't use the last frame.");
|
||||
}
|
||||
}
|
||||
|
||||
// Undo the above: take every attached last frame back off the clip.
|
||||
async function stopUsingLastFrame(clip: SceneClip) {
|
||||
try {
|
||||
let prompt = clip.job.prompt;
|
||||
let references = clip.job.references ?? [];
|
||||
// Newest first: only the most recently added image can be removed cleanly.
|
||||
for (const { source } of attachedLastFrames(references).reverse()) {
|
||||
const result = detachLastFrame(prompt, references, source);
|
||||
if (!result.ok) throw new Error(result.error);
|
||||
prompt = result.prompt;
|
||||
references = result.references;
|
||||
}
|
||||
await updateJob(clip.job.id, { references, prompt });
|
||||
toast.success(`Clip ${clip.position} no longer starts from a previous clip's last frame.`);
|
||||
reload();
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't remove the last frame.");
|
||||
}
|
||||
}
|
||||
|
||||
// Join the scene's clips, in the order shown (the take chosen for each), into one video.
|
||||
async function mergeSceneClips(scene: Scene) {
|
||||
try {
|
||||
const result = await mergeScene(scene.clips.map((c) => c.job.id), scene.title);
|
||||
setMerged({ ...result, title: scene.title });
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't merge the scene.");
|
||||
}
|
||||
}
|
||||
|
||||
async function queueClip(clip: SceneClip) {
|
||||
try {
|
||||
await queueJobs([clip.job.id]);
|
||||
toast.success(`Queued clip ${clip.position}.`);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't queue the clip.");
|
||||
}
|
||||
reload();
|
||||
}
|
||||
|
||||
async function dequeueClip(clip: SceneClip) {
|
||||
try {
|
||||
await dequeueJob(clip.job.id);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't remove the clip from the queue.");
|
||||
}
|
||||
reload();
|
||||
}
|
||||
|
||||
// Link clips to the frames before them (if asked), then queue them all in scene order.
|
||||
async function queueScene(plan: SceneQueuePlan) {
|
||||
try {
|
||||
for (const { clip, previous } of plan.chain) {
|
||||
await attachPreviousFrame(clip, previous).catch((e) => {
|
||||
throw new Error(`Clip ${clip.position}: ${e instanceof Error ? e.message : e}. Nothing was queued.`);
|
||||
});
|
||||
}
|
||||
await queueJobs(plan.queue.map((c) => c.job.id));
|
||||
const n = plan.queue.length;
|
||||
toast.success(`Queued ${n} clip${n === 1 ? '' : 's'}.`);
|
||||
setQueueTarget(null);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't queue the scene.");
|
||||
}
|
||||
reload();
|
||||
}
|
||||
|
||||
const scenes = React.useMemo(() => buildScenes(jobs, choices), [jobs, choices]);
|
||||
const scene = scenes.find((s) => s.key === selectedKey) ?? scenes[0];
|
||||
|
||||
return (
|
||||
<div className="mx-auto w-full max-w-[1200px] px-4 pb-12 pt-4">
|
||||
<Card className="p-6">
|
||||
<div className="mb-1 flex flex-wrap items-center gap-3">
|
||||
<h2 className="text-2xl font-semibold text-foreground">Scenes</h2>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="ml-auto gap-1.5"
|
||||
onClick={reload}
|
||||
disabled={isLoading}
|
||||
>
|
||||
<RefreshCw className={isLoading ? 'size-3.5 animate-spin' : 'size-3.5'} aria-hidden />
|
||||
Refresh
|
||||
</Button>
|
||||
</div>
|
||||
<p className="mb-6 text-sm text-muted-foreground">
|
||||
Every shot across your jobs, in order. Jobs named with a shared stem and a number, like{' '}
|
||||
<code className="font-mono text-xs">wolf-lunch-clip-01</code>,{' '}
|
||||
<code className="font-mono text-xs">wolf-lunch-clip-02</code>, are grouped into one scene.
|
||||
</p>
|
||||
|
||||
{isLoading && jobs.length === 0 ? (
|
||||
<div className="flex items-center gap-3 p-8 text-muted-foreground">
|
||||
<Loader2 className="h-6 w-6 animate-spin text-primary" />
|
||||
<span>Loading scenes…</span>
|
||||
</div>
|
||||
) : error && jobs.length === 0 ? (
|
||||
<div role="alert" className="flex flex-col items-center gap-3 py-8 text-center">
|
||||
<AlertTriangle className="size-6 text-destructive" aria-hidden />
|
||||
<p className="max-w-md text-sm text-muted-foreground">{error}</p>
|
||||
<Button type="button" variant="outline" onClick={reload}>
|
||||
Try Again
|
||||
</Button>
|
||||
</div>
|
||||
) : !scene ? (
|
||||
<p className="py-8 text-center text-muted-foreground">
|
||||
No inference jobs yet. Create some and they'll show up here.
|
||||
</p>
|
||||
) : (
|
||||
<div className="flex flex-col gap-6">
|
||||
{scenes.length > 1 && (
|
||||
<label className="flex max-w-md flex-col gap-1 text-xs text-muted-foreground">
|
||||
Scene
|
||||
<NativeSelect value={scene.key} onChange={(e) => setSelectedKey(e.target.value)}>
|
||||
{scenes.map((s) => (
|
||||
<option key={s.key} value={s.key}>
|
||||
{s.title} ({s.clips.length} clip{s.clips.length === 1 ? '' : 's'})
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</label>
|
||||
)}
|
||||
<SceneBoard
|
||||
scene={scene}
|
||||
onOpenJob={setOpenJob}
|
||||
onUseLastFrame={useLastFrame}
|
||||
onDetachLastFrame={stopUsingLastFrame}
|
||||
onQueueScene={setQueueTarget}
|
||||
onMergeScene={mergeSceneClips}
|
||||
onQueueClip={queueClip}
|
||||
onDequeueClip={dequeueClip}
|
||||
onTrimClip={(clip) => setTrimTarget(clip.job)}
|
||||
onChooseTake={(takeKey, jobId) => setChoices((c) => ({ ...c, [takeKey]: jobId }))}
|
||||
videoVersionFor={(jobId) => videoVersions[jobId] ?? 0}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</Card>
|
||||
|
||||
<MergedSceneDialog merged={merged} onClose={() => setMerged(null)} />
|
||||
<SceneQueueDialog scene={queueTarget} onConfirm={queueScene} onClose={() => setQueueTarget(null)} />
|
||||
<TrimClipDialog
|
||||
clip={trimTarget}
|
||||
videoVersion={trimTarget ? (videoVersions[trimTarget.id] ?? 0) : 0}
|
||||
onChanged={(job) => {
|
||||
setTrimTarget(job);
|
||||
bumpVideoVersion(job.id);
|
||||
getJobsList('inference').then(setJobs).catch(() => {});
|
||||
}}
|
||||
onClose={() => setTrimTarget(null)}
|
||||
/>
|
||||
|
||||
{openJob && (
|
||||
<CreateJobModal
|
||||
isOpen
|
||||
readOnly={!EDITABLE_STATUSES.includes(openJob.status)}
|
||||
editingJob={openJob}
|
||||
jobType={(openJob.job_type ?? 'inference') as never}
|
||||
workloadType={openJob.workload_type ?? 't2v'}
|
||||
onClose={() => setOpenJob(null)}
|
||||
onSuccess={() => {
|
||||
setOpenJob(null);
|
||||
reload();
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -1,19 +1,29 @@
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { toast } from 'sonner';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import CreateJobButton from './CreateJobButton';
|
||||
|
||||
vi.mock('sonner', () => ({ toast: { error: vi.fn(), warning: vi.fn() } }));
|
||||
|
||||
vi.mock('./CreateJobModal', () => ({
|
||||
default: ({
|
||||
isOpen,
|
||||
workloadType,
|
||||
prefillJob,
|
||||
}: {
|
||||
isOpen: boolean;
|
||||
workloadType: string;
|
||||
prefillJob?: { name?: string; prompt: string } | null;
|
||||
}) =>
|
||||
isOpen ? (
|
||||
<div role="dialog" data-workload-type={workloadType}>
|
||||
<div
|
||||
role="dialog"
|
||||
data-workload-type={workloadType}
|
||||
data-prefill-name={prefillJob?.name ?? ''}
|
||||
data-prefill-prompt={prefillJob?.prompt ?? ''}
|
||||
>
|
||||
Create job form
|
||||
</div>
|
||||
) : null,
|
||||
@@ -27,7 +37,11 @@ describe('CreateJobButton', () => {
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(screen.getByRole('menuitem', { name: /I2V/i }));
|
||||
|
||||
expect(screen.getByRole('dialog')).toHaveAttribute(
|
||||
// The dialog opens a frame after the menu selection (see CreateJobButton's
|
||||
// openModal: deferred via requestAnimationFrame to dodge a Radix
|
||||
// DropdownMenu + Dialog pointer-events race), so wait for it rather than
|
||||
// asserting synchronously.
|
||||
expect(await screen.findByRole('dialog')).toHaveAttribute(
|
||||
'data-workload-type',
|
||||
'i2v',
|
||||
);
|
||||
@@ -45,9 +59,75 @@ describe('CreateJobButton', () => {
|
||||
expect(firstItem).toHaveFocus();
|
||||
await user.keyboard('{Enter}');
|
||||
|
||||
expect(screen.getByRole('dialog')).toHaveAttribute(
|
||||
expect(await screen.findByRole('dialog')).toHaveAttribute(
|
||||
'data-workload-type',
|
||||
't2v',
|
||||
);
|
||||
});
|
||||
|
||||
describe('importing a job file', () => {
|
||||
const file = (contents: string, name = 'job.json') =>
|
||||
new File([contents], name, { type: 'application/json' });
|
||||
|
||||
it('opens the form for the file\'s workload with the job pre-filled', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<CreateJobButton jobType="inference" />);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(screen.getByRole('menuitem', { name: /Import job JSON/ }));
|
||||
await user.upload(
|
||||
screen.getByLabelText('Import job JSON file'),
|
||||
file(
|
||||
JSON.stringify({
|
||||
model_id: 'MiniMaxAI/MiniMax-H3',
|
||||
name: 'from-file',
|
||||
workload_type: 'i2v',
|
||||
prompt: ['summary:', 'Hi.'],
|
||||
}),
|
||||
),
|
||||
);
|
||||
|
||||
const dialog = await screen.findByRole('dialog');
|
||||
expect(dialog).toHaveAttribute('data-workload-type', 'i2v');
|
||||
expect(dialog).toHaveAttribute('data-prefill-name', 'from-file');
|
||||
expect(dialog).toHaveAttribute('data-prefill-prompt', 'summary:\nHi.');
|
||||
});
|
||||
|
||||
it('reports an unreadable file instead of opening the form', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<CreateJobButton jobType="inference" />);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(screen.getByRole('menuitem', { name: /Import job JSON/ }));
|
||||
await user.upload(screen.getByLabelText('Import job JSON file'), file('not json', 'bad.json'));
|
||||
|
||||
await vi.waitFor(() => expect(toast.error).toHaveBeenCalled());
|
||||
expect(vi.mocked(toast.error).mock.calls.at(-1)?.[0]).toContain("bad.json");
|
||||
expect(screen.queryByRole('dialog')).not.toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('a normal open after an import starts clean', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<CreateJobButton jobType="inference" />);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(screen.getByRole('menuitem', { name: /Import job JSON/ }));
|
||||
await user.upload(
|
||||
screen.getByLabelText('Import job JSON file'),
|
||||
file(JSON.stringify({ model_id: 'm', name: 'from-file', prompt: 'p' })),
|
||||
);
|
||||
expect(await screen.findByRole('dialog')).toHaveAttribute('data-prefill-name', 'from-file');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await user.click(await screen.findByRole('menuitem', { name: /T2V/i }));
|
||||
await vi.waitFor(() => expect(screen.getByRole('dialog')).toHaveAttribute('data-prefill-name', ''));
|
||||
});
|
||||
|
||||
it('is not offered on the training pages', async () => {
|
||||
const user = userEvent.setup();
|
||||
render(<CreateJobButton jobType="finetuning" />);
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
expect(screen.queryByRole('menuitem', { name: /Import job JSON/ })).not.toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
@@ -6,7 +6,10 @@ import * as DropdownMenu from '@radix-ui/react-dropdown-menu';
|
||||
|
||||
import CreateJobModal from '@/components/jobs/CreateJobModal';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { toast } from 'sonner';
|
||||
import { WORKLOAD_OPTIONS } from '@/lib/jobConfig';
|
||||
import { parseJobJson } from '@/lib/jobJson';
|
||||
import type { JobLike } from '@/lib/jobToFields';
|
||||
import type { JobType } from '@/lib/types';
|
||||
import { triggerRefresh } from '@/stores/jobsRefresh';
|
||||
|
||||
@@ -22,9 +25,32 @@ export default function CreateJobButton({ jobType }: CreateJobButtonProps) {
|
||||
options[0]?.type ?? 't2v',
|
||||
);
|
||||
|
||||
function openModal(type: string) {
|
||||
// An imported job file pre-fills the form; a normal open starts clean.
|
||||
const [prefillJob, setPrefillJob] = React.useState<JobLike | null>(null);
|
||||
const fileInputRef = React.useRef<HTMLInputElement>(null);
|
||||
|
||||
async function handleImportFile(e: React.ChangeEvent<HTMLInputElement>) {
|
||||
const file = e.target.files?.[0];
|
||||
e.target.value = ''; // allow re-picking the same file
|
||||
if (!file) return;
|
||||
const result = parseJobJson(await file.text());
|
||||
if (!result.ok) {
|
||||
toast.error(`Couldn't import ${file.name}: ${result.error}`);
|
||||
return;
|
||||
}
|
||||
openModal(result.workloadType, result.job);
|
||||
}
|
||||
|
||||
function openModal(type: string, prefill: JobLike | null = null) {
|
||||
setWorkloadType(type);
|
||||
setModalOpen(true);
|
||||
setPrefillJob(prefill);
|
||||
// Opening the Dialog synchronously inside the DropdownMenu.Item's
|
||||
// onSelect races the two Radix layers' pointer-events body lock/unlock:
|
||||
// the dropdown's own close cleanup can lose that race, leaving <body>
|
||||
// stuck with `pointer-events: none` after this dialog later closes --
|
||||
// silently blocking every click, including this same button. Deferring
|
||||
// to the next frame lets the dropdown finish closing first.
|
||||
requestAnimationFrame(() => setModalOpen(true));
|
||||
}
|
||||
|
||||
function handleSuccess() {
|
||||
@@ -60,15 +86,39 @@ export default function CreateJobButton({ jobType }: CreateJobButtonProps) {
|
||||
</span>
|
||||
</DropdownMenu.Item>
|
||||
))}
|
||||
{jobType === 'inference' && (
|
||||
<>
|
||||
<DropdownMenu.Separator className="my-1 h-px bg-border" />
|
||||
<DropdownMenu.Item
|
||||
onSelect={() => fileInputRef.current?.click()}
|
||||
className="flex min-h-11 cursor-pointer select-none flex-col justify-center px-4 py-2 text-left text-sm font-medium outline-none data-[highlighted]:bg-secondary"
|
||||
>
|
||||
Import job JSON…
|
||||
<span className="mt-0.5 block text-xs font-normal text-muted-foreground">
|
||||
Fill the form from a job file
|
||||
</span>
|
||||
</DropdownMenu.Item>
|
||||
</>
|
||||
)}
|
||||
</DropdownMenu.Content>
|
||||
</DropdownMenu.Portal>
|
||||
</DropdownMenu.Root>
|
||||
<input
|
||||
ref={fileInputRef}
|
||||
type="file"
|
||||
accept=".json,application/json"
|
||||
aria-label="Import job JSON file"
|
||||
tabIndex={-1}
|
||||
className="sr-only"
|
||||
onChange={handleImportFile}
|
||||
/>
|
||||
<CreateJobModal
|
||||
isOpen={modalOpen}
|
||||
onClose={() => setModalOpen(false)}
|
||||
onSuccess={handleSuccess}
|
||||
jobType={jobType}
|
||||
workloadType={workloadType}
|
||||
prefillJob={prefillJob}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
|
||||
@@ -4,10 +4,13 @@ import userEvent from '@testing-library/user-event';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import CreateJobModal from './CreateJobModal';
|
||||
import { toast } from 'sonner';
|
||||
import { createJob, getDatasets, getModels, uploadImage } from '@/lib/api';
|
||||
import { defaultOptionsStore } from '@/stores/defaultOptions';
|
||||
import { DEFAULT_OPTIONS } from '@/lib/defaultOptions';
|
||||
|
||||
vi.mock('sonner', () => ({ toast: { warning: vi.fn(), error: vi.fn(), success: vi.fn() } }));
|
||||
|
||||
vi.mock('@/lib/api', () => ({
|
||||
createJob: vi.fn(),
|
||||
getModels: vi.fn(),
|
||||
@@ -206,4 +209,179 @@ describe('CreateJobModal', () => {
|
||||
|
||||
await waitFor(() => expect(onSuccess).toHaveBeenCalledTimes(1));
|
||||
});
|
||||
|
||||
it('shows the H3 speech-tag console only for an H3 model and inserts at the prompt cursor', async () => {
|
||||
vi.mocked(getModels).mockResolvedValue([
|
||||
...MODELS,
|
||||
{ id: 'minimax-h3/i2v', label: 'MiniMax H3', type: 'i2v' },
|
||||
]);
|
||||
const user = userEvent.setup();
|
||||
renderModal({ workloadType: 'i2v' });
|
||||
|
||||
await screen.findByRole('option', { name: 'Wan T2V (wan/t2v-1.3b)' });
|
||||
expect(screen.queryByText(/Speech tags/)).not.toBeInTheDocument();
|
||||
|
||||
await user.selectOptions(
|
||||
screen.getByLabelText('Model'),
|
||||
'minimax-h3/i2v',
|
||||
);
|
||||
expect(await screen.findByText(/Speech tags/)).toBeInTheDocument();
|
||||
|
||||
const promptField = screen.getByLabelText('Prompt') as HTMLTextAreaElement;
|
||||
await user.type(promptField, 'Hello there.');
|
||||
promptField.setSelectionRange(5, 5);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: '<pause>' }));
|
||||
|
||||
expect(promptField.value).toBe('Hello <pause> there.');
|
||||
// Focus and cursor return to the prompt field so tags can be chained.
|
||||
await waitFor(() => expect(promptField).toHaveFocus());
|
||||
});
|
||||
});
|
||||
|
||||
const H3 = { id: 'minimax-h3/i2v', label: 'MiniMax H3', type: 'i2v' };
|
||||
|
||||
const BASE_PROMPT = [
|
||||
'For the target video, <Picture 1> is fully referenced.',
|
||||
'',
|
||||
'integrated_multimodal_description: [Shot 1] A close-up. He (S1) says: <d>[English] Sorry.</d> <cutoff>',
|
||||
'',
|
||||
'overall_soundscape: Room tone.',
|
||||
'',
|
||||
'non_diegetic_music: N/A',
|
||||
].join('\n');
|
||||
|
||||
describe('CreateJobModal with a prefilled job', () => {
|
||||
beforeEach(() => {
|
||||
vi.mocked(getModels).mockResolvedValue([...MODELS, H3]);
|
||||
vi.mocked(toast.warning).mockClear();
|
||||
});
|
||||
|
||||
it('fills the form from the job and creates a new job on submit', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onSuccess } = renderModal({
|
||||
workloadType: 'i2v',
|
||||
prefillJob: {
|
||||
id: 'imported-1',
|
||||
model_id: H3.id,
|
||||
name: 'from-file',
|
||||
prompt: BASE_PROMPT,
|
||||
workload_type: 'i2v',
|
||||
references: [{ source: '/data/inputs/face.png', media_type: 'image' }],
|
||||
num_frames: 243,
|
||||
seed: 7,
|
||||
},
|
||||
});
|
||||
|
||||
await screen.findByRole('option', { name: /MiniMax H3/ });
|
||||
await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(H3.id));
|
||||
expect(screen.getByLabelText('Name (optional)')).toHaveValue('from-file');
|
||||
expect(screen.getByLabelText('Prompt')).toHaveValue(BASE_PROMPT);
|
||||
expect(screen.getByText('face.png')).toBeInTheDocument();
|
||||
expect(screen.getByText('New Inference Job (I2V)')).toBeInTheDocument();
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Create Job' }));
|
||||
await waitFor(() => expect(onSuccess).toHaveBeenCalledTimes(1));
|
||||
const payload = vi.mocked(createJob).mock.calls[0][0] as unknown as Record<string, unknown>;
|
||||
expect(payload).toMatchObject({
|
||||
model_id: H3.id,
|
||||
name: 'from-file',
|
||||
prompt: BASE_PROMPT,
|
||||
num_frames: 243,
|
||||
seed: 7,
|
||||
references: [{ source: '/data/inputs/face.png', media_type: 'image' }],
|
||||
});
|
||||
});
|
||||
|
||||
it("warns when the file's model isn't offered for the workload", async () => {
|
||||
renderModal({
|
||||
workloadType: 't2v',
|
||||
prefillJob: { id: 'imported-2', model_id: 'not/a-real-model', prompt: 'p' },
|
||||
});
|
||||
await waitFor(() => expect(toast.warning).toHaveBeenCalled());
|
||||
expect(vi.mocked(toast.warning).mock.calls[0][0]).toContain('not/a-real-model');
|
||||
expect(screen.getByLabelText('Model')).toHaveValue(MODELS[0].id);
|
||||
});
|
||||
});
|
||||
|
||||
describe('CreateJobModal shot list on a raw prompt', () => {
|
||||
beforeEach(() => {
|
||||
vi.mocked(getModels).mockResolvedValue([...MODELS, H3]);
|
||||
});
|
||||
|
||||
async function openShotList(user: ReturnType<typeof userEvent.setup>) {
|
||||
await user.click(screen.getByRole('button', { name: 'Open shot list' }));
|
||||
return screen.findByRole('dialog', { name: 'Shot list' });
|
||||
}
|
||||
|
||||
it('edits dialogue inside integrated_multimodal_description and leaves the rest of the prompt alone', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderModal({
|
||||
workloadType: 'i2v',
|
||||
prefillJob: {
|
||||
id: 'imported-3',
|
||||
model_id: H3.id,
|
||||
prompt: BASE_PROMPT,
|
||||
references: [{ source: '/data/face.png', media_type: 'image' }],
|
||||
},
|
||||
});
|
||||
await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(H3.id));
|
||||
|
||||
const dialog = await openShotList(user);
|
||||
expect(within(dialog).getAllByText(/integrated_multimodal_description/).length).toBeGreaterThan(0);
|
||||
expect(within(dialog).queryByText('Retention analysis')).not.toBeInTheDocument();
|
||||
|
||||
const line = within(dialog).getByPlaceholderText('What the speaker says') as HTMLTextAreaElement;
|
||||
expect(line.value).toBe('Sorry.');
|
||||
await user.type(line, ' Really.');
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Apply' }));
|
||||
|
||||
await waitFor(() =>
|
||||
expect(screen.getByLabelText('Prompt')).toHaveValue(BASE_PROMPT.replace('Sorry.', 'Sorry. Really.')),
|
||||
);
|
||||
});
|
||||
|
||||
it('treats a plain prompt as the shot text and writes shot markers back over it', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderModal({ workloadType: 't2v' });
|
||||
await screen.findByRole('option', { name: /MiniMax H3/ });
|
||||
await user.selectOptions(screen.getByLabelText('Model'), H3.id);
|
||||
await user.type(screen.getByLabelText('Prompt'), 'A dock at dawn.');
|
||||
|
||||
const dialog = await openShotList(user);
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Apply' }));
|
||||
await waitFor(() => expect(screen.getByLabelText('Prompt')).toHaveValue('[Shot 1] A dock at dawn.'));
|
||||
});
|
||||
|
||||
it('keeps retention analysis in step when the raw prompt has a detailed_description and retention_analysis', async () => {
|
||||
const user = userEvent.setup();
|
||||
const sections = [
|
||||
'subject_definitions:',
|
||||
'<Subject 1> is the courier.',
|
||||
'',
|
||||
'retention_analysis:',
|
||||
'<Subject 1> (appears in [Shot 1]): fully_preserved - jacket.',
|
||||
'',
|
||||
'detailed_description:',
|
||||
'[Shot 1] Wide shot. A rooftop with <Subject 1>.',
|
||||
].join('\n');
|
||||
renderModal({ workloadType: 't2v', prefillJob: { id: 'imported-4', model_id: H3.id, prompt: sections } });
|
||||
await waitFor(() => expect(screen.getByLabelText('Model')).toHaveValue(H3.id));
|
||||
|
||||
const dialog = await openShotList(user);
|
||||
expect(within(dialog).getByText('Retention analysis')).toBeInTheDocument();
|
||||
await user.click(within(dialog).getAllByRole('button', { name: /Add shot/ })[0]);
|
||||
await user.click(within(dialog).getAllByRole('button', { name: '<Subject 1>' })[1]);
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Apply' }));
|
||||
|
||||
const out = (screen.getByLabelText('Prompt') as HTMLTextAreaElement).value;
|
||||
expect(out).toContain('<Subject 1> (appears in [Shot 1], [Shot 2]): fully_preserved - jacket.');
|
||||
expect(out).toContain('subject_definitions:\n<Subject 1> is the courier.');
|
||||
});
|
||||
|
||||
it('is not offered for a non-H3 model', async () => {
|
||||
renderModal({ workloadType: 't2v' });
|
||||
await screen.findByRole('option', { name: 'Wan T2V (wan/t2v-1.3b)' });
|
||||
expect(screen.queryByRole('button', { name: 'Open shot list' })).not.toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -49,8 +49,18 @@ import {
|
||||
parseH3Prompt,
|
||||
serializeH3Prompt,
|
||||
type H3PromptFields,
|
||||
type H3PromptSection,
|
||||
} from '@/lib/h3Prompt';
|
||||
import { applyH3Tag, type H3Tag } from '@/lib/h3Tags';
|
||||
import { H3TagConsole } from '@/components/jobs/H3TagConsole';
|
||||
import { H3DialogueEditor } from '@/components/jobs/H3DialogueEditor';
|
||||
import { H3ShotListEditor } from '@/components/jobs/H3ShotListEditor';
|
||||
import { jobToFormFields, type JobLike } from '@/lib/jobToFields';
|
||||
import { readField, shotTarget, writeField, writeShotTarget } from '@/lib/h3PromptFields';
|
||||
import { toast } from 'sonner';
|
||||
|
||||
/** Prompt fields the speech-tag console can insert into. */
|
||||
type TagTarget = 'prompt' | H3PromptSection;
|
||||
|
||||
export interface CreateJobModalProps {
|
||||
isOpen: boolean;
|
||||
@@ -60,6 +70,8 @@ export interface CreateJobModalProps {
|
||||
workloadType: string;
|
||||
/** When set, the modal edits this pending job instead of creating a new one. */
|
||||
editingJob?: JobLike | null;
|
||||
/** Seed the form from this job (an imported job file) but still create a new job on submit. */
|
||||
prefillJob?: JobLike | null;
|
||||
/** Show the configuration without allowing changes (started/finished jobs). */
|
||||
readOnly?: boolean;
|
||||
}
|
||||
@@ -71,6 +83,7 @@ export default function CreateJobModal({
|
||||
jobType,
|
||||
workloadType,
|
||||
editingJob,
|
||||
prefillJob,
|
||||
readOnly = false,
|
||||
}: CreateJobModalProps) {
|
||||
const { options } = useStore(defaultOptionsStore);
|
||||
@@ -121,11 +134,85 @@ export default function CreateJobModal({
|
||||
// H3 is the only registered model with an end frame or references.
|
||||
const supportsLastImage = modelId.toLowerCase().includes('minimax-h3');
|
||||
const usingReferences = supportsLastImage && references.length > 0;
|
||||
// Same check, named for where it gates H3-only prompt features (speech tags)
|
||||
// rather than the end-frame/reference upload UI.
|
||||
const isH3Model = supportsLastImage;
|
||||
|
||||
// Which prompt field the speech-tag console inserts into: whichever one is
|
||||
// actually visible, tracked via each field's onFocus below.
|
||||
const [activeTagField, setActiveTagField] = React.useState<TagTarget>('prompt');
|
||||
const tagFieldRefs = React.useRef<Partial<Record<TagTarget, HTMLTextAreaElement | null>>>({});
|
||||
const showGuidedFields = usingReferences && useGuidedPrompt;
|
||||
React.useEffect(() => {
|
||||
setActiveTagField(showGuidedFields ? 'detailed_description' : 'prompt');
|
||||
}, [showGuidedFields]);
|
||||
|
||||
function getTagFieldValue(field: TagTarget): string {
|
||||
return field === 'prompt' ? prompt : promptFields[field];
|
||||
}
|
||||
function setTagFieldValue(field: TagTarget, value: string) {
|
||||
if (field === 'prompt') setPrompt(value);
|
||||
else setPromptField(field, value);
|
||||
}
|
||||
|
||||
function insertH3Tag(tag: H3Tag) {
|
||||
const field = activeTagField;
|
||||
const el = tagFieldRefs.current[field] ?? null;
|
||||
const currentValue = getTagFieldValue(field);
|
||||
const start = el?.selectionStart ?? currentValue.length;
|
||||
const end = el?.selectionEnd ?? currentValue.length;
|
||||
const result = applyH3Tag(currentValue, start, end, tag);
|
||||
setTagFieldValue(field, result.value);
|
||||
// The textarea re-renders with the new value before this runs; restore
|
||||
// focus and the cursor position so tags can be chained without reaching
|
||||
// for the mouse each time.
|
||||
requestAnimationFrame(() => {
|
||||
const node = tagFieldRefs.current[field];
|
||||
if (node) {
|
||||
node.focus();
|
||||
node.setSelectionRange(result.start, result.end);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
// Full-screen dialogue editor: opens with a copy of whichever field the
|
||||
// speech-tag console would currently insert into, and on Apply overwrites
|
||||
// that same field wholesale (not just an insertion at the cursor).
|
||||
const [dialogueEditorOpen, setDialogueEditorOpen] = React.useState(false);
|
||||
const dialogueEditorField = activeTagField;
|
||||
|
||||
// Shot list: writes into detailed_description AND retention_analysis
|
||||
// together, so unlike the speech-tag console/dialogue editor it isn't
|
||||
// scoped to "whichever field is focused" -- it's specific to those two.
|
||||
const [shotListOpen, setShotListOpen] = React.useState(false);
|
||||
// 'sections' edits the guided Detailed description; 'prompt' edits the shots
|
||||
// inside the raw prompt (base format, or a plain prompt).
|
||||
const [shotListMode, setShotListMode] = React.useState<'sections' | 'prompt'>('sections');
|
||||
function applyShotList(detailedDescription: string, retentionAnalysis: string) {
|
||||
setPromptFields((prev) => ({
|
||||
...prev,
|
||||
detailed_description: detailedDescription,
|
||||
retention_analysis: retentionAnalysis,
|
||||
}));
|
||||
}
|
||||
|
||||
const promptShots = React.useMemo(() => shotTarget(prompt), [prompt]);
|
||||
const promptSubjects = React.useMemo(() => readField(prompt, 'subject_definitions') ?? '', [prompt]);
|
||||
const promptRetention = React.useMemo(() => readField(prompt, 'retention_analysis'), [prompt]);
|
||||
const promptSyncsRetention = promptShots.field === 'detailed_description' && promptRetention !== null;
|
||||
|
||||
function applyShotListToPrompt(shotText: string, retentionAnalysis: string) {
|
||||
let next = writeShotTarget(prompt, promptShots, shotText);
|
||||
if (promptSyncsRetention) next = writeField(next, 'retention_analysis', retentionAnalysis);
|
||||
setPrompt(next);
|
||||
}
|
||||
|
||||
// JobCard re-renders on every job-list poll, so `editingJob` is a fresh
|
||||
// object each time. Effects must depend on these, never on the object.
|
||||
const editingJobId = editingJob?.id ?? null;
|
||||
const editingJobModelId = editingJob?.model_id ?? null;
|
||||
const seedJob = editingJob ?? prefillJob ?? null;
|
||||
const seedJobId = seedJob?.id ?? null;
|
||||
const seedJobModelId = seedJob?.model_id ?? null;
|
||||
const isPrefill = !editingJob && !!prefillJob;
|
||||
|
||||
// Layerwise offload and FSDP compete for the DiT weights and FastVideoArgs
|
||||
// silently picks a winner (fastvideo_args.py:859); resolve it visibly here.
|
||||
@@ -204,10 +291,10 @@ export default function CreateJobModal({
|
||||
const justOpened = isOpen && !justOpenedRef.current;
|
||||
justOpenedRef.current = isOpen;
|
||||
if (!justOpened) return;
|
||||
if (editingJob) {
|
||||
if (seedJob) {
|
||||
// Must not fall through to the defaults below: a partially-seeded
|
||||
// form silently edits values the user never saw.
|
||||
const f = jobToFormFields(editingJob);
|
||||
const f = jobToFormFields(seedJob);
|
||||
setModelId(f.modelId);
|
||||
setName(f.name);
|
||||
setPrompt(f.prompt);
|
||||
@@ -296,7 +383,7 @@ export default function CreateJobModal({
|
||||
setRealScoreModelPath('');
|
||||
setFakeScoreModelPath('');
|
||||
}
|
||||
}, [isOpen, workloadType, inferenceWorkload, options, editingJobId]);
|
||||
}, [isOpen, workloadType, inferenceWorkload, options, seedJobId]);
|
||||
|
||||
// Load the models available for this workload.
|
||||
React.useEffect(() => {
|
||||
@@ -319,7 +406,7 @@ export default function CreateJobModal({
|
||||
// When editing, the job's own model wins over the workload default --
|
||||
// this resolves after the seeding effect, so choosing a default here
|
||||
// would silently swap the model out from under the user.
|
||||
const editedId = editingJobModelId;
|
||||
const editedId = seedJobModelId;
|
||||
const chosen =
|
||||
editedId && ids.includes(editedId)
|
||||
? editedId
|
||||
@@ -327,6 +414,11 @@ export default function CreateJobModal({
|
||||
? defaultId
|
||||
: (list[0]?.id ?? '');
|
||||
setModelId(chosen);
|
||||
if (isPrefill && editedId && chosen !== editedId) {
|
||||
toast.warning(
|
||||
`${editedId} isn't available for ${inferenceWorkload.toUpperCase()}${chosen ? `, so ${chosen} is selected instead` : ''}.`,
|
||||
);
|
||||
}
|
||||
if (workloadType === 'dmd_t2v') {
|
||||
setRealScoreModelPath(chosen);
|
||||
setFakeScoreModelPath(chosen);
|
||||
@@ -347,7 +439,7 @@ export default function CreateJobModal({
|
||||
return () => {
|
||||
stale = true;
|
||||
};
|
||||
}, [isOpen, inferenceWorkload, workloadType, editingJobModelId]);
|
||||
}, [isOpen, inferenceWorkload, workloadType, seedJobModelId, isPrefill]);
|
||||
|
||||
// Training jobs need a dataset; load the ready datasets when relevant.
|
||||
React.useEffect(() => {
|
||||
@@ -857,6 +949,22 @@ export default function CreateJobModal({
|
||||
</FieldRow>
|
||||
)}
|
||||
|
||||
{isH3Model && isInference && workloadType !== 't2i' && (
|
||||
<div className="flex flex-col gap-2">
|
||||
<H3TagConsole onInsert={insertH3Tag} disabled={isSubmitting} />
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="self-start"
|
||||
onClick={() => setDialogueEditorOpen(true)}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
Open dialogue editor
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{usingReferences && useGuidedPrompt ? (
|
||||
/* Six-section format from the model's reference prompt guide. */
|
||||
<>
|
||||
@@ -868,12 +976,31 @@ export default function CreateJobModal({
|
||||
>
|
||||
<Textarea
|
||||
id={`modal-prompt-${section}`}
|
||||
ref={(el) => {
|
||||
tagFieldRefs.current[section] = el;
|
||||
}}
|
||||
value={promptFields[section]}
|
||||
onChange={(e) => setPromptField(section, e.target.value)}
|
||||
onFocus={() => setActiveTagField(section)}
|
||||
rows={section === 'detailed_description' ? 5 : 2}
|
||||
placeholder={H3_SECTION_HINTS[section]}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
{section === 'detailed_description' && (
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="self-start"
|
||||
onClick={() => {
|
||||
setShotListMode('sections');
|
||||
setShotListOpen(true);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
Open shot list
|
||||
</Button>
|
||||
)}
|
||||
</FieldRow>
|
||||
))}
|
||||
<button
|
||||
@@ -893,8 +1020,12 @@ export default function CreateJobModal({
|
||||
>
|
||||
<Textarea
|
||||
id="modal-prompt"
|
||||
ref={(el) => {
|
||||
tagFieldRefs.current.prompt = el;
|
||||
}}
|
||||
value={prompt}
|
||||
onChange={(e) => setPrompt(e.target.value)}
|
||||
onFocus={() => setActiveTagField('prompt')}
|
||||
rows={isInference ? 3 : 2}
|
||||
placeholder={
|
||||
isInference
|
||||
@@ -904,6 +1035,21 @@ export default function CreateJobModal({
|
||||
required={!(usingReferences && useGuidedPrompt)}
|
||||
disabled={isSubmitting}
|
||||
/>
|
||||
{isH3Model && isInference && workloadType !== 't2i' && (
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
className="self-start"
|
||||
onClick={() => {
|
||||
setShotListMode('prompt');
|
||||
setShotListOpen(true);
|
||||
}}
|
||||
disabled={isSubmitting}
|
||||
>
|
||||
Open shot list
|
||||
</Button>
|
||||
)}
|
||||
</FieldRow>
|
||||
{usingReferences && (
|
||||
<button
|
||||
@@ -1379,6 +1525,38 @@ export default function CreateJobModal({
|
||||
</Button>
|
||||
</div>
|
||||
</form>
|
||||
|
||||
{/* Nested in the React tree (still portaled to <body>) so Radix treats
|
||||
clicks and focus inside these as inside this dialog. As siblings
|
||||
they counted as "outside", and closing one could close this too. */}
|
||||
<H3DialogueEditor
|
||||
open={dialogueEditorOpen}
|
||||
initialValue={getTagFieldValue(dialogueEditorField)}
|
||||
onApply={(value) => setTagFieldValue(dialogueEditorField, value)}
|
||||
onClose={() => setDialogueEditorOpen(false)}
|
||||
/>
|
||||
<H3ShotListEditor
|
||||
open={shotListOpen}
|
||||
subjectDefinitions={shotListMode === 'sections' ? promptFields.subject_definitions : promptSubjects}
|
||||
initialDetailedDescription={
|
||||
shotListMode === 'sections' ? promptFields.detailed_description : promptShots.body
|
||||
}
|
||||
initialRetentionAnalysis={
|
||||
shotListMode === 'sections' ? promptFields.retention_analysis : (promptRetention ?? '')
|
||||
}
|
||||
targetLabel={
|
||||
shotListMode === 'sections'
|
||||
? undefined
|
||||
: promptShots.field === 'integrated_multimodal_description'
|
||||
? 'integrated_multimodal_description'
|
||||
: promptShots.field === 'detailed_description'
|
||||
? 'Detailed description'
|
||||
: 'the prompt'
|
||||
}
|
||||
withRetention={shotListMode === 'sections' || promptSyncsRetention}
|
||||
onApply={shotListMode === 'sections' ? applyShotList : applyShotListToPrompt}
|
||||
onClose={() => setShotListOpen(false)}
|
||||
/>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
|
||||
@@ -0,0 +1,346 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { H3DialoguePreview } from '@/components/jobs/H3DialoguePreview';
|
||||
import { H3TagPalette } from '@/components/jobs/H3TagPalette';
|
||||
import { useUndoableText } from '@/hooks/useUndoableText';
|
||||
import { applyH3Preset, applyH3Tag, type H3Preset, type H3Tag } from '@/lib/h3Tags';
|
||||
import {
|
||||
SPEECH_RATES,
|
||||
dialogueTree,
|
||||
estimateDuration,
|
||||
type DialogueNode,
|
||||
type SpeechRate,
|
||||
} from '@/lib/h3DialogueEstimate';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface H3DialogueEditorProps {
|
||||
open: boolean;
|
||||
/** Seeded from whichever prompt field the editor was opened for. */
|
||||
initialValue: string;
|
||||
onApply: (value: string) => void;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
/** Every `<d>...</d>` block's inner text -- what's actually spoken, ignoring surrounding shot/camera prose. */
|
||||
function extractDialogueBlocks(text: string): string[] {
|
||||
const blocks: string[] = [];
|
||||
const re = /<d>([\s\S]*?)<\/d>/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(text))) blocks.push(m[1].replace(/^\s*\[[^\]]*\]\s*/, '')); // drop a leading [Language] marker
|
||||
return blocks;
|
||||
}
|
||||
|
||||
/**
|
||||
* Full-screen dialogue editor for MiniMax-H3 speech tags: the same tag
|
||||
* registry as `H3TagConsole`, plus a duration estimate (scoped to `<d>`
|
||||
* blocks so scene-description prose doesn't inflate it), a styled
|
||||
* performance preview, and emotion presets. Opens with a copy of the target
|
||||
* field's text; `onApply` hands the edited text back, `onClose` discards it.
|
||||
* The shot list editor reuses the same palette, preview, and undo history.
|
||||
*/
|
||||
export function H3DialogueEditor({ open, initialValue, onApply, onClose }: H3DialogueEditorProps) {
|
||||
const { text, type, commit, undo, redo, reset, canUndo, canRedo } = useUndoableText(initialValue);
|
||||
const [pace, setPace] = React.useState<SpeechRate>('Natural');
|
||||
const [calib, setCalib] = React.useState(1);
|
||||
const [actualSeconds, setActualSeconds] = React.useState('');
|
||||
const taRef = React.useRef<HTMLTextAreaElement>(null);
|
||||
const pending = React.useRef<[number, number] | null>(null);
|
||||
|
||||
function restoreCaretAtEnd(restored: string | null) {
|
||||
if (restored != null) pending.current = [restored.length, restored.length];
|
||||
}
|
||||
|
||||
// Re-seed from the field's current value each time the editor opens, the
|
||||
// same "seed on the open transition only" guard CreateJobModal itself
|
||||
// uses -- otherwise an in-progress edit here could be clobbered by a
|
||||
// stray parent re-render while open. History resets with it: undo
|
||||
// shouldn't reach back into a previous time this editor was opened.
|
||||
const wasOpenRef = React.useRef(false);
|
||||
React.useEffect(() => {
|
||||
if (open && !wasOpenRef.current) {
|
||||
reset(initialValue);
|
||||
setActualSeconds('');
|
||||
setCalib(1);
|
||||
}
|
||||
wasOpenRef.current = open;
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [open]);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (pending.current && taRef.current) {
|
||||
const [s, e] = pending.current;
|
||||
taRef.current.focus();
|
||||
taRef.current.setSelectionRange(s, e);
|
||||
pending.current = null;
|
||||
}
|
||||
});
|
||||
|
||||
function handleKeyDown(e: React.KeyboardEvent<HTMLTextAreaElement>) {
|
||||
if (!(e.metaKey || e.ctrlKey) || e.key.toLowerCase() !== 'z') return;
|
||||
e.preventDefault();
|
||||
restoreCaretAtEnd(e.shiftKey ? redo() : undo());
|
||||
}
|
||||
|
||||
function insertTag(tag: H3Tag) {
|
||||
const ta = taRef.current;
|
||||
const result = applyH3Tag(text, ta?.selectionStart ?? text.length, ta?.selectionEnd ?? text.length, tag);
|
||||
commit(result.value);
|
||||
pending.current = [result.start, result.end];
|
||||
}
|
||||
|
||||
function applyPreset(p: H3Preset) {
|
||||
const ta = taRef.current;
|
||||
const result = applyH3Preset(text, ta?.selectionStart ?? text.length, ta?.selectionEnd ?? text.length, p);
|
||||
commit(result.value);
|
||||
pending.current = [result.start, result.end];
|
||||
}
|
||||
|
||||
const dialogueBlocks = React.useMemo(() => extractDialogueBlocks(text), [text]);
|
||||
// Falls back to the whole field only when it contains no <d> blocks at
|
||||
// all yet -- e.g. a fresh field the user is about to write dialogue into.
|
||||
const segments = dialogueBlocks.length > 0 ? dialogueBlocks : [text];
|
||||
const estimates = React.useMemo(
|
||||
() => segments.map((s) => estimateDuration(s, SPEECH_RATES[pace], calib)),
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
[segments.join('\n'), pace, calib],
|
||||
);
|
||||
const totalSeconds = estimates.reduce((sum, e) => sum + e.total, 0);
|
||||
|
||||
function calibrate() {
|
||||
const actual = parseFloat(actualSeconds);
|
||||
if (!actual || actual <= 0 || segments.length === 0) return;
|
||||
const base = segments.reduce((sum, s) => sum + estimateDuration(s, SPEECH_RATES[pace], 1).total, 0);
|
||||
if (base > 0) setCalib(actual / base);
|
||||
}
|
||||
|
||||
/* ---- rough browser-voice preview, for rhythm/pacing only ---- */
|
||||
const synth = typeof window !== 'undefined' ? window.speechSynthesis : null;
|
||||
const [playing, setPlaying] = React.useState(false);
|
||||
const playTokenRef = React.useRef({ cancelled: true });
|
||||
|
||||
function stopPreview() {
|
||||
playTokenRef.current.cancelled = true;
|
||||
synth?.cancel();
|
||||
setPlaying(false);
|
||||
}
|
||||
|
||||
function playPreview() {
|
||||
if (!synth) return;
|
||||
stopPreview();
|
||||
const token = { cancelled: false };
|
||||
playTokenRef.current = token;
|
||||
setPlaying(true);
|
||||
const VOX: Record<string, string> = {
|
||||
laughs: 'ha ha ha', chuckle: 'heh', sighs: 'haah', gasp: 'ah', uh: 'uh', mhm: 'mhm', phew: 'phew',
|
||||
};
|
||||
const atoms: { say?: string; gap?: number }[] = [];
|
||||
for (const [i, seg] of segments.entries()) {
|
||||
if (i > 0) atoms.push({ gap: 0.6 });
|
||||
const walk = (nodes: DialogueNode[]) => {
|
||||
for (const n of nodes) {
|
||||
if (n.t === 'text') {
|
||||
const t = n.v.replace(/\s+/g, ' ').trim();
|
||||
if (t) atoms.push({ say: t });
|
||||
} else if (n.t === 'point') {
|
||||
atoms.push(VOX[n.name] ? { say: VOX[n.name] } : { gap: 0.25 });
|
||||
} else {
|
||||
if (n.name === 'humming') atoms.push({ say: 'hmm hmm' });
|
||||
else walk(n.children);
|
||||
}
|
||||
}
|
||||
};
|
||||
walk(dialogueTree(seg));
|
||||
}
|
||||
let i = 0;
|
||||
const step = () => {
|
||||
if (token.cancelled) return;
|
||||
if (i >= atoms.length) {
|
||||
setPlaying(false);
|
||||
return;
|
||||
}
|
||||
const a = atoms[i++];
|
||||
if (a.gap != null) {
|
||||
setTimeout(() => { if (!token.cancelled) step(); }, a.gap * 1000);
|
||||
return;
|
||||
}
|
||||
const u = new SpeechSynthesisUtterance(a.say);
|
||||
u.rate = 0.95;
|
||||
u.onend = () => { if (!token.cancelled) step(); };
|
||||
u.onerror = () => { if (!token.cancelled) step(); };
|
||||
synth.speak(u);
|
||||
};
|
||||
step();
|
||||
}
|
||||
|
||||
React.useEffect(() => () => stopPreview(), []); // eslint-disable-line react-hooks/exhaustive-deps
|
||||
|
||||
function handleApply() {
|
||||
onApply(text);
|
||||
onClose();
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(o) => { if (!o) onClose(); }}>
|
||||
<DialogContent className="flex h-[92vh] max-h-[920px] w-[96vw] max-w-[1180px] flex-col gap-0 overflow-hidden p-0">
|
||||
<DialogHeader className="border-b border-border px-5 py-4">
|
||||
<DialogTitle>Dialogue editor</DialogTitle>
|
||||
<DialogDescription>
|
||||
Click a tag to insert it at the cursor, or select text first to wrap it. Applying writes the edited text
|
||||
back into the field you opened this from.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
<div className="flex min-h-0 flex-1">
|
||||
{/* palette */}
|
||||
<H3TagPalette
|
||||
onInsertTag={insertTag}
|
||||
onApplyPreset={applyPreset}
|
||||
className="w-64 shrink-0 overflow-y-auto border-r border-border p-3"
|
||||
/>
|
||||
|
||||
{/* editor + preview */}
|
||||
<div className="flex min-w-0 flex-1 flex-col gap-4 overflow-y-auto p-4">
|
||||
{/* duration estimate */}
|
||||
<div className="rounded-xl border border-border bg-secondary/30 p-4">
|
||||
<div className="flex flex-wrap items-baseline justify-between gap-3">
|
||||
<div className="flex items-baseline gap-3">
|
||||
<span className="text-2xl font-semibold tracking-tight text-foreground">
|
||||
{totalSeconds.toFixed(1)}<span className="ml-1 text-sm font-normal text-muted-foreground">s spoken</span>
|
||||
</span>
|
||||
<span className="font-mono text-xs text-muted-foreground">
|
||||
{dialogueBlocks.length > 0
|
||||
? `across ${dialogueBlocks.length} <d> block${dialogueBlocks.length === 1 ? '' : 's'}`
|
||||
: 'no <d> blocks yet — estimating the whole field'}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex items-center gap-1 rounded-lg border border-border bg-card p-1">
|
||||
{(Object.keys(SPEECH_RATES) as SpeechRate[]).map((p) => (
|
||||
<button
|
||||
key={p}
|
||||
type="button"
|
||||
onClick={() => setPace(p)}
|
||||
className={cn(
|
||||
'rounded-md px-2.5 py-1 text-xs transition-colors',
|
||||
pace === p ? 'bg-secondary text-foreground' : 'text-muted-foreground hover:bg-accent',
|
||||
)}
|
||||
>
|
||||
{p}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{segments.length > 1 && (
|
||||
<div className="mt-3 flex flex-wrap gap-1.5">
|
||||
{estimates.map((e, i) => (
|
||||
<span key={i} className="rounded-md border border-border bg-card px-2 py-0.5 font-mono text-[11px] text-muted-foreground">
|
||||
block {i + 1}: {e.total.toFixed(1)}s
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="mt-3 flex flex-wrap items-center gap-2 text-xs text-muted-foreground">
|
||||
<span title="Enter the real length of a clip you already generated, to tune the estimate to your setup">
|
||||
calibrate
|
||||
</span>
|
||||
<input
|
||||
value={actualSeconds}
|
||||
onChange={(e) => setActualSeconds(e.target.value)}
|
||||
placeholder="actual s"
|
||||
inputMode="decimal"
|
||||
className="w-16 rounded-md border border-input bg-card px-2 py-1 text-foreground"
|
||||
/>
|
||||
<Button type="button" variant="outline" size="sm" onClick={calibrate}>Set</Button>
|
||||
{Math.abs(calib - 1) > 0.001 && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => { setCalib(1); setActualSeconds(''); }}
|
||||
className="font-mono text-muted-foreground/80 hover:text-foreground"
|
||||
title="Reset calibration"
|
||||
>
|
||||
×{calib.toFixed(2)}
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* audio preview */}
|
||||
{synth && (
|
||||
<div className="flex items-center gap-3 rounded-xl border border-border bg-secondary/30 p-3">
|
||||
<Button type="button" size="sm" onClick={playing ? stopPreview : playPreview}>
|
||||
{playing ? '■ Stop' : '▶ Play preview'}
|
||||
</Button>
|
||||
<span className="text-xs text-muted-foreground">
|
||||
A stand-in browser voice for rhythm only — it won't match MiniMax's voice.
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* the field itself */}
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-xs font-semibold text-foreground">Text</span>
|
||||
<div className="flex items-center gap-1">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => restoreCaretAtEnd(undo())}
|
||||
disabled={!canUndo}
|
||||
title="Undo (⌘/Ctrl+Z)"
|
||||
>
|
||||
↶ Undo
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => restoreCaretAtEnd(redo())}
|
||||
disabled={!canRedo}
|
||||
title="Redo (⌘/Ctrl+Shift+Z)"
|
||||
>
|
||||
↷ Redo
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
<Textarea
|
||||
ref={taRef}
|
||||
value={text}
|
||||
onChange={(e) => type(e.target.value)}
|
||||
onKeyDown={handleKeyDown}
|
||||
spellCheck={false}
|
||||
rows={8}
|
||||
className="flex-1 font-mono text-sm"
|
||||
/>
|
||||
|
||||
{/* performance preview */}
|
||||
<div>
|
||||
<div className="mb-2 text-xs font-semibold text-foreground">Performance preview</div>
|
||||
<H3DialoguePreview
|
||||
text={text}
|
||||
className="rounded-xl border border-border bg-card p-4 text-sm leading-7 text-foreground"
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<DialogFooter className="border-t border-border px-5 py-3">
|
||||
<Button type="button" variant="outline" onClick={onClose}>Cancel</Button>
|
||||
<Button type="button" onClick={handleApply}>Apply to field</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,44 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { CATEGORY_STYLE } from '@/components/jobs/H3TagPalette';
|
||||
import { dialogueTree, type DialogueNode } from '@/lib/h3DialogueEstimate';
|
||||
import { H3_TAG_CATEGORIES } from '@/lib/h3Tags';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const CATEGORY_BY_TAG_NAME: Record<string, string> = Object.fromEntries(
|
||||
H3_TAG_CATEGORIES.flatMap((c) => c.tags.map((t) => [t.open.replace(/[<>]/g, ''), c.id])),
|
||||
);
|
||||
|
||||
function renderPreviewNodes(nodes: DialogueNode[], keyPrefix = ''): React.ReactNode[] {
|
||||
return nodes.map((n, i) => {
|
||||
const key = `${keyPrefix}${i}`;
|
||||
if (n.t === 'text') return <span key={key}>{n.v}</span>;
|
||||
if (n.t === 'point') {
|
||||
const cat = CATEGORY_BY_TAG_NAME[n.name] ?? 'body';
|
||||
const style = CATEGORY_STYLE[cat] ?? CATEGORY_STYLE.body;
|
||||
return (
|
||||
<span
|
||||
key={key}
|
||||
className={cn(
|
||||
'mx-0.5 inline-block rounded border px-1.5 py-0.5 font-mono text-[11px] leading-none',
|
||||
style.chip,
|
||||
)}
|
||||
>
|
||||
{n.name}
|
||||
</span>
|
||||
);
|
||||
}
|
||||
const inner = renderPreviewNodes(n.children, `${key}-`);
|
||||
if (n.name === 'i') return <strong key={key} className="italic">{inner}</strong>;
|
||||
if (n.name === 'whisper') return <span key={key} className="text-[0.92em] tracking-wide opacity-60">{inner}</span>;
|
||||
if (n.name === 'humming') return <em key={key} className="underline decoration-wavy decoration-emerald-500 underline-offset-4">{inner}</em>;
|
||||
return <span key={key}>{inner}</span>;
|
||||
});
|
||||
}
|
||||
|
||||
/** Styled "performance" rendering of speech tags: chips for point tags, styling for wrapping ones. */
|
||||
export function H3DialoguePreview({ text, className }: { text: string; className?: string }) {
|
||||
return <div className={className}>{renderPreviewNodes(dialogueTree(text))}</div>;
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
import { render, screen, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { H3DialogueEditor } from './H3DialogueEditor';
|
||||
import { H3ShotListEditor } from './H3ShotListEditor';
|
||||
|
||||
const SUBJECTS = '<Subject 1> is the courier.\n<Subject 2> is the rival.';
|
||||
const DETAILED = [
|
||||
'[Shot 1] Wide shot. A rooftop at night.',
|
||||
'[Shot 2] Close-up shot, handheld. He turns. (S2) <d>[English] I had no choice.</d>',
|
||||
].join('\n\n');
|
||||
|
||||
describe('H3DialogueEditor (shared palette + undo)', () => {
|
||||
it('inserts a tag at the cursor, undoes it, and applies the edited text', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = vi.fn();
|
||||
render(<H3DialogueEditor open initialValue="Hello there" onApply={onApply} onClose={() => {}} />);
|
||||
|
||||
const textarea = screen.getByDisplayValue('Hello there') as HTMLTextAreaElement;
|
||||
textarea.focus();
|
||||
textarea.setSelectionRange(5, 5);
|
||||
await user.click(screen.getByRole('button', { name: /<pause>/ }));
|
||||
expect(textarea.value).toBe('Hello <pause> there');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /Undo/ }));
|
||||
expect(textarea.value).toBe('Hello there');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Nervous' }));
|
||||
expect(textarea.value).toContain('<uh> <stutter>');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Apply to field' }));
|
||||
expect(onApply).toHaveBeenCalledWith(expect.stringContaining('<uh> <stutter>'));
|
||||
});
|
||||
});
|
||||
|
||||
describe('H3ShotListEditor dialogue tooling', () => {
|
||||
function renderEditor(onApply = vi.fn()) {
|
||||
render(
|
||||
<H3ShotListEditor
|
||||
open
|
||||
subjectDefinitions={SUBJECTS}
|
||||
initialDetailedDescription={DETAILED}
|
||||
initialRetentionAnalysis=""
|
||||
onApply={onApply}
|
||||
onClose={() => {}}
|
||||
/>,
|
||||
);
|
||||
return onApply;
|
||||
}
|
||||
|
||||
it('disables the palette until a shot dialogue is focused', () => {
|
||||
renderEditor();
|
||||
expect(screen.getByText(/Click into a shot's dialogue/)).toBeInTheDocument();
|
||||
expect(screen.getByRole('button', { name: /<pause>/ })).toBeDisabled();
|
||||
});
|
||||
|
||||
it('inserts a palette tag into the focused shot dialogue and writes it on apply', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderEditor();
|
||||
|
||||
const dialogueBoxes = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
expect(dialogueBoxes).toHaveLength(1);
|
||||
expect(dialogueBoxes[0].value).toBe('I had no choice.');
|
||||
|
||||
await user.click(dialogueBoxes[0]);
|
||||
dialogueBoxes[0].setSelectionRange(1, 1);
|
||||
expect(screen.getByText('Tags insert into Shot 2 dialogue')).toBeInTheDocument();
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /<pause>/ }));
|
||||
expect(dialogueBoxes[0].value).toBe('I <pause> had no choice.');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply).toHaveBeenCalledTimes(1);
|
||||
expect(onApply.mock.calls[0][0]).toContain('<d>[English] I <pause> had no choice.</d>');
|
||||
});
|
||||
|
||||
it('adds and removes dialogue lines, writing every line on apply', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderEditor();
|
||||
|
||||
await user.click(screen.getAllByRole('button', { name: '+ Add line' })[1]);
|
||||
const boxes = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
await user.type(boxes[1], 'Then go.');
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toContain(
|
||||
'(S2) <d>[English] I had no choice.</d> (S2) <d>[English] Then go.</d>',
|
||||
);
|
||||
});
|
||||
|
||||
it('shows the styled preview once a shot dialogue contains tags', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
const boxes = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
await user.click(boxes[0]);
|
||||
await user.click(screen.getByRole('button', { name: /<long pause>/ }));
|
||||
|
||||
const shot2 = boxes[0].closest('[id^="shot-card-"]') as HTMLElement;
|
||||
expect(within(shot2).getByText('long pause')).toBeInTheDocument();
|
||||
});
|
||||
});
|
||||
|
||||
describe('H3ShotListEditor ordered body', () => {
|
||||
function renderWith(detailed: string) {
|
||||
const onApply = vi.fn();
|
||||
render(
|
||||
<H3ShotListEditor
|
||||
open
|
||||
subjectDefinitions={SUBJECTS}
|
||||
initialDetailedDescription={detailed}
|
||||
initialRetentionAnalysis=""
|
||||
onApply={onApply}
|
||||
onClose={() => {}}
|
||||
/>,
|
||||
);
|
||||
return onApply;
|
||||
}
|
||||
|
||||
it('shows hand-written dialogue as an editable line between its prose and applies the text unchanged', async () => {
|
||||
const user = userEvent.setup();
|
||||
const text = "[Shot 1] He (S1) says: <d>[English] I think you're sorry.</d> <cutoff>";
|
||||
const onApply = renderWith(text);
|
||||
|
||||
expect(screen.getByText(/keeps its original wording/)).toBeInTheDocument();
|
||||
const lines = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
expect(lines).toHaveLength(1);
|
||||
expect(lines[0].value).toBe("I think you're sorry.");
|
||||
const prose = screen.getAllByPlaceholderText("What's happening in this shot") as HTMLTextAreaElement[];
|
||||
expect(prose.map((p) => p.value)).toEqual(['He (S1) says: ', ' <cutoff>']);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe(text);
|
||||
});
|
||||
|
||||
it('editing a hand-written line keeps the tag after it where it was', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith("[Shot 1] He (S1) says: <d>[English] Sorry.</d> <cutoff>");
|
||||
const [line] = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
await user.type(line, ' Really.');
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe("[Shot 1] He (S1) says: <d>[English] Sorry. Really.</d> <cutoff>");
|
||||
});
|
||||
|
||||
it('reorders dialogue lines with the move buttons', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith('[Shot 1] Wide shot. (S1) <d>[English] One.</d> (S2) <d>[English] Two.</d>');
|
||||
await user.click(screen.getByRole('button', { name: 'Move line 2 up' }));
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe(
|
||||
'[Shot 1] Wide shot. (S2) <d>[English] Two.</d> (S1) <d>[English] One.</d>',
|
||||
);
|
||||
});
|
||||
|
||||
it('adds a text block after the existing content', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith('[Shot 1] Wide shot. A dock.');
|
||||
await user.click(screen.getByRole('button', { name: '+ Add text' }));
|
||||
const prose = screen.getAllByPlaceholderText("What's happening in this shot") as HTMLTextAreaElement[];
|
||||
expect(prose).toHaveLength(2);
|
||||
await user.type(prose[1], 'Rain falls.');
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe('[Shot 1] Wide shot. A dock. Rain falls.');
|
||||
});
|
||||
|
||||
it('removes a text block', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith('[Shot 1] Wide shot. A dock. (S1) <d>[English] Hi.</d>');
|
||||
await user.click(screen.getByRole('button', { name: 'Remove text 1' }));
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe('[Shot 1] Wide shot. (S1) <d>[English] Hi.</d>');
|
||||
});
|
||||
|
||||
it('adds a new line at the end of the shot, after any closing prose', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderWith('[Shot 1] Wide shot. He speaks. (S1) <d>[English] First.</d> He pauses.');
|
||||
await user.click(screen.getByRole('button', { name: '+ Add line' }));
|
||||
const lines = screen.getAllByPlaceholderText('What the speaker says') as HTMLTextAreaElement[];
|
||||
await user.type(lines[1], 'Later.');
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
expect(onApply.mock.calls[0][0]).toBe(
|
||||
'[Shot 1] Wide shot. He speaks. (S1) <d>[English] First.</d> He pauses. (S1) <d>[English] Later.</d>',
|
||||
);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,123 @@
|
||||
import { render, screen, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { H3ShotListEditor } from './H3ShotListEditor';
|
||||
|
||||
// Original test material -- not from any real screenplay.
|
||||
const SCRIPT = [
|
||||
'CAPTAIN: Sit down. (EXHALES) We need to talk about the ledger.',
|
||||
'',
|
||||
'CLERK: Yes, sir.',
|
||||
'',
|
||||
'CAPTAIN: Three crates are missing and nobody on this dock saw a thing.',
|
||||
].join('\n');
|
||||
|
||||
const SUBJECTS = '<Subject 1> is the captain.\n<Subject 2> is the clerk.';
|
||||
|
||||
function renderEditor(detailed = '', onApply = vi.fn()) {
|
||||
render(
|
||||
<H3ShotListEditor
|
||||
open
|
||||
subjectDefinitions={SUBJECTS}
|
||||
initialDetailedDescription={detailed}
|
||||
initialRetentionAnalysis=""
|
||||
onApply={onApply}
|
||||
onClose={() => {}}
|
||||
/>,
|
||||
);
|
||||
return onApply;
|
||||
}
|
||||
|
||||
async function importScript(user: ReturnType<typeof userEvent.setup>) {
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
const dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
await user.click(within(dialog).getByLabelText('Script'));
|
||||
await user.paste(SCRIPT);
|
||||
return dialog;
|
||||
}
|
||||
|
||||
describe('script import', () => {
|
||||
it('maps speakers to subjects by default and previews the clip', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
const dialog = await importScript(user);
|
||||
|
||||
expect(within(dialog).getByLabelText('CAPTAIN subject')).toHaveValue('<Subject 1>');
|
||||
expect(within(dialog).getByLabelText('CLERK subject')).toHaveValue('<Subject 2>');
|
||||
expect(within(dialog).getByLabelText('CAPTAIN speaker tag')).toHaveValue('S1');
|
||||
expect(within(dialog).getByLabelText('CLERK speaker tag')).toHaveValue('S2');
|
||||
expect(within(dialog).getByRole('option', { name: /Clip 1 of \d+/ })).toBeInTheDocument();
|
||||
expect(within(dialog).getByText('Establishing')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('loads the chosen clip into the shot list and writes it on apply', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onApply = renderEditor();
|
||||
const dialog = await importScript(user);
|
||||
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
|
||||
const boxes = (await screen.findAllByPlaceholderText('What the speaker says')) as HTMLTextAreaElement[];
|
||||
expect(boxes[0].value).toBe('Sit down. <exhale> We need to talk about the ledger.');
|
||||
expect(boxes[1].value).toBe('Yes, sir.');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
const [detailed, retention] = onApply.mock.calls[0];
|
||||
expect(detailed).toContain('[Shot 1] Establishing shot.');
|
||||
expect(detailed).toContain('(S1) <d>[English] Sit down. <exhale> We need to talk about the ledger.</d> (S2) <d>[English] Yes, sir.</d>');
|
||||
expect(retention).toContain('<Subject 1> (appears in');
|
||||
expect(retention).toContain('<Subject 2> (appears in');
|
||||
});
|
||||
|
||||
it('warns that loading replaces the shots already in the list', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor('[Shot 1] Wide shot. A dock.\n\n[Shot 2] Close-up shot. A face.');
|
||||
const dialog = await importScript(user);
|
||||
expect(within(dialog).getByText('Replaces the 2 shots currently in the list.')).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('reloading the same clip after edits restores the imported text', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
let dialog = await importScript(user);
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
|
||||
let boxes = (await screen.findAllByPlaceholderText('What the speaker says')) as HTMLTextAreaElement[];
|
||||
await user.type(boxes[1], ' EDITED');
|
||||
expect(boxes[1].value).toBe('Yes, sir. EDITED');
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
// Loading a clip advances the picker to the next one; choose clip 1 again.
|
||||
await user.selectOptions(within(dialog).getByLabelText('Clip to load'), '0');
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
|
||||
boxes = (await screen.findAllByPlaceholderText('What the speaker says')) as HTMLTextAreaElement[];
|
||||
expect(boxes[1].value).toBe('Yes, sir.');
|
||||
});
|
||||
|
||||
it('advances to the next clip after loading one', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
let dialog = await importScript(user);
|
||||
const picker = within(dialog).getByLabelText('Clip to load') as HTMLSelectElement;
|
||||
expect(picker.options.length).toBeGreaterThan(1);
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
expect((within(dialog).getByLabelText('Clip to load') as HTMLSelectElement).value).toBe('1');
|
||||
});
|
||||
|
||||
it('explains when the text has no speaker turns', async () => {
|
||||
const user = userEvent.setup();
|
||||
renderEditor();
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
const dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
await user.click(within(dialog).getByLabelText('Script'));
|
||||
await user.paste('just some prose');
|
||||
expect(within(dialog).getByRole('alert')).toHaveTextContent(/No "NAME: line" turns found/);
|
||||
expect(within(dialog).getByRole('button', { name: 'Load clip' })).toBeDisabled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,247 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { buildClips, parseScript, scriptSpeakers, type ImportSpeaker } from '@/lib/h3ScriptImport';
|
||||
import { SHOT_TYPES, shotLines, shotProse, type Shot } from '@/lib/h3Shots';
|
||||
|
||||
const TYPE_LABEL = Object.fromEntries(SHOT_TYPES.map((t) => [t.id, t.label]));
|
||||
const DEFAULT_CLIP_SECONDS = 11;
|
||||
|
||||
export interface H3ScriptImportDialogProps {
|
||||
open: boolean;
|
||||
subjectLabels: string[];
|
||||
/** shots currently in the list -- loading a clip replaces them */
|
||||
existingShotCount: number;
|
||||
onLoad: (shots: Shot[]) => void;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
function preview(shot: Shot): string {
|
||||
const line = shotLines(shot).find((l) => l.text.trim());
|
||||
const text = line ? `${line.speaker ? `${line.speaker}: ` : ''}“${line.text.trim()}”` : shotProse(shot);
|
||||
return text.length > 90 ? `${text.slice(0, 89)}…` : text;
|
||||
}
|
||||
|
||||
/**
|
||||
* Paste (or load) a "NAME: line" script and pick one clip-sized chunk of it
|
||||
* to load into the shot list. The script and speaker mapping persist between
|
||||
* openings, so working through a long scene is: load clip, apply, next clip.
|
||||
*/
|
||||
export function H3ScriptImportDialog({
|
||||
open,
|
||||
subjectLabels,
|
||||
existingShotCount,
|
||||
onLoad,
|
||||
onClose,
|
||||
}: H3ScriptImportDialogProps) {
|
||||
const [script, setScript] = React.useState('');
|
||||
const [clipSecondsInput, setClipSecondsInput] = React.useState(String(DEFAULT_CLIP_SECONDS));
|
||||
const [overrides, setOverrides] = React.useState<Record<string, Partial<ImportSpeaker>>>({});
|
||||
const [clipIndex, setClipIndex] = React.useState(0);
|
||||
|
||||
const turns = React.useMemo(() => parseScript(script), [script]);
|
||||
const names = React.useMemo(() => scriptSpeakers(turns), [turns]);
|
||||
|
||||
const speakers = React.useMemo(() => {
|
||||
const map: Record<string, ImportSpeaker> = {};
|
||||
names.forEach((name, i) => {
|
||||
const o = overrides[name] ?? {};
|
||||
map[name] = {
|
||||
subject: 'subject' in o ? (o.subject ?? null) : (subjectLabels[i] ?? null),
|
||||
tag: o.tag ?? `S${i + 1}`,
|
||||
};
|
||||
});
|
||||
return map;
|
||||
}, [names, overrides, subjectLabels]);
|
||||
|
||||
const parsedSeconds = Number(clipSecondsInput);
|
||||
const targetSeconds = Number.isFinite(parsedSeconds) ? Math.min(Math.max(parsedSeconds, 4), 12) : DEFAULT_CLIP_SECONDS;
|
||||
|
||||
const clips = React.useMemo(
|
||||
() => (turns.length ? buildClips(turns, { speakers, targetSeconds }) : []),
|
||||
[turns, speakers, targetSeconds],
|
||||
);
|
||||
const selected = Math.min(clipIndex, Math.max(clips.length - 1, 0));
|
||||
const clip = clips[selected];
|
||||
|
||||
function setOverride(name: string, patch: Partial<ImportSpeaker>) {
|
||||
setOverrides((prev) => ({ ...prev, [name]: { ...prev[name], ...patch } }));
|
||||
}
|
||||
|
||||
async function handleFile(e: React.ChangeEvent<HTMLInputElement>) {
|
||||
const file = e.target.files?.[0];
|
||||
e.target.value = '';
|
||||
if (!file) return;
|
||||
setScript(await file.text());
|
||||
setClipIndex(0);
|
||||
}
|
||||
|
||||
function handleLoad() {
|
||||
if (!clip) return;
|
||||
onLoad(clip.shots);
|
||||
setClipIndex(Math.min(selected + 1, clips.length - 1));
|
||||
onClose();
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(o) => { if (!o) onClose(); }}>
|
||||
<DialogContent className="flex max-h-[90vh] w-[94vw] max-w-[760px] flex-col gap-0 overflow-hidden p-0">
|
||||
<DialogHeader className="border-b border-border px-5 py-4">
|
||||
<DialogTitle>Import script</DialogTitle>
|
||||
<DialogDescription>
|
||||
Paste a script of NAME: line turns. It's split into clip-sized chunks; load one at a time into the
|
||||
shot list, then set the camera and descriptions.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
<div className="flex min-h-0 flex-1 flex-col gap-4 overflow-y-auto p-5">
|
||||
<div className="flex flex-col gap-2">
|
||||
<label htmlFor="script-import-text" className="text-xs font-semibold text-foreground">
|
||||
Script
|
||||
</label>
|
||||
<Textarea
|
||||
id="script-import-text"
|
||||
value={script}
|
||||
onChange={(e) => {
|
||||
setScript(e.target.value);
|
||||
setClipIndex(0);
|
||||
}}
|
||||
rows={8}
|
||||
spellCheck={false}
|
||||
placeholder={'MARK: First line...\n\nJORDAN: Reply...'}
|
||||
className="font-mono text-xs"
|
||||
/>
|
||||
<Input
|
||||
type="file"
|
||||
accept=".md,.txt,.text"
|
||||
aria-label="Load script file"
|
||||
onChange={handleFile}
|
||||
className="h-auto py-2 file:mr-3 file:cursor-pointer file:rounded-md file:border-0 file:bg-secondary file:px-2 file:py-1 file:text-sm file:text-secondary-foreground"
|
||||
/>
|
||||
</div>
|
||||
|
||||
{script.trim() && turns.length === 0 && (
|
||||
<p role="alert" className="text-sm text-destructive">
|
||||
No "NAME: line" turns found. Each turn should start with a speaker name and a colon.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{names.length > 0 && (
|
||||
<div className="flex flex-col gap-2">
|
||||
<span className="text-xs font-semibold text-foreground">Speakers</span>
|
||||
{names.map((name) => (
|
||||
<div key={name} className="grid grid-cols-[1fr_1fr_5rem] items-center gap-2">
|
||||
<span className="truncate text-sm text-foreground">{name}</span>
|
||||
<NativeSelect
|
||||
aria-label={`${name} subject`}
|
||||
value={speakers[name].subject ?? ''}
|
||||
onChange={(e) => setOverride(name, { subject: e.target.value || null })}
|
||||
>
|
||||
<option value="">Not a tracked subject</option>
|
||||
{subjectLabels.map((label) => (
|
||||
<option key={label} value={label}>{label}</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
<Input
|
||||
aria-label={`${name} speaker tag`}
|
||||
value={speakers[name].tag}
|
||||
onChange={(e) => setOverride(name, { tag: e.target.value })}
|
||||
className="font-mono"
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
{subjectLabels.length === 0 && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
No subjects are defined yet, so speakers can't be linked. Add them under Subject definitions
|
||||
to track who appears in each shot.
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{turns.length > 0 && (
|
||||
<>
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Clip length target (seconds)
|
||||
<Input
|
||||
type="number"
|
||||
min={4}
|
||||
max={12}
|
||||
step={1}
|
||||
value={clipSecondsInput}
|
||||
onChange={(e) => {
|
||||
setClipSecondsInput(e.target.value);
|
||||
setClipIndex(0);
|
||||
}}
|
||||
className="w-28"
|
||||
/>
|
||||
<span>One generation runs about 11 s (277 frames at 24 fps), so each clip becomes its own job.</span>
|
||||
</label>
|
||||
|
||||
<div className="flex flex-col gap-2">
|
||||
<label htmlFor="script-import-clip" className="text-xs font-semibold text-foreground">
|
||||
Clip to load
|
||||
</label>
|
||||
<NativeSelect
|
||||
id="script-import-clip"
|
||||
value={selected}
|
||||
onChange={(e) => setClipIndex(Number(e.target.value))}
|
||||
>
|
||||
{clips.map((c, i) => (
|
||||
<option key={i} value={i}>
|
||||
Clip {i + 1} of {clips.length} · {c.shots.length} shot{c.shots.length === 1 ? '' : 's'}
|
||||
{c.continuation ? ' · continues previous shot' : ''}
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
|
||||
{clip?.continuation && (
|
||||
<p className="rounded-lg border border-border bg-secondary/30 px-3 py-2 text-xs text-muted-foreground">
|
||||
This clip continues the previous clip's last shot, cut short because that speech didn't
|
||||
fit one generation. Generate it from the previous clip's last frame (as the start Image)
|
||||
with the same references.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{clip && (
|
||||
<ol className="flex flex-col gap-1 rounded-xl border border-border bg-secondary/20 p-3">
|
||||
{clip.shots.map((shot, i) => (
|
||||
<li key={shot.id} className="flex gap-2 text-xs">
|
||||
<span className="w-6 shrink-0 font-mono text-muted-foreground">{i + 1}</span>
|
||||
<span className="w-24 shrink-0 font-medium text-foreground">{TYPE_LABEL[shot.type]}</span>
|
||||
<span className="min-w-0 text-muted-foreground">{preview(shot)}</span>
|
||||
</li>
|
||||
))}
|
||||
</ol>
|
||||
)}
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<DialogFooter className="items-center border-t border-border px-5 py-3">
|
||||
{existingShotCount > 0 && clip && (
|
||||
<span className="mr-auto text-xs text-muted-foreground">
|
||||
Replaces the {existingShotCount} shot{existingShotCount === 1 ? '' : 's'} currently in the list.
|
||||
</span>
|
||||
)}
|
||||
<Button type="button" variant="outline" onClick={onClose}>Cancel</Button>
|
||||
<Button type="button" onClick={handleLoad} disabled={!clip}>Load clip</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,92 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { H3DialoguePreview } from '@/components/jobs/H3DialoguePreview';
|
||||
import { applyH3Preset, applyH3Tag, type H3Preset, type H3Tag } from '@/lib/h3Tags';
|
||||
|
||||
/** What the shared tag palette drives: the last-focused shot dialogue field. */
|
||||
export interface H3ShotDialogueHandle {
|
||||
insertTag: (tag: H3Tag) => void;
|
||||
applyPreset: (preset: H3Preset) => void;
|
||||
}
|
||||
|
||||
export interface H3ShotDialogueFieldProps {
|
||||
/** The line's text. The parent owns it (and the undo history), so this is fully controlled. */
|
||||
value: string;
|
||||
/** Typing. */
|
||||
onChange: (value: string) => void;
|
||||
/** A tag or preset inserted from the palette -- a discrete edit, unlike typing. */
|
||||
onInsert: (value: string) => void;
|
||||
onFocus: () => void;
|
||||
active?: boolean;
|
||||
label?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* One spoken line: a textarea plus, once it contains tags, a styled preview.
|
||||
* The shared palette inserts at the cursor through the ref. Undo/redo isn't
|
||||
* here -- the shot list editor keeps one history for everything in it.
|
||||
*/
|
||||
export const H3ShotDialogueField = React.forwardRef<H3ShotDialogueHandle, H3ShotDialogueFieldProps>(
|
||||
function H3ShotDialogueField({ value, onChange, onInsert, onFocus, active, label = 'Dialogue' }, ref) {
|
||||
const taRef = React.useRef<HTMLTextAreaElement>(null);
|
||||
// Selection to restore once the inserted text has rendered.
|
||||
const pending = React.useRef<[number, number] | null>(null);
|
||||
|
||||
React.useEffect(() => {
|
||||
if (pending.current && taRef.current) {
|
||||
const [s, e] = pending.current;
|
||||
taRef.current.focus();
|
||||
taRef.current.setSelectionRange(s, e);
|
||||
pending.current = null;
|
||||
}
|
||||
});
|
||||
|
||||
function selection(): [number, number] {
|
||||
const ta = taRef.current;
|
||||
return [ta?.selectionStart ?? value.length, ta?.selectionEnd ?? value.length];
|
||||
}
|
||||
|
||||
React.useImperativeHandle(ref, () => ({
|
||||
insertTag(tag) {
|
||||
const [s, e] = selection();
|
||||
const result = applyH3Tag(value, s, e, tag);
|
||||
pending.current = [result.start, result.end];
|
||||
onInsert(result.value);
|
||||
},
|
||||
applyPreset(preset) {
|
||||
const [s, e] = selection();
|
||||
const result = applyH3Preset(value, s, e, preset);
|
||||
pending.current = [result.start, result.end];
|
||||
onInsert(result.value);
|
||||
},
|
||||
}));
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-1">
|
||||
<div className="text-xs text-muted-foreground">
|
||||
{label}
|
||||
{active && <span className="ml-2 text-accent-blue">tags insert here</span>}
|
||||
</div>
|
||||
<Textarea
|
||||
ref={taRef}
|
||||
value={value}
|
||||
onChange={(e) => onChange(e.target.value)}
|
||||
onFocus={onFocus}
|
||||
rows={2}
|
||||
spellCheck={false}
|
||||
placeholder="What the speaker says"
|
||||
className="font-mono text-sm"
|
||||
/>
|
||||
{value.includes('<') && (
|
||||
<H3DialoguePreview
|
||||
text={value}
|
||||
className="rounded-lg border border-border bg-card px-3 py-2 text-sm leading-7 text-foreground"
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
},
|
||||
);
|
||||
@@ -0,0 +1,690 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { Textarea } from '@/components/ui/textarea';
|
||||
import { H3ShotDialogueField, type H3ShotDialogueHandle } from '@/components/jobs/H3ShotDialogueField';
|
||||
import { H3ScriptImportDialog } from '@/components/jobs/H3ScriptImportDialog';
|
||||
import { H3ShotStoryboard } from '@/components/jobs/H3ShotStoryboard';
|
||||
import { H3TagPalette } from '@/components/jobs/H3TagPalette';
|
||||
import type { H3Preset, H3Tag } from '@/lib/h3Tags';
|
||||
import {
|
||||
CAMERA_MOVEMENTS,
|
||||
SHOT_TYPES,
|
||||
detailedDescriptionFromShots,
|
||||
formatCutTime,
|
||||
linePart,
|
||||
newLine,
|
||||
newShot,
|
||||
newTextPart,
|
||||
parseCutTime,
|
||||
parseDetailedDescription,
|
||||
parseSubjectLabels,
|
||||
resolveShotStarts,
|
||||
retentionAnalysisFromShots,
|
||||
shotLines,
|
||||
withFreshIds,
|
||||
type CameraMovement,
|
||||
type DialogueLine,
|
||||
type LinePart,
|
||||
type ShotPart,
|
||||
type Shot,
|
||||
type ShotType,
|
||||
} from '@/lib/h3Shots';
|
||||
import { useUndoableState } from '@/hooks/useUndoableState';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface H3ShotListEditorProps {
|
||||
open: boolean;
|
||||
/** Raw subject_definitions text -- parsed for <Subject N> labels to track per shot. */
|
||||
subjectDefinitions: string;
|
||||
initialDetailedDescription: string;
|
||||
initialRetentionAnalysis: string;
|
||||
onApply: (detailedDescription: string, retentionAnalysis: string) => void;
|
||||
onClose: () => void;
|
||||
/** Where the shots get written, for the description text. */
|
||||
targetLabel?: string;
|
||||
/** False when there is no retention analysis to keep in sync (base-format and plain prompts). */
|
||||
withRetention?: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* Structured camera-direction editor for MiniMax-H3: an ordered shot list
|
||||
* (type, movement, which subjects appear, action, optional dialogue) that
|
||||
* writes into detailed_description's [Shot N] blocks and keeps
|
||||
* retention_analysis's "appears in [...]" lines in sync automatically,
|
||||
* instead of hand-maintaining both fields' free text in agreement.
|
||||
*/
|
||||
/** Cut-in time field: blank means "automatic" (shown as the placeholder); Enter or blur commits. */
|
||||
function CutTimeInput({
|
||||
value,
|
||||
auto,
|
||||
disabled,
|
||||
onCommit,
|
||||
}: {
|
||||
value: number | null;
|
||||
auto: number;
|
||||
disabled?: boolean;
|
||||
onCommit: (seconds: number | null) => void;
|
||||
}) {
|
||||
const show = (v: number | null) => (v == null ? '' : formatCutTime(v));
|
||||
const [text, setText] = React.useState(show(value));
|
||||
React.useEffect(() => setText(show(value)), [value]);
|
||||
|
||||
function commit() {
|
||||
const trimmed = text.trim();
|
||||
if (!trimmed) return onCommit(null);
|
||||
const parsed = parseCutTime(trimmed);
|
||||
if (parsed == null) return setText(show(value));
|
||||
onCommit(parsed);
|
||||
}
|
||||
|
||||
return (
|
||||
<Input
|
||||
value={text}
|
||||
onChange={(e) => setText(e.target.value)}
|
||||
onBlur={commit}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === 'Enter') {
|
||||
e.preventDefault();
|
||||
commit();
|
||||
}
|
||||
}}
|
||||
placeholder={disabled ? 'Opening shot' : formatCutTime(auto)}
|
||||
disabled={disabled}
|
||||
inputMode="decimal"
|
||||
className="font-mono"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
interface EditorState {
|
||||
shots: Shot[];
|
||||
preamble: string;
|
||||
}
|
||||
|
||||
export function H3ShotListEditor({
|
||||
open,
|
||||
subjectDefinitions,
|
||||
initialDetailedDescription,
|
||||
initialRetentionAnalysis,
|
||||
onApply,
|
||||
onClose,
|
||||
targetLabel = 'Detailed description',
|
||||
withRetention = true,
|
||||
}: H3ShotListEditorProps) {
|
||||
// One history covers everything editable here: the shots (structure, fields and
|
||||
// every line of text) and the scene notes. Typing carries a key so a burst
|
||||
// merges into one step; everything else is a step of its own.
|
||||
const history = useUndoableState<EditorState>({ shots: [], preamble: '' });
|
||||
const { shots, preamble } = history.state;
|
||||
// Returning the same list (or text) hands the history back the same state object,
|
||||
// which it treats as "nothing changed" and does not record.
|
||||
const editShots = (fn: (shots: Shot[]) => Shot[], key: string | null = null) =>
|
||||
history.set((s) => {
|
||||
const next = fn(s.shots);
|
||||
return next === s.shots ? s : { ...s, shots: next };
|
||||
}, key);
|
||||
const setPreamble = (value: string) =>
|
||||
history.set((s) => (s.preamble === value ? s : { ...s, preamble: value }), 'preamble');
|
||||
const [selectedId, setSelectedId] = React.useState<string | null>(null);
|
||||
// The shared tag palette inserts into whichever shot's dialogue was last focused.
|
||||
const [dialogueTargetId, setDialogueTargetId] = React.useState<string | null>(null);
|
||||
const dialogueFields = React.useRef(new Map<string, H3ShotDialogueHandle>());
|
||||
const [importOpen, setImportOpen] = React.useState(false);
|
||||
const subjectLabels = React.useMemo(() => parseSubjectLabels(subjectDefinitions), [subjectDefinitions]);
|
||||
|
||||
// Re-seed from the field's current text each time the editor opens, the
|
||||
// same "seed on the open transition only" guard H3DialogueEditor uses --
|
||||
// otherwise a stray parent re-render while open could clobber in-progress edits.
|
||||
const wasOpenRef = React.useRef(false);
|
||||
React.useEffect(() => {
|
||||
if (open && !wasOpenRef.current) {
|
||||
const { preamble: seededPreamble, shots: seeded } = parseDetailedDescription(
|
||||
initialDetailedDescription,
|
||||
subjectLabels,
|
||||
);
|
||||
history.reset({ preamble: seededPreamble, shots: seeded });
|
||||
setSelectedId(seeded[0]?.id ?? null);
|
||||
setDialogueTargetId(null);
|
||||
}
|
||||
wasOpenRef.current = open;
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [open]);
|
||||
|
||||
function addShot() {
|
||||
const shot = newShot();
|
||||
editShots((prev) => [...prev, shot]);
|
||||
selectShot(shot.id);
|
||||
}
|
||||
|
||||
// Storyboard panel -> jump to that shot's editor card.
|
||||
function selectShot(id: string) {
|
||||
setSelectedId(id);
|
||||
requestAnimationFrame(() =>
|
||||
document.getElementById(`shot-card-${id}`)?.scrollIntoView({ behavior: 'smooth', block: 'nearest' }),
|
||||
);
|
||||
}
|
||||
|
||||
function removeShot(id: string) {
|
||||
editShots((prev) => prev.filter((s) => s.id !== id));
|
||||
setDialogueTargetId((cur) => (cur === id ? null : cur));
|
||||
}
|
||||
|
||||
function insertTag(tag: H3Tag) {
|
||||
if (dialogueTargetId) dialogueFields.current.get(dialogueTargetId)?.insertTag(tag);
|
||||
}
|
||||
|
||||
function applyPreset(preset: H3Preset) {
|
||||
if (dialogueTargetId) dialogueFields.current.get(dialogueTargetId)?.applyPreset(preset);
|
||||
}
|
||||
|
||||
const partKey = (p: ShotPart) => (p.kind === 'text' ? p.id : p.line.id);
|
||||
|
||||
// An edit that changes nothing returns the same list and so records no undo step
|
||||
// (a blur that re-commits the same value, for instance).
|
||||
function updateBody(shotId: string, fn: (body: ShotPart[]) => ShotPart[], editKey: string | null = null) {
|
||||
editShots((prev) => {
|
||||
let changed = false;
|
||||
const next = prev.map((s) => {
|
||||
if (s.id !== shotId) return s;
|
||||
const body = fn(s.body);
|
||||
if (body === s.body) return s;
|
||||
changed = true;
|
||||
return { ...s, body };
|
||||
});
|
||||
return changed ? next : prev;
|
||||
}, editKey);
|
||||
}
|
||||
|
||||
function addLine(shotId: string) {
|
||||
updateBody(shotId, (body) => {
|
||||
const lastSpeaker = [...body].reverse().find((p): p is LinePart => p.kind === 'line')?.line.speaker;
|
||||
return [...body, linePart(newLine({ speaker: lastSpeaker ?? 'S1' }))];
|
||||
});
|
||||
}
|
||||
|
||||
function addText(shotId: string) {
|
||||
updateBody(shotId, (body) => [...body, newTextPart()]);
|
||||
}
|
||||
|
||||
function removePart(shotId: string, key: string) {
|
||||
updateBody(shotId, (body) => body.filter((p) => partKey(p) !== key));
|
||||
setDialogueTargetId((cur) => (cur === key ? null : cur));
|
||||
}
|
||||
|
||||
function movePart(shotId: string, key: string, dir: -1 | 1) {
|
||||
updateBody(shotId, (body) => {
|
||||
const i = body.findIndex((p) => partKey(p) === key);
|
||||
const j = i + dir;
|
||||
if (i < 0 || j < 0 || j >= body.length) return body;
|
||||
const next = [...body];
|
||||
[next[i], next[j]] = [next[j], next[i]];
|
||||
return next;
|
||||
});
|
||||
}
|
||||
|
||||
function updateText(shotId: string, key: string, text: string) {
|
||||
updateBody(
|
||||
shotId,
|
||||
(body) => {
|
||||
const part = body.find((p) => p.kind === 'text' && p.id === key);
|
||||
if (!part || (part.kind === 'text' && part.text === text)) return body;
|
||||
return body.map((p) => (p === part ? { ...part, text } : p));
|
||||
},
|
||||
`text:${key}`,
|
||||
);
|
||||
}
|
||||
|
||||
/** `editKey` groups typing into one undo step; omit it for a discrete edit like a tag insertion. */
|
||||
function updateLine(shotId: string, lineId: string, patch: Partial<DialogueLine>, editKey?: string) {
|
||||
updateBody(
|
||||
shotId,
|
||||
(body) => {
|
||||
const part = body.find((p) => p.kind === 'line' && p.line.id === lineId);
|
||||
if (!part || part.kind !== 'line') return body;
|
||||
const unchanged = (Object.keys(patch) as (keyof DialogueLine)[]).every((k) => part.line[k] === patch[k]);
|
||||
return unchanged ? body : body.map((p) => (p === part ? { ...part, line: { ...part.line, ...patch } } : p));
|
||||
},
|
||||
editKey ?? null,
|
||||
);
|
||||
}
|
||||
|
||||
const undo = () => void history.undo();
|
||||
const redo = () => void history.redo();
|
||||
|
||||
// Undo/redo for the whole list, from anywhere inside it -- including from a textbox,
|
||||
// where the browser's own undo would only know about that one box. Not while the
|
||||
// script import (nested inside this dialog) is open: its box keeps native undo.
|
||||
function handleKeyDown(e: React.KeyboardEvent) {
|
||||
if (importOpen || e.altKey || !(e.metaKey || e.ctrlKey)) return;
|
||||
const k = e.key.toLowerCase();
|
||||
if (k === 'z') {
|
||||
e.preventDefault();
|
||||
if (e.shiftKey) redo();
|
||||
else undo();
|
||||
} else if (k === 'y' && !e.shiftKey) {
|
||||
e.preventDefault();
|
||||
redo();
|
||||
}
|
||||
}
|
||||
|
||||
// An imported clip replaces the list wholesale, as one undoable step.
|
||||
function loadImportedShots(imported: Shot[]) {
|
||||
const next = withFreshIds(imported);
|
||||
editShots(() => next);
|
||||
setSelectedId(next[0]?.id ?? null);
|
||||
setDialogueTargetId(null);
|
||||
}
|
||||
|
||||
const dialogueTarget = (() => {
|
||||
for (let i = 0; i < shots.length; i += 1) {
|
||||
const lines = shotLines(shots[i]);
|
||||
const li = lines.findIndex((l) => l.id === dialogueTargetId);
|
||||
if (li >= 0) return { shot: i + 1, line: li + 1, multi: lines.length > 1 };
|
||||
}
|
||||
return null;
|
||||
})();
|
||||
|
||||
function moveShot(id: string, dir: -1 | 1) {
|
||||
editShots((prev) => {
|
||||
const i = prev.findIndex((s) => s.id === id);
|
||||
const j = i + dir;
|
||||
if (i < 0 || j < 0 || j >= prev.length) return prev;
|
||||
const next = [...prev];
|
||||
[next[i], next[j]] = [next[j], next[i]];
|
||||
return next;
|
||||
});
|
||||
}
|
||||
|
||||
function updateShot(id: string, patch: Partial<Shot>) {
|
||||
editShots((prev) => {
|
||||
const shot = prev.find((s) => s.id === id);
|
||||
const unchanged = shot && (Object.keys(patch) as (keyof Shot)[]).every((k) => shot[k] === patch[k]);
|
||||
return !shot || unchanged ? prev : prev.map((s) => (s === shot ? { ...s, ...patch } : s));
|
||||
});
|
||||
}
|
||||
|
||||
function toggleSubject(id: string, label: string) {
|
||||
editShots((prev) =>
|
||||
prev.map((s) =>
|
||||
s.id === id
|
||||
? {
|
||||
...s,
|
||||
subjectIds: s.subjectIds.includes(label)
|
||||
? s.subjectIds.filter((l) => l !== label)
|
||||
: [...s.subjectIds, label],
|
||||
}
|
||||
: s,
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
const starts = React.useMemo(() => resolveShotStarts(shots), [shots]);
|
||||
const previewDetailedDescription = React.useMemo(
|
||||
() => detailedDescriptionFromShots(shots, preamble),
|
||||
[shots, preamble],
|
||||
);
|
||||
const previewRetentionAnalysis = React.useMemo(
|
||||
() => (withRetention ? retentionAnalysisFromShots(shots, subjectLabels, initialRetentionAnalysis) : ''),
|
||||
[withRetention, shots, subjectLabels, initialRetentionAnalysis],
|
||||
);
|
||||
|
||||
function handleApply() {
|
||||
onApply(previewDetailedDescription, previewRetentionAnalysis);
|
||||
onClose();
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(o) => { if (!o) onClose(); }}>
|
||||
<DialogContent
|
||||
onKeyDown={handleKeyDown}
|
||||
className="flex h-[92vh] max-h-[920px] w-[96vw] max-w-[1100px] flex-col gap-0 overflow-hidden p-0"
|
||||
>
|
||||
<DialogHeader className="border-b border-border px-5 py-4">
|
||||
<DialogTitle>Shot list</DialogTitle>
|
||||
<DialogDescription>
|
||||
Build the scene shot by shot. Applying writes [Shot N] blocks into {targetLabel}
|
||||
{withRetention ? ' and keeps Retention analysis\u2019s \u201cappears in\u201d lines in sync' : ''}.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
<div className="flex shrink-0 flex-col gap-2 border-b border-border bg-secondary/10 px-4 py-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-xs font-semibold text-foreground">Storyboard</span>
|
||||
<span className="flex items-center gap-1.5">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={undo}
|
||||
disabled={!history.canUndo}
|
||||
title="Undo (Ctrl/Cmd+Z)"
|
||||
>
|
||||
↶ Undo
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={redo}
|
||||
disabled={!history.canRedo}
|
||||
title="Redo (Ctrl/Cmd+Shift+Z)"
|
||||
>
|
||||
↷ Redo
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => setImportOpen(true)}>
|
||||
Import script…
|
||||
</Button>
|
||||
</span>
|
||||
</div>
|
||||
<H3ShotStoryboard shots={shots} selectedId={selectedId} onSelect={selectShot} onAdd={addShot} />
|
||||
</div>
|
||||
|
||||
<div className="flex min-h-0 flex-1 flex-col md:flex-row">
|
||||
<div className="flex min-h-0 min-w-0 flex-1 flex-col gap-4 overflow-y-auto p-4">
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Scene notes (optional, written before Shot 1 -- style, lighting, anything true of the whole clip)
|
||||
<Textarea
|
||||
value={preamble}
|
||||
onChange={(e) => setPreamble(e.target.value)}
|
||||
rows={2}
|
||||
placeholder="e.g. Live-action, cinematic, shallow depth of field."
|
||||
/>
|
||||
</label>
|
||||
|
||||
{shots.length === 0 && (
|
||||
<p className="rounded-xl border border-dashed border-border p-4 text-sm text-muted-foreground">
|
||||
No shots yet.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{shots.map((shot, i) => (
|
||||
<div
|
||||
key={shot.id}
|
||||
id={`shot-card-${shot.id}`}
|
||||
onFocusCapture={() => setSelectedId(shot.id)}
|
||||
className={cn(
|
||||
'flex flex-col gap-3 rounded-xl border bg-secondary/20 p-3',
|
||||
shot.id === selectedId ? 'border-accent-blue' : 'border-border',
|
||||
)}
|
||||
>
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-sm font-semibold text-foreground">
|
||||
Shot {i + 1}
|
||||
{i > 0 && (
|
||||
<span className="ml-2 font-mono text-xs font-normal text-muted-foreground">
|
||||
cuts in at {formatCutTime(starts[i])}
|
||||
</span>
|
||||
)}
|
||||
</span>
|
||||
<div className="flex items-center gap-1">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => moveShot(shot.id, -1)}
|
||||
disabled={i === 0}
|
||||
title="Move up"
|
||||
>
|
||||
↑
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => moveShot(shot.id, 1)}
|
||||
disabled={i === shots.length - 1}
|
||||
title="Move down"
|
||||
>
|
||||
↓
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => removeShot(shot.id)}>
|
||||
Remove
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 gap-2 sm:grid-cols-3">
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Shot type
|
||||
<NativeSelect
|
||||
value={shot.type}
|
||||
onChange={(e) => updateShot(shot.id, { type: e.target.value as ShotType, freeLead: false })}
|
||||
>
|
||||
{SHOT_TYPES.map((t) => (
|
||||
<option key={t.id} value={t.id}>{t.label}</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</label>
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Camera movement
|
||||
<NativeSelect
|
||||
value={shot.movement}
|
||||
onChange={(e) => updateShot(shot.id, { movement: e.target.value as CameraMovement, freeLead: false })}
|
||||
>
|
||||
{CAMERA_MOVEMENTS.map((m) => (
|
||||
<option key={m.id} value={m.id}>{m.label}</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
</label>
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Cut time (blank = automatic)
|
||||
<CutTimeInput
|
||||
value={shot.startSeconds}
|
||||
auto={starts[i]}
|
||||
disabled={i === 0}
|
||||
onCommit={(startSeconds) => updateShot(shot.id, { startSeconds })}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
{shot.freeLead && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
This shot keeps its original wording, so type and movement aren't written. Changing either
|
||||
adds a generated lead-in in front of the text below.
|
||||
</p>
|
||||
)}
|
||||
|
||||
{subjectLabels.length > 0 && (
|
||||
<div className="flex flex-col gap-1">
|
||||
<span className="text-xs text-muted-foreground">Subjects in this shot</span>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{subjectLabels.map((label) => {
|
||||
const active = shot.subjectIds.includes(label);
|
||||
return (
|
||||
<button
|
||||
key={label}
|
||||
type="button"
|
||||
aria-pressed={active}
|
||||
onClick={() => toggleSubject(shot.id, label)}
|
||||
className={cn(
|
||||
'rounded-full border px-2 py-0.5 font-mono text-[11px] transition-colors',
|
||||
active
|
||||
? 'border-accent-blue bg-accent-blue/15 text-accent-blue'
|
||||
: 'border-input bg-card text-muted-foreground hover:bg-accent',
|
||||
)}
|
||||
>
|
||||
{label}
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="flex flex-col gap-2">
|
||||
<span className="text-xs text-muted-foreground">
|
||||
Description and dialogue, in the order they play. Dialogue is wrapped in <d> on apply.
|
||||
</span>
|
||||
{(() => {
|
||||
let lineNo = 0;
|
||||
let textNo = 0;
|
||||
return shot.body.map((part, bi) => {
|
||||
const key = partKey(part);
|
||||
const noun = part.kind === 'line' ? `line ${(lineNo += 1)}` : `text ${(textNo += 1)}`;
|
||||
const controls = (
|
||||
<div className="mt-6 flex shrink-0 gap-1">
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => movePart(shot.id, key, -1)}
|
||||
disabled={bi === 0}
|
||||
aria-label={`Move ${noun} up`}
|
||||
title="Move up"
|
||||
>
|
||||
↑
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => movePart(shot.id, key, 1)}
|
||||
disabled={bi === shot.body.length - 1}
|
||||
aria-label={`Move ${noun} down`}
|
||||
title="Move down"
|
||||
>
|
||||
↓
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => removePart(shot.id, key)}
|
||||
aria-label={`Remove ${noun}`}
|
||||
title="Remove"
|
||||
>
|
||||
×
|
||||
</Button>
|
||||
</div>
|
||||
);
|
||||
|
||||
if (part.kind === 'text') {
|
||||
return (
|
||||
<div key={key} className="flex items-start gap-2">
|
||||
<label className="flex min-w-0 flex-1 flex-col gap-1 text-xs text-muted-foreground">
|
||||
{textNo === 1 ? 'Description (composition, action, environment)' : `Text ${textNo}`}
|
||||
<Textarea
|
||||
value={part.text}
|
||||
onChange={(e) => updateText(shot.id, key, e.target.value)}
|
||||
rows={2}
|
||||
placeholder="What's happening in this shot"
|
||||
/>
|
||||
</label>
|
||||
{controls}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const line = part.line;
|
||||
return (
|
||||
<div key={key} className="flex items-start gap-2">
|
||||
<Input
|
||||
value={line.speaker}
|
||||
onChange={(e) => updateLine(shot.id, line.id, { speaker: e.target.value }, `speaker:${line.id}`)}
|
||||
placeholder="S1"
|
||||
aria-label={`Line ${lineNo} speaker`}
|
||||
className="mt-7 w-16 shrink-0 font-mono"
|
||||
/>
|
||||
<div className="min-w-0 flex-1">
|
||||
<H3ShotDialogueField
|
||||
ref={(handle) => {
|
||||
if (handle) dialogueFields.current.set(line.id, handle);
|
||||
else dialogueFields.current.delete(line.id);
|
||||
}}
|
||||
value={line.text}
|
||||
onChange={(text) => updateLine(shot.id, line.id, { text }, `line:${line.id}`)}
|
||||
onInsert={(text) => updateLine(shot.id, line.id, { text })}
|
||||
onFocus={() => setDialogueTargetId(line.id)}
|
||||
active={line.id === dialogueTargetId}
|
||||
label={`Line ${lineNo}`}
|
||||
/>
|
||||
</div>
|
||||
{controls}
|
||||
</div>
|
||||
);
|
||||
});
|
||||
})()}
|
||||
<div className="flex gap-2">
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => addText(shot.id)}>
|
||||
+ Add text
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => addLine(shot.id)}>
|
||||
+ Add line
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
|
||||
<Button type="button" variant="outline" onClick={addShot} className="self-start">
|
||||
+ Add shot
|
||||
</Button>
|
||||
|
||||
<div className="flex flex-col gap-2 rounded-xl border border-border bg-card p-3">
|
||||
<span className="text-xs font-semibold text-foreground">Preview</span>
|
||||
<div className="flex flex-col gap-1">
|
||||
<span className="text-[11px] font-medium uppercase tracking-wide text-muted-foreground">
|
||||
{targetLabel}
|
||||
</span>
|
||||
<pre className="whitespace-pre-wrap rounded-lg bg-secondary/30 p-2 font-mono text-[11px] text-foreground">
|
||||
{previewDetailedDescription || '—'}
|
||||
</pre>
|
||||
</div>
|
||||
{withRetention && (
|
||||
<div className="flex flex-col gap-1">
|
||||
<span className="text-[11px] font-medium uppercase tracking-wide text-muted-foreground">
|
||||
Retention analysis
|
||||
</span>
|
||||
<pre className="whitespace-pre-wrap rounded-lg bg-secondary/30 p-2 font-mono text-[11px] text-foreground">
|
||||
{previewRetentionAnalysis || '—'}
|
||||
</pre>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<aside className="flex max-h-56 shrink-0 flex-col border-t border-border md:max-h-none md:w-64 md:border-l md:border-t-0">
|
||||
<p className="border-b border-border px-3 py-2 text-xs text-muted-foreground">
|
||||
{dialogueTarget
|
||||
? `Tags insert into Shot ${dialogueTarget.shot}${dialogueTarget.multi ? `, line ${dialogueTarget.line}` : ''} dialogue`
|
||||
: 'Click into a shot\'s dialogue to insert tags'}
|
||||
</p>
|
||||
<H3TagPalette
|
||||
onInsertTag={insertTag}
|
||||
onApplyPreset={applyPreset}
|
||||
disabled={!dialogueTarget}
|
||||
className="overflow-y-auto p-3"
|
||||
/>
|
||||
</aside>
|
||||
</div>
|
||||
|
||||
<DialogFooter className="border-t border-border px-5 py-3">
|
||||
<Button type="button" variant="outline" onClick={onClose}>Cancel</Button>
|
||||
<Button type="button" onClick={handleApply}>Apply</Button>
|
||||
</DialogFooter>
|
||||
|
||||
{/* Nested so Radix treats it as part of this dialog (see CreateJobModal). */}
|
||||
<H3ScriptImportDialog
|
||||
open={importOpen}
|
||||
subjectLabels={subjectLabels}
|
||||
existingShotCount={shots.length}
|
||||
onLoad={loadImportedShots}
|
||||
onClose={() => setImportOpen(false)}
|
||||
/>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,214 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import {
|
||||
CAMERA_MOVEMENTS,
|
||||
SHOT_TYPES,
|
||||
shotLines,
|
||||
shotProse,
|
||||
type CameraMovement,
|
||||
type Shot,
|
||||
type ShotType,
|
||||
} from '@/lib/h3Shots';
|
||||
|
||||
const TYPE_LABEL = Object.fromEntries(SHOT_TYPES.map((t) => [t.id, t.label])) as Record<ShotType, string>;
|
||||
const MOVEMENT_LABEL = Object.fromEntries(CAMERA_MOVEMENTS.map((m) => [m.id, m.label])) as Record<
|
||||
CameraMovement,
|
||||
string
|
||||
>;
|
||||
|
||||
// Schematic framing per shot type: `scale` sizes the figures and `baseY` is
|
||||
// where their shoulders bottom out in the 160x90 frame, so a wide shot reads
|
||||
// as small distant figures and a close-up as a head filling the frame.
|
||||
const FRAMING: Partial<Record<ShotType, { scale: number; baseY: number }>> = {
|
||||
establishing: { scale: 0.45, baseY: 72 },
|
||||
wide: { scale: 0.8, baseY: 78 },
|
||||
medium: { scale: 1.5, baseY: 92 },
|
||||
'close-up': { scale: 2.6, baseY: 118 },
|
||||
};
|
||||
|
||||
function figureXs(count: number): number[] {
|
||||
if (count <= 1) return [80];
|
||||
if (count === 2) return [55, 105];
|
||||
return [38, 80, 122];
|
||||
}
|
||||
|
||||
function Figure({ x, baseY, scale, opacity }: { x: number; baseY: number; scale: number; opacity: number }) {
|
||||
return (
|
||||
<g fill="currentColor" opacity={opacity}>
|
||||
<circle cx={x} cy={baseY - 30 * scale} r={8 * scale} />
|
||||
<ellipse cx={x} cy={baseY - 12 * scale} rx={14 * scale} ry={12 * scale} />
|
||||
</g>
|
||||
);
|
||||
}
|
||||
|
||||
export function ShotFrame({ shot }: { shot: Shot }) {
|
||||
const count = Math.min(Math.max(shot.subjectIds.length, 1), 3);
|
||||
const empty = shot.subjectIds.length === 0;
|
||||
const opacity = empty ? 0.25 : 0.6;
|
||||
const framing = FRAMING[shot.type];
|
||||
|
||||
let scene: React.ReactNode;
|
||||
if (shot.type === 'extreme-close-up') {
|
||||
scene = (
|
||||
<g fill="currentColor" opacity={opacity}>
|
||||
<circle cx={80} cy={52} r={62} />
|
||||
<ellipse cx={60} cy={42} rx={7} ry={4} className="fill-background" />
|
||||
<ellipse cx={100} cy={42} rx={7} ry={4} className="fill-background" />
|
||||
</g>
|
||||
);
|
||||
} else if (shot.type === 'over-the-shoulder') {
|
||||
scene = (
|
||||
<>
|
||||
<Figure x={22} baseY={135} scale={3} opacity={0.85} />
|
||||
<Figure x={116} baseY={80} scale={1} opacity={opacity} />
|
||||
</>
|
||||
);
|
||||
} else if (shot.type === 'pov') {
|
||||
scene = (
|
||||
<>
|
||||
<Figure x={80} baseY={72} scale={0.9} opacity={opacity} />
|
||||
<path
|
||||
d="M0 0H160V90H0Z M80 45m-52 0a52 40 0 1 0 104 0a52 40 0 1 0 -104 0"
|
||||
fill="currentColor"
|
||||
fillRule="evenodd"
|
||||
opacity={0.3}
|
||||
/>
|
||||
<path d="M74 45H86M80 39V51" stroke="currentColor" strokeWidth={1.5} opacity={0.6} />
|
||||
</>
|
||||
);
|
||||
} else if (shot.type === 'insert') {
|
||||
scene = (
|
||||
<g opacity={0.6} fill="none" stroke="currentColor" strokeWidth={2.5}>
|
||||
<rect x={52} y={16} width={56} height={58} rx={8} />
|
||||
<path d="M64 34H96M64 46H96M64 58H84" strokeLinecap="round" />
|
||||
</g>
|
||||
);
|
||||
} else if (framing) {
|
||||
scene = (
|
||||
<>
|
||||
{shot.type === 'establishing' && (
|
||||
<path d="M0 66H160" stroke="currentColor" strokeWidth={1} opacity={0.35} />
|
||||
)}
|
||||
{figureXs(count).map((x, i) => (
|
||||
<Figure
|
||||
key={x}
|
||||
x={shot.type === 'close-up' ? 80 : x}
|
||||
baseY={framing.baseY}
|
||||
scale={framing.scale}
|
||||
opacity={opacity * (i === 0 ? 1 : 0.75)}
|
||||
/>
|
||||
))}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<svg viewBox="0 0 160 90" className="block h-full w-full text-foreground" aria-hidden>
|
||||
{scene}
|
||||
<MovementOverlay movement={shot.movement} />
|
||||
</svg>
|
||||
);
|
||||
}
|
||||
|
||||
function MovementOverlay({ movement }: { movement: CameraMovement }) {
|
||||
const g = (children: React.ReactNode) => (
|
||||
<g
|
||||
className="text-accent-blue"
|
||||
stroke="currentColor"
|
||||
fill="none"
|
||||
strokeWidth={2.5}
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
>
|
||||
{children}
|
||||
</g>
|
||||
);
|
||||
switch (movement) {
|
||||
case 'pan':
|
||||
return g(<path d="M28 80H132M28 80l6-4M28 80l6 4M132 80l-6-4M132 80l-6 4" />);
|
||||
case 'tilt':
|
||||
return g(<path d="M146 18V72M146 18l-4 6M146 18l4 6M146 72l-4-6M146 72l4-6" />);
|
||||
case 'dolly':
|
||||
return g(<path d="M10 86L54 56M150 86L106 56M10 86H150" strokeDasharray="4 4" />);
|
||||
case 'zoom':
|
||||
return g(<path d="M16 26V14H28M132 14H144V26M16 64V76H28M132 76H144V64" />);
|
||||
case 'handheld':
|
||||
return g(
|
||||
<rect x={7} y={7} width={146} height={76} rx={4} strokeDasharray="3 4" transform="rotate(-1.6 80 45)" />,
|
||||
);
|
||||
case 'tracking':
|
||||
return g(
|
||||
<>
|
||||
<path d="M22 82H134M134 82l-6-4M134 82l-6 4" />
|
||||
<circle cx={30} cy={82} r={3} fill="currentColor" />
|
||||
</>,
|
||||
);
|
||||
case 'crane':
|
||||
return g(<path d="M24 80L134 20M134 20h-9M134 20l-3 8" />);
|
||||
default:
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function subjectShortLabel(label: string): string {
|
||||
const m = label.match(/(\d+)/);
|
||||
return m ? `Subj ${m[1]}` : label;
|
||||
}
|
||||
|
||||
export interface H3ShotPanelProps {
|
||||
shot: Shot;
|
||||
/** the number shown in the badge */
|
||||
number: number | string;
|
||||
/** time printed on the frame, e.g. "00:03.500" */
|
||||
timeLabel: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* The contents of one storyboard panel: badge, shot type and camera move, a
|
||||
* schematic frame, who is in it, what is said, and the shot's prose. Callers
|
||||
* supply the surrounding card (a button in the editor, plain in the scene view).
|
||||
*/
|
||||
export function H3ShotPanel({ shot, number, timeLabel }: H3ShotPanelProps) {
|
||||
const spoken = shotLines(shot).filter((l) => l.text.trim());
|
||||
return (
|
||||
<>
|
||||
<div className="flex items-center gap-1.5">
|
||||
<span className="flex size-5 shrink-0 items-center justify-center rounded-full bg-secondary text-[11px] font-semibold text-foreground">
|
||||
{number}
|
||||
</span>
|
||||
<span className="truncate text-xs font-medium text-foreground">{TYPE_LABEL[shot.type]}</span>
|
||||
<span className="ml-auto shrink-0 text-[10px] text-muted-foreground">{MOVEMENT_LABEL[shot.movement]}</span>
|
||||
</div>
|
||||
|
||||
<div className="relative aspect-video overflow-hidden rounded-md border border-border bg-secondary/40">
|
||||
<ShotFrame shot={shot} />
|
||||
<span className="absolute bottom-1 left-1 rounded bg-background/80 px-1 font-mono text-[9px] text-muted-foreground">
|
||||
{timeLabel}
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{shot.subjectIds.length > 0 && (
|
||||
<div className="flex flex-wrap gap-1">
|
||||
{shot.subjectIds.map((label) => (
|
||||
<span
|
||||
key={label}
|
||||
className="rounded-full border border-input bg-secondary/50 px-1.5 py-px font-mono text-[10px] text-muted-foreground"
|
||||
>
|
||||
{subjectShortLabel(label)}
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{spoken.length > 0 && (
|
||||
<p className="line-clamp-3 rounded-md bg-accent-blue/10 px-1.5 py-1 text-[11px] italic text-foreground">
|
||||
{spoken.map((l) => `${l.speaker.trim() ? `${l.speaker.trim()}: ` : ''}“${l.text.trim()}”`).join(' ')}
|
||||
</p>
|
||||
)}
|
||||
|
||||
<p className="line-clamp-2 text-[11px] text-muted-foreground">{shotProse(shot) || 'No description yet'}</p>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { H3ShotStoryboard } from './H3ShotStoryboard';
|
||||
import { CAMERA_MOVEMENTS, SHOT_TYPES, makeShot, newLine, type Shot, type ShotInit } from '@/lib/h3Shots';
|
||||
|
||||
function shot(init: ShotInit): Shot {
|
||||
return makeShot(init);
|
||||
}
|
||||
|
||||
describe('H3ShotStoryboard', () => {
|
||||
it('lays shots out in order with type, movement, subjects, and dialogue', () => {
|
||||
const shots = [
|
||||
shot({ type: 'establishing', description: 'Wide rooftop.', subjectIds: ['<Subject 1>', '<Subject 2>'] }),
|
||||
shot({ type: 'close-up', movement: 'handheld', lines: [newLine({ speaker: 'S2', text: 'I had no choice.' }), newLine({ speaker: 'S1', text: 'Then go.' })], subjectIds: ['<Subject 2>'] }),
|
||||
];
|
||||
render(<H3ShotStoryboard shots={shots} selectedId={shots[1].id} onSelect={() => {}} onAdd={() => {}} />);
|
||||
|
||||
expect(screen.getByText('Establishing')).toBeInTheDocument();
|
||||
expect(screen.getByText('Close-up')).toBeInTheDocument();
|
||||
expect(screen.getByText('Handheld')).toBeInTheDocument();
|
||||
expect(screen.getByText(/S2: “I had no choice.” S1: “Then go.”/)).toBeInTheDocument();
|
||||
expect(screen.getAllByText('Subj 2')).toHaveLength(2);
|
||||
|
||||
const panels = screen.getAllByRole('button').filter((b) => b.hasAttribute('aria-current') || b.textContent?.match(/^\d/));
|
||||
expect(panels[0]).toHaveTextContent(/^1Establishing/);
|
||||
expect(screen.getByRole('button', { name: /Close-up/ })).toHaveAttribute('aria-current', 'true');
|
||||
});
|
||||
|
||||
it('selects a shot on click and adds via the trailing tile', async () => {
|
||||
const user = userEvent.setup();
|
||||
const onSelect = vi.fn();
|
||||
const onAdd = vi.fn();
|
||||
const shots = [shot({ description: 'First.' }), shot({ description: 'Second.' })];
|
||||
render(<H3ShotStoryboard shots={shots} selectedId={null} onSelect={onSelect} onAdd={onAdd} />);
|
||||
|
||||
await user.click(screen.getByRole('button', { name: /Second\./ }));
|
||||
expect(onSelect).toHaveBeenCalledWith(shots[1].id);
|
||||
await user.click(screen.getByRole('button', { name: /Add shot/ }));
|
||||
expect(onAdd).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('renders every shot type and camera movement without crashing', () => {
|
||||
const shots = SHOT_TYPES.flatMap((t) =>
|
||||
CAMERA_MOVEMENTS.map((m) => shot({ type: t.id, movement: m.id, subjectIds: ['<Subject 1>'] })),
|
||||
);
|
||||
const { container } = render(<H3ShotStoryboard shots={shots} selectedId={null} onSelect={() => {}} onAdd={() => {}} />);
|
||||
expect(container.querySelectorAll('svg')).toHaveLength(SHOT_TYPES.length * CAMERA_MOVEMENTS.length);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,61 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { H3ShotPanel } from '@/components/jobs/H3ShotPanel';
|
||||
import { formatCutTime, resolveShotStarts, type Shot } from '@/lib/h3Shots';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface H3ShotStoryboardProps {
|
||||
shots: Shot[];
|
||||
selectedId: string | null;
|
||||
onSelect: (id: string) => void;
|
||||
onAdd: () => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* The shot list as a storyboard: one panel per shot, left to right in
|
||||
* playback order. Panels are schematic (no generated imagery exists yet) --
|
||||
* subject size reflects shot type and an overlay marks camera movement --
|
||||
* so the sequence, who's in frame, and where the dialogue falls read at a
|
||||
* glance without scrolling the per-shot editors.
|
||||
*/
|
||||
export function H3ShotStoryboard({ shots, selectedId, onSelect, onAdd }: H3ShotStoryboardProps) {
|
||||
const starts = React.useMemo(() => resolveShotStarts(shots), [shots]);
|
||||
return (
|
||||
<div className="flex items-stretch gap-2 overflow-x-auto pb-1" aria-label="Storyboard">
|
||||
{shots.map((shot, i) => {
|
||||
const selected = shot.id === selectedId;
|
||||
return (
|
||||
<React.Fragment key={shot.id}>
|
||||
{i > 0 && (
|
||||
<span className="flex shrink-0 items-center text-muted-foreground/60" aria-hidden>
|
||||
›
|
||||
</span>
|
||||
)}
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => onSelect(shot.id)}
|
||||
aria-current={selected ? 'true' : undefined}
|
||||
className={cn(
|
||||
'flex w-52 shrink-0 flex-col gap-1.5 rounded-xl border bg-card p-2 text-left transition-colors hover:bg-accent/40',
|
||||
selected ? 'border-accent-blue ring-2 ring-accent-blue/50' : 'border-border',
|
||||
)}
|
||||
>
|
||||
<H3ShotPanel shot={shot} number={i + 1} timeLabel={formatCutTime(starts[i])} />
|
||||
</button>
|
||||
</React.Fragment>
|
||||
);
|
||||
})}
|
||||
|
||||
<button
|
||||
type="button"
|
||||
onClick={onAdd}
|
||||
className="flex min-h-32 w-28 shrink-0 flex-col items-center justify-center gap-1 rounded-xl border border-dashed border-border text-xs text-muted-foreground transition-colors hover:bg-accent/40 hover:text-foreground"
|
||||
>
|
||||
<span className="text-lg leading-none">+</span>
|
||||
Add shot
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,57 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { H3_TAG_CATEGORIES, type H3Tag } from '@/lib/h3Tags';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface H3TagConsoleProps {
|
||||
/** Called with the tag to insert into whichever field is currently focused. */
|
||||
onInsert: (tag: H3Tag) => void;
|
||||
disabled?: boolean;
|
||||
className?: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* A microexpression "console" for MiniMax-H3 speech tags: click a tag to
|
||||
* insert it at the cursor of whichever prompt field was last focused. Kept
|
||||
* as plain buttons (not a menu) so every tag stays one click away.
|
||||
*/
|
||||
export function H3TagConsole({ onInsert, disabled, className }: H3TagConsoleProps) {
|
||||
return (
|
||||
<div
|
||||
className={cn(
|
||||
'flex flex-col gap-1.5 rounded-xl border border-input bg-secondary/40 p-2',
|
||||
className,
|
||||
)}
|
||||
>
|
||||
<span className="px-0.5 text-[11px] font-medium uppercase tracking-wide text-muted-foreground">
|
||||
Speech tags — click to insert at the cursor
|
||||
</span>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{H3_TAG_CATEGORIES.map((category) => (
|
||||
<div key={category.id} className="flex flex-wrap items-center gap-1">
|
||||
<span className="mr-0.5 w-full shrink-0 text-[10px] text-muted-foreground/80 sm:w-auto">
|
||||
{category.label}
|
||||
</span>
|
||||
{category.tags.map((tag) => (
|
||||
<button
|
||||
key={tag.id}
|
||||
type="button"
|
||||
title={`${tag.description} · e.g. "${tag.example}"`}
|
||||
// Keep the target textarea focused (and its selection intact)
|
||||
// so the click lands where the user was typing, not nowhere.
|
||||
onMouseDown={(e) => e.preventDefault()}
|
||||
onClick={() => onInsert(tag)}
|
||||
disabled={disabled}
|
||||
className="rounded-full border border-input bg-card px-2 py-0.5 font-mono text-[11px] text-foreground transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
{tag.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { H3_PRESETS, H3_TAG_CATEGORIES, type H3Preset, type H3Tag } from '@/lib/h3Tags';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
// Reuses the app's existing status hues as a purely visual category key --
|
||||
// no status meaning implied, just four distinguishable accents that already
|
||||
// work in both themes.
|
||||
export const CATEGORY_STYLE: Record<string, { chip: string; dot: string }> = {
|
||||
pace: { chip: 'border-blue-400/35 bg-blue-500/10 text-blue-900 dark:text-blue-100', dot: 'bg-blue-500' },
|
||||
delivery: { chip: 'border-amber-400/35 bg-amber-500/10 text-amber-900 dark:text-amber-100', dot: 'bg-amber-500' },
|
||||
vocal: { chip: 'border-emerald-400/35 bg-emerald-500/10 text-emerald-900 dark:text-emerald-100', dot: 'bg-emerald-500' },
|
||||
body: { chip: 'border-rose-400/35 bg-rose-500/10 text-rose-900 dark:text-rose-100', dot: 'bg-rose-500' },
|
||||
};
|
||||
|
||||
export interface H3TagPaletteProps {
|
||||
onInsertTag: (tag: H3Tag) => void;
|
||||
onApplyPreset: (preset: H3Preset) => void;
|
||||
disabled?: boolean;
|
||||
className?: string;
|
||||
}
|
||||
|
||||
/** Full speech-tag palette + emotion presets. Buttons keep the target textarea focused so the click lands at its cursor. */
|
||||
export function H3TagPalette({ onInsertTag, onApplyPreset, disabled, className }: H3TagPaletteProps) {
|
||||
return (
|
||||
<div className={className}>
|
||||
{H3_TAG_CATEGORIES.map((cat) => {
|
||||
const style = CATEGORY_STYLE[cat.id] ?? CATEGORY_STYLE.body;
|
||||
return (
|
||||
<div key={cat.id} className="mb-4">
|
||||
<div className="mb-1.5 flex items-center gap-2 px-0.5">
|
||||
<span className={cn('size-2 rounded-full', style.dot)} />
|
||||
<span className="text-xs font-semibold text-foreground">{cat.label}</span>
|
||||
</div>
|
||||
<div className="flex flex-col gap-1">
|
||||
{cat.tags.map((tag) => (
|
||||
<button
|
||||
key={tag.id}
|
||||
type="button"
|
||||
title={tag.example}
|
||||
disabled={disabled}
|
||||
onMouseDown={(e) => e.preventDefault()}
|
||||
onClick={() => onInsertTag(tag)}
|
||||
className="flex items-baseline justify-between gap-2 rounded-lg border border-input bg-card px-2.5 py-1.5 text-left transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
<span className="font-mono text-[11.5px] text-foreground">{tag.label}</span>
|
||||
<span className="truncate text-[10.5px] text-muted-foreground">{tag.description}</span>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
|
||||
<div className="mb-1.5 mt-5 px-0.5 text-xs font-semibold text-foreground">Emotion presets</div>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{H3_PRESETS.map((p) => (
|
||||
<button
|
||||
key={p.label}
|
||||
type="button"
|
||||
disabled={disabled}
|
||||
onMouseDown={(e) => e.preventDefault()}
|
||||
onClick={() => onApplyPreset(p)}
|
||||
className="rounded-full border border-input bg-card px-2.5 py-1 text-[11.5px] text-foreground transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
{p.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -5,7 +5,9 @@ import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import JobCard from '@/components/jobs/JobCard';
|
||||
import {
|
||||
deleteJob,
|
||||
dequeueJob,
|
||||
downloadJobVideo,
|
||||
queueJobs,
|
||||
startJob,
|
||||
stopJob,
|
||||
} from '@/lib/api';
|
||||
@@ -18,6 +20,8 @@ vi.mock('@/lib/api', () => ({
|
||||
stopJob: vi.fn(),
|
||||
deleteJob: vi.fn(),
|
||||
downloadJobVideo: vi.fn(),
|
||||
queueJobs: vi.fn(),
|
||||
dequeueJob: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock('@/lib/utils', async (importOriginal) => {
|
||||
@@ -47,6 +51,8 @@ beforeEach(() => {
|
||||
setActiveJobId(null);
|
||||
vi.mocked(startJob).mockResolvedValue({} as Job);
|
||||
vi.mocked(stopJob).mockResolvedValue({} as Job);
|
||||
vi.mocked(queueJobs).mockResolvedValue([]);
|
||||
vi.mocked(dequeueJob).mockResolvedValue({} as Job);
|
||||
vi.mocked(deleteJob).mockResolvedValue(undefined);
|
||||
vi.mocked(downloadJobVideo).mockResolvedValue(new Blob());
|
||||
vi.spyOn(window, 'confirm').mockReturnValue(true);
|
||||
@@ -91,6 +97,46 @@ describe('JobCard', () => {
|
||||
expect(onJobUpdated).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it.each(['pending', 'failed', 'stopped'])('queues a %s job and notifies the parent', async (status) => {
|
||||
const onJobUpdated = vi.fn();
|
||||
render(<JobCard job={makeJob({ status })} onJobUpdated={onJobUpdated} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Queue' }));
|
||||
await waitFor(() => expect(queueJobs).toHaveBeenCalledWith(['job-1']));
|
||||
expect(onJobUpdated).toHaveBeenCalled();
|
||||
expect(startJob).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('reports a job that cannot be queued', async () => {
|
||||
vi.mocked(queueJobs).mockRejectedValue(new Error('Jobs would wait on each other'));
|
||||
render(<JobCard job={makeJob({ status: 'pending' })} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Queue' }));
|
||||
await waitFor(() => expect(window.alert).toHaveBeenCalledWith('Jobs would wait on each other'));
|
||||
});
|
||||
|
||||
it('shows a queued job as queued, and takes it back out of the queue', async () => {
|
||||
const onJobUpdated = vi.fn();
|
||||
render(<JobCard job={makeJob({ status: 'queued' })} onJobUpdated={onJobUpdated} />);
|
||||
|
||||
expect(screen.getByText('queued')).toBeInTheDocument();
|
||||
// nothing to start or queue again; it can't be edited either
|
||||
for (const name of ['Start', 'Restart', 'Queue', 'Stop', 'Edit']) {
|
||||
expect(screen.queryByRole('button', { name })).not.toBeInTheDocument();
|
||||
}
|
||||
expect(screen.getByRole('button', { name: 'View' })).toBeInTheDocument();
|
||||
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Remove from queue' }));
|
||||
await waitFor(() => expect(dequeueJob).toHaveBeenCalledWith('job-1'));
|
||||
expect(onJobUpdated).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('does not offer to queue a running or completed job', () => {
|
||||
for (const status of ['running', 'completed']) {
|
||||
const { unmount } = render(<JobCard job={makeJob({ status, started_at: Date.now() })} />);
|
||||
expect(screen.queryByRole('button', { name: 'Queue' })).not.toBeInTheDocument();
|
||||
unmount();
|
||||
}
|
||||
});
|
||||
|
||||
it('stops a running job', async () => {
|
||||
render(<JobCard job={makeJob({ status: 'running', started_at: Date.now() })} />);
|
||||
await userEvent.click(screen.getByRole('button', { name: 'Stop' }));
|
||||
|
||||
@@ -10,7 +10,9 @@ import { useStore } from '@/hooks/useStore';
|
||||
import {
|
||||
deleteJob,
|
||||
duplicateJob,
|
||||
dequeueJob,
|
||||
downloadJobVideo,
|
||||
queueJobs,
|
||||
startJob,
|
||||
stopJob,
|
||||
} from '@/lib/api';
|
||||
@@ -51,6 +53,7 @@ function computeElapsed(job: Job, currentTime: number): string | null {
|
||||
|
||||
const BADGE_VARIANTS: Record<string, BadgeProps['variant']> = {
|
||||
pending: 'secondary',
|
||||
queued: 'default',
|
||||
running: 'warning',
|
||||
completed: 'success',
|
||||
ready: 'success',
|
||||
@@ -92,6 +95,36 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
}
|
||||
}
|
||||
|
||||
async function handleQueue(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await queueJobs([job.id]);
|
||||
onJobUpdated?.();
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to queue job');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDequeue(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
if (isLoading) return;
|
||||
setIsLoading(true);
|
||||
try {
|
||||
await dequeueJob(job.id);
|
||||
onJobUpdated?.();
|
||||
} catch (err) {
|
||||
alert(err instanceof Error ? err.message : 'Failed to remove job from the queue');
|
||||
} finally {
|
||||
setIsLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function handleStop(e: React.MouseEvent) {
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
@@ -238,7 +271,30 @@ export default function JobCard({ job, onJobUpdated }: JobCardProps) {
|
||||
>
|
||||
Start
|
||||
</Button>
|
||||
) : job.status === 'queued' ? (
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={handleDequeue}
|
||||
disabled={isLoading}
|
||||
title="Take this job out of the queue"
|
||||
>
|
||||
Remove from queue
|
||||
</Button>
|
||||
) : null}
|
||||
{(job.status === 'pending' ||
|
||||
job.status === 'failed' ||
|
||||
job.status === 'stopped') && (
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={handleQueue}
|
||||
disabled={isLoading}
|
||||
title="Run this job when the ones queued before it have finished"
|
||||
>
|
||||
Queue
|
||||
</Button>
|
||||
)}
|
||||
{job.status === 'completed' &&
|
||||
job.output_path &&
|
||||
job.job_type === 'inference' && (
|
||||
|
||||
@@ -77,7 +77,7 @@ export default function JobDetailsSidebar({
|
||||
const isRestarting =
|
||||
previousJobId.current === job.id &&
|
||||
wasTerminal &&
|
||||
(job.status === 'pending' || job.status === 'running');
|
||||
(job.status === 'pending' || job.status === 'queued' || job.status === 'running');
|
||||
|
||||
if (previousJobId.current !== job.id || isRestarting) {
|
||||
stateRef.current.text = '';
|
||||
@@ -89,7 +89,10 @@ export default function JobDetailsSidebar({
|
||||
}, [job.id, job.status]);
|
||||
|
||||
React.useEffect(() => {
|
||||
const shouldPoll = job.status === 'running' || job.status === 'pending';
|
||||
const shouldPoll =
|
||||
job.status === 'running' ||
|
||||
job.status === 'pending' ||
|
||||
job.status === 'queued';
|
||||
let pollInterval: ReturnType<typeof setInterval> | null = null;
|
||||
let mounted = true;
|
||||
// Effect-local lock so this job's first fetch is never blocked by a
|
||||
|
||||
@@ -108,7 +108,8 @@ export default function JobQueue({ jobType, jobTypesForList }: JobQueueProps) {
|
||||
|
||||
// Poll every second while any job is running/pending; stop otherwise.
|
||||
const hasActive = jobs.some(
|
||||
(j) => j.status === 'running' || j.status === 'pending',
|
||||
(j) =>
|
||||
j.status === 'running' || j.status === 'pending' || j.status === 'queued',
|
||||
);
|
||||
React.useEffect(() => {
|
||||
if (!hasActive) return;
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
import * as React from 'react';
|
||||
import { flushSync } from 'react-dom';
|
||||
import { render, screen, waitFor, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import CreateJobModal from './CreateJobModal';
|
||||
import { createJob, getDatasets, getModels } from '@/lib/api';
|
||||
import { defaultOptionsStore } from '@/stores/defaultOptions';
|
||||
import { DEFAULT_OPTIONS } from '@/lib/defaultOptions';
|
||||
|
||||
vi.mock('sonner', () => ({ toast: { warning: vi.fn(), error: vi.fn(), success: vi.fn() } }));
|
||||
vi.mock('@/lib/api', () => ({
|
||||
createJob: vi.fn(),
|
||||
getModels: vi.fn(),
|
||||
getDatasets: vi.fn(),
|
||||
uploadImage: vi.fn(),
|
||||
getSettings: vi.fn(),
|
||||
updateSettings: vi.fn(),
|
||||
}));
|
||||
|
||||
const H3 = { id: 'minimax-h3/i2v', label: 'MiniMax H3', type: 'i2v' };
|
||||
|
||||
// A browser runs a microtask checkpoint -- so React flushes -- between the
|
||||
// listeners of a single native event; jsdom's synchronous dispatch never does.
|
||||
// Radix decides whether a parent dialog should dismiss from the layer stack at
|
||||
// the moment its listener runs, so without this the bug can't show up here.
|
||||
const wrapped = new WeakMap<object, EventListener>();
|
||||
beforeEach(() => {
|
||||
const realAdd = Document.prototype.addEventListener;
|
||||
const realRemove = Document.prototype.removeEventListener;
|
||||
vi.spyOn(document, 'addEventListener').mockImplementation(function (
|
||||
this: Document,
|
||||
type: string,
|
||||
listener: unknown,
|
||||
options?: unknown,
|
||||
) {
|
||||
if (typeof listener === 'function' && ['keydown', 'pointerdown', 'click'].includes(type)) {
|
||||
const flushing: EventListener = (e) => {
|
||||
(listener as EventListener)(e);
|
||||
flushSync(() => {});
|
||||
};
|
||||
wrapped.set(listener, flushing);
|
||||
return realAdd.call(this, type, flushing, options as AddEventListenerOptions);
|
||||
}
|
||||
return realAdd.call(this, type, listener as EventListener, options as AddEventListenerOptions);
|
||||
} as typeof document.addEventListener);
|
||||
vi.spyOn(document, 'removeEventListener').mockImplementation(function (
|
||||
this: Document,
|
||||
type: string,
|
||||
listener: unknown,
|
||||
options?: unknown,
|
||||
) {
|
||||
return realRemove.call(
|
||||
this,
|
||||
type,
|
||||
(wrapped.get(listener as object) ?? listener) as EventListener,
|
||||
options as EventListenerOptions,
|
||||
);
|
||||
} as typeof document.removeEventListener);
|
||||
|
||||
defaultOptionsStore.set({ options: DEFAULT_OPTIONS });
|
||||
vi.mocked(getModels).mockResolvedValue([H3]);
|
||||
vi.mocked(getDatasets).mockResolvedValue([]);
|
||||
vi.mocked(createJob).mockResolvedValue({ id: 'j' } as never);
|
||||
});
|
||||
|
||||
function Harness() {
|
||||
const [open, setOpen] = React.useState(true);
|
||||
return (
|
||||
<>
|
||||
<div data-testid="state">{open ? 'open' : 'closed'}</div>
|
||||
<CreateJobModal
|
||||
isOpen={open}
|
||||
onClose={() => setOpen(false)}
|
||||
onSuccess={() => {}}
|
||||
jobType="inference"
|
||||
workloadType="i2v"
|
||||
/>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
type User = ReturnType<typeof userEvent.setup>;
|
||||
type Closer = (user: User, dialog: HTMLElement) => Promise<void>;
|
||||
|
||||
async function renderH3Modal(user: User) {
|
||||
render(<Harness />);
|
||||
await screen.findByRole('option', { name: /MiniMax H3/ });
|
||||
await user.selectOptions(screen.getByLabelText('Model'), H3.id);
|
||||
}
|
||||
|
||||
// Radix marks every dialog under the topmost one aria-hidden, which also hides its
|
||||
// accessible name, so the job dialog is identified by its title text instead.
|
||||
const expectParentStillOpen = () => {
|
||||
expect(screen.getByTestId('state')).toHaveTextContent('open');
|
||||
expect(screen.getByText(/New Inference Job/)).toBeInTheDocument();
|
||||
};
|
||||
// [0] is the job dialog's overlay; the last is the topmost dialog's.
|
||||
const topOverlay = () => [...document.querySelectorAll('.bg-black\\/70')].at(-1) as Element;
|
||||
|
||||
const CLOSERS: [string, Closer][] = [
|
||||
['Cancel', (u, d) => u.click(within(d).getByRole('button', { name: 'Cancel' }))],
|
||||
['the X button', (u, d) => u.click(within(d).getByRole('button', { name: 'Close' }))],
|
||||
['Escape', (u) => u.keyboard('{Escape}')],
|
||||
['a click on the overlay', (u) => u.click(topOverlay())],
|
||||
];
|
||||
|
||||
describe('closing a dialog opened from the job form', () => {
|
||||
describe.each([
|
||||
['shot list', 'Open shot list', 'Shot list', [...CLOSERS, ['Apply', (u: User, d: HTMLElement) => u.click(within(d).getByRole('button', { name: 'Apply' }))] as [string, Closer]]],
|
||||
['dialogue editor', 'Open dialogue editor', 'Dialogue editor', [...CLOSERS, ['Apply to field', (u: User, d: HTMLElement) => u.click(within(d).getByRole('button', { name: 'Apply to field' }))] as [string, Closer]]],
|
||||
] as const)('%s', (_label, openButton, title, closers) => {
|
||||
it.each(closers)('closing with %s leaves the job form open', async (_name, close) => {
|
||||
const user = userEvent.setup();
|
||||
await renderH3Modal(user);
|
||||
await user.click(await screen.findByRole('button', { name: openButton }));
|
||||
const child = await screen.findByRole('dialog', { name: title });
|
||||
|
||||
await close(user, child);
|
||||
|
||||
await waitFor(() => expect(screen.queryByRole('dialog', { name: title })).not.toBeInTheDocument());
|
||||
expectParentStillOpen();
|
||||
});
|
||||
});
|
||||
|
||||
it('the job form is still usable afterwards', async () => {
|
||||
const user = userEvent.setup();
|
||||
await renderH3Modal(user);
|
||||
await user.click(await screen.findByRole('button', { name: 'Open shot list' }));
|
||||
const child = await screen.findByRole('dialog', { name: 'Shot list' });
|
||||
await user.click(within(child).getByRole('button', { name: 'Cancel' }));
|
||||
await waitFor(() => expect(screen.queryByRole('dialog', { name: 'Shot list' })).not.toBeInTheDocument());
|
||||
|
||||
await user.type(screen.getByLabelText('Name (optional)'), 'still here');
|
||||
expect(screen.getByLabelText('Name (optional)')).toHaveValue('still here');
|
||||
// ...and the dialog can be opened again
|
||||
await user.click(screen.getByRole('button', { name: 'Open shot list' }));
|
||||
expect(await screen.findByRole('dialog', { name: 'Shot list' })).toBeInTheDocument();
|
||||
});
|
||||
|
||||
describe('the script import inside the shot list', () => {
|
||||
it.each(CLOSERS.slice(0, 4))('closing it with %s leaves the shot list and the job form open', async (_name, close) => {
|
||||
const user = userEvent.setup();
|
||||
await renderH3Modal(user);
|
||||
await user.click(await screen.findByRole('button', { name: 'Open shot list' }));
|
||||
const shotList = await screen.findByRole('dialog', { name: 'Shot list' });
|
||||
await user.click(within(shotList).getByRole('button', { name: 'Import script…' }));
|
||||
const importDialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
|
||||
await close(user, importDialog);
|
||||
|
||||
await waitFor(() => expect(screen.queryByRole('dialog', { name: 'Import script' })).not.toBeInTheDocument());
|
||||
expect(screen.getByRole('dialog', { name: 'Shot list' })).toBeInTheDocument();
|
||||
expectParentStillOpen();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,253 @@
|
||||
import { render, screen, within } from '@testing-library/react';
|
||||
import userEvent from '@testing-library/user-event';
|
||||
import { describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { H3ShotListEditor } from './H3ShotListEditor';
|
||||
|
||||
const TWO_SHOTS = '[Shot 1] Wide shot. A dock.\n[Shot 2] At 00:02.000, the shot cuts to a close-up shot. A face.';
|
||||
const WITH_LINE = '[Shot 1] Wide shot. (S1) <d>[English] Hi.</d>';
|
||||
|
||||
type User = ReturnType<typeof userEvent.setup>;
|
||||
|
||||
function setup(detailed: string, extra: { open?: boolean } = {}) {
|
||||
const onApply = vi.fn();
|
||||
const ui = (open: boolean) => (
|
||||
<H3ShotListEditor
|
||||
open={open}
|
||||
subjectDefinitions="<Subject 1> is a man."
|
||||
initialDetailedDescription={detailed}
|
||||
initialRetentionAnalysis=""
|
||||
onApply={onApply}
|
||||
onClose={() => {}}
|
||||
/>
|
||||
);
|
||||
const view = render(ui(extra.open ?? true));
|
||||
return { onApply, view, ui };
|
||||
}
|
||||
|
||||
const undoButton = () => screen.getByRole('button', { name: /Undo/ });
|
||||
const redoButton = () => screen.getByRole('button', { name: /Redo/ });
|
||||
const applied = (onApply: ReturnType<typeof vi.fn>) => onApply.mock.calls.at(-1)?.[0] as string;
|
||||
const apply = (user: User) => user.click(screen.getByRole('button', { name: 'Apply' }));
|
||||
const lineBox = () => screen.getByPlaceholderText('What the speaker says') as HTMLTextAreaElement;
|
||||
const ctrl = (key: string) => `{Control>}${key}{/Control}`;
|
||||
|
||||
describe('shot list undo/redo', () => {
|
||||
it('starts with nothing to undo or redo', () => {
|
||||
setup(TWO_SHOTS);
|
||||
expect(undoButton()).toBeDisabled();
|
||||
expect(redoButton()).toBeDisabled();
|
||||
});
|
||||
|
||||
it('undoes and redoes moving a shot', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
expect(undoButton()).toBeEnabled();
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(redoButton()).toBeEnabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
|
||||
it('redo reapplies what was undone', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
await user.click(undoButton());
|
||||
await user.click(redoButton());
|
||||
await apply(user);
|
||||
expect(applied(onApply).startsWith('[Shot 1] Close-up shot. A face.')).toBe(true);
|
||||
});
|
||||
|
||||
it('undoes removing a shot, dialogue and all', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(`${WITH_LINE}\n[Shot 2] At 00:02.500, the shot cuts to a wide shot. B.`);
|
||||
await user.click(screen.getAllByRole('button', { name: 'Remove' })[0]);
|
||||
expect(screen.queryByPlaceholderText('What the speaker says')).not.toBeInTheDocument();
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(lineBox().value).toBe('Hi.');
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toContain('(S1) <d>[English] Hi.</d>');
|
||||
});
|
||||
|
||||
it('undoes an add, a type change, a subject toggle, and a cut time in turn', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
|
||||
await user.click(screen.getAllByRole('button', { name: /Add shot/ })[0]);
|
||||
await user.selectOptions(screen.getAllByLabelText('Shot type')[0], 'pov');
|
||||
await user.click(screen.getAllByRole('button', { name: '<Subject 1>' })[0]);
|
||||
const cut = screen.getAllByPlaceholderText(/\d\d:\d\d\.\d{3}/)[0] as HTMLInputElement;
|
||||
await user.type(cut, '5{Enter}');
|
||||
|
||||
// four discrete steps, undone one at a time
|
||||
for (let i = 0; i < 4; i += 1) await user.click(undoButton());
|
||||
expect(undoButton()).toBeDisabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
|
||||
it('committing a cut time that has not changed is not a step', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(TWO_SHOTS);
|
||||
const cut = screen.getAllByPlaceholderText(/\d\d:\d\d\.\d{3}/)[0] as HTMLInputElement;
|
||||
await user.type(cut, '5{Enter}');
|
||||
await user.click(undoButton()); // the click blurs the box, which re-commits 00:05.000
|
||||
expect(undoButton()).toBeDisabled();
|
||||
});
|
||||
|
||||
it('a burst of typing is one step', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(WITH_LINE);
|
||||
await user.type(lineBox(), ' there my friend');
|
||||
expect(lineBox().value).toBe('Hi. there my friend');
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(lineBox().value).toBe('Hi.');
|
||||
expect(undoButton()).toBeDisabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toContain('<d>[English] Hi.</d>');
|
||||
});
|
||||
|
||||
it('a tag inserted from the palette is its own step, separate from typing', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(WITH_LINE);
|
||||
await user.type(lineBox(), ' there');
|
||||
lineBox().setSelectionRange(lineBox().value.length, lineBox().value.length);
|
||||
await user.click(screen.getByRole('button', { name: /<pause>/ }));
|
||||
expect(lineBox().value).toBe('Hi. there <pause>');
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(lineBox().value).toBe('Hi. there');
|
||||
await user.click(undoButton());
|
||||
expect(lineBox().value).toBe('Hi.');
|
||||
});
|
||||
|
||||
it('undoes edits to the description and the scene notes', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup('[Shot 1] Wide shot. A dock.');
|
||||
await user.type(screen.getByPlaceholderText(/Live-action, cinematic/), 'Noir.');
|
||||
await user.type(screen.getByPlaceholderText("What's happening in this shot"), ' At dawn.');
|
||||
|
||||
await user.click(undoButton());
|
||||
await user.click(undoButton());
|
||||
expect(undoButton()).toBeDisabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe('[Shot 1] Wide shot. A dock.');
|
||||
});
|
||||
|
||||
it('typing in different boxes makes separate steps', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup('[Shot 1] Wide shot. A dock.');
|
||||
await user.type(screen.getByPlaceholderText(/Live-action, cinematic/), 'Noir.');
|
||||
await user.type(screen.getByPlaceholderText("What's happening in this shot"), ' At dawn.');
|
||||
|
||||
await user.click(undoButton());
|
||||
expect((screen.getByPlaceholderText("What's happening in this shot") as HTMLTextAreaElement).value).toBe('A dock.');
|
||||
expect((screen.getByPlaceholderText(/Live-action, cinematic/) as HTMLTextAreaElement).value).toBe('Noir.');
|
||||
});
|
||||
|
||||
it('a new edit clears what could be redone', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
await user.click(undoButton());
|
||||
expect(redoButton()).toBeEnabled();
|
||||
await user.click(screen.getAllByRole('button', { name: /Add shot/ })[0]);
|
||||
expect(redoButton()).toBeDisabled();
|
||||
});
|
||||
});
|
||||
|
||||
describe('shot list undo/redo shortcuts', () => {
|
||||
it('Ctrl+Z from inside a textbox undoes the whole list, not just that box', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]); // move a shot
|
||||
// focus a different textbox, then undo from there
|
||||
const box = screen.getAllByPlaceholderText("What's happening in this shot")[0] as HTMLTextAreaElement;
|
||||
await user.click(box);
|
||||
await user.keyboard(ctrl('z'));
|
||||
expect(undoButton()).toBeDisabled();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
|
||||
it('redoes with Ctrl+Shift+Z and with Ctrl+Y', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
const box = screen.getAllByPlaceholderText("What's happening in this shot")[0];
|
||||
await user.click(box);
|
||||
|
||||
await user.keyboard(ctrl('z'));
|
||||
expect(redoButton()).toBeEnabled();
|
||||
await user.keyboard('{Control>}{Shift>}z{/Shift}{/Control}');
|
||||
expect(redoButton()).toBeDisabled();
|
||||
|
||||
await user.keyboard(ctrl('z'));
|
||||
await user.keyboard(ctrl('y'));
|
||||
expect(redoButton()).toBeDisabled();
|
||||
expect(undoButton()).toBeEnabled();
|
||||
});
|
||||
|
||||
it('Cmd+Z works too', async () => {
|
||||
const user = userEvent.setup();
|
||||
setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
await user.click(screen.getAllByPlaceholderText("What's happening in this shot")[0]);
|
||||
await user.keyboard('{Meta>}z{/Meta}');
|
||||
expect(undoButton()).toBeDisabled();
|
||||
});
|
||||
|
||||
it('leaves the script import box to its own undo', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
expect(undoButton()).toBeEnabled();
|
||||
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
const dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
const box = within(dialog).getByLabelText('Script');
|
||||
await user.click(box);
|
||||
await user.keyboard(ctrl('z'));
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Cancel' }));
|
||||
|
||||
// the shot list's history was untouched: the move is still there to undo
|
||||
await user.click(await screen.findByRole('button', { name: /Undo/ }));
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
});
|
||||
|
||||
describe('shot list history across imports and reopening', () => {
|
||||
it('loading a script clip is one undoable step', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { onApply } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getByRole('button', { name: 'Import script…' }));
|
||||
const dialog = await screen.findByRole('dialog', { name: 'Import script' });
|
||||
await user.click(within(dialog).getByLabelText('Script'));
|
||||
await user.paste('CAPTAIN: Sit down and hear me out.\n\nCLERK: Yes, sir.');
|
||||
await user.click(within(dialog).getByRole('button', { name: 'Load clip' }));
|
||||
await screen.findAllByPlaceholderText('What the speaker says');
|
||||
|
||||
await user.click(undoButton());
|
||||
expect(screen.queryByPlaceholderText('What the speaker says')).not.toBeInTheDocument();
|
||||
await apply(user);
|
||||
expect(applied(onApply)).toBe(TWO_SHOTS);
|
||||
});
|
||||
|
||||
it('opening the editor again starts a fresh history', async () => {
|
||||
const user = userEvent.setup();
|
||||
const { view, ui } = setup(TWO_SHOTS);
|
||||
await user.click(screen.getAllByRole('button', { name: '↑' })[1]);
|
||||
expect(undoButton()).toBeEnabled();
|
||||
|
||||
view.rerender(ui(false));
|
||||
view.rerender(ui(true));
|
||||
expect(await screen.findByRole('button', { name: /Undo/ })).toBeDisabled();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,111 @@
|
||||
import { fireEvent, render, screen } from '@testing-library/react';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { BeforeAfterGrade } from './BeforeAfterGrade';
|
||||
|
||||
// jsdom has no real video decoder, canvas 2D context, or `ImageData` constructor,
|
||||
// so all three are stubbed: the video always "has" an 8x6 frame ready, the canvas
|
||||
// records what was drawn/read/written instead of actually rasterizing anything,
|
||||
// and ImageData is a plain data holder (its real job -- backing a canvas -- is
|
||||
// already covered by the getContext stub).
|
||||
if (typeof globalThis.ImageData === 'undefined') {
|
||||
class FakeImageData {
|
||||
data: Uint8ClampedArray;
|
||||
width: number;
|
||||
height: number;
|
||||
constructor(data: Uint8ClampedArray, width: number, height: number) {
|
||||
this.data = data;
|
||||
this.width = width;
|
||||
this.height = height;
|
||||
}
|
||||
}
|
||||
// @ts-expect-error -- test-only polyfill for a browser global jsdom doesn't provide
|
||||
globalThis.ImageData = FakeImageData;
|
||||
}
|
||||
|
||||
function stubVideoFrame() {
|
||||
Object.defineProperty(HTMLMediaElement.prototype, 'readyState', { configurable: true, get: () => 4 });
|
||||
Object.defineProperty(HTMLVideoElement.prototype, 'videoWidth', { configurable: true, get: () => 8 });
|
||||
Object.defineProperty(HTMLVideoElement.prototype, 'videoHeight', { configurable: true, get: () => 6 });
|
||||
}
|
||||
|
||||
function stubCanvasContext() {
|
||||
const data = new Uint8ClampedArray(8 * 6 * 4).fill(100);
|
||||
const putImageData = vi.fn();
|
||||
const drawImage = vi.fn();
|
||||
const getImageData = vi.fn(() => ({ width: 8, height: 6, data }));
|
||||
vi.spyOn(HTMLCanvasElement.prototype, 'getContext').mockReturnValue({
|
||||
drawImage,
|
||||
getImageData,
|
||||
putImageData,
|
||||
} as unknown as CanvasRenderingContext2D);
|
||||
return { drawImage, getImageData, putImageData, data };
|
||||
}
|
||||
|
||||
describe('BeforeAfterGrade', () => {
|
||||
beforeEach(() => {
|
||||
stubVideoFrame();
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks();
|
||||
});
|
||||
|
||||
it('shows the "load a frame" hint until a video frame is captured', () => {
|
||||
const ref = { current: null };
|
||||
render(<BeforeAfterGrade videoRef={ref} brightness={0} contrast={1} saturation={1} />);
|
||||
expect(screen.getByText(/Play or seek the preview above/)).toBeInTheDocument();
|
||||
});
|
||||
|
||||
it('captures a frame once the video has data, and draws it into both canvases', () => {
|
||||
const { getImageData, putImageData } = stubCanvasContext();
|
||||
const video = document.createElement('video');
|
||||
document.body.appendChild(video);
|
||||
const ref = { current: video };
|
||||
|
||||
render(<BeforeAfterGrade videoRef={ref} brightness={0} contrast={1} saturation={1} />);
|
||||
fireEvent.loadedData(video);
|
||||
|
||||
expect(getImageData).toHaveBeenCalled();
|
||||
expect(putImageData).toHaveBeenCalled();
|
||||
expect(screen.queryByText(/Play or seek the preview above/)).not.toBeInTheDocument();
|
||||
document.body.removeChild(video);
|
||||
});
|
||||
|
||||
it('re-grades the "after" canvas when the color values change, without touching the video again', () => {
|
||||
const { putImageData } = stubCanvasContext();
|
||||
const video = document.createElement('video');
|
||||
document.body.appendChild(video);
|
||||
const ref = { current: video };
|
||||
|
||||
const { rerender } = render(<BeforeAfterGrade videoRef={ref} brightness={0} contrast={1} saturation={1} />);
|
||||
fireEvent.loadedData(video);
|
||||
const callsAfterCapture = putImageData.mock.calls.length;
|
||||
|
||||
rerender(<BeforeAfterGrade videoRef={ref} brightness={50} contrast={1} saturation={1} />);
|
||||
expect(putImageData.mock.calls.length).toBeGreaterThan(callsAfterCapture);
|
||||
|
||||
// Neutral values were drawn as-is; the brightened redraw's pixel buffer must differ.
|
||||
const neutralData = putImageData.mock.calls[0][0].data as Uint8ClampedArray;
|
||||
const brightenedData = putImageData.mock.calls[putImageData.mock.calls.length - 1][0].data as Uint8ClampedArray;
|
||||
expect(brightenedData[0]).toBeGreaterThan(neutralData[0]);
|
||||
|
||||
document.body.removeChild(video);
|
||||
});
|
||||
|
||||
it('"Use current frame" re-captures the video at its current position', () => {
|
||||
const { drawImage } = stubCanvasContext();
|
||||
const video = document.createElement('video');
|
||||
document.body.appendChild(video);
|
||||
const ref = { current: video };
|
||||
|
||||
render(<BeforeAfterGrade videoRef={ref} brightness={0} contrast={1} saturation={1} />);
|
||||
fireEvent.loadedData(video);
|
||||
const callsAfterAutoCapture = drawImage.mock.calls.length;
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: 'Use current frame' }));
|
||||
expect(drawImage.mock.calls.length).toBeGreaterThan(callsAfterAutoCapture);
|
||||
|
||||
document.body.removeChild(video);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,113 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { applyGradeInPlace, isNeutralGrade } from '@/lib/grading';
|
||||
|
||||
export interface BeforeAfterGradeProps {
|
||||
/** the video the snapshot is captured from -- must be `crossOrigin="anonymous"` for pixel access to work */
|
||||
videoRef: React.RefObject<HTMLVideoElement | null>;
|
||||
brightness: number;
|
||||
contrast: number;
|
||||
saturation: number;
|
||||
/** called with the video's currentTime whenever a frame is (re)captured, so a caller with multiple
|
||||
* sections can tell which one the previewed frame falls into and keep its own selection in sync */
|
||||
onFrameCaptured?: (timeSeconds: number) => void;
|
||||
}
|
||||
|
||||
// Downscaled so the per-pixel pass stays instant on every slider tick.
|
||||
const PREVIEW_WIDTH = 320;
|
||||
|
||||
/**
|
||||
* A still frame from `videoRef`, shown twice: untouched, and with the current
|
||||
* color values applied via `applyGradeInPlace` -- entirely in the browser, so
|
||||
* dragging a slider never waits on the server. Captures a frame automatically
|
||||
* once the video has one to show; "Use current frame" re-captures whatever the
|
||||
* video is currently paused/seeked to, for comparing a specific moment.
|
||||
*/
|
||||
export function BeforeAfterGrade({ videoRef, brightness, contrast, saturation, onFrameCaptured }: BeforeAfterGradeProps) {
|
||||
const beforeRef = React.useRef<HTMLCanvasElement>(null);
|
||||
const afterRef = React.useRef<HTMLCanvasElement>(null);
|
||||
const sourceRef = React.useRef<ImageData | null>(null);
|
||||
const [ready, setReady] = React.useState(false);
|
||||
// Kept in a ref, not a `capture` dependency, so an inline arrow function prop
|
||||
// doesn't force the video-listener effect below to tear down and re-add itself every render.
|
||||
const onFrameCapturedRef = React.useRef(onFrameCaptured);
|
||||
onFrameCapturedRef.current = onFrameCaptured;
|
||||
|
||||
const capture = React.useCallback(() => {
|
||||
const video = videoRef.current;
|
||||
const before = beforeRef.current;
|
||||
if (!video || !before || video.readyState < 2 || !video.videoWidth) return;
|
||||
|
||||
const scale = PREVIEW_WIDTH / video.videoWidth;
|
||||
const width = PREVIEW_WIDTH;
|
||||
const height = Math.round(video.videoHeight * scale);
|
||||
before.width = width;
|
||||
before.height = height;
|
||||
const ctx = before.getContext('2d');
|
||||
if (!ctx) return;
|
||||
ctx.drawImage(video, 0, 0, width, height);
|
||||
try {
|
||||
sourceRef.current = ctx.getImageData(0, 0, width, height);
|
||||
setReady(true);
|
||||
onFrameCapturedRef.current?.(video.currentTime);
|
||||
} catch {
|
||||
// A cross-origin frame the browser wouldn't expose pixels for -- comparison stays unavailable.
|
||||
sourceRef.current = null;
|
||||
setReady(false);
|
||||
}
|
||||
}, [videoRef]);
|
||||
|
||||
React.useEffect(() => {
|
||||
const video = videoRef.current;
|
||||
if (!video) return;
|
||||
if (video.readyState >= 2) capture();
|
||||
video.addEventListener('loadeddata', capture);
|
||||
video.addEventListener('seeked', capture);
|
||||
return () => {
|
||||
video.removeEventListener('loadeddata', capture);
|
||||
video.removeEventListener('seeked', capture);
|
||||
};
|
||||
}, [videoRef, capture]);
|
||||
|
||||
React.useEffect(() => {
|
||||
const source = sourceRef.current;
|
||||
const after = afterRef.current;
|
||||
if (!source || !after) return;
|
||||
after.width = source.width;
|
||||
after.height = source.height;
|
||||
const ctx = after.getContext('2d');
|
||||
if (!ctx) return;
|
||||
const copy = new ImageData(new Uint8ClampedArray(source.data), source.width, source.height);
|
||||
if (!isNeutralGrade(brightness, contrast, saturation)) {
|
||||
applyGradeInPlace(copy.data, brightness, contrast, saturation);
|
||||
}
|
||||
ctx.putImageData(copy, 0, 0);
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps -- redraws whenever a new frame is captured (`ready`) too
|
||||
}, [brightness, contrast, saturation, ready]);
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-2">
|
||||
<div className="flex items-center justify-between">
|
||||
<p className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Before / after</p>
|
||||
<button type="button" onClick={capture} className="text-xs text-accent-blue hover:underline">
|
||||
Use current frame
|
||||
</button>
|
||||
</div>
|
||||
<div className="grid grid-cols-2 gap-2">
|
||||
<div className="flex flex-col gap-1">
|
||||
<canvas ref={beforeRef} role="img" aria-label="Before color grading" className="w-full rounded-md border border-border bg-black" />
|
||||
<span className="text-center text-[11px] text-muted-foreground">Before</span>
|
||||
</div>
|
||||
<div className="flex flex-col gap-1">
|
||||
<canvas ref={afterRef} role="img" aria-label="After color grading" className="w-full rounded-md border border-border bg-black" />
|
||||
<span className="text-center text-[11px] text-muted-foreground">After</span>
|
||||
</div>
|
||||
</div>
|
||||
{!ready && (
|
||||
<p className="text-xs text-muted-foreground">Play or seek the preview above to load a frame to compare.</p>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,72 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { toast } from 'sonner';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { downloadMergedVideo, getMergedVideoUrl, type MergedScene } from '@/lib/api';
|
||||
import { formatClock } from '@/lib/scenes';
|
||||
import { downloadBlob } from '@/lib/utils';
|
||||
|
||||
export interface MergedSceneDialogProps {
|
||||
/** the merged video to show, or null when closed */
|
||||
merged: (MergedScene & { title: string }) | null;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
/** The finished, merged scene: play it, and save it. */
|
||||
export function MergedSceneDialog({ merged, onClose }: MergedSceneDialogProps) {
|
||||
const [downloading, setDownloading] = React.useState(false);
|
||||
|
||||
async function download() {
|
||||
if (!merged) return;
|
||||
setDownloading(true);
|
||||
try {
|
||||
downloadBlob(await downloadMergedVideo(merged.filename), merged.filename);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't download the video.");
|
||||
} finally {
|
||||
setDownloading(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={!!merged} onOpenChange={(open) => !open && onClose()}>
|
||||
<DialogContent className="max-w-3xl">
|
||||
<DialogHeader>
|
||||
<DialogTitle>{merged?.title} merged</DialogTitle>
|
||||
<DialogDescription>
|
||||
{merged &&
|
||||
`${merged.clips} clip${merged.clips === 1 ? '' : 's'} joined in order · ${formatClock(merged.seconds)}`}
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
{merged && (
|
||||
<video
|
||||
src={getMergedVideoUrl(merged.filename)}
|
||||
aria-label={`Merged video of ${merged.title}`}
|
||||
className="max-h-[60vh] w-full rounded-md border border-border bg-black"
|
||||
controls
|
||||
playsInline
|
||||
preload="metadata"
|
||||
/>
|
||||
)}
|
||||
<DialogFooter>
|
||||
<Button type="button" variant="outline" onClick={onClose}>
|
||||
Close
|
||||
</Button>
|
||||
<Button type="button" onClick={download} disabled={downloading}>
|
||||
{downloading ? 'Downloading…' : 'Download'}
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,453 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { H3ShotPanel } from '@/components/jobs/H3ShotPanel';
|
||||
import { Badge, type BadgeProps } from '@/components/ui/badge';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { NativeSelect } from '@/components/ui/native-select';
|
||||
import { getJobVideoUrl } from '@/lib/api';
|
||||
import { attachedLastFrames, whyCannotAttach } from '@/lib/continuation';
|
||||
import { formatCutTime } from '@/lib/h3Shots';
|
||||
import { QUEUEABLE_STATUSES } from '@/lib/sceneQueue';
|
||||
import { formatClock, type Scene, type SceneClip } from '@/lib/scenes';
|
||||
import type { Job } from '@/lib/types';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
const BADGE_VARIANTS: Record<string, BadgeProps['variant']> = {
|
||||
pending: 'secondary',
|
||||
queued: 'default',
|
||||
running: 'warning',
|
||||
completed: 'success',
|
||||
failed: 'destructive',
|
||||
stopped: 'secondary',
|
||||
};
|
||||
|
||||
const SEGMENT_TONE: Record<string, string> = {
|
||||
completed: 'bg-emerald-500/70',
|
||||
running: 'bg-amber-500/70',
|
||||
queued: 'bg-sky-500/60',
|
||||
failed: 'bg-rose-500/70',
|
||||
};
|
||||
|
||||
/** Statuses a job can still be edited in (mirrors the Jobs page). */
|
||||
export const EDITABLE_STATUSES = ['pending', 'failed', 'stopped'];
|
||||
|
||||
function clipLabel(clip: SceneClip): string {
|
||||
return clip.job.name?.trim() || `Job ${clip.job.id.slice(0, 8)}`;
|
||||
}
|
||||
|
||||
function takeLabel(job: Job, index: number, total: number): string {
|
||||
const ms = job.created_at < 1e12 ? job.created_at * 1000 : job.created_at;
|
||||
const when = new Date(ms).toLocaleString(undefined, {
|
||||
month: 'short',
|
||||
day: 'numeric',
|
||||
hour: '2-digit',
|
||||
minute: '2-digit',
|
||||
});
|
||||
return `Take ${index + 1} of ${total} · ${job.status} · ${when}`;
|
||||
}
|
||||
|
||||
function scrollToClip(id: string) {
|
||||
document.getElementById(`clip-${id}`)?.scrollIntoView({ behavior: 'smooth', block: 'start' });
|
||||
}
|
||||
|
||||
function Timeline({ scene }: { scene: Scene }) {
|
||||
return (
|
||||
<div className="flex h-4 w-full gap-px overflow-hidden rounded-md" role="group" aria-label="Scene timeline">
|
||||
{scene.clips.map((clip) => (
|
||||
<button
|
||||
key={clip.job.id}
|
||||
type="button"
|
||||
onClick={() => scrollToClip(clip.job.id)}
|
||||
aria-label={`Go to clip ${clip.position}: ${clipLabel(clip)}`}
|
||||
title={`Clip ${clip.position} · ${clipLabel(clip)} · ${clip.job.status} · ${formatClock(clip.startSeconds)}–${formatClock(clip.startSeconds + clip.durationSeconds)}`}
|
||||
style={{ flexGrow: Math.max(clip.durationSeconds, 0.1), flexBasis: 0 }}
|
||||
className={cn(
|
||||
'min-w-[3px] transition-opacity hover:opacity-70',
|
||||
SEGMENT_TONE[clip.job.status] ?? 'bg-muted-foreground/40',
|
||||
)}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function ClipPreview({ job, videoVersion }: { job: Job; videoVersion: number }) {
|
||||
const isImage = job.output_path?.toLowerCase().endsWith('.png') ?? false;
|
||||
const src = getJobVideoUrl(job.id, videoVersion);
|
||||
// Sets the *initial* muted state only, imperatively, so it doesn't fight the
|
||||
// user's own unmute via the native controls: `muted` as a plain JSX prop is
|
||||
// controlled, and React re-applies it on every re-render of this component
|
||||
// (e.g. the scene board's poll while a job is queued/running), silently
|
||||
// re-muting a video the user just unmuted.
|
||||
const videoRef = React.useRef<HTMLVideoElement | null>(null);
|
||||
React.useEffect(() => {
|
||||
if (videoRef.current) videoRef.current.muted = true;
|
||||
}, []);
|
||||
return isImage ? (
|
||||
// eslint-disable-next-line @next/next/no-img-element
|
||||
<img src={src} alt={job.name ?? 'Generated output'} className="max-h-56 rounded-md border border-border" />
|
||||
) : (
|
||||
<video
|
||||
ref={videoRef}
|
||||
src={src}
|
||||
aria-label={`Generated video for ${job.name ?? job.id}`}
|
||||
className="max-h-56 rounded-md border border-border bg-black"
|
||||
controls
|
||||
loop
|
||||
playsInline
|
||||
preload="metadata"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
function ClipRow({
|
||||
clip,
|
||||
previous,
|
||||
firstShotNumber,
|
||||
onOpenJob,
|
||||
onChooseTake,
|
||||
onUseLastFrame,
|
||||
onDetachLastFrame,
|
||||
onQueueClip,
|
||||
onDequeueClip,
|
||||
onTrimClip,
|
||||
videoVersionFor,
|
||||
}: {
|
||||
clip: SceneClip;
|
||||
/** the clip before this one in the scene, if any */
|
||||
previous: SceneClip | undefined;
|
||||
firstShotNumber: number;
|
||||
onOpenJob: (job: Job) => void;
|
||||
onChooseTake: (takeKey: string, jobId: string) => void;
|
||||
onUseLastFrame: (clip: SceneClip, previous: SceneClip) => Promise<void>;
|
||||
onDetachLastFrame: (clip: SceneClip) => Promise<void>;
|
||||
onQueueClip: (clip: SceneClip) => Promise<void>;
|
||||
onDequeueClip: (clip: SceneClip) => Promise<void>;
|
||||
onTrimClip: (clip: SceneClip) => void;
|
||||
videoVersionFor: (jobId: string) => number;
|
||||
}) {
|
||||
const [previewing, setPreviewing] = React.useState(false);
|
||||
const [attaching, setAttaching] = React.useState(false);
|
||||
const [queueing, setQueueing] = React.useState(false);
|
||||
const { job } = clip;
|
||||
const canPreview = job.status === 'completed' && !!job.output_path;
|
||||
const editable = EDITABLE_STATUSES.includes(job.status);
|
||||
|
||||
const queued = job.status === 'queued';
|
||||
const canQueue = QUEUEABLE_STATUSES.includes(job.status);
|
||||
|
||||
// Starting from the previous clip's last frame: offered on clips that can still be
|
||||
// edited. The frame is taken when this clip runs, so the previous clip needn't be done yet.
|
||||
const frames = attachedLastFrames(job.references);
|
||||
const fromPrevious = previous && frames.some((f) => f.jobId === previous.job.id);
|
||||
const fromAnotherTake = frames.some((f) => f.jobId !== previous?.job.id);
|
||||
const previousDone = !!previous && previous.job.status === 'completed';
|
||||
const cannotUseFrame = previous ? whyCannotAttach(job.prompt, job.references) : null;
|
||||
|
||||
const usingLastFrame = frames.length > 0;
|
||||
const lastFrameBlocked = attaching || (!usingLastFrame && !!cannotUseFrame);
|
||||
|
||||
async function toggleLastFrame() {
|
||||
if (!previous) return;
|
||||
setAttaching(true);
|
||||
try {
|
||||
await (usingLastFrame ? onDetachLastFrame(clip) : onUseLastFrame(clip, previous));
|
||||
} finally {
|
||||
setAttaching(false);
|
||||
}
|
||||
}
|
||||
|
||||
async function queueOrDequeue(action: (clip: SceneClip) => Promise<void>) {
|
||||
setQueueing(true);
|
||||
try {
|
||||
await action(clip);
|
||||
} finally {
|
||||
setQueueing(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<li id={`clip-${job.id}`} className="scroll-mt-32 rounded-xl border border-border bg-card p-3">
|
||||
<div className="mb-2 flex flex-wrap items-center gap-2">
|
||||
<span className="flex size-6 shrink-0 items-center justify-center rounded-md bg-secondary text-xs font-semibold text-foreground">
|
||||
{clip.position}
|
||||
</span>
|
||||
<h3 className="min-w-0 truncate text-sm font-semibold text-foreground">{clipLabel(clip)}</h3>
|
||||
<Badge variant={BADGE_VARIANTS[job.status] ?? 'secondary'}>{job.status}</Badge>
|
||||
<span className="font-mono text-xs text-muted-foreground">
|
||||
{formatClock(clip.startSeconds)}–{formatClock(clip.startSeconds + clip.durationSeconds)}
|
||||
</span>
|
||||
{clip.continuation && (
|
||||
<Badge variant="outline" className="normal-case tracking-normal">
|
||||
continues previous shot
|
||||
</Badge>
|
||||
)}
|
||||
<span className="ml-auto flex flex-wrap items-center justify-end gap-2">
|
||||
{previous && queued && fromPrevious && (
|
||||
<Badge variant="outline" className="normal-case tracking-normal">
|
||||
starts from clip {previous.position}'s last frame
|
||||
</Badge>
|
||||
)}
|
||||
{previous && queued && fromPrevious && !previousDone && (
|
||||
<Badge variant="outline" className="normal-case tracking-normal">
|
||||
waiting for clip {previous.position}
|
||||
</Badge>
|
||||
)}
|
||||
{previous && editable && (
|
||||
<label
|
||||
className={cn(
|
||||
'flex items-center gap-1.5 text-xs text-foreground',
|
||||
lastFrameBlocked && 'text-muted-foreground',
|
||||
)}
|
||||
title={
|
||||
usingLastFrame
|
||||
? "Stop starting this clip from the previous clip's last frame"
|
||||
: (cannotUseFrame ??
|
||||
`Attach the last frame of clip ${previous.position} as this clip's opening frame${
|
||||
previousDone ? '' : ` (taken when this clip runs, after clip ${previous.position} has finished)`
|
||||
}`)
|
||||
}
|
||||
>
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={usingLastFrame}
|
||||
onChange={toggleLastFrame}
|
||||
disabled={lastFrameBlocked}
|
||||
className="size-4"
|
||||
/>
|
||||
{attaching
|
||||
? usingLastFrame
|
||||
? 'Removing…'
|
||||
: 'Attaching…'
|
||||
: fromPrevious || !usingLastFrame
|
||||
? `Use last frame of clip ${previous.position}`
|
||||
: "Starts from another take's last frame"}
|
||||
</label>
|
||||
)}
|
||||
{clip.takeKey && clip.takes.length > 1 && (
|
||||
<NativeSelect
|
||||
aria-label={`Take for clip ${clip.position}`}
|
||||
value={job.id}
|
||||
onChange={(e) => onChooseTake(clip.takeKey as string, e.target.value)}
|
||||
className="h-8 w-auto py-0 text-xs"
|
||||
>
|
||||
{clip.takes.map((take, i) => (
|
||||
<option key={take.id} value={take.id}>
|
||||
{takeLabel(take, i, clip.takes.length)}
|
||||
</option>
|
||||
))}
|
||||
</NativeSelect>
|
||||
)}
|
||||
{canPreview && (
|
||||
<Button type="button" size="sm" variant="outline" onClick={() => setPreviewing((p) => !p)}>
|
||||
{previewing ? 'Hide video' : 'Preview'}
|
||||
</Button>
|
||||
)}
|
||||
{canPreview && (
|
||||
<Button type="button" size="sm" variant="outline" onClick={() => onTrimClip(clip)}>
|
||||
Edit video
|
||||
</Button>
|
||||
)}
|
||||
{canQueue && (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={() => queueOrDequeue(onQueueClip)}
|
||||
disabled={queueing}
|
||||
title="Run this clip when the ones queued before it have finished"
|
||||
>
|
||||
Queue
|
||||
</Button>
|
||||
)}
|
||||
{queued && (
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={() => queueOrDequeue(onDequeueClip)}
|
||||
disabled={queueing}
|
||||
>
|
||||
Remove from queue
|
||||
</Button>
|
||||
)}
|
||||
<Button type="button" size="sm" variant="outline" onClick={() => onOpenJob(job)}>
|
||||
{editable ? 'Edit' : 'View'}
|
||||
</Button>
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{previewing && canPreview && (
|
||||
<div className="mb-2">
|
||||
<ClipPreview job={job} videoVersion={videoVersionFor(job.id)} />
|
||||
</div>
|
||||
)}
|
||||
|
||||
{clip.shots.length === 0 ? (
|
||||
<p className="rounded-lg border border-dashed border-border p-3 text-sm text-muted-foreground">
|
||||
This job's prompt has no shots.
|
||||
</p>
|
||||
) : (
|
||||
<div className="flex items-stretch gap-2 overflow-x-auto pb-1" aria-label={`Shots in ${clipLabel(clip)}`}>
|
||||
{clip.shots.map((shot, i) => (
|
||||
<div
|
||||
key={shot.id}
|
||||
className="flex w-52 shrink-0 flex-col gap-1.5 rounded-xl border border-border bg-background p-2"
|
||||
>
|
||||
<H3ShotPanel
|
||||
shot={shot}
|
||||
number={firstShotNumber + i}
|
||||
timeLabel={formatCutTime(clip.shotStarts[i])}
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</li>
|
||||
);
|
||||
}
|
||||
|
||||
export interface SceneBoardProps {
|
||||
scene: Scene;
|
||||
onOpenJob: (job: Job) => void;
|
||||
/** start a clip from the last frame of the one before it; resolves when done (errors are reported by the caller) */
|
||||
onUseLastFrame: (clip: SceneClip, previous: SceneClip) => Promise<void>;
|
||||
/** pick which re-run of a clip number to show */
|
||||
onChooseTake: (takeKey: string, jobId: string) => void;
|
||||
/** open the confirmation for queueing every runnable clip of the scene */
|
||||
onQueueScene: (scene: Scene) => void;
|
||||
/** join the scene's finished clips into one video; resolves when done (errors are reported by the caller) */
|
||||
onMergeScene: (scene: Scene) => Promise<void>;
|
||||
/** stop starting a clip from a previous clip's last frame; resolves when done */
|
||||
onDetachLastFrame: (clip: SceneClip) => Promise<void>;
|
||||
/** queue / unqueue one clip; resolve when done (errors are reported by the caller) */
|
||||
onQueueClip: (clip: SceneClip) => Promise<void>;
|
||||
onDequeueClip: (clip: SceneClip) => Promise<void>;
|
||||
/** open the trim dialog for one clip's video */
|
||||
onTrimClip: (clip: SceneClip) => void;
|
||||
/** cache-busting version for a job's video preview, bumped after each edit */
|
||||
videoVersionFor: (jobId: string) => number;
|
||||
}
|
||||
|
||||
/** One scene: a timeline of its clips, then every clip's shots in order, numbered straight through. */
|
||||
export function SceneBoard({
|
||||
scene,
|
||||
onOpenJob,
|
||||
onChooseTake,
|
||||
onUseLastFrame,
|
||||
onDetachLastFrame,
|
||||
onQueueScene,
|
||||
onMergeScene,
|
||||
onQueueClip,
|
||||
onDequeueClip,
|
||||
onTrimClip,
|
||||
videoVersionFor,
|
||||
}: SceneBoardProps) {
|
||||
let shotNumber = 1;
|
||||
const statuses = Object.entries(scene.statusCounts);
|
||||
const anythingToQueue = scene.clips.some((c) => QUEUEABLE_STATUSES.includes(c.job.status));
|
||||
|
||||
// Merging needs every clip's video, so a scene missing one can't pass for the whole thing.
|
||||
const unfinished = scene.clips.filter((c) => c.job.status !== 'completed' || !c.job.output_path).length;
|
||||
const [merging, setMerging] = React.useState(false);
|
||||
async function merge() {
|
||||
setMerging(true);
|
||||
try {
|
||||
await onMergeScene(scene);
|
||||
} finally {
|
||||
setMerging(false);
|
||||
}
|
||||
}
|
||||
|
||||
// The header stays where it is until it reaches the top of the scrolling area,
|
||||
// then sticks there. A 1px sentinel sitting just above it tells us which: once
|
||||
// the sentinel has scrolled out of view, the header is stuck.
|
||||
const sentinelRef = React.useRef<HTMLDivElement>(null);
|
||||
const [stuck, setStuck] = React.useState(false);
|
||||
React.useEffect(() => {
|
||||
const el = sentinelRef.current;
|
||||
if (!el || typeof IntersectionObserver === 'undefined') return;
|
||||
const observer = new IntersectionObserver(([entry]) => setStuck(!entry.isIntersecting));
|
||||
observer.observe(el);
|
||||
return () => observer.disconnect();
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<section aria-label={scene.title} className="flex flex-col gap-4">
|
||||
<div ref={sentinelRef} aria-hidden className="-mb-4 h-px" />
|
||||
<div
|
||||
data-stuck={stuck}
|
||||
className={cn(
|
||||
'sticky top-0 z-30 -mx-3 flex flex-col gap-2 bg-card px-3 pb-3 pt-2 transition-shadow',
|
||||
stuck && 'border-b border-border shadow-md',
|
||||
)}
|
||||
>
|
||||
<div className="flex flex-wrap items-baseline gap-x-4 gap-y-1">
|
||||
<h2 className="text-xl font-semibold text-foreground">{scene.title}</h2>
|
||||
<p className="text-sm text-muted-foreground">
|
||||
{scene.clips.length} clip{scene.clips.length === 1 ? '' : 's'} · {scene.shotCount} shot
|
||||
{scene.shotCount === 1 ? '' : 's'} · about {formatClock(scene.totalSeconds)}
|
||||
</p>
|
||||
<span className="flex flex-wrap gap-1.5">
|
||||
{statuses.map(([status, count]) => (
|
||||
<Badge key={status} variant={BADGE_VARIANTS[status] ?? 'secondary'}>
|
||||
{count} {status}
|
||||
</Badge>
|
||||
))}
|
||||
</span>
|
||||
<span className="ml-auto flex flex-wrap gap-2">
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="outline"
|
||||
onClick={merge}
|
||||
disabled={unfinished > 0 || merging}
|
||||
title={
|
||||
unfinished > 0
|
||||
? `${unfinished} clip${unfinished === 1 ? " hasn't" : "s haven't"} finished yet`
|
||||
: 'Join all the clips, in order, into one video'
|
||||
}
|
||||
>
|
||||
{merging ? 'Merging…' : 'Merge scene'}
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
onClick={() => onQueueScene(scene)}
|
||||
disabled={!anythingToQueue}
|
||||
title={anythingToQueue ? 'Queue every clip that has not run yet' : 'Every clip has already run or is queued'}
|
||||
>
|
||||
Queue scene
|
||||
</Button>
|
||||
</span>
|
||||
</div>
|
||||
<Timeline scene={scene} />
|
||||
</div>
|
||||
|
||||
<ol className="flex list-none flex-col gap-3 p-0">
|
||||
{scene.clips.map((clip, index) => {
|
||||
const first = shotNumber;
|
||||
shotNumber += clip.shots.length;
|
||||
return (
|
||||
<ClipRow
|
||||
key={clip.takeKey ?? clip.job.id}
|
||||
clip={clip}
|
||||
previous={scene.clips[index - 1]}
|
||||
firstShotNumber={first}
|
||||
onOpenJob={onOpenJob}
|
||||
onChooseTake={onChooseTake}
|
||||
onUseLastFrame={onUseLastFrame}
|
||||
onDetachLastFrame={onDetachLastFrame}
|
||||
onQueueClip={onQueueClip}
|
||||
onDequeueClip={onDequeueClip}
|
||||
onTrimClip={onTrimClip}
|
||||
videoVersionFor={videoVersionFor}
|
||||
/>
|
||||
);
|
||||
})}
|
||||
</ol>
|
||||
</section>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,132 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { attachedLastFrames } from '@/lib/continuation';
|
||||
import { planSceneQueue, QUEUEABLE_STATUSES, type SceneQueuePlan } from '@/lib/sceneQueue';
|
||||
import type { Scene, SceneClip } from '@/lib/scenes';
|
||||
|
||||
const plural = (n: number, word: string) => `${n} ${word}${n === 1 ? '' : 's'}`;
|
||||
const clipNumbers = (clips: SceneClip[]) => clips.map((c) => c.position).join(', ');
|
||||
|
||||
export interface SceneQueueDialogProps {
|
||||
scene: Scene | null;
|
||||
onConfirm: (plan: SceneQueuePlan) => Promise<void>;
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
/**
|
||||
* Confirm queueing a whole scene: which clips are queued, and whether each
|
||||
* starts from the previous clip's last frame. Nothing is sent until "Queue".
|
||||
*/
|
||||
export function SceneQueueDialog({ scene, onConfirm, onClose }: SceneQueueDialogProps) {
|
||||
// Off by default: the per-clip "Use last frame" boxes are the source of truth, and
|
||||
// this would add links to every clip that doesn't have one.
|
||||
const [chain, setChain] = React.useState(false);
|
||||
const [busy, setBusy] = React.useState(false);
|
||||
|
||||
// Each opening starts from the default choice.
|
||||
React.useEffect(() => {
|
||||
if (scene) setChain(false);
|
||||
}, [scene]);
|
||||
|
||||
const plan = React.useMemo(() => (scene ? planSceneQueue(scene.clips, { chain }) : null), [scene, chain]);
|
||||
const alreadyLinked = React.useMemo(
|
||||
() =>
|
||||
scene
|
||||
? scene.clips.filter(
|
||||
(c) => QUEUEABLE_STATUSES.includes(c.job.status) && attachedLastFrames(c.job.references).length > 0,
|
||||
)
|
||||
: [],
|
||||
[scene],
|
||||
);
|
||||
// Whether the option would do anything is independent of the checkbox.
|
||||
const linkable = React.useMemo(
|
||||
() => (scene ? planSceneQueue(scene.clips, { chain: true }) : null),
|
||||
[scene],
|
||||
);
|
||||
|
||||
async function confirm() {
|
||||
if (!plan) return;
|
||||
setBusy(true);
|
||||
try {
|
||||
await onConfirm(plan);
|
||||
} finally {
|
||||
setBusy(false);
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Dialog open={!!scene} onOpenChange={(open) => !open && !busy && onClose()}>
|
||||
<DialogContent>
|
||||
<DialogHeader>
|
||||
<DialogTitle>Queue {scene?.title}</DialogTitle>
|
||||
<DialogDescription>
|
||||
{plan && plan.queue.length > 0
|
||||
? `${plural(plan.queue.length, 'clip')} will be queued and run one at a time, in order.`
|
||||
: 'There is nothing in this scene to queue.'}
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
{plan && linkable && (
|
||||
<div className="flex flex-col gap-3 text-sm">
|
||||
{alreadyLinked.length > 0 && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Clip {clipNumbers(alreadyLinked)} already start{alreadyLinked.length === 1 ? 's' : ''} from a previous
|
||||
frame (as you set on the clip{alreadyLinked.length === 1 ? '' : 's'}). Other clips are queued as they are.
|
||||
</p>
|
||||
)}
|
||||
{linkable.chain.length > 0 && (
|
||||
<label className="flex items-start gap-2">
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={chain}
|
||||
onChange={(e) => setChain(e.target.checked)}
|
||||
className="mt-1 size-4"
|
||||
/>
|
||||
<span>
|
||||
Start each clip from the previous clip's last frame
|
||||
<span className="block text-xs text-muted-foreground">
|
||||
Also links {plural(linkable.chain.length, 'clip')} that don't have it (
|
||||
{clipNumbers(linkable.chain.map((c) => c.clip))}), rewriting their prompts. Each waits for the clip
|
||||
before it to finish.
|
||||
</span>
|
||||
</span>
|
||||
</label>
|
||||
)}
|
||||
{chain && plan.cannotChain.length > 0 && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Clip {clipNumbers(plan.cannotChain.map((c) => c.clip))} can't be linked automatically (its prompt
|
||||
isn't in the six-section format) and will start without the previous frame.
|
||||
</p>
|
||||
)}
|
||||
{plan.skipped.length > 0 && (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Skipping{' '}
|
||||
{plan.skipped.map((s) => `clip ${s.clip.position} (${s.reason})`).join(', ')}.
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<DialogFooter>
|
||||
<Button type="button" variant="outline" onClick={onClose} disabled={busy}>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button type="button" onClick={confirm} disabled={busy || !plan || plan.queue.length === 0}>
|
||||
{busy ? 'Queueing…' : plan ? `Queue ${plural(plan.queue.length, 'clip')}` : 'Queue'}
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,500 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
import { toast } from 'sonner';
|
||||
|
||||
import { BeforeAfterGrade } from '@/components/scenes/BeforeAfterGrade';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from '@/components/ui/dialog';
|
||||
import { Input } from '@/components/ui/input';
|
||||
import { Label } from '@/components/ui/label';
|
||||
import { Slider } from '@/components/ui/slider';
|
||||
import { getJobVideoUrl, restoreJobVideo, trimJob } from '@/lib/api';
|
||||
import { BRIGHTNESS_BOUNDS, CONTRAST_BOUNDS, NEUTRAL_GRADE, SATURATION_BOUNDS } from '@/lib/grading';
|
||||
import { clipSeconds, DEFAULT_FPS, formatClock } from '@/lib/scenes';
|
||||
import type { GradeSegment, Job } from '@/lib/types';
|
||||
import { cn } from '@/lib/utils';
|
||||
|
||||
export interface TrimClipDialogProps {
|
||||
/** the clip to edit, or null when closed */
|
||||
clip: Job | null;
|
||||
/**
|
||||
* Cache-busts the preview after an edit (the video keeps the same URL, only
|
||||
* the file changes). Owned by the caller, not this component -- a version
|
||||
* counter kept here gets reset by the very re-render `onChanged` causes
|
||||
* (a fresh `clip` object comes back from the edit), before the browser
|
||||
* ever sees the new URL.
|
||||
*/
|
||||
videoVersion: number;
|
||||
onClose: () => void;
|
||||
/** called with the job as the edit/restore left it, so the caller can refresh */
|
||||
onChanged: (job: Job) => void;
|
||||
}
|
||||
|
||||
/** A section as edited locally -- same shape as GradeSegment plus a stable React key. */
|
||||
interface EditableSection extends GradeSegment {
|
||||
key: string;
|
||||
}
|
||||
|
||||
let sectionSeq = 0;
|
||||
function makeSectionKey(): string {
|
||||
return `section-${Date.now()}-${sectionSeq++}`;
|
||||
}
|
||||
|
||||
const NEUTRAL_SECTION: Omit<EditableSection, 'key'> = { end_seconds: null, ...NEUTRAL_GRADE };
|
||||
|
||||
/** The clip's last-applied sections, or one neutral section covering everything if it was never edited. */
|
||||
function sectionsFromJob(job: Job): EditableSection[] {
|
||||
const segments = job.edit_segments;
|
||||
if (segments && segments.length > 0) {
|
||||
return segments.map((s) => ({ ...s, key: makeSectionKey() }));
|
||||
}
|
||||
return [{ ...NEUTRAL_SECTION, key: makeSectionKey() }];
|
||||
}
|
||||
|
||||
// A split closer than this to an existing edge would leave a sliver too thin
|
||||
// to be a meaningful section (and trim.py would likely refuse it outright).
|
||||
const MIN_SPLIT_MARGIN_SECONDS = 0.1;
|
||||
|
||||
/**
|
||||
* Cut a finished clip to a chosen in/out range and grade it section by
|
||||
* section -- like a razor tool: "Split at current time" cuts the clip where
|
||||
* the preview is paused, and each resulting section gets its own brightness,
|
||||
* contrast and saturation. Previous/Next frame step the preview by exactly
|
||||
* one frame (at the clip's own fps), for lining a cut up precisely -- the
|
||||
* native scrub bar alone isn't frame-accurate. The server always keeps the
|
||||
* untouched original and re-renders from it with the full set of values
|
||||
* shown here, so range and sections are applied together and neither is
|
||||
* lost by changing the other --
|
||||
* as long as both are set the way you want them before you click Apply. The
|
||||
* fields keep whatever you last applied while this dialog stays open for the
|
||||
* same clip, so a second, smaller tweak doesn't also reset the range. Opening
|
||||
* it fresh for a clip (or reopening it later) shows exactly what's currently
|
||||
* applied -- the server records the range/sections of the last edit on the
|
||||
* job itself (`edit_*`), so this isn't guessing from the clip's current
|
||||
* length. A never-edited clip shows one neutral section covering its full
|
||||
* length. Restoring clears the fields back to that same neutral state, since
|
||||
* that's what it just did to the video.
|
||||
*/
|
||||
export function TrimClipDialog({ clip, videoVersion, onClose, onChanged }: TrimClipDialogProps) {
|
||||
const duration = clip ? clipSeconds(clip) : 0;
|
||||
const fps = clip?.fps && clip.fps > 0 ? clip.fps : DEFAULT_FPS;
|
||||
const [start, setStart] = React.useState('0');
|
||||
const [end, setEnd] = React.useState('');
|
||||
const [sections, setSections] = React.useState<EditableSection[]>([]);
|
||||
const [previewIndex, setPreviewIndex] = React.useState(0);
|
||||
const [busy, setBusy] = React.useState<'apply' | 'restore' | null>(null);
|
||||
const [displayTime, setDisplayTime] = React.useState(0);
|
||||
|
||||
// Reset the fields only when a *different* clip opens -- onChanged() re-renders
|
||||
// this dialog with a fresh job object for the same edit, which must not reset
|
||||
// what the user is about to apply next.
|
||||
const openClipId = React.useRef<string | null>(null);
|
||||
React.useEffect(() => {
|
||||
if (clip && clip.id !== openClipId.current) {
|
||||
openClipId.current = clip.id;
|
||||
setStart(String(clip.edit_start_seconds ?? 0));
|
||||
setEnd(clip.edit_end_seconds != null ? clip.edit_end_seconds.toFixed(2) : clipSeconds(clip).toFixed(2));
|
||||
setSections(sectionsFromJob(clip));
|
||||
setPreviewIndex(0);
|
||||
} else if (!clip) {
|
||||
openClipId.current = null;
|
||||
}
|
||||
}, [clip]);
|
||||
|
||||
// This dialog is always mounted by the page (it just renders closed when
|
||||
// `clip` is null), so a `useEffect(fn, [])` here would only ever fire once,
|
||||
// the very first time -- before `clip` is ever set and before the <video>
|
||||
// below exists at all -- and never again on later opens. A callback ref
|
||||
// fires on every real mount/unmount of the DOM node itself, however many
|
||||
// times that happens across this component's one long lifetime, which is
|
||||
// what both of these actually need:
|
||||
// - setting the *initial* muted state imperatively, so it doesn't fight the
|
||||
// user's own unmute via the native controls (`muted` as a plain JSX prop
|
||||
// is controlled, and React would re-apply it on every re-render instead)
|
||||
// - keeping the frame-step readout in sync with wherever the video is,
|
||||
// whether that's a step button, a Preview click, or the native scrub bar
|
||||
const videoRef = React.useRef<HTMLVideoElement | null>(null);
|
||||
const setVideoNode = React.useCallback((node: HTMLVideoElement | null) => {
|
||||
videoRef.current = node;
|
||||
if (!node) return;
|
||||
node.muted = true;
|
||||
const onTimeChange = () => setDisplayTime(node.currentTime);
|
||||
node.addEventListener('timeupdate', onTimeChange);
|
||||
node.addEventListener('seeked', onTimeChange);
|
||||
return () => {
|
||||
node.removeEventListener('timeupdate', onTimeChange);
|
||||
node.removeEventListener('seeked', onTimeChange);
|
||||
};
|
||||
}, []);
|
||||
|
||||
if (!clip) {
|
||||
return <Dialog open={false} onOpenChange={() => {}} />;
|
||||
}
|
||||
|
||||
const startSeconds = Number(start);
|
||||
const endSeconds = end.trim() === '' ? undefined : Number(end);
|
||||
|
||||
const rangeError =
|
||||
!Number.isFinite(startSeconds) || startSeconds < 0
|
||||
? 'The start must be 0 or later.'
|
||||
: endSeconds !== undefined && (!Number.isFinite(endSeconds) || endSeconds <= startSeconds)
|
||||
? 'The end must come after the start.'
|
||||
: null;
|
||||
// Color has no equivalent error: the sliders can't produce a value outside their bounds,
|
||||
// and sections are only ever built by split/merge below, which keep them in valid order.
|
||||
const invalid = !!rangeError;
|
||||
|
||||
/** The [start, end) of section `i`, in the kept range's own seconds -- the same timeline the preview plays. */
|
||||
function sectionBounds(i: number): [number, number] {
|
||||
const start = i === 0 ? 0 : (sections[i - 1].end_seconds ?? duration);
|
||||
const end = sections[i].end_seconds ?? duration;
|
||||
return [start, end];
|
||||
}
|
||||
|
||||
function updateSection(i: number, patch: Partial<Omit<EditableSection, 'key' | 'end_seconds'>>) {
|
||||
setSections((prev) => prev.map((s, idx) => (idx === i ? { ...s, ...patch } : s)));
|
||||
}
|
||||
|
||||
/**
|
||||
* Seeks the preview to `target`, clamped to the clip's length. A seek
|
||||
* requested before the video has loaded metadata (readyState is still
|
||||
* HAVE_NOTHING) is unreliable -- browsers commonly drop or silently reset it
|
||||
* once metadata actually arrives -- so this defers to 'loadedmetadata' when
|
||||
* that's the case, instead of jumping to the wrong spot (often 0).
|
||||
*/
|
||||
function seekTo(target: number) {
|
||||
const video = videoRef.current;
|
||||
if (!video) return;
|
||||
const clamped = Math.max(0, Math.min(duration, target));
|
||||
const apply = () => {
|
||||
video.currentTime = clamped;
|
||||
};
|
||||
if (video.readyState >= 1) {
|
||||
apply();
|
||||
} else {
|
||||
video.addEventListener('loadedmetadata', apply, { once: true });
|
||||
}
|
||||
}
|
||||
|
||||
/** Steps the preview by exactly one frame (at the clip's own fps), pausing playback first. */
|
||||
function stepFrame(direction: 1 | -1) {
|
||||
const video = videoRef.current;
|
||||
if (!video) return;
|
||||
video.pause();
|
||||
seekTo(video.currentTime + direction / fps);
|
||||
}
|
||||
|
||||
function splitAtCurrentTime() {
|
||||
const video = videoRef.current;
|
||||
if (!video) return;
|
||||
const t = video.currentTime;
|
||||
for (let i = 0; i < sections.length; i++) {
|
||||
const [sectionStart, sectionEnd] = sectionBounds(i);
|
||||
if (t <= sectionStart + MIN_SPLIT_MARGIN_SECONDS || t >= sectionEnd - MIN_SPLIT_MARGIN_SECONDS) continue;
|
||||
setSections((prev) => {
|
||||
const next = [...prev];
|
||||
next.splice(
|
||||
i,
|
||||
1,
|
||||
{ ...prev[i], end_seconds: t },
|
||||
{ ...NEUTRAL_SECTION, end_seconds: prev[i].end_seconds, key: makeSectionKey() },
|
||||
);
|
||||
return next;
|
||||
});
|
||||
setPreviewIndex(i + 1);
|
||||
return;
|
||||
}
|
||||
toast.error('Pause the preview somewhere inside a section (away from its edges) to split it there.');
|
||||
}
|
||||
|
||||
function mergeWithNext(i: number) {
|
||||
setSections((prev) => {
|
||||
const next = [...prev];
|
||||
next[i] = { ...next[i], end_seconds: next[i + 1].end_seconds };
|
||||
next.splice(i + 1, 1);
|
||||
return next;
|
||||
});
|
||||
setPreviewIndex((p) => Math.min(p, i));
|
||||
}
|
||||
|
||||
/** Seeks the preview to this section's start and previews it; BeforeAfterGrade re-captures on 'seeked'. */
|
||||
function previewSection(i: number) {
|
||||
const [sectionStart] = sectionBounds(i);
|
||||
seekTo(sectionStart);
|
||||
setPreviewIndex(i);
|
||||
}
|
||||
|
||||
/** Keeps the preview panel's selected section in sync with wherever the video actually is. */
|
||||
function handleFrameCaptured(t: number) {
|
||||
for (let i = 0; i < sections.length; i++) {
|
||||
const [, sectionEnd] = sectionBounds(i);
|
||||
if (t < sectionEnd || i === sections.length - 1) {
|
||||
setPreviewIndex(i);
|
||||
return;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function apply() {
|
||||
if (!clip || invalid) return;
|
||||
setBusy('apply');
|
||||
try {
|
||||
const updated = await trimJob(clip.id, {
|
||||
startSeconds,
|
||||
endSeconds,
|
||||
segments: sections.map(({ key: _key, ...segment }) => segment),
|
||||
});
|
||||
toast.success(`Applied: ${formatClock(startSeconds)}–${endSeconds === undefined ? 'end' : formatClock(endSeconds)}.`);
|
||||
onChanged(updated);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't edit the clip.");
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
}
|
||||
|
||||
async function restore() {
|
||||
if (!clip) return;
|
||||
setBusy('restore');
|
||||
try {
|
||||
const updated = await restoreJobVideo(clip.id);
|
||||
// Restoring doesn't open a *different* clip, so the reset effect above
|
||||
// won't fire on its own -- but the fields have to reflect it too, since
|
||||
// that's exactly what just happened to the video.
|
||||
setStart('0');
|
||||
setEnd(clipSeconds(updated).toFixed(2));
|
||||
setSections([{ ...NEUTRAL_SECTION, key: makeSectionKey() }]);
|
||||
setPreviewIndex(0);
|
||||
toast.success('Restored the original video.');
|
||||
onChanged(updated);
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Couldn't restore the original video.");
|
||||
} finally {
|
||||
setBusy(null);
|
||||
}
|
||||
}
|
||||
|
||||
const preview = sections[previewIndex] ?? sections[0];
|
||||
|
||||
return (
|
||||
<Dialog open onOpenChange={(open) => !open && !busy && onClose()}>
|
||||
<DialogContent className="max-h-[90vh] max-w-lg overflow-y-auto">
|
||||
<DialogHeader>
|
||||
<DialogTitle>Edit {clip.name?.trim() || clip.id.slice(0, 8)}</DialogTitle>
|
||||
<DialogDescription>
|
||||
Currently {formatClock(duration)}. Editing keeps the untouched original, so this can be adjusted or undone.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
<video
|
||||
ref={setVideoNode}
|
||||
src={getJobVideoUrl(clip.id, videoVersion)}
|
||||
aria-label={`Preview of ${clip.name ?? clip.id}`}
|
||||
className="max-h-64 w-full rounded-md border border-border bg-black"
|
||||
controls
|
||||
playsInline
|
||||
preload="metadata"
|
||||
crossOrigin="anonymous"
|
||||
/>
|
||||
|
||||
<div className="flex items-center justify-between gap-2">
|
||||
<div className="flex gap-2">
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => stepFrame(-1)} disabled={!!busy}>
|
||||
◂ Previous frame
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={() => stepFrame(1)} disabled={!!busy}>
|
||||
Next frame ▸
|
||||
</Button>
|
||||
</div>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">
|
||||
Frame {Math.round(displayTime * fps)} · {displayTime.toFixed(2)}s
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<div className="flex flex-col gap-2">
|
||||
<p className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Range</p>
|
||||
<div className="flex items-end gap-3">
|
||||
<div className="flex flex-1 flex-col gap-1 text-sm">
|
||||
<Label htmlFor="trim-start">Start (seconds)</Label>
|
||||
<Input
|
||||
id="trim-start"
|
||||
type="number"
|
||||
min={0}
|
||||
step={0.1}
|
||||
value={start}
|
||||
onChange={(e) => setStart(e.target.value)}
|
||||
disabled={!!busy}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex flex-1 flex-col gap-1 text-sm">
|
||||
<Label htmlFor="trim-end">End (seconds, optional)</Label>
|
||||
<Input
|
||||
id="trim-end"
|
||||
type="number"
|
||||
min={0}
|
||||
step={0.1}
|
||||
placeholder="end of clip"
|
||||
value={end}
|
||||
onChange={(e) => setEnd(e.target.value)}
|
||||
disabled={!!busy}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
{rangeError && <p className="text-sm text-destructive">{rangeError}</p>}
|
||||
</div>
|
||||
|
||||
<div className="flex flex-col gap-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<p className="text-xs font-medium uppercase tracking-wide text-muted-foreground">Sections</p>
|
||||
<Button type="button" variant="outline" size="sm" onClick={splitAtCurrentTime} disabled={!!busy}>
|
||||
Split at current time
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{sections.map((section, i) => {
|
||||
const [sectionStart, sectionEnd] = sectionBounds(i);
|
||||
return (
|
||||
<div
|
||||
key={section.key}
|
||||
className={cn(
|
||||
'flex flex-col gap-2 rounded-md border p-3',
|
||||
i === previewIndex ? 'border-accent-blue' : 'border-border',
|
||||
)}
|
||||
>
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-xs font-medium text-muted-foreground">
|
||||
Section {i + 1} · {formatClock(sectionStart)}–{section.end_seconds == null ? 'end' : formatClock(sectionEnd)}
|
||||
</span>
|
||||
<div className="flex gap-3">
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => previewSection(i)}
|
||||
disabled={!!busy}
|
||||
className="text-xs text-accent-blue underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
Preview
|
||||
</button>
|
||||
{i < sections.length - 1 && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => mergeWithNext(i)}
|
||||
disabled={!!busy}
|
||||
className="text-xs text-accent-blue underline-offset-2 hover:underline disabled:cursor-not-allowed disabled:opacity-50"
|
||||
>
|
||||
Merge with next
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
<GradeSlider
|
||||
id={`grade-brightness-${section.key}`}
|
||||
label="Brightness"
|
||||
ariaLabel={`Section ${i + 1} Brightness`}
|
||||
bounds={BRIGHTNESS_BOUNDS}
|
||||
step={5}
|
||||
value={section.brightness}
|
||||
onChange={(v) => updateSection(i, { brightness: v })}
|
||||
format={(v) => String(v)}
|
||||
disabled={!!busy}
|
||||
/>
|
||||
<GradeSlider
|
||||
id={`grade-contrast-${section.key}`}
|
||||
label="Contrast"
|
||||
ariaLabel={`Section ${i + 1} Contrast`}
|
||||
bounds={CONTRAST_BOUNDS}
|
||||
step={0.1}
|
||||
value={section.contrast}
|
||||
onChange={(v) => updateSection(i, { contrast: v })}
|
||||
format={(v) => `${v.toFixed(1)}x`}
|
||||
disabled={!!busy}
|
||||
/>
|
||||
<GradeSlider
|
||||
id={`grade-saturation-${section.key}`}
|
||||
label="Saturation"
|
||||
ariaLabel={`Section ${i + 1} Saturation`}
|
||||
bounds={SATURATION_BOUNDS}
|
||||
step={0.1}
|
||||
value={section.saturation}
|
||||
onChange={(v) => updateSection(i, { saturation: v })}
|
||||
format={(v) => `${v.toFixed(1)}x`}
|
||||
disabled={!!busy}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
|
||||
<BeforeAfterGrade
|
||||
videoRef={videoRef}
|
||||
brightness={preview?.brightness ?? NEUTRAL_GRADE.brightness}
|
||||
contrast={preview?.contrast ?? NEUTRAL_GRADE.contrast}
|
||||
saturation={preview?.saturation ?? NEUTRAL_GRADE.saturation}
|
||||
onFrameCaptured={handleFrameCaptured}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<DialogFooter className="justify-between sm:justify-between">
|
||||
<Button type="button" variant="outline" onClick={restore} disabled={!!busy}>
|
||||
{busy === 'restore' ? 'Restoring…' : 'Restore original'}
|
||||
</Button>
|
||||
<div className="flex gap-2">
|
||||
<Button type="button" variant="outline" onClick={onClose} disabled={!!busy}>
|
||||
Close
|
||||
</Button>
|
||||
<Button type="button" onClick={apply} disabled={!!busy || invalid}>
|
||||
{busy === 'apply' ? 'Applying…' : 'Apply'}
|
||||
</Button>
|
||||
</div>
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
|
||||
function GradeSlider({
|
||||
id,
|
||||
label,
|
||||
ariaLabel,
|
||||
bounds,
|
||||
step,
|
||||
value,
|
||||
onChange,
|
||||
format,
|
||||
disabled,
|
||||
}: {
|
||||
id: string;
|
||||
label: string;
|
||||
ariaLabel: string;
|
||||
bounds: readonly [number, number];
|
||||
step: number;
|
||||
value: number;
|
||||
onChange: (v: number) => void;
|
||||
format: (v: number) => string;
|
||||
disabled?: boolean;
|
||||
}) {
|
||||
return (
|
||||
<div className="flex flex-col gap-1 text-sm">
|
||||
<div className="flex items-center justify-between">
|
||||
<Label htmlFor={id}>{label}</Label>
|
||||
<span className="text-xs tabular-nums text-muted-foreground">{format(value)}</span>
|
||||
</div>
|
||||
<Slider
|
||||
id={id}
|
||||
min={bounds[0]}
|
||||
max={bounds[1]}
|
||||
step={step}
|
||||
value={[value]}
|
||||
onValueChange={(v) => onChange(v[0])}
|
||||
disabled={disabled}
|
||||
aria-label={ariaLabel}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -13,6 +13,7 @@ const TAB_TITLES: Record<string, string> = {
|
||||
'/distillation': 'Jobs',
|
||||
'/datasets': 'Datasets',
|
||||
'/gallery': 'Gallery',
|
||||
'/scenes': 'Scenes',
|
||||
'/gpus': 'GPUs',
|
||||
'/settings': 'Settings',
|
||||
};
|
||||
|
||||
@@ -150,6 +150,14 @@ export default function PrimarySidebar({
|
||||
>
|
||||
Datasets
|
||||
</Link>
|
||||
<Link
|
||||
href="/scenes"
|
||||
aria-current={pathname === '/scenes' ? 'page' : undefined}
|
||||
onClick={onMobileClose}
|
||||
className={cn(TAB_BASE, pathname === '/scenes' && TAB_ACTIVE)}
|
||||
>
|
||||
Scenes
|
||||
</Link>
|
||||
<Link
|
||||
href="/gallery"
|
||||
aria-current={pathname === '/gallery' ? 'page' : undefined}
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
import { render, screen } from '@testing-library/react';
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { Dialog, DialogContent, DialogDescription, DialogTitle } from '@/components/ui/dialog';
|
||||
|
||||
// The app header is `fixed` at z-[100] (components/shell/Header.tsx).
|
||||
const HEADER_Z = 100;
|
||||
|
||||
function zOf(el: Element | null): number {
|
||||
const m = el?.className.match(/\bz-\[(\d+)\]/);
|
||||
return m ? Number(m[1]) : 0;
|
||||
}
|
||||
|
||||
describe('Dialog stacking', () => {
|
||||
it('renders its overlay and content above the fixed app header', () => {
|
||||
render(
|
||||
<Dialog open>
|
||||
<DialogContent>
|
||||
<DialogTitle>Tall dialog</DialogTitle>
|
||||
<DialogDescription>Body</DialogDescription>
|
||||
</DialogContent>
|
||||
</Dialog>,
|
||||
);
|
||||
const content = screen.getByRole('dialog');
|
||||
const overlay = document.querySelector('[data-state="open"].bg-black\\/70');
|
||||
|
||||
expect(zOf(content)).toBeGreaterThan(HEADER_Z);
|
||||
expect(zOf(overlay)).toBeGreaterThan(HEADER_Z);
|
||||
});
|
||||
});
|
||||
@@ -11,6 +11,8 @@ const DialogTrigger = DialogPrimitive.Trigger;
|
||||
const DialogPortal = DialogPrimitive.Portal;
|
||||
const DialogClose = DialogPrimitive.Close;
|
||||
|
||||
// Dialogs sit above the app header (z-[100], fixed). Below it, a tall dialog's
|
||||
// top edge slides under the header and, being fixed itself, can't be scrolled clear.
|
||||
const DialogOverlay = React.forwardRef<
|
||||
React.ElementRef<typeof DialogPrimitive.Overlay>,
|
||||
React.ComponentPropsWithoutRef<typeof DialogPrimitive.Overlay>
|
||||
@@ -18,7 +20,7 @@ const DialogOverlay = React.forwardRef<
|
||||
<DialogPrimitive.Overlay
|
||||
ref={ref}
|
||||
className={cn(
|
||||
'fixed inset-0 z-50 bg-black/70 backdrop-blur-sm data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
|
||||
'fixed inset-0 z-[110] bg-black/70 backdrop-blur-sm data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
@@ -36,7 +38,7 @@ const DialogContent = React.forwardRef<
|
||||
ref={ref}
|
||||
data-modal=""
|
||||
className={cn(
|
||||
'fixed left-1/2 top-1/2 z-50 grid w-full max-w-lg -translate-x-1/2 -translate-y-1/2 gap-4 rounded-2xl border border-border bg-card p-6 text-card-foreground shadow-lg duration-200 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
|
||||
'fixed left-1/2 top-1/2 z-[110] grid w-full max-w-lg -translate-x-1/2 -translate-y-1/2 gap-4 rounded-2xl border border-border bg-card p-6 text-card-foreground shadow-lg duration-200 data-[state=open]:animate-in data-[state=closed]:animate-out data-[state=closed]:fade-out-0 data-[state=open]:fade-in-0',
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
import { act, renderHook } from '@testing-library/react';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { COALESCE_MS, useUndoableState } from './useUndoableState';
|
||||
|
||||
describe('useUndoableState', () => {
|
||||
let now = 1_000_000;
|
||||
beforeEach(() => {
|
||||
now = 1_000_000;
|
||||
vi.spyOn(Date, 'now').mockImplementation(() => now);
|
||||
});
|
||||
afterEach(() => vi.restoreAllMocks());
|
||||
|
||||
const tick = (ms: number) => {
|
||||
now += ms;
|
||||
};
|
||||
|
||||
it('makes each unkeyed change its own step', () => {
|
||||
const { result } = renderHook(() => useUndoableState({ n: 0 }));
|
||||
act(() => result.current.set({ n: 1 }));
|
||||
act(() => result.current.set({ n: 2 }));
|
||||
expect(result.current.state.n).toBe(2);
|
||||
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state.n).toBe(1);
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state.n).toBe(0);
|
||||
expect(result.current.canUndo).toBe(false);
|
||||
});
|
||||
|
||||
it('merges same-key edits made close together into one step', () => {
|
||||
const { result } = renderHook(() => useUndoableState('a'));
|
||||
act(() => result.current.set('ab', 'text'));
|
||||
tick(100);
|
||||
act(() => result.current.set('abc', 'text'));
|
||||
tick(100);
|
||||
act(() => result.current.set('abcd', 'text'));
|
||||
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state).toBe('a');
|
||||
expect(result.current.canUndo).toBe(false);
|
||||
});
|
||||
|
||||
it('starts a new step after a pause, or when the key changes', () => {
|
||||
const { result } = renderHook(() => useUndoableState('a'));
|
||||
act(() => result.current.set('ab', 'text'));
|
||||
tick(COALESCE_MS + 1);
|
||||
act(() => result.current.set('abc', 'text'));
|
||||
act(() => result.current.set('abcX', 'other'));
|
||||
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state).toBe('abc');
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state).toBe('ab');
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state).toBe('a');
|
||||
});
|
||||
|
||||
it('never merges an unkeyed change into typing, or typing into it', () => {
|
||||
const { result } = renderHook(() => useUndoableState('a'));
|
||||
act(() => result.current.set('ab', 'text'));
|
||||
act(() => result.current.set('AB'));
|
||||
act(() => result.current.set('ABc', 'text'));
|
||||
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state).toBe('AB');
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state).toBe('ab');
|
||||
});
|
||||
|
||||
it('typing after an undo begins a fresh step instead of merging into the undone one', () => {
|
||||
const { result } = renderHook(() => useUndoableState('a'));
|
||||
act(() => result.current.set('ab', 'text'));
|
||||
act(() => void result.current.undo());
|
||||
act(() => result.current.set('ax', 'text'));
|
||||
expect(result.current.canUndo).toBe(true);
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state).toBe('a');
|
||||
});
|
||||
|
||||
it('redoes, and a new change clears what could be redone', () => {
|
||||
const { result } = renderHook(() => useUndoableState(0));
|
||||
act(() => result.current.set(1));
|
||||
act(() => result.current.set(2));
|
||||
act(() => void result.current.undo());
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.canRedo).toBe(true);
|
||||
|
||||
act(() => void result.current.redo());
|
||||
expect(result.current.state).toBe(1);
|
||||
act(() => result.current.set(9));
|
||||
expect(result.current.canRedo).toBe(false);
|
||||
expect(result.current.redo()).toBe(false);
|
||||
});
|
||||
|
||||
it('returns false when there is nothing to undo or redo', () => {
|
||||
const { result } = renderHook(() => useUndoableState(0));
|
||||
expect(result.current.undo()).toBe(false);
|
||||
expect(result.current.redo()).toBe(false);
|
||||
});
|
||||
|
||||
it('chains functional updates made in the same event', () => {
|
||||
const { result } = renderHook(() => useUndoableState(0));
|
||||
act(() => {
|
||||
result.current.set((n) => n + 1);
|
||||
result.current.set((n) => n + 1);
|
||||
});
|
||||
expect(result.current.state).toBe(2);
|
||||
act(() => void result.current.undo());
|
||||
expect(result.current.state).toBe(1);
|
||||
});
|
||||
|
||||
it('ignores a set that produces the same value', () => {
|
||||
const { result } = renderHook(() => useUndoableState('x'));
|
||||
act(() => result.current.set('x'));
|
||||
expect(result.current.canUndo).toBe(false);
|
||||
});
|
||||
|
||||
it('reset replaces the value and drops all history', () => {
|
||||
const { result } = renderHook(() => useUndoableState(0));
|
||||
act(() => result.current.set(1));
|
||||
act(() => void result.current.undo());
|
||||
act(() => result.current.reset(7));
|
||||
expect(result.current.state).toBe(7);
|
||||
expect(result.current.canUndo).toBe(false);
|
||||
expect(result.current.canRedo).toBe(false);
|
||||
});
|
||||
|
||||
it('keeps at most 100 steps', () => {
|
||||
const { result } = renderHook(() => useUndoableState(0));
|
||||
for (let i = 1; i <= 130; i += 1) act(() => result.current.set(i));
|
||||
let undone = 0;
|
||||
while (result.current.canUndo) {
|
||||
act(() => void result.current.undo());
|
||||
undone += 1;
|
||||
}
|
||||
expect(undone).toBe(100);
|
||||
expect(result.current.state).toBe(30);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,86 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
const MAX_HISTORY = 100;
|
||||
// Edits made under the same key, this close together, merge into one undo step.
|
||||
export const COALESCE_MS = 500;
|
||||
|
||||
interface Snapshot<T> {
|
||||
present: T;
|
||||
past: T[];
|
||||
future: T[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Undo/redo over a whole state value. Every `set` is one undo step, except
|
||||
* that consecutive sets with the same `key` inside COALESCE_MS merge -- pass a
|
||||
* key for typing so a burst is one step, and none for discrete changes (moves,
|
||||
* adds, selects) so each is its own. Values are treated as immutable, so
|
||||
* snapshots share structure and are cheap to keep.
|
||||
*/
|
||||
export function useUndoableState<T>(initial: T) {
|
||||
const [snapshot, setSnapshot] = React.useState<Snapshot<T>>({ present: initial, past: [], future: [] });
|
||||
// Mirrors the snapshot synchronously so several sets in one event chain correctly.
|
||||
const ref = React.useRef(snapshot);
|
||||
const last = React.useRef<{ key: string | null; at: number }>({ key: null, at: 0 });
|
||||
|
||||
const apply = React.useCallback((next: Snapshot<T>) => {
|
||||
ref.current = next;
|
||||
setSnapshot(next);
|
||||
}, []);
|
||||
|
||||
const set = React.useCallback(
|
||||
(update: T | ((prev: T) => T), key: string | null = null) => {
|
||||
const cur = ref.current;
|
||||
const next = typeof update === 'function' ? (update as (prev: T) => T)(cur.present) : update;
|
||||
if (Object.is(next, cur.present)) return;
|
||||
|
||||
const now = Date.now();
|
||||
const merges = key !== null && last.current.key === key && now - last.current.at < COALESCE_MS;
|
||||
last.current = { key, at: now };
|
||||
|
||||
if (merges) {
|
||||
apply({ ...cur, present: next });
|
||||
} else {
|
||||
apply({ present: next, past: [...cur.past.slice(-(MAX_HISTORY - 1)), cur.present], future: [] });
|
||||
}
|
||||
},
|
||||
[apply],
|
||||
);
|
||||
|
||||
const undo = React.useCallback(() => {
|
||||
const cur = ref.current;
|
||||
if (cur.past.length === 0) return false;
|
||||
last.current = { key: null, at: 0 };
|
||||
apply({ present: cur.past[cur.past.length - 1], past: cur.past.slice(0, -1), future: [cur.present, ...cur.future] });
|
||||
return true;
|
||||
}, [apply]);
|
||||
|
||||
const redo = React.useCallback(() => {
|
||||
const cur = ref.current;
|
||||
if (cur.future.length === 0) return false;
|
||||
last.current = { key: null, at: 0 };
|
||||
apply({ present: cur.future[0], past: [...cur.past, cur.present], future: cur.future.slice(1) });
|
||||
return true;
|
||||
}, [apply]);
|
||||
|
||||
/** Replace the value and forget all history (e.g. when an editor reopens). */
|
||||
const reset = React.useCallback(
|
||||
(value: T) => {
|
||||
last.current = { key: null, at: 0 };
|
||||
apply({ present: value, past: [], future: [] });
|
||||
},
|
||||
[apply],
|
||||
);
|
||||
|
||||
return {
|
||||
state: snapshot.present,
|
||||
set,
|
||||
undo,
|
||||
redo,
|
||||
reset,
|
||||
canUndo: snapshot.past.length > 0,
|
||||
canRedo: snapshot.future.length > 0,
|
||||
};
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
import { act, renderHook } from '@testing-library/react';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { useUndoableText } from './useUndoableText';
|
||||
|
||||
describe('useUndoableText', () => {
|
||||
beforeEach(() => vi.useFakeTimers());
|
||||
afterEach(() => vi.useRealTimers());
|
||||
|
||||
it('coalesces a typing burst into one undo step', () => {
|
||||
const { result } = renderHook(() => useUndoableText('a'));
|
||||
act(() => {
|
||||
result.current.type('ab');
|
||||
result.current.type('abc');
|
||||
});
|
||||
act(() => {
|
||||
vi.advanceTimersByTime(600);
|
||||
});
|
||||
expect(result.current.canUndo).toBe(true);
|
||||
|
||||
let restored: string | null = null;
|
||||
act(() => {
|
||||
restored = result.current.undo();
|
||||
});
|
||||
expect(restored).toBe('a');
|
||||
expect(result.current.text).toBe('a');
|
||||
expect(result.current.canUndo).toBe(false);
|
||||
});
|
||||
|
||||
it('gives a committed edit its own step and supports redo', () => {
|
||||
const { result } = renderHook(() => useUndoableText('hi'));
|
||||
act(() => result.current.commit('hi <pause>'));
|
||||
expect(result.current.text).toBe('hi <pause>');
|
||||
|
||||
act(() => {
|
||||
result.current.undo();
|
||||
});
|
||||
expect(result.current.text).toBe('hi');
|
||||
expect(result.current.canRedo).toBe(true);
|
||||
|
||||
let redone: string | null = null;
|
||||
act(() => {
|
||||
redone = result.current.redo();
|
||||
});
|
||||
expect(redone).toBe('hi <pause>');
|
||||
expect(result.current.text).toBe('hi <pause>');
|
||||
});
|
||||
|
||||
it('flushes pending typing before a commit so undo steps stay separate', () => {
|
||||
const { result } = renderHook(() => useUndoableText(''));
|
||||
act(() => result.current.type('hello'));
|
||||
act(() => result.current.commit('hello <pause>'));
|
||||
|
||||
act(() => {
|
||||
result.current.undo();
|
||||
});
|
||||
expect(result.current.text).toBe('hello');
|
||||
act(() => {
|
||||
result.current.undo();
|
||||
});
|
||||
expect(result.current.text).toBe('');
|
||||
});
|
||||
|
||||
it('returns null when there is nothing to undo or redo', () => {
|
||||
const { result } = renderHook(() => useUndoableText('x'));
|
||||
let undone: string | null = 'sentinel';
|
||||
let redone: string | null = 'sentinel';
|
||||
act(() => {
|
||||
undone = result.current.undo();
|
||||
redone = result.current.redo();
|
||||
});
|
||||
expect(undone).toBeNull();
|
||||
expect(redone).toBeNull();
|
||||
});
|
||||
|
||||
it('reset replaces the text and drops history', () => {
|
||||
const { result } = renderHook(() => useUndoableText('a'));
|
||||
act(() => result.current.commit('b'));
|
||||
act(() => result.current.reset('fresh'));
|
||||
expect(result.current.text).toBe('fresh');
|
||||
expect(result.current.canUndo).toBe(false);
|
||||
expect(result.current.canRedo).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,109 @@
|
||||
'use client';
|
||||
|
||||
import * as React from 'react';
|
||||
|
||||
const MAX_HISTORY = 100;
|
||||
// How long a pause in typing has to be before it becomes its own undo step.
|
||||
// Keeps undo from removing text one keystroke at a time while still giving
|
||||
// each tag/preset insertion (the actual ask) a guaranteed, immediate step.
|
||||
const TYPING_COALESCE_MS = 500;
|
||||
|
||||
/**
|
||||
* Text state with undo/redo. Typing is coalesced into bursts; discrete edits
|
||||
* (`commit`, e.g. a tag insertion) always get their own step. `textRef`
|
||||
* mirrors `text` synchronously because history bookkeeping needs the true
|
||||
* current value the instant an edit happens, not whatever the last render
|
||||
* closed over.
|
||||
*/
|
||||
export function useUndoableText(initial: string) {
|
||||
const [text, setTextState] = React.useState(initial);
|
||||
const textRef = React.useRef(initial);
|
||||
const [past, setPast] = React.useState<string[]>([]);
|
||||
const [future, setFuture] = React.useState<string[]>([]);
|
||||
// Start of the in-progress typing burst that hasn't become a history step
|
||||
// yet; flushed into `past` once typing pauses or a discrete edit happens.
|
||||
const typingBaseRef = React.useRef<string | null>(null);
|
||||
const typingTimerRef = React.useRef<ReturnType<typeof setTimeout> | null>(null);
|
||||
|
||||
const setText = React.useCallback((next: string) => {
|
||||
textRef.current = next;
|
||||
setTextState(next);
|
||||
}, []);
|
||||
|
||||
const pushHistory = React.useCallback((previous: string) => {
|
||||
setPast((p) => (p.length >= MAX_HISTORY ? [...p.slice(1), previous] : [...p, previous]));
|
||||
setFuture([]);
|
||||
}, []);
|
||||
|
||||
const clearTypingTimer = React.useCallback(() => {
|
||||
if (typingTimerRef.current) {
|
||||
clearTimeout(typingTimerRef.current);
|
||||
typingTimerRef.current = null;
|
||||
}
|
||||
}, []);
|
||||
|
||||
const flushTypingCheckpoint = React.useCallback(() => {
|
||||
clearTypingTimer();
|
||||
if (typingBaseRef.current != null && typingBaseRef.current !== textRef.current) {
|
||||
pushHistory(typingBaseRef.current);
|
||||
}
|
||||
typingBaseRef.current = null;
|
||||
}, [clearTypingTimer, pushHistory]);
|
||||
|
||||
/** Free typing: coalesced into one undo step per burst. */
|
||||
const type = React.useCallback(
|
||||
(next: string) => {
|
||||
if (typingBaseRef.current == null) typingBaseRef.current = textRef.current;
|
||||
setText(next);
|
||||
clearTypingTimer();
|
||||
typingTimerRef.current = setTimeout(flushTypingCheckpoint, TYPING_COALESCE_MS);
|
||||
},
|
||||
[setText, clearTypingTimer, flushTypingCheckpoint],
|
||||
);
|
||||
|
||||
/** Discrete edit (tag/preset insertion): always its own undo step. */
|
||||
const commit = React.useCallback(
|
||||
(next: string) => {
|
||||
flushTypingCheckpoint();
|
||||
pushHistory(textRef.current);
|
||||
setText(next);
|
||||
},
|
||||
[flushTypingCheckpoint, pushHistory, setText],
|
||||
);
|
||||
|
||||
/** Returns the restored text (so the caller can place the caret), or null if nothing to undo. */
|
||||
function undo(): string | null {
|
||||
flushTypingCheckpoint();
|
||||
if (past.length === 0) return null;
|
||||
const prev = past[past.length - 1];
|
||||
setPast(past.slice(0, -1));
|
||||
setFuture([...future, textRef.current]);
|
||||
setText(prev);
|
||||
return prev;
|
||||
}
|
||||
|
||||
function redo(): string | null {
|
||||
if (future.length === 0) return null;
|
||||
const next = future[future.length - 1];
|
||||
setFuture(future.slice(0, -1));
|
||||
setPast([...past, textRef.current]);
|
||||
setText(next);
|
||||
return next;
|
||||
}
|
||||
|
||||
/** Replace the text and drop all history (e.g. when an editor is reopened). */
|
||||
const reset = React.useCallback(
|
||||
(value: string) => {
|
||||
setText(value);
|
||||
setPast([]);
|
||||
setFuture([]);
|
||||
typingBaseRef.current = null;
|
||||
clearTypingTimer();
|
||||
},
|
||||
[setText, clearTypingTimer],
|
||||
);
|
||||
|
||||
React.useEffect(() => clearTypingTimer, [clearTypingTimer]);
|
||||
|
||||
return { text, type, commit, undo, redo, reset, canUndo: past.length > 0, canRedo: future.length > 0 };
|
||||
}
|
||||
@@ -5,7 +5,7 @@ import {
|
||||
loadDefaultOptions,
|
||||
type DefaultOptions,
|
||||
} from "./defaultOptions";
|
||||
import type { Job, JobType } from "./types";
|
||||
import type { GradeSegment, Job, JobType } from "./types";
|
||||
|
||||
const DEFAULT_API_BASE_URL = "http://localhost:8189/api";
|
||||
|
||||
@@ -50,8 +50,15 @@ function stripTrailingSlash(url: string): string {
|
||||
}
|
||||
|
||||
/** Full URL for streaming a job's video/image output (for <video> or <img> src). */
|
||||
export function getJobVideoUrl(jobId: string): string {
|
||||
return `${getApiBaseUrl()}/jobs/${jobId}/video`;
|
||||
/**
|
||||
* `version` busts the browser's cache after editing a job's video in place
|
||||
* (Trim/Edit video): the URL is otherwise identical before and after an edit,
|
||||
* even though the file on disk changed, so without this the same cached
|
||||
* response keeps showing. Omit it where staleness doesn't matter.
|
||||
*/
|
||||
export function getJobVideoUrl(jobId: string, version?: number): string {
|
||||
const base = `${getApiBaseUrl()}/jobs/${jobId}/video`;
|
||||
return version ? `${base}?v=${version}` : base;
|
||||
}
|
||||
|
||||
export interface CreateJobRequest {
|
||||
@@ -194,6 +201,84 @@ export async function updateJob(
|
||||
return response.json();
|
||||
}
|
||||
|
||||
/**
|
||||
* Ask the server to save the last frame of a finished clip and return its path,
|
||||
* ready to attach to another job as an image reference.
|
||||
*/
|
||||
export async function extractLastFrame(jobId: string): Promise<{ path: string; media_type: MediaType }> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/${jobId}/last-frame`, { method: "POST" });
|
||||
if (!response.ok) {
|
||||
const err = await response.json().catch(() => ({ detail: "Could not get the last frame" }));
|
||||
throw new Error(err.detail || "Could not get the last frame");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export interface EditVideoParams {
|
||||
startSeconds: number;
|
||||
/** omitted (or undefined) keeps to the end of the video */
|
||||
endSeconds?: number;
|
||||
/**
|
||||
* Ordered, contiguous razor-cut sections covering the whole kept range --
|
||||
* each section's `end_seconds` is relative to that kept range, not the
|
||||
* original file. Omitted (or empty) means one neutral section, i.e. a
|
||||
* plain range trim with no grading.
|
||||
*/
|
||||
segments?: GradeSegment[];
|
||||
}
|
||||
|
||||
/**
|
||||
* Cut a completed job's video to a range and/or grade it section by section,
|
||||
* in place. The untouched original is kept server-side, so this always
|
||||
* renders fresh from it with the *given* parameters -- pass every value you
|
||||
* want kept, not just the one you changed, or the rest reset to their neutral
|
||||
* defaults. Can be undone with restoreJobVideo.
|
||||
*/
|
||||
export async function trimJob(jobId: string, params: EditVideoParams): Promise<Job> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/${jobId}/trim`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
start_seconds: params.startSeconds,
|
||||
end_seconds: params.endSeconds ?? null,
|
||||
segments: params.segments ?? [],
|
||||
}),
|
||||
});
|
||||
if (!response.ok) {
|
||||
const err = await response.json().catch(() => ({ detail: "Could not edit the clip" }));
|
||||
throw new Error(err.detail || "Could not edit the clip");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
/** Undo every trim and color adjustment on a job's video, back to what it originally generated. */
|
||||
export async function restoreJobVideo(jobId: string): Promise<Job> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/${jobId}/restore-video`, { method: "POST" });
|
||||
if (!response.ok) {
|
||||
const err = await response.json().catch(() => ({ detail: "Could not restore the original video" }));
|
||||
throw new Error(err.detail || "Could not restore the original video");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
/**
|
||||
* Ask the server to save the last second or so of a finished clip and return its
|
||||
* path, ready to attach to another job as a video reference -- so the next
|
||||
* clip continues with real motion instead of guessing it from a single frame.
|
||||
*/
|
||||
export async function extractLastClip(jobId: string): Promise<{ path: string; media_type: MediaType }> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/${jobId}/last-clip`, { method: "POST" });
|
||||
if (!response.ok) {
|
||||
const err = await response.json().catch(() => ({ detail: "Could not get the end of the clip" }));
|
||||
throw new Error(err.detail || "Could not get the end of the clip");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export async function uploadMedia(
|
||||
file: File,
|
||||
): Promise<{ path: string; media_type: MediaType }> {
|
||||
@@ -313,6 +398,80 @@ export async function startJob(id: string): Promise<Job> {
|
||||
return response.json();
|
||||
}
|
||||
|
||||
/**
|
||||
* Queue jobs, in the order given, to run one after another as the server has room.
|
||||
* A job that starts from another job's last frame waits for that job to finish.
|
||||
* All-or-nothing: if any job can't be queued, none are.
|
||||
*/
|
||||
export async function queueJobs(ids: string[]): Promise<Job[]> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/queue`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ job_ids: ids }),
|
||||
});
|
||||
if (!response.ok) {
|
||||
const error = await response
|
||||
.json()
|
||||
.catch(() => ({ detail: "Failed to queue jobs" }));
|
||||
throw new Error(error.detail || "Failed to queue jobs");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
/** Take a queued job out of the queue; it goes back to pending. */
|
||||
export async function dequeueJob(id: string): Promise<Job> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/${id}/dequeue`, {
|
||||
method: "POST",
|
||||
});
|
||||
if (!response.ok) {
|
||||
const error = await response
|
||||
.json()
|
||||
.catch(() => ({ detail: "Failed to remove job from the queue" }));
|
||||
throw new Error(error.detail || "Failed to remove job from the queue");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export interface MergedScene {
|
||||
/** file name on the server; pass to getMergedVideoUrl / downloadMergedVideo */
|
||||
filename: string;
|
||||
clips: number;
|
||||
/** length of the merged video */
|
||||
seconds: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Join the finished clips of a scene, in the order given, into one video. Fails,
|
||||
* making nothing, if any clip hasn't finished.
|
||||
*/
|
||||
export async function mergeScene(jobIds: string[], name: string): Promise<MergedScene> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/scenes/merge`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ job_ids: jobIds, name }),
|
||||
});
|
||||
if (!response.ok) {
|
||||
const error = await response.json().catch(() => ({ detail: "Failed to merge the scene" }));
|
||||
throw new Error(error.detail || "Failed to merge the scene");
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export function getMergedVideoUrl(filename: string): string {
|
||||
return `${getApiBaseUrl()}/merged/${encodeURIComponent(filename)}`;
|
||||
}
|
||||
|
||||
export async function downloadMergedVideo(filename: string): Promise<Blob> {
|
||||
const response = await fetch(getMergedVideoUrl(filename));
|
||||
if (!response.ok) {
|
||||
throw new Error("Failed to download the merged video");
|
||||
}
|
||||
return response.blob();
|
||||
}
|
||||
|
||||
export async function stopJob(id: string): Promise<Job> {
|
||||
const baseApiUrl = getApiBaseUrl();
|
||||
const response = await fetch(`${baseApiUrl}/jobs/${id}/stop`, {
|
||||
|
||||
@@ -0,0 +1,478 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
attachLastClip,
|
||||
attachLastFrame,
|
||||
attachedLastClips,
|
||||
attachedLastFrames,
|
||||
deferredLastClipSource,
|
||||
deferredLastFrameSource,
|
||||
detachLastClip,
|
||||
detachLastFrame,
|
||||
lastClipSourceJobId,
|
||||
lastFrameSourceJobId,
|
||||
whyCannotAttach,
|
||||
whyCannotAttachClip,
|
||||
} from "@/lib/continuation";
|
||||
import { parseH3Prompt } from "@/lib/h3Prompt";
|
||||
import { readField } from "@/lib/h3PromptFields";
|
||||
|
||||
const FRAME = "/data/uploads/last_frames/last_frame_job-5.png";
|
||||
const REFS = [
|
||||
{ source: "/refs/veteran.png", media_type: "image" },
|
||||
{ source: "/refs/young.png", media_type: "image" },
|
||||
];
|
||||
|
||||
const CLIP = [
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is the veteran broker from <Picture 1>: a confident man.",
|
||||
"<Subject 2> is the young broker from <Picture 2>: eager.",
|
||||
"",
|
||||
"summary:",
|
||||
"[reference generation] The target video shows <Subject 1> and <Subject 2> at a lunch.",
|
||||
"",
|
||||
"retention_analysis:",
|
||||
"<Subject 1> (appears in [Shot 1]): fully_preserved - his suit.",
|
||||
"<Subject 2> (appears in [Shot 1]): fully_preserved - his suit.",
|
||||
"",
|
||||
"detailed_description:",
|
||||
"[Shot 1] Close-up shot, dolly. Continuing the same shot on <Subject 1>, still speaking. (S1) <d>[English] And then, well.</d>",
|
||||
"[Shot 2] At 00:04.000, the shot cuts to a wide shot. Both men at the table.",
|
||||
"",
|
||||
"overall_soundscape:",
|
||||
"Muted restaurant ambience.",
|
||||
"",
|
||||
"non_diegetic_music:",
|
||||
"N/A",
|
||||
].join("\n");
|
||||
|
||||
function attach(prompt = CLIP, refs = REFS, frame = FRAME) {
|
||||
const r = attachLastFrame(prompt, refs, frame);
|
||||
if (!r.ok) throw new Error(r.error);
|
||||
return r;
|
||||
}
|
||||
|
||||
describe("attachLastFrame", () => {
|
||||
it("adds the frame as the next image reference and names it by its picture number", () => {
|
||||
const r = attach();
|
||||
expect(r.label).toBe("<Picture 3>");
|
||||
expect(r.references).toEqual([...REFS, { source: FRAME, media_type: "image" }]);
|
||||
});
|
||||
|
||||
it("defines the frame as the first frame of Shot 1, after the existing definitions", () => {
|
||||
const defs = readField(attach().prompt, "subject_definitions") as string;
|
||||
expect(defs.split("\n")).toEqual([
|
||||
"<Subject 1> is the veteran broker from <Picture 1>: a confident man.",
|
||||
"<Subject 2> is the young broker from <Picture 2>: eager.",
|
||||
"<Picture 3> is the first frame of [Shot 1]: the last frame of the previous clip, showing the scene, positions, and lighting exactly as that clip ended.",
|
||||
]);
|
||||
});
|
||||
|
||||
it("tags the summary as keyframe completion alongside its existing task type, and names the frame", () => {
|
||||
const summary = readField(attach().prompt, "summary") as string;
|
||||
expect(summary.startsWith("[reference generation + keyframe completion] The target video shows")).toBe(true);
|
||||
expect(summary.endsWith("The target video starts from <Picture 3>, the last frame of the previous clip.")).toBe(true);
|
||||
});
|
||||
|
||||
it("adds a first-frame line to the retention analysis", () => {
|
||||
const lines = (readField(attach().prompt, "retention_analysis") as string).split("\n");
|
||||
expect(lines).toHaveLength(3);
|
||||
expect(lines[2]).toBe(
|
||||
"<Picture 3> ([Shot 1] first frame): fully_preserved - the scene, positions, and lighting of the previous clip's last frame are retained.",
|
||||
);
|
||||
});
|
||||
|
||||
it("starts the first shot from the frame and leaves the other shots alone", () => {
|
||||
const shots = (readField(attach().prompt, "detailed_description") as string).split("\n");
|
||||
expect(shots[0]).toBe(
|
||||
"[Shot 1] Close-up shot, dolly. The shot begins from <Picture 3>. Continuing the same shot on <Subject 1>, still speaking. (S1) <d>[English] And then, well.</d>",
|
||||
);
|
||||
expect(shots[1]).toBe("[Shot 2] At 00:04.000, the shot cuts to a wide shot. Both men at the table.");
|
||||
});
|
||||
|
||||
it("leaves every other section untouched", () => {
|
||||
const out = attach().prompt;
|
||||
expect(readField(out, "overall_soundscape")).toBe("Muted restaurant ambience.");
|
||||
expect(readField(out, "non_diegetic_music")).toBe("N/A");
|
||||
});
|
||||
|
||||
it("still reads as a six-section prompt", () => {
|
||||
expect(parseH3Prompt(attach().prompt)).not.toBeNull();
|
||||
});
|
||||
|
||||
it("counts only images when numbering, so a video reference doesn't shift the label", () => {
|
||||
const refs = [REFS[0], { source: "/refs/clip.mp4", media_type: "video" }, REFS[1]];
|
||||
expect(attach(CLIP, refs).label).toBe("<Picture 3>");
|
||||
expect(attach(CLIP, [{ source: "/refs/clip.mp4", media_type: "video" }]).label).toBe("<Picture 1>");
|
||||
});
|
||||
|
||||
it("does not mutate what it was given", () => {
|
||||
const refs = [...REFS];
|
||||
attach(CLIP, refs);
|
||||
expect(refs).toEqual(REFS);
|
||||
});
|
||||
});
|
||||
|
||||
describe("attachLastFrame: summary wording", () => {
|
||||
const summaryOf = (summary: string) =>
|
||||
readField(attach(CLIP.replace("[reference generation] The target video shows <Subject 1> and <Subject 2> at a lunch.", summary)).prompt, "summary") as string;
|
||||
|
||||
it("adds the tag when the summary has none", () => {
|
||||
expect(summaryOf("Two men at a lunch.").startsWith("[keyframe completion] Two men at a lunch.")).toBe(true);
|
||||
});
|
||||
|
||||
it("extends a combined tag", () => {
|
||||
expect(summaryOf("[video editing + reference generation] Edit.").startsWith("[video editing + reference generation + keyframe completion] Edit.")).toBe(true);
|
||||
});
|
||||
|
||||
it("does not repeat a tag that is already there", () => {
|
||||
const s = summaryOf("[keyframe completion + reference generation] Both.");
|
||||
expect(s.startsWith("[keyframe completion + reference generation] Both.")).toBe(true);
|
||||
expect(s.match(/keyframe completion/g)).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe("attachLastFrame: other prompt shapes", () => {
|
||||
it("replaces an N/A subject_definitions instead of appending under it", () => {
|
||||
const out = attach(CLIP.replace(/subject_definitions:\n[\s\S]*?\n\nsummary:/, "subject_definitions:\nN/A\n\nsummary:")).prompt;
|
||||
expect(readField(out, "subject_definitions")).toBe(
|
||||
"<Picture 3> is the first frame of [Shot 1]: the last frame of the previous clip, showing the scene, positions, and lighting exactly as that clip ended.",
|
||||
);
|
||||
});
|
||||
|
||||
it("keeps hand-written shots verbatim, only prefixing the first", () => {
|
||||
const hand = CLIP.replace(
|
||||
/\[Shot 1\][^\n]*\n\[Shot 2\][^\n]*/,
|
||||
"[Shot 1] A handheld medium shot holds <Subject 1> at frame left.\n[Shot 2] At 00:03.800, the shot cuts to a low tracking shot behind him.",
|
||||
);
|
||||
const shots = (readField(attach(hand).prompt, "detailed_description") as string).split("\n");
|
||||
expect(shots).toEqual([
|
||||
"[Shot 1] The shot begins from <Picture 3>. A handheld medium shot holds <Subject 1> at frame left.",
|
||||
"[Shot 2] At 00:03.800, the shot cuts to a low tracking shot behind him.",
|
||||
]);
|
||||
});
|
||||
|
||||
it("adds the sentence to a first shot that is only dialogue", () => {
|
||||
const dialogueOnly = CLIP.replace(
|
||||
"[Shot 1] Close-up shot, dolly. Continuing the same shot on <Subject 1>, still speaking. (S1) <d>[English] And then, well.</d>",
|
||||
"[Shot 1] Wide shot. (S1) <d>[English] Hi.</d>",
|
||||
);
|
||||
const first = (readField(attach(dialogueOnly).prompt, "detailed_description") as string).split("\n")[0];
|
||||
expect(first).toBe("[Shot 1] Wide shot. The shot begins from <Picture 3>. (S1) <d>[English] Hi.</d>");
|
||||
});
|
||||
});
|
||||
|
||||
describe("attachLastFrame: refusals", () => {
|
||||
it("won't attach the same frame twice", () => {
|
||||
const r = attachLastFrame(CLIP, [...REFS, { source: FRAME, media_type: "image" }], FRAME);
|
||||
expect(r).toEqual({ ok: false, error: "That frame is already attached to this clip." });
|
||||
});
|
||||
|
||||
it("won't rewrite a prompt that isn't in the six-section format", () => {
|
||||
const r = attachLastFrame("A toy car drives into a plush dog.", REFS, FRAME);
|
||||
expect(r.ok).toBe(false);
|
||||
expect(!r.ok && r.error).toMatch(/six-section/);
|
||||
});
|
||||
|
||||
it("won't rewrite a base-format prompt either", () => {
|
||||
const base = "For the target video, <Picture 1> is used.\n\nintegrated_multimodal_description: [Shot 1] A.\n\noverall_soundscape: Quiet.";
|
||||
expect(whyCannotAttach(base, REFS)).toMatch(/six-section/);
|
||||
});
|
||||
|
||||
it("stops at nine image references", () => {
|
||||
const nine = Array.from({ length: 9 }, (_, i) => ({ source: `/r/${i}.png`, media_type: "image" }));
|
||||
const r = attachLastFrame(CLIP, nine, FRAME);
|
||||
expect(!r.ok && r.error).toMatch(/at most 9 image/);
|
||||
});
|
||||
|
||||
it("reports a prompt with no shots", () => {
|
||||
const empty = CLIP.replace(/detailed_description:\n[\s\S]*?\n\noverall_soundscape:/, "detailed_description:\n\n\noverall_soundscape:");
|
||||
const r = attachLastFrame(empty, REFS, FRAME);
|
||||
expect(r.ok).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe("recognizing an attached last frame", () => {
|
||||
it("reads the job id out of the saved path", () => {
|
||||
expect(lastFrameSourceJobId(FRAME)).toBe("job-5");
|
||||
expect(lastFrameSourceJobId("last_frame_abc-123.png")).toBe("abc-123");
|
||||
expect(lastFrameSourceJobId("/refs/veteran.png")).toBeNull();
|
||||
expect(lastFrameSourceJobId("/x/not_last_frame_job-5.png")).toBeNull();
|
||||
});
|
||||
|
||||
it("finds last-frame references among a job's references", () => {
|
||||
expect(attachedLastFrames([...REFS, { source: FRAME, media_type: "image" }])).toEqual([{ source: FRAME, jobId: "job-5" }]);
|
||||
expect(attachedLastFrames(REFS)).toEqual([]);
|
||||
expect(attachedLastFrames(null)).toEqual([]);
|
||||
});
|
||||
|
||||
it("reads the job id out of a reference the server resolves when the job runs", () => {
|
||||
expect(deferredLastFrameSource("job-5")).toBe("job-last-frame:job-5");
|
||||
expect(lastFrameSourceJobId(deferredLastFrameSource("job-5"))).toBe("job-5");
|
||||
expect(lastFrameSourceJobId("job-last-frame:")).toBeNull();
|
||||
expect(attachedLastFrames([...REFS, { source: deferredLastFrameSource("job-5"), media_type: "image" }])).toEqual([
|
||||
{ source: "job-last-frame:job-5", jobId: "job-5" },
|
||||
]);
|
||||
});
|
||||
|
||||
it("attaches a deferred frame exactly like a saved one", () => {
|
||||
const result = attachLastFrame(CLIP, REFS, deferredLastFrameSource("job-5"));
|
||||
expect(result.ok).toBe(true);
|
||||
if (result.ok) {
|
||||
expect(result.references.at(-1)).toEqual({ source: "job-last-frame:job-5", media_type: "image" });
|
||||
expect(result.label).toBe("<Picture 3>");
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("detachLastFrame", () => {
|
||||
const roundTrip = (prompt: string, frame = FRAME) => {
|
||||
const attached = attach(prompt, REFS, frame);
|
||||
const r = detachLastFrame(attached.prompt, attached.references, frame);
|
||||
if (!r.ok) throw new Error(r.error);
|
||||
return r;
|
||||
};
|
||||
|
||||
it("takes back exactly what attaching added", () => {
|
||||
const r = roundTrip(CLIP);
|
||||
expect(r.references).toEqual(REFS);
|
||||
expect(r.prompt).toBe(CLIP);
|
||||
});
|
||||
|
||||
it("works for a frame the server resolves when the job runs", () => {
|
||||
expect(roundTrip(CLIP, deferredLastFrameSource("job-5")).prompt).toBe(CLIP);
|
||||
});
|
||||
|
||||
it.each([
|
||||
["a summary with no tag", "Two men at a lunch."],
|
||||
["a combined tag", "[video editing + reference generation] Edit."],
|
||||
])("restores %s", (_name, summary) => {
|
||||
const prompt = CLIP.replace("[reference generation] The target video shows <Subject 1> and <Subject 2> at a lunch.", summary);
|
||||
expect(roundTrip(prompt).prompt).toBe(prompt);
|
||||
});
|
||||
|
||||
it("restores hand-written shots and an N/A section", () => {
|
||||
const hand = CLIP.replace(/\[Shot 1\][^\n]*\n\[Shot 2\][^\n]*/, "[Shot 1] A handheld medium shot.\n[Shot 2] At 00:03.800, the shot cuts to a low tracking shot.").replace(
|
||||
/subject_definitions:\n[\s\S]*?\n\nsummary:/,
|
||||
"subject_definitions:\nN/A\n\nsummary:",
|
||||
);
|
||||
expect(roundTrip(hand).prompt).toBe(hand);
|
||||
});
|
||||
|
||||
it("removes the sentence from a first shot that is only dialogue", () => {
|
||||
const dialogueOnly = CLIP.replace(
|
||||
"[Shot 1] Close-up shot, dolly. Continuing the same shot on <Subject 1>, still speaking. (S1) <d>[English] And then, well.</d>",
|
||||
"[Shot 1] Wide shot. (S1) <d>[English] Hi.</d>",
|
||||
);
|
||||
expect(roundTrip(dialogueOnly).prompt).toBe(dialogueOnly);
|
||||
});
|
||||
|
||||
it("only removes the reference from a prompt that was never rewritten", () => {
|
||||
const r = detachLastFrame("A plain prompt.", [...REFS, { source: FRAME, media_type: "image" }], FRAME);
|
||||
expect(r).toEqual({ ok: true, prompt: "A plain prompt.", references: REFS });
|
||||
});
|
||||
|
||||
it("refuses when other images were added after the frame", () => {
|
||||
const attached = attach();
|
||||
const r = detachLastFrame(attached.prompt, [...attached.references, { source: "/refs/extra.png", media_type: "image" }], FRAME);
|
||||
expect(r.ok).toBe(false);
|
||||
});
|
||||
|
||||
it("refuses a frame that isn't attached", () => {
|
||||
expect(detachLastFrame(CLIP, REFS, FRAME)).toEqual({ ok: false, error: "That frame isn't attached to this clip." });
|
||||
});
|
||||
});
|
||||
|
||||
const CLIP_VIDEO = "/data/uploads/last_clips/last_clip_job-5.mp4";
|
||||
|
||||
function attachClip(prompt = CLIP, refs = REFS, clip = CLIP_VIDEO) {
|
||||
const r = attachLastClip(prompt, refs, clip);
|
||||
if (!r.ok) throw new Error(r.error);
|
||||
return r;
|
||||
}
|
||||
|
||||
describe("attachLastClip", () => {
|
||||
it("adds the clip as the next video reference and names it by its video number", () => {
|
||||
const r = attachClip();
|
||||
expect(r.label).toBe("<Video 1>");
|
||||
expect(r.references).toEqual([...REFS, { source: CLIP_VIDEO, media_type: "video" }]);
|
||||
});
|
||||
|
||||
it("numbers a second video reference after the first", () => {
|
||||
const withOne = attachClip();
|
||||
const r = attachLastClip(withOne.prompt, withOne.references, "/refs/another.mp4");
|
||||
if (!r.ok) throw new Error(r.error);
|
||||
expect(r.label).toBe("<Video 2>");
|
||||
});
|
||||
|
||||
it("declares the video standalone, not bound to a <Subject> the way pictures and audio are", () => {
|
||||
const defs = readField(attachClip().prompt, "subject_definitions") as string;
|
||||
expect(defs).toContain("<Video 1> is the source video for [Shot 1]");
|
||||
expect(defs).not.toMatch(/<Subject \d+> is .*<Video 1>/);
|
||||
});
|
||||
|
||||
it("adds a retention line with the fully_preserved status", () => {
|
||||
const line = (readField(attachClip().prompt, "retention_analysis") as string).split("\n").at(-1);
|
||||
expect(line).toBe(
|
||||
"<Video 1> (continuation source): fully_preserved - the motion, scene, positions, and lighting of the previous clip's ending are continued exactly.",
|
||||
);
|
||||
});
|
||||
|
||||
it("prefixes the first shot with a continuation lead, keeping hand-written shots verbatim otherwise", () => {
|
||||
const hand = CLIP.replace(
|
||||
/\[Shot 1\][^\n]*\n\[Shot 2\][^\n]*/,
|
||||
"[Shot 1] A handheld medium shot holds <Subject 1> at frame left.\n[Shot 2] At 00:03.800, the shot cuts to a low tracking shot.",
|
||||
);
|
||||
const shots = (readField(attachClip(hand).prompt, "detailed_description") as string).split("\n");
|
||||
expect(shots).toEqual([
|
||||
"[Shot 1] The shot continues from <Video 1>. A handheld medium shot holds <Subject 1> at frame left.",
|
||||
"[Shot 2] At 00:03.800, the shot cuts to a low tracking shot.",
|
||||
]);
|
||||
});
|
||||
|
||||
describe("summary wording", () => {
|
||||
const summaryOf = (summary: string) =>
|
||||
readField(attachClip(CLIP.replace("[reference generation] The target video shows <Subject 1> and <Subject 2> at a lunch.", summary)).prompt, "summary") as string;
|
||||
|
||||
it("adds the tag when the summary has none", () => {
|
||||
expect(summaryOf("Two men at a lunch.")).toBe("[video continuation] Two men at a lunch.");
|
||||
});
|
||||
|
||||
it("replaces reference generation, not adds to it", () => {
|
||||
expect(summaryOf("[reference generation] Two men.")).toBe("[video continuation] Two men.");
|
||||
});
|
||||
|
||||
it("keeps a combined audio reference tag, per the guide's documented combination", () => {
|
||||
expect(summaryOf("[reference generation + audio reference] Two men.")).toBe("[video continuation + audio reference] Two men.");
|
||||
});
|
||||
|
||||
it("drops keyframe completion when switching from a picture-based continuation", () => {
|
||||
expect(summaryOf("[reference generation + keyframe completion] Two men.")).toBe("[video continuation] Two men.");
|
||||
});
|
||||
});
|
||||
|
||||
describe("refusals", () => {
|
||||
it("won't attach the same clip twice", () => {
|
||||
const r = attachLastClip(CLIP, [...REFS, { source: CLIP_VIDEO, media_type: "video" }], CLIP_VIDEO);
|
||||
expect(r).toEqual({ ok: false, error: "That clip is already attached to this job." });
|
||||
});
|
||||
|
||||
it("refuses a prompt that is not in the six-section format", () => {
|
||||
const r = attachLastClip("A plain prompt.", REFS, CLIP_VIDEO);
|
||||
expect(r).toEqual({ ok: false, error: "This prompt isn't in the six-section format, so it can't be rewritten automatically. Attach the clip in Edit instead." });
|
||||
});
|
||||
|
||||
it("refuses past H3's video reference limit", () => {
|
||||
const many = Array.from({ length: 3 }, (_, i) => ({ source: `/v${i}.mp4`, media_type: "video" }));
|
||||
const r = attachLastClip(CLIP, [...REFS, ...many], CLIP_VIDEO);
|
||||
expect(r).toEqual({ ok: false, error: "A job can have at most 3 video references." });
|
||||
});
|
||||
|
||||
it("refuses a prompt with no shots", () => {
|
||||
const noShots = CLIP.replace(/detailed_description:\n[\s\S]*?\n\noverall_soundscape:/, "detailed_description:\n\noverall_soundscape:");
|
||||
const r = attachLastClip(noShots, REFS, CLIP_VIDEO);
|
||||
expect(r).toEqual({ ok: false, error: "This prompt has no shots to continue from the clip." });
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("recognizing an attached last clip", () => {
|
||||
it("reads the job id out of the saved path", () => {
|
||||
expect(lastClipSourceJobId(CLIP_VIDEO)).toBe("job-5");
|
||||
expect(lastClipSourceJobId("last_clip_abc-123.mp4")).toBe("abc-123");
|
||||
expect(lastClipSourceJobId("/refs/veteran.png")).toBeNull();
|
||||
});
|
||||
|
||||
it("reads the job id out of a reference the server resolves when the job runs", () => {
|
||||
expect(deferredLastClipSource("job-5")).toBe("job-last-clip:job-5");
|
||||
expect(lastClipSourceJobId(deferredLastClipSource("job-5"))).toBe("job-5");
|
||||
});
|
||||
|
||||
it("finds last-clip references among a job's references, separately from last-frame ones", () => {
|
||||
const frame = { source: "/data/uploads/last_frames/last_frame_job-2.png", media_type: "image" };
|
||||
expect(attachedLastClips([...REFS, frame, { source: CLIP_VIDEO, media_type: "video" }])).toEqual([
|
||||
{ source: CLIP_VIDEO, jobId: "job-5" },
|
||||
]);
|
||||
expect(attachedLastClips([frame])).toEqual([]);
|
||||
expect(attachedLastClips(null)).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("whyCannotAttachClip", () => {
|
||||
it("blocks a prompt not in the six-section format", () => {
|
||||
expect(whyCannotAttachClip("A plain prompt.", REFS)).toMatch(/six-section/);
|
||||
});
|
||||
|
||||
it("allows a normal prompt under the video limit", () => {
|
||||
expect(whyCannotAttachClip(CLIP, REFS)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("detachLastClip", () => {
|
||||
const roundTrip = (prompt: string, clip = CLIP_VIDEO) => {
|
||||
const attached = attachClip(prompt, REFS, clip);
|
||||
const r = detachLastClip(attached.prompt, attached.references, clip);
|
||||
if (!r.ok) throw new Error(r.error);
|
||||
return r;
|
||||
};
|
||||
|
||||
it("takes back exactly what attaching added", () => {
|
||||
const r = roundTrip(CLIP);
|
||||
expect(r.references).toEqual(REFS);
|
||||
expect(r.prompt).toBe(CLIP);
|
||||
});
|
||||
|
||||
it("works for a clip the server resolves when the job runs", () => {
|
||||
expect(roundTrip(CLIP, deferredLastClipSource("job-5")).prompt).toBe(CLIP);
|
||||
});
|
||||
|
||||
it("round-trips a starting tag combined with audio reference", () => {
|
||||
// A realistic starting point: audio references already attached (see continuation.test.ts's
|
||||
// own audio-reference wiring), before any video continuation is attached.
|
||||
const prompt = CLIP.replace(
|
||||
"[reference generation] The target video shows <Subject 1> and <Subject 2> at a lunch.",
|
||||
"[reference generation + audio reference] Two men at a lunch.",
|
||||
);
|
||||
expect(roundTrip(prompt).prompt).toBe(prompt);
|
||||
});
|
||||
|
||||
// Unlike attachLastFrame's "keyframe completion" (only ever added alongside the existing base
|
||||
// tag), "video continuation" *replaces* the base tag -- so once attached, nothing remembers
|
||||
// what was there before, and detaching lands on the default "reference generation", not back
|
||||
// on "no tag".
|
||||
it("restores a summary with no tag to the default reference-generation tag, not back to no tag", () => {
|
||||
const prompt = CLIP.replace("[reference generation] The target video shows <Subject 1> and <Subject 2> at a lunch.", "Two men at a lunch.");
|
||||
expect(roundTrip(prompt).prompt).toBe(CLIP.replace("The target video shows <Subject 1> and <Subject 2> at a lunch.", "Two men at a lunch."));
|
||||
});
|
||||
|
||||
it("restores hand-written shots and an N/A section", () => {
|
||||
const hand = CLIP.replace(/\[Shot 1\][^\n]*\n\[Shot 2\][^\n]*/, "[Shot 1] A handheld medium shot.\n[Shot 2] At 00:03.800, the shot cuts to a low tracking shot.").replace(
|
||||
/subject_definitions:\n[\s\S]*?\n\nsummary:/,
|
||||
"subject_definitions:\nN/A\n\nsummary:",
|
||||
);
|
||||
expect(roundTrip(hand).prompt).toBe(hand);
|
||||
});
|
||||
|
||||
it("only removes the reference from a prompt that was never rewritten", () => {
|
||||
const r = detachLastClip("A plain prompt.", [...REFS, { source: CLIP_VIDEO, media_type: "video" }], CLIP_VIDEO);
|
||||
expect(r).toEqual({ ok: true, prompt: "A plain prompt.", references: REFS });
|
||||
});
|
||||
|
||||
it("refuses when other videos were added after the clip", () => {
|
||||
const attached = attachClip();
|
||||
const r = detachLastClip(attached.prompt, [...attached.references, { source: "/refs/extra.mp4", media_type: "video" }], CLIP_VIDEO);
|
||||
expect(r.ok).toBe(false);
|
||||
});
|
||||
|
||||
it("refuses a clip that isn't attached", () => {
|
||||
expect(detachLastClip(CLIP, REFS, CLIP_VIDEO)).toEqual({ ok: false, error: "That clip isn't attached to this job." });
|
||||
});
|
||||
|
||||
it("does not remove a last-frame attachment, and vice versa", () => {
|
||||
const withFrame = attach(CLIP, REFS, FRAME);
|
||||
const withBoth = attachClip(withFrame.prompt, withFrame.references, CLIP_VIDEO);
|
||||
const r = detachLastClip(withBoth.prompt, withBoth.references, CLIP_VIDEO);
|
||||
if (!r.ok) throw new Error(r.error);
|
||||
expect(attachedLastFrames(r.references)).toEqual([{ source: FRAME, jobId: "job-5" }]);
|
||||
expect(attachedLastClips(r.references)).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,334 @@
|
||||
/**
|
||||
* Start a clip from how the one before it ended -- either its last frame (a
|
||||
* still) or its last second or so (a short video).
|
||||
*
|
||||
* A scene is generated as many short clips. To make one pick up where the
|
||||
* previous ended, the previous clip's ending is attached as a reference, and
|
||||
* the prompt is rewritten the way the prompt guide describes for a picture
|
||||
* that anchors the opening frame (`attachLastFrame`) or a video that the shot
|
||||
* continues from (`attachLastClip`). The wording is kept together here so it
|
||||
* is easy to change if the guide is read differently.
|
||||
*
|
||||
* The two approaches trade off differently: a still is a hard, exact opening
|
||||
* frame, but the model has only a pose to guess the continuing motion from. A
|
||||
* short video carries real motion, at the cost of one more reference slot and
|
||||
* (per H3's own limits) at most 3 per job instead of 9.
|
||||
*/
|
||||
import { readField, writeField } from "@/lib/h3PromptFields";
|
||||
import { H3_REFERENCE_LIMITS } from "@/lib/h3References";
|
||||
import {
|
||||
detailedDescriptionFromShots,
|
||||
newTextPart,
|
||||
parseDetailedDescription,
|
||||
} from "@/lib/h3Shots";
|
||||
|
||||
export interface Reference {
|
||||
source: string;
|
||||
media_type: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* A reference that means "the last frame of that job's video", resolved by the
|
||||
* server when this job runs -- so it can be attached before that job has
|
||||
* finished, and the server's queue knows this job must wait for it.
|
||||
*/
|
||||
export const DEFERRED_LAST_FRAME_PREFIX = "job-last-frame:";
|
||||
/** As above, but for the previous job's trailing video instead of its last frame. */
|
||||
export const DEFERRED_LAST_CLIP_PREFIX = "job-last-clip:";
|
||||
|
||||
export const deferredLastFrameSource = (jobId: string) => `${DEFERRED_LAST_FRAME_PREFIX}${jobId}`;
|
||||
export const deferredLastClipSource = (jobId: string) => `${DEFERRED_LAST_CLIP_PREFIX}${jobId}`;
|
||||
|
||||
/** Saved by the server as `.../last_frames/last_frame_<job id>.png`. */
|
||||
const LAST_FRAME_RE = /(?:^|\/)last_frame_([^/]+)\.png$/;
|
||||
/** Saved by the server as `.../last_clips/last_clip_<job id>.mp4`. */
|
||||
const LAST_CLIP_RE = /(?:^|\/)last_clip_([^/]+)\.mp4$/;
|
||||
|
||||
const REQUIRED_SECTIONS = [
|
||||
"subject_definitions",
|
||||
"summary",
|
||||
"retention_analysis",
|
||||
"detailed_description",
|
||||
] as const;
|
||||
|
||||
/** The job a last-frame reference came from, or null if `source` isn't one. */
|
||||
export function lastFrameSourceJobId(source: string): string | null {
|
||||
if (source.startsWith(DEFERRED_LAST_FRAME_PREFIX)) return source.slice(DEFERRED_LAST_FRAME_PREFIX.length) || null;
|
||||
return source.match(LAST_FRAME_RE)?.[1] ?? null;
|
||||
}
|
||||
|
||||
/** Last-frame references already on a job, with the job each came from. */
|
||||
export function attachedLastFrames(references: Reference[] | null | undefined): { source: string; jobId: string }[] {
|
||||
return (references ?? []).flatMap((r) => {
|
||||
const jobId = lastFrameSourceJobId(r.source);
|
||||
return jobId ? [{ source: r.source, jobId }] : [];
|
||||
});
|
||||
}
|
||||
|
||||
/** The job a last-clip reference came from, or null if `source` isn't one. */
|
||||
export function lastClipSourceJobId(source: string): string | null {
|
||||
if (source.startsWith(DEFERRED_LAST_CLIP_PREFIX)) return source.slice(DEFERRED_LAST_CLIP_PREFIX.length) || null;
|
||||
return source.match(LAST_CLIP_RE)?.[1] ?? null;
|
||||
}
|
||||
|
||||
/** Last-clip references already on a job, with the job each came from. */
|
||||
export function attachedLastClips(references: Reference[] | null | undefined): { source: string; jobId: string }[] {
|
||||
return (references ?? []).flatMap((r) => {
|
||||
const jobId = lastClipSourceJobId(r.source);
|
||||
return jobId ? [{ source: r.source, jobId }] : [];
|
||||
});
|
||||
}
|
||||
|
||||
/** Why a last frame can't be attached to this job, or null if it can. */
|
||||
export function whyCannotAttach(prompt: string, references: Reference[] | null | undefined): string | null {
|
||||
const missing = REQUIRED_SECTIONS.filter((s) => readField(prompt, s) === null);
|
||||
if (missing.length > 0) {
|
||||
return "This prompt isn't in the six-section format, so it can't be rewritten automatically. Attach the frame in Edit instead.";
|
||||
}
|
||||
const images = (references ?? []).filter((r) => r.media_type === "image").length;
|
||||
if (images >= H3_REFERENCE_LIMITS.image) {
|
||||
return `A job can have at most ${H3_REFERENCE_LIMITS.image} image references.`;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** "[reference generation]" -> "[reference generation + keyframe completion]"; no tag -> "[keyframe completion] ...". */
|
||||
function withKeyframeCompletion(summary: string): string {
|
||||
const tag = summary.match(/^\[([^\]]*)\]\s*/);
|
||||
if (!tag) return `[keyframe completion] ${summary}`.trim();
|
||||
const types = tag[1].split("+").map((t) => t.trim()).filter(Boolean);
|
||||
if (types.includes("keyframe completion")) return summary;
|
||||
return `[${[...types, "keyframe completion"].join(" + ")}] ${summary.slice(tag[0].length)}`.trim();
|
||||
}
|
||||
|
||||
const isBlank = (text: string) => !text.trim() || text.trim().toUpperCase() === "N/A";
|
||||
const appendLine = (body: string, line: string) => (isBlank(body) ? line : `${body}\n${line}`);
|
||||
|
||||
export type AttachResult =
|
||||
| { ok: true; prompt: string; references: Reference[]; label: string }
|
||||
| { ok: false; error: string };
|
||||
|
||||
/**
|
||||
* Add `framePath` as the next image reference and rewrite the prompt to use it
|
||||
* as the opening frame of Shot 1. Fails, changing nothing, if the frame is
|
||||
* already attached or the prompt isn't in a form that can be rewritten safely.
|
||||
*/
|
||||
export function attachLastFrame(prompt: string, references: Reference[], framePath: string): AttachResult {
|
||||
if (references.some((r) => r.source === framePath)) {
|
||||
return { ok: false, error: "That frame is already attached to this clip." };
|
||||
}
|
||||
const blocked = whyCannotAttach(prompt, references);
|
||||
if (blocked) return { ok: false, error: blocked };
|
||||
|
||||
// Labels count per media type in reference order, so the new image is the next <Picture N>.
|
||||
const label = `<Picture ${references.filter((r) => r.media_type === "image").length + 1}>`;
|
||||
|
||||
const { preamble, shots } = parseDetailedDescription(readField(prompt, "detailed_description") ?? "", []);
|
||||
if (shots.length === 0) return { ok: false, error: "This prompt has no shots to start from the frame." };
|
||||
const first = shots[0];
|
||||
const firstText = first.body.find((p) => p.kind === "text");
|
||||
if (firstText && firstText.kind === "text") firstText.text = `The shot begins from ${label}. ${firstText.text}`;
|
||||
else first.body.unshift(newTextPart(`The shot begins from ${label}.`));
|
||||
|
||||
let next = prompt;
|
||||
next = writeField(
|
||||
next,
|
||||
"subject_definitions",
|
||||
appendLine(
|
||||
readField(next, "subject_definitions") ?? "",
|
||||
`${label} is the first frame of [Shot 1]: the last frame of the previous clip, showing the scene, positions, and lighting exactly as that clip ended.`,
|
||||
),
|
||||
);
|
||||
next = writeField(
|
||||
next,
|
||||
"summary",
|
||||
`${withKeyframeCompletion(readField(next, "summary") ?? "")} The target video starts from ${label}, the last frame of the previous clip.`,
|
||||
);
|
||||
next = writeField(
|
||||
next,
|
||||
"retention_analysis",
|
||||
appendLine(
|
||||
readField(next, "retention_analysis") ?? "",
|
||||
`${label} ([Shot 1] first frame): fully_preserved - the scene, positions, and lighting of the previous clip's last frame are retained.`,
|
||||
),
|
||||
);
|
||||
next = writeField(next, "detailed_description", detailedDescriptionFromShots(shots, preamble));
|
||||
|
||||
return { ok: true, prompt: next, references: [...references, { source: framePath, media_type: "image" }], label };
|
||||
}
|
||||
|
||||
/** Drop the lines `attachLastFrame` added, leaving "N/A" if nothing else was there. */
|
||||
function removeLine(body: string, startsWith: string): string {
|
||||
const kept = body.split("\n").filter((l) => !l.trim().startsWith(startsWith));
|
||||
const text = kept.join("\n").trim();
|
||||
return text || "N/A";
|
||||
}
|
||||
|
||||
export type DetachResult = { ok: true; prompt: string; references: Reference[] } | { ok: false; error: string };
|
||||
|
||||
/**
|
||||
* Undo `attachLastFrame`: remove the reference at `source` and take back what was
|
||||
* added to the prompt for it. Only the most recently added image can be removed --
|
||||
* removing an earlier one would renumber the pictures after it.
|
||||
*/
|
||||
export function detachLastFrame(prompt: string, references: Reference[], source: string): DetachResult {
|
||||
const images = references.filter((r) => r.media_type === "image");
|
||||
const index = images.findIndex((r) => r.source === source);
|
||||
if (index < 0) return { ok: false, error: "That frame isn't attached to this clip." };
|
||||
if (index !== images.length - 1) {
|
||||
return { ok: false, error: "Other images were added after that frame. Remove it in Edit instead." };
|
||||
}
|
||||
const remaining = references.filter((r) => r.source !== source);
|
||||
|
||||
// A prompt that isn't in the six-section form was never rewritten; only the reference goes.
|
||||
if (REQUIRED_SECTIONS.some((s) => readField(prompt, s) === null)) {
|
||||
return { ok: true, prompt, references: remaining };
|
||||
}
|
||||
|
||||
const label = `<Picture ${index + 1}>`;
|
||||
let next = prompt;
|
||||
next = writeField(next, "subject_definitions", removeLine(readField(next, "subject_definitions") ?? "", `${label} is the first frame of [Shot 1]`));
|
||||
next = writeField(next, "retention_analysis", removeLine(readField(next, "retention_analysis") ?? "", `${label} ([Shot 1] first frame)`));
|
||||
|
||||
const summary = readField(next, "summary") ?? "";
|
||||
const sentence = ` The target video starts from ${label}, the last frame of the previous clip.`;
|
||||
if (summary.includes(sentence)) {
|
||||
let restored = summary.replace(sentence, "");
|
||||
const tag = restored.match(/^\[([^\]]*)\]\s*/);
|
||||
if (tag) {
|
||||
const types = tag[1].split("+").map((t) => t.trim()).filter((t) => t && t !== "keyframe completion");
|
||||
restored = types.length > 0 ? `[${types.join(" + ")}] ${restored.slice(tag[0].length)}` : restored.slice(tag[0].length);
|
||||
}
|
||||
next = writeField(next, "summary", restored.trim());
|
||||
}
|
||||
|
||||
const { preamble, shots } = parseDetailedDescription(readField(next, "detailed_description") ?? "", []);
|
||||
const lead = `The shot begins from ${label}.`;
|
||||
const first = shots[0];
|
||||
const part = first?.body.find((p) => p.kind === "text" && p.text.startsWith(lead));
|
||||
if (first && part && part.kind === "text") {
|
||||
part.text = part.text.slice(lead.length).trimStart();
|
||||
if (!part.text) first.body = first.body.filter((p) => p !== part);
|
||||
next = writeField(next, "detailed_description", detailedDescriptionFromShots(shots, preamble));
|
||||
}
|
||||
return { ok: true, prompt: next, references: remaining };
|
||||
}
|
||||
|
||||
/** Why a trailing clip can't be attached to this job, or null if it can. */
|
||||
export function whyCannotAttachClip(prompt: string, references: Reference[] | null | undefined): string | null {
|
||||
const missing = REQUIRED_SECTIONS.filter((s) => readField(prompt, s) === null);
|
||||
if (missing.length > 0) {
|
||||
return "This prompt isn't in the six-section format, so it can't be rewritten automatically. Attach the clip in Edit instead.";
|
||||
}
|
||||
const videos = (references ?? []).filter((r) => r.media_type === "video").length;
|
||||
if (videos >= H3_REFERENCE_LIMITS.video) {
|
||||
return `A job can have at most ${H3_REFERENCE_LIMITS.video} video references.`;
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** "[reference generation + audio reference]" -> "[video continuation + audio reference]"; drops "keyframe completion". */
|
||||
function withVideoContinuation(summary: string): string {
|
||||
const tag = summary.match(/^\[([^\]]*)\]\s*/);
|
||||
const rest = tag ? summary.slice(tag[0].length) : summary;
|
||||
const kept = tag
|
||||
? tag[1].split("+").map((t) => t.trim()).filter((t) => t && t !== "reference generation" && t !== "keyframe completion")
|
||||
: [];
|
||||
if (!kept.includes("video continuation")) kept.unshift("video continuation");
|
||||
return `[${kept.join(" + ")}] ${rest}`.trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* Add `clipPath` as the next video reference and rewrite the prompt so Shot 1
|
||||
* continues from it, per the guide's video-reference form: a standalone
|
||||
* `<Video N>` declaration (videos aren't bound to a `<Subject>` the way
|
||||
* pictures and audio are), a `[video continuation]` task tag, a retention
|
||||
* line, and "the shot continues from <Video N>" in the first shot.
|
||||
*/
|
||||
export function attachLastClip(prompt: string, references: Reference[], clipPath: string): AttachResult {
|
||||
if (references.some((r) => r.source === clipPath)) {
|
||||
return { ok: false, error: "That clip is already attached to this job." };
|
||||
}
|
||||
const blocked = whyCannotAttachClip(prompt, references);
|
||||
if (blocked) return { ok: false, error: blocked };
|
||||
|
||||
const label = `<Video ${references.filter((r) => r.media_type === "video").length + 1}>`;
|
||||
|
||||
const { preamble, shots } = parseDetailedDescription(readField(prompt, "detailed_description") ?? "", []);
|
||||
if (shots.length === 0) return { ok: false, error: "This prompt has no shots to continue from the clip." };
|
||||
const first = shots[0];
|
||||
const lead = `The shot continues from ${label}.`;
|
||||
const firstText = first.body.find((p) => p.kind === "text");
|
||||
if (firstText && firstText.kind === "text") firstText.text = `${lead} ${firstText.text}`;
|
||||
else first.body.unshift(newTextPart(lead));
|
||||
|
||||
let next = prompt;
|
||||
next = writeField(
|
||||
next,
|
||||
"subject_definitions",
|
||||
appendLine(
|
||||
readField(next, "subject_definitions") ?? "",
|
||||
`${label} is the source video for [Shot 1]: the last second of the previous clip, providing the continuation starting point and its motion.`,
|
||||
),
|
||||
);
|
||||
next = writeField(next, "summary", withVideoContinuation(readField(next, "summary") ?? ""));
|
||||
next = writeField(
|
||||
next,
|
||||
"retention_analysis",
|
||||
appendLine(
|
||||
readField(next, "retention_analysis") ?? "",
|
||||
`${label} (continuation source): fully_preserved - the motion, scene, positions, and lighting of the previous clip's ending are continued exactly.`,
|
||||
),
|
||||
);
|
||||
next = writeField(next, "detailed_description", detailedDescriptionFromShots(shots, preamble));
|
||||
|
||||
return { ok: true, prompt: next, references: [...references, { source: clipPath, media_type: "video" }], label };
|
||||
}
|
||||
|
||||
/**
|
||||
* Undo `attachLastClip`: remove the reference at `source` and take back what
|
||||
* was added to the prompt for it. Only the most recently added video can be
|
||||
* removed -- removing an earlier one would renumber the videos after it.
|
||||
*/
|
||||
export function detachLastClip(prompt: string, references: Reference[], source: string): DetachResult {
|
||||
const videos = references.filter((r) => r.media_type === "video");
|
||||
const index = videos.findIndex((r) => r.source === source);
|
||||
if (index < 0) return { ok: false, error: "That clip isn't attached to this job." };
|
||||
if (index !== videos.length - 1) {
|
||||
return { ok: false, error: "Other videos were added after that clip. Remove it in Edit instead." };
|
||||
}
|
||||
const remaining = references.filter((r) => r.source !== source);
|
||||
|
||||
if (REQUIRED_SECTIONS.some((s) => readField(prompt, s) === null)) {
|
||||
return { ok: true, prompt, references: remaining };
|
||||
}
|
||||
|
||||
const label = `<Video ${index + 1}>`;
|
||||
let next = prompt;
|
||||
next = writeField(next, "subject_definitions", removeLine(readField(next, "subject_definitions") ?? "", `${label} is the source video for [Shot 1]`));
|
||||
next = writeField(next, "retention_analysis", removeLine(readField(next, "retention_analysis") ?? "", `${label} (continuation source)`));
|
||||
|
||||
// "video continuation" replaced "reference generation" as the base tag (attachLastClip drops
|
||||
// it outright, unlike "keyframe completion" which is only ever added alongside it) -- so
|
||||
// undoing it means putting "reference generation" back, not just deleting the tag.
|
||||
const summaryTag = (readField(next, "summary") ?? "").match(/^\[([^\]]*)\]\s*/);
|
||||
if (summaryTag && summaryTag[1].split("+").map((t) => t.trim()).includes("video continuation")) {
|
||||
const rest = (readField(next, "summary") ?? "").slice(summaryTag[0].length);
|
||||
const types = summaryTag[1]
|
||||
.split("+")
|
||||
.map((t) => t.trim())
|
||||
.map((t) => (t === "video continuation" ? "reference generation" : t));
|
||||
next = writeField(next, "summary", `[${types.join(" + ")}] ${rest}`.trim());
|
||||
}
|
||||
|
||||
const { preamble, shots } = parseDetailedDescription(readField(next, "detailed_description") ?? "", []);
|
||||
const lead = `The shot continues from ${label}.`;
|
||||
const first = shots[0];
|
||||
const part = first?.body.find((p) => p.kind === "text" && p.text.startsWith(lead));
|
||||
if (first && part && part.kind === "text") {
|
||||
part.text = part.text.slice(lead.length).trimStart();
|
||||
if (!part.text) first.body = first.body.filter((p) => p !== part);
|
||||
next = writeField(next, "detailed_description", detailedDescriptionFromShots(shots, preamble));
|
||||
}
|
||||
return { ok: true, prompt: next, references: remaining };
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { applyGradeInPlace, isNeutralGrade, NEUTRAL_GRADE } from './grading';
|
||||
|
||||
function solidRgba(rgb: [number, number, number], count = 4, alpha = 255): Uint8ClampedArray {
|
||||
const data = new Uint8ClampedArray(count * 4);
|
||||
for (let i = 0; i < count; i++) {
|
||||
data[i * 4] = rgb[0];
|
||||
data[i * 4 + 1] = rgb[1];
|
||||
data[i * 4 + 2] = rgb[2];
|
||||
data[i * 4 + 3] = alpha;
|
||||
}
|
||||
return data;
|
||||
}
|
||||
|
||||
function meanRgb(data: Uint8ClampedArray): [number, number, number] {
|
||||
let r = 0, g = 0, b = 0;
|
||||
const n = data.length / 4;
|
||||
for (let i = 0; i < data.length; i += 4) {
|
||||
r += data[i];
|
||||
g += data[i + 1];
|
||||
b += data[i + 2];
|
||||
}
|
||||
return [r / n, g / n, b / n];
|
||||
}
|
||||
|
||||
describe('isNeutralGrade', () => {
|
||||
it('is true only for brightness 0, contrast 1, saturation 1', () => {
|
||||
expect(isNeutralGrade(0, 1, 1)).toBe(true);
|
||||
expect(isNeutralGrade(NEUTRAL_GRADE.brightness, NEUTRAL_GRADE.contrast, NEUTRAL_GRADE.saturation)).toBe(true);
|
||||
expect(isNeutralGrade(1, 1, 1)).toBe(false);
|
||||
expect(isNeutralGrade(0, 1.1, 1)).toBe(false);
|
||||
expect(isNeutralGrade(0, 1, 0.9)).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('applyGradeInPlace', () => {
|
||||
it('leaves neutral values as a no-op', () => {
|
||||
const data = solidRgba([200, 100, 50]);
|
||||
const before = Array.from(data);
|
||||
applyGradeInPlace(data, 0, 1, 1);
|
||||
expect(Array.from(data)).toEqual(before);
|
||||
});
|
||||
|
||||
it('leaves alpha untouched', () => {
|
||||
const data = solidRgba([200, 100, 50], 1, 137);
|
||||
applyGradeInPlace(data, 50, 1.5, 0.5);
|
||||
expect(data[3]).toBe(137);
|
||||
});
|
||||
|
||||
it('positive brightness lightens every channel', () => {
|
||||
const data = solidRgba([200, 100, 50]);
|
||||
const [r0, g0, b0] = meanRgb(data);
|
||||
applyGradeInPlace(data, 50, 1, 1);
|
||||
const [r1, g1, b1] = meanRgb(data);
|
||||
expect(r1).toBeGreaterThan(r0);
|
||||
expect(g1).toBeGreaterThan(g0);
|
||||
expect(b1).toBeGreaterThan(b0);
|
||||
});
|
||||
|
||||
it('negative brightness darkens', () => {
|
||||
const data = solidRgba([200, 100, 50]);
|
||||
const [r0, g0, b0] = meanRgb(data);
|
||||
applyGradeInPlace(data, -50, 1, 1);
|
||||
const [r1, g1, b1] = meanRgb(data);
|
||||
expect(r1).toBeLessThan(r0);
|
||||
expect(g1).toBeLessThan(g0);
|
||||
expect(b1).toBeLessThan(b0);
|
||||
});
|
||||
|
||||
it('clamps brightness at the top of the range instead of wrapping', () => {
|
||||
const data = solidRgba([240, 240, 240]);
|
||||
applyGradeInPlace(data, 100, 1, 1);
|
||||
expect(Array.from(data.subarray(0, 3))).toEqual([255, 255, 255]);
|
||||
});
|
||||
|
||||
it('clamps brightness at the bottom of the range instead of wrapping', () => {
|
||||
const data = solidRgba([10, 10, 10]);
|
||||
applyGradeInPlace(data, -100, 1, 1);
|
||||
expect(Array.from(data.subarray(0, 3))).toEqual([0, 0, 0]);
|
||||
});
|
||||
|
||||
it('zero saturation makes every channel converge on the luma', () => {
|
||||
const data = solidRgba([200, 100, 50]);
|
||||
applyGradeInPlace(data, 0, 1, 0);
|
||||
const [r, g, b] = meanRgb(data);
|
||||
expect(Math.abs(r - g)).toBeLessThan(2);
|
||||
expect(Math.abs(g - b)).toBeLessThan(2);
|
||||
});
|
||||
|
||||
it('saturation above one widens the channel spread', () => {
|
||||
const base = solidRgba([200, 100, 50]);
|
||||
const [baseR, , baseB] = meanRgb(base);
|
||||
const data = solidRgba([200, 100, 50]);
|
||||
applyGradeInPlace(data, 0, 1, 2);
|
||||
const [r, , b] = meanRgb(data);
|
||||
expect(r - b).toBeGreaterThan(baseR - baseB);
|
||||
});
|
||||
|
||||
it('high contrast pushes values away from mid-gray, in both directions', () => {
|
||||
const base = solidRgba([200, 100, 50]);
|
||||
const [baseR, baseG, baseB] = meanRgb(base);
|
||||
const data = solidRgba([200, 100, 50]);
|
||||
applyGradeInPlace(data, 0, 2, 1);
|
||||
const [r, g, b] = meanRgb(data);
|
||||
expect(r).toBeGreaterThan(baseR); // 200 sits above 128, pushed further up
|
||||
expect(g).toBeLessThan(baseG); // 100 sits below 128, pushed further down
|
||||
expect(b).toBeLessThan(baseB); // 50 sits below 128, pushed further down
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,61 @@
|
||||
/**
|
||||
* Client-side mirror of trim.py's `_apply_grade`, so the "before/after" preview in
|
||||
* the Edit video dialog can show the effect of a color grade instantly, without a
|
||||
* server round trip. Same bounds, same order of operations (saturation, then
|
||||
* contrast, then brightness). Not pixel-exact with the server's numpy path --
|
||||
* `Uint8ClampedArray` rounds to the nearest integer on assignment where numpy
|
||||
* truncates -- but the difference is well under a visible unit per channel, and
|
||||
* this is only ever a preview; the server's render (via `POST .../trim`) is what
|
||||
* actually gets saved.
|
||||
*/
|
||||
|
||||
export const BRIGHTNESS_BOUNDS = [-100, 100] as const;
|
||||
export const CONTRAST_BOUNDS = [0, 3] as const;
|
||||
export const SATURATION_BOUNDS = [0, 3] as const;
|
||||
|
||||
export const NEUTRAL_GRADE = { brightness: 0, contrast: 1, saturation: 1 } as const;
|
||||
|
||||
// Perceptual luma weights, matching trim.py's _LUMA_WEIGHTS.
|
||||
const LUMA_R = 0.2126;
|
||||
const LUMA_G = 0.7152;
|
||||
const LUMA_B = 0.0722;
|
||||
|
||||
/** True for the values that leave every pixel unchanged -- mirrors trim.py's `graded` flag. */
|
||||
export function isNeutralGrade(brightness: number, contrast: number, saturation: number): boolean {
|
||||
return brightness === NEUTRAL_GRADE.brightness && contrast === NEUTRAL_GRADE.contrast && saturation === NEUTRAL_GRADE.saturation;
|
||||
}
|
||||
|
||||
/**
|
||||
* Adjusts an RGBA buffer (as in `ImageData.data`) in place; alpha is left alone.
|
||||
* Skipping the call entirely when `isNeutralGrade` is true (as callers should)
|
||||
* avoids the per-pixel work for the common no-op case.
|
||||
*/
|
||||
export function applyGradeInPlace(data: Uint8ClampedArray, brightness: number, contrast: number, saturation: number): void {
|
||||
for (let i = 0; i < data.length; i += 4) {
|
||||
let r = data[i];
|
||||
let g = data[i + 1];
|
||||
let b = data[i + 2];
|
||||
|
||||
if (saturation !== 1) {
|
||||
const luma = r * LUMA_R + g * LUMA_G + b * LUMA_B;
|
||||
r = luma + (r - luma) * saturation;
|
||||
g = luma + (g - luma) * saturation;
|
||||
b = luma + (b - luma) * saturation;
|
||||
}
|
||||
if (contrast !== 1) {
|
||||
r = (r - 128) * contrast + 128;
|
||||
g = (g - 128) * contrast + 128;
|
||||
b = (b - 128) * contrast + 128;
|
||||
}
|
||||
if (brightness !== 0) {
|
||||
r += brightness;
|
||||
g += brightness;
|
||||
b += brightness;
|
||||
}
|
||||
|
||||
// Assigning into a Uint8ClampedArray clamps to [0, 255] and rounds on its own.
|
||||
data[i] = r;
|
||||
data[i + 1] = g;
|
||||
data[i + 2] = b;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,171 @@
|
||||
/**
|
||||
* Ballpark spoken-duration estimate for MiniMax-H3 dialogue text, plus the
|
||||
* lexer/tree used to render a styled "performance preview" of the tags.
|
||||
*
|
||||
* Numbers here are heuristic (syllable counting + fixed per-tag seconds from
|
||||
* `H3_TAGS[].seconds`), not measured against the model -- good enough to
|
||||
* flag "this clip is clearly too long for one generation," calibrated
|
||||
* against a real render via `calibrationFactor`.
|
||||
*/
|
||||
|
||||
import { H3_TAGS, type H3Tag } from "@/lib/h3Tags";
|
||||
|
||||
/** Bare tag name, e.g. "<long pause>" -> "long pause". */
|
||||
function bareName(bracketed: string): string {
|
||||
return bracketed.replace(/[<>/]/g, "").trim();
|
||||
}
|
||||
|
||||
const WRAP_TAG_NAMES = new Set(H3_TAGS.filter((t) => t.kind === "wrap").map((t) => bareName(t.open)));
|
||||
const TAG_SECONDS: Record<string, number> = Object.fromEntries(
|
||||
H3_TAGS.map((t) => [bareName(t.open), t.seconds ?? 0]),
|
||||
);
|
||||
/** Bare tag name -> its registry entry, so callers (the preview renderer) can look up category/description. */
|
||||
export const TAG_BY_NAME: Record<string, H3Tag> = Object.fromEntries(H3_TAGS.map((t) => [bareName(t.open), t]));
|
||||
|
||||
/* -------------------------------------------------------------------- */
|
||||
/* Lexer / tree — mirrors the tag registry so preview & estimate always */
|
||||
/* agree with what the console/editor can actually insert. */
|
||||
/* -------------------------------------------------------------------- */
|
||||
|
||||
export type LexToken =
|
||||
| { t: "text"; v: string }
|
||||
| { t: "point"; name: string }
|
||||
| { t: "open"; name: string }
|
||||
| { t: "close"; name: string };
|
||||
|
||||
const TAG_RE = /<(\/?)([a-zA-Z ]+?)>/g;
|
||||
|
||||
export function lexDialogue(text: string): LexToken[] {
|
||||
const tokens: LexToken[] = [];
|
||||
let last = 0;
|
||||
let m: RegExpExecArray | null;
|
||||
TAG_RE.lastIndex = 0;
|
||||
while ((m = TAG_RE.exec(text))) {
|
||||
if (m.index > last) tokens.push({ t: "text", v: text.slice(last, m.index) });
|
||||
const closing = m[1] === "/";
|
||||
const name = m[2].trim();
|
||||
if (WRAP_TAG_NAMES.has(name)) tokens.push({ t: closing ? "close" : "open", name });
|
||||
else if (!closing) tokens.push({ t: "point", name }); // stray/unknown closers are ignored
|
||||
last = TAG_RE.lastIndex;
|
||||
}
|
||||
if (last < text.length) tokens.push({ t: "text", v: text.slice(last) });
|
||||
return tokens;
|
||||
}
|
||||
|
||||
export interface DialogueTextNode {
|
||||
t: "text";
|
||||
v: string;
|
||||
}
|
||||
export interface DialoguePointNode {
|
||||
t: "point";
|
||||
name: string;
|
||||
}
|
||||
export interface DialogueSpanNode {
|
||||
t: "span";
|
||||
name: string;
|
||||
children: DialogueNode[];
|
||||
}
|
||||
export type DialogueNode = DialogueTextNode | DialoguePointNode | DialogueSpanNode;
|
||||
|
||||
export function dialogueTree(text: string): DialogueNode[] {
|
||||
const root: { children: DialogueNode[] } = { children: [] };
|
||||
const stack: { children: DialogueNode[] }[] = [root];
|
||||
for (const tk of lexDialogue(text)) {
|
||||
const top = stack[stack.length - 1];
|
||||
if (tk.t === "text") top.children.push({ t: "text", v: tk.v });
|
||||
else if (tk.t === "point") top.children.push({ t: "point", name: tk.name });
|
||||
else if (tk.t === "open") {
|
||||
const node: DialogueSpanNode = { t: "span", name: tk.name, children: [] };
|
||||
top.children.push(node);
|
||||
stack.push(node);
|
||||
} else if (tk.t === "close" && stack.length > 1) stack.pop();
|
||||
}
|
||||
return root.children;
|
||||
}
|
||||
|
||||
/* -------------------------------------------------------------------- */
|
||||
/* Duration estimate */
|
||||
/* -------------------------------------------------------------------- */
|
||||
|
||||
export const SPEECH_RATES = { Measured: 3.2, Natural: 4.0, Quick: 4.8 } as const; // syllables/sec
|
||||
export type SpeechRate = keyof typeof SPEECH_RATES;
|
||||
|
||||
export function syllables(word: string): number {
|
||||
let w = word.toLowerCase().replace(/[^a-z]/g, "");
|
||||
if (!w) return 0;
|
||||
if (w.length <= 3) return 1;
|
||||
w = w.replace(/(?:[^laeiouy]es|ed|[^laeiouy]e)$/, "");
|
||||
w = w.replace(/^y/, "");
|
||||
const m = w.match(/[aeiouy]{1,2}/g);
|
||||
return m ? m.length : 1;
|
||||
}
|
||||
|
||||
export interface DurationEstimate {
|
||||
total: number;
|
||||
speech: number;
|
||||
punct: number;
|
||||
tags: number;
|
||||
syl: number;
|
||||
words: number;
|
||||
low: number;
|
||||
high: number;
|
||||
}
|
||||
|
||||
export function estimateDuration(text: string, rate: number, calib: number): DurationEstimate {
|
||||
const spoken = text.replace(/<\/?[a-zA-Z ]+?>/g, " ");
|
||||
const words = spoken.split(/\s+/).filter(Boolean);
|
||||
const syl = words.reduce((n, w) => n + syllables(w), 0);
|
||||
const speech = syl / rate;
|
||||
|
||||
const sentence = (spoken.match(/[.!?]/g) || []).length * 0.35;
|
||||
const commas = (spoken.match(/[,;:]/g) || []).length * 0.18;
|
||||
const ellipsis = (spoken.match(/\.\.\.|…/g) || []).length * 0.5;
|
||||
const dashes = (spoken.match(/—/g) || []).length * 0.2;
|
||||
const punct = sentence + commas + ellipsis + dashes;
|
||||
|
||||
let tags = 0;
|
||||
const re = /<(\/?)([a-zA-Z ]+?)>/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(text))) {
|
||||
if (m[1] === "/") continue;
|
||||
tags += TAG_SECONDS[m[2].trim()] ?? 0;
|
||||
}
|
||||
|
||||
const total = (speech + punct + tags) * calib;
|
||||
return {
|
||||
total,
|
||||
speech: speech * calib,
|
||||
punct: punct * calib,
|
||||
tags: tags * calib,
|
||||
syl,
|
||||
words: words.length,
|
||||
low: total * 0.86,
|
||||
high: total * 1.16,
|
||||
};
|
||||
}
|
||||
|
||||
/** Find the best sentence/pause boundary at which to split text that runs past `ceiling` seconds. */
|
||||
export function suggestSplit(text: string, rate: number, calib: number, ceiling: number): number {
|
||||
const re = /(<\/?[a-zA-Z ]+?>)|([.!?]+)|([,;:]+)|(\s+)|([^\s<.!?,;:]+)/g;
|
||||
let cum = 0;
|
||||
let boundary = 0;
|
||||
let m: RegExpExecArray | null;
|
||||
const budget = ceiling - 1.5;
|
||||
while ((m = re.exec(text))) {
|
||||
if (m[1]) {
|
||||
const closing = m[1].startsWith("</");
|
||||
const name = bareName(m[1]);
|
||||
if (!closing) cum += (TAG_SECONDS[name] ?? 0) * calib;
|
||||
if (name === "pause" || name === "long pause") boundary = re.lastIndex;
|
||||
} else if (m[2]) {
|
||||
cum += 0.35 * calib;
|
||||
boundary = re.lastIndex;
|
||||
} else if (m[3]) {
|
||||
cum += 0.18 * calib;
|
||||
} else if (m[5]) {
|
||||
cum += (syllables(m[5]) / rate) * calib;
|
||||
}
|
||||
if (cum > budget && boundary > 0) return boundary;
|
||||
}
|
||||
return boundary > 0 ? boundary : -1;
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { readField, shotTarget, writeField, writeShotTarget } from "@/lib/h3PromptFields";
|
||||
|
||||
const BASE = [
|
||||
"For the target video, <Picture 1> is fully referenced.",
|
||||
"",
|
||||
"integrated_multimodal_description: [Shot 1] A close-up. He (S1) says: <d>[English] Hi.</d> <cutoff>",
|
||||
"",
|
||||
"overall_soundscape: Room tone.",
|
||||
"",
|
||||
"non_diegetic_music: N/A",
|
||||
].join("\n");
|
||||
|
||||
const SECTIONS = [
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is the courier.",
|
||||
"",
|
||||
"retention_analysis:",
|
||||
"<Subject 1> (appears in [Shot 1]): fully_preserved - x.",
|
||||
"",
|
||||
"detailed_description:",
|
||||
"[Shot 1] Wide shot. A rooftop.",
|
||||
"[Shot 2] At 00:02.000, the shot cuts to a close-up shot. A face.",
|
||||
"",
|
||||
"overall_soundscape:",
|
||||
"Rain.",
|
||||
].join("\n");
|
||||
|
||||
describe("readField / writeField", () => {
|
||||
it("reads a field whose text follows the heading on the same line", () => {
|
||||
expect(readField(BASE, "integrated_multimodal_description")).toBe(
|
||||
"[Shot 1] A close-up. He (S1) says: <d>[English] Hi.</d> <cutoff>",
|
||||
);
|
||||
});
|
||||
|
||||
it("reads a field whose text starts on the line after the heading", () => {
|
||||
expect(readField(SECTIONS, "detailed_description")).toBe(
|
||||
"[Shot 1] Wide shot. A rooftop.\n[Shot 2] At 00:02.000, the shot cuts to a close-up shot. A face.",
|
||||
);
|
||||
expect(readField(SECTIONS, "subject_definitions")).toBe("<Subject 1> is the courier.");
|
||||
});
|
||||
|
||||
it("returns null for a field that isn't there", () => {
|
||||
expect(readField(BASE, "detailed_description")).toBeNull();
|
||||
});
|
||||
|
||||
it("rewrites only that field, leaving the rest byte-identical (inline style)", () => {
|
||||
const out = writeField(BASE, "integrated_multimodal_description", "[Shot 1] New.");
|
||||
expect(out).toBe(BASE.replace("[Shot 1] A close-up. He (S1) says: <d>[English] Hi.</d> <cutoff>", "[Shot 1] New."));
|
||||
});
|
||||
|
||||
it("rewrites only that field (own-line style)", () => {
|
||||
const out = writeField(SECTIONS, "detailed_description", "[Shot 1] Only.");
|
||||
expect(out).toContain("detailed_description:\n[Shot 1] Only.\n\noverall_soundscape:\nRain.");
|
||||
expect(out.startsWith("subject_definitions:\n<Subject 1> is the courier.")).toBe(true);
|
||||
});
|
||||
|
||||
it("writing a field's own text back is a no-op", () => {
|
||||
for (const [prompt, field] of [
|
||||
[BASE, "integrated_multimodal_description"],
|
||||
[SECTIONS, "detailed_description"],
|
||||
[SECTIONS, "retention_analysis"],
|
||||
] as const) {
|
||||
expect(writeField(prompt, field, readField(prompt, field) as string)).toBe(prompt);
|
||||
}
|
||||
});
|
||||
|
||||
it("ignores a heading that isn't at the start of a line", () => {
|
||||
expect(readField("see detailed_description: not a header", "detailed_description")).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("shotTarget", () => {
|
||||
it("prefers integrated_multimodal_description", () => {
|
||||
expect(shotTarget(BASE).field).toBe("integrated_multimodal_description");
|
||||
});
|
||||
|
||||
it("falls back to detailed_description", () => {
|
||||
expect(shotTarget(SECTIONS).field).toBe("detailed_description");
|
||||
});
|
||||
|
||||
it("treats a plain prompt as the shot text itself", () => {
|
||||
const t = shotTarget("A toy car drives into a plush dog.");
|
||||
expect(t).toEqual({ field: null, body: "A toy car drives into a plush dog." });
|
||||
expect(writeShotTarget("A toy car drives into a plush dog.", t, "[Shot 1] X.")).toBe("[Shot 1] X.");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,77 @@
|
||||
/**
|
||||
* Locate and rewrite a single named field inside a raw MiniMax-H3 prompt,
|
||||
* leaving every other character of the prompt untouched. Handles both header
|
||||
* styles seen in real prompts: the heading on its own line (six-section
|
||||
* reference format) and the heading followed by its text on the same line
|
||||
* (`integrated_multimodal_description: [Shot 1] ...`, the base format).
|
||||
*/
|
||||
|
||||
const FIELD_NAMES = [
|
||||
"subject_definitions",
|
||||
"summary",
|
||||
"retention_analysis",
|
||||
"detailed_description",
|
||||
"integrated_multimodal_description",
|
||||
"overall_soundscape",
|
||||
"non_diegetic_music",
|
||||
] as const;
|
||||
|
||||
export type H3PromptFieldName = (typeof FIELD_NAMES)[number];
|
||||
|
||||
const HEADER_RE = new RegExp(`^(${FIELD_NAMES.join("|")}):[ \\t]*`, "gm");
|
||||
|
||||
interface Span {
|
||||
start: number;
|
||||
end: number;
|
||||
}
|
||||
|
||||
function locate(prompt: string, field: H3PromptFieldName): Span | null {
|
||||
const headers = [...prompt.matchAll(HEADER_RE)];
|
||||
const i = headers.findIndex((h) => h[1] === field);
|
||||
if (i < 0) return null;
|
||||
|
||||
const header = headers[i];
|
||||
let start = (header.index ?? 0) + header[0].length;
|
||||
// Own-line style: nothing follows the colon, so the body starts on the next line.
|
||||
const newline = prompt.slice(start).match(/^\r?\n/);
|
||||
if (newline) start += newline[0].length;
|
||||
|
||||
const nextHeader = i + 1 < headers.length ? (headers[i + 1].index ?? prompt.length) : prompt.length;
|
||||
const end = start + Math.max(0, prompt.slice(start, nextHeader).trimEnd().length);
|
||||
return { start, end: Math.max(start, end) };
|
||||
}
|
||||
|
||||
/** The field's text exactly as written, or null if the prompt has no such field. */
|
||||
export function readField(prompt: string, field: H3PromptFieldName): string | null {
|
||||
const span = locate(prompt, field);
|
||||
return span ? prompt.slice(span.start, span.end) : null;
|
||||
}
|
||||
|
||||
/** Replace only the field's text. Returns the prompt unchanged if the field isn't there. */
|
||||
export function writeField(prompt: string, field: H3PromptFieldName, body: string): string {
|
||||
const span = locate(prompt, field);
|
||||
return span ? prompt.slice(0, span.start) + body + prompt.slice(span.end) : prompt;
|
||||
}
|
||||
|
||||
export interface ShotTarget {
|
||||
/** the field the [Shot N] blocks live in, or null when the whole prompt is the shot text */
|
||||
field: "integrated_multimodal_description" | "detailed_description" | null;
|
||||
body: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* Where a prompt's shots live: the base format's integrated_multimodal_description,
|
||||
* else the reference format's detailed_description, else the prompt as a whole.
|
||||
*/
|
||||
export function shotTarget(prompt: string): ShotTarget {
|
||||
for (const field of ["integrated_multimodal_description", "detailed_description"] as const) {
|
||||
const body = readField(prompt, field);
|
||||
if (body !== null) return { field, body };
|
||||
}
|
||||
return { field: null, body: prompt };
|
||||
}
|
||||
|
||||
/** Write shot text back to wherever shotTarget found it. */
|
||||
export function writeShotTarget(prompt: string, target: ShotTarget, body: string): string {
|
||||
return target.field ? writeField(prompt, target.field, body) : body;
|
||||
}
|
||||
@@ -0,0 +1,253 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
buildClips,
|
||||
convertDirections,
|
||||
parseScript,
|
||||
scriptSpeakers,
|
||||
type ImportSpeaker,
|
||||
} from "@/lib/h3ScriptImport";
|
||||
import { detailedDescriptionFromShots, shotLines, shotProse } from "@/lib/h3Shots";
|
||||
|
||||
// Original test material -- not from any real screenplay.
|
||||
const SCRIPT = [
|
||||
"INT. HARBOR OFFICE - DAY",
|
||||
"",
|
||||
"CAPTAIN: Sit down. (EXHALES) We need to talk about the ledger.",
|
||||
"",
|
||||
"CLERK: Yes, sir.",
|
||||
"",
|
||||
"CAPTAIN: Three crates are missing, and nobody on this dock saw a thing.",
|
||||
"Not one of you. That does not happen by accident.",
|
||||
"",
|
||||
"CLERK: I counted twice.",
|
||||
"",
|
||||
"CAPTAIN: (WHISPERING) Then count a third time.",
|
||||
"",
|
||||
"CLERK: (TAPPING TABLE) Right away.",
|
||||
].join("\n");
|
||||
|
||||
const SPEAKERS: Record<string, ImportSpeaker> = {
|
||||
CAPTAIN: { subject: "<Subject 1>", tag: "S1" },
|
||||
CLERK: { subject: "<Subject 2>", tag: "S2" },
|
||||
};
|
||||
|
||||
describe("parseScript", () => {
|
||||
it("reads NAME: line turns and ignores headings between turns", () => {
|
||||
const turns = parseScript(SCRIPT);
|
||||
expect(turns.map((t) => t.speaker)).toEqual(["CAPTAIN", "CLERK", "CAPTAIN", "CLERK", "CAPTAIN", "CLERK"]);
|
||||
expect(turns.some((t) => t.text.includes("HARBOR"))).toBe(false);
|
||||
});
|
||||
|
||||
it("joins continuation lines directly under a turn", () => {
|
||||
const turns = parseScript(SCRIPT);
|
||||
expect(turns[2].text).toBe(
|
||||
"Three crates are missing, and nobody on this dock saw a thing. Not one of you. That does not happen by accident.",
|
||||
);
|
||||
});
|
||||
|
||||
it("tolerates markdown bold speaker names", () => {
|
||||
expect(parseScript("**MARK:** hello there")).toEqual([{ speaker: "MARK", text: "hello there" }]);
|
||||
});
|
||||
|
||||
it("lists speakers in order of first appearance", () => {
|
||||
expect(scriptSpeakers(parseScript(SCRIPT))).toEqual(["CAPTAIN", "CLERK"]);
|
||||
});
|
||||
|
||||
it("returns nothing for text with no speaker turns", () => {
|
||||
expect(parseScript("just some prose without turns")).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("convertDirections", () => {
|
||||
it("turns known parentheticals into speech tags in place", () => {
|
||||
expect(convertDirections("Sit down. (EXHALES) We talk.").text).toBe("Sit down. <exhale> We talk.");
|
||||
expect(convertDirections("Well (BEAT) fine.").text).toBe("Well <pause> fine.");
|
||||
});
|
||||
|
||||
it("wraps the line for a whispering direction", () => {
|
||||
expect(convertDirections("(WHISPERING) Keep it down.").text).toBe("<whisper>Keep it down.</whisper>");
|
||||
});
|
||||
|
||||
it("keeps unmapped directions out of the dialogue but reports them", () => {
|
||||
const r = convertDirections("(TAPPING TABLE) Right away.");
|
||||
expect(r.text).toBe("Right away.");
|
||||
expect(r.directions).toEqual(["tapping table"]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildClips", () => {
|
||||
it("opens the first clip with an establishing shot", () => {
|
||||
const [clip] = buildClips(parseScript(SCRIPT), { speakers: SPEAKERS, targetSeconds: 60 });
|
||||
expect(clip.shots[0].type).toBe("establishing");
|
||||
expect(shotLines(clip.shots[0])).toHaveLength(0);
|
||||
expect(clip.shots[0].subjectIds).toEqual(["<Subject 1>", "<Subject 2>"]);
|
||||
});
|
||||
|
||||
it("folds a short reaction into the shot it reacts to", () => {
|
||||
const [clip] = buildClips(parseScript(SCRIPT), { speakers: SPEAKERS, targetSeconds: 60 });
|
||||
const first = clip.shots[1];
|
||||
expect(shotLines(first).map((l) => [l.speaker, l.text])).toEqual([
|
||||
["S1", "Sit down. <exhale> We need to talk about the ledger."],
|
||||
["S2", "Yes, sir."],
|
||||
]);
|
||||
expect(first.type).toBe("medium");
|
||||
expect(first.subjectIds).toEqual(["<Subject 1>", "<Subject 2>"]);
|
||||
});
|
||||
|
||||
it("never adds camera movement, however long the shot runs", () => {
|
||||
const long = "We keep the clients on the wheel and the wheel keeps turning, every day of the week. ".repeat(4).trim();
|
||||
const clips = buildClips(
|
||||
[
|
||||
{ speaker: "CAPTAIN", text: long },
|
||||
{ speaker: "CLERK", text: long },
|
||||
{ speaker: "CAPTAIN", text: "Yes." },
|
||||
],
|
||||
{ speakers: SPEAKERS, targetSeconds: 60 },
|
||||
);
|
||||
const shots = clips.flatMap((c) => c.shots);
|
||||
expect(shots.length).toBeGreaterThan(2);
|
||||
expect(shots.every((s) => s.movement === "static")).toBe(true);
|
||||
});
|
||||
|
||||
it("uses a close-up when only one person speaks in a shot", () => {
|
||||
const [clip] = buildClips([{ speaker: "CAPTAIN", text: "Nobody leaves until the count is done." }], {
|
||||
speakers: SPEAKERS,
|
||||
targetSeconds: 60,
|
||||
});
|
||||
const shot = clip.shots[1];
|
||||
expect(shot.type).toBe("close-up");
|
||||
expect(shot.subjectIds).toEqual(["<Subject 1>"]);
|
||||
expect(shotProse(shot)).toBe("Close on <Subject 1>, speaking.");
|
||||
});
|
||||
|
||||
it("records unmapped stage directions on the shot they belong to", () => {
|
||||
const [clip] = buildClips(parseScript(SCRIPT), { speakers: SPEAKERS, targetSeconds: 60 });
|
||||
const last = clip.shots[clip.shots.length - 1];
|
||||
expect(shotLines(last).map((l) => l.text)).toEqual(["<whisper>Then count a third time.</whisper>", "Right away."]);
|
||||
expect(shotProse(last)).toContain("Stage direction: tapping table.");
|
||||
});
|
||||
|
||||
it("splits a long scene into several clips, each with dialogue", () => {
|
||||
const clips = buildClips(parseScript(SCRIPT), { speakers: SPEAKERS, targetSeconds: 5 });
|
||||
expect(clips.length).toBeGreaterThan(1);
|
||||
for (const c of clips) expect(c.shots.some((s) => shotLines(s).length > 0)).toBe(true);
|
||||
expect(clips[1].shots[0].type).not.toBe("establishing");
|
||||
});
|
||||
|
||||
it("keeps every spoken word across clips", () => {
|
||||
const clips = buildClips(parseScript(SCRIPT), { speakers: SPEAKERS, targetSeconds: 5 });
|
||||
const all = clips.flatMap((c) => c.shots.flatMap((s) => shotLines(s).map((l) => l.text))).join(" ");
|
||||
for (const word of ["ledger", "crates", "accident", "counted", "third", "Right away"]) {
|
||||
expect(all).toContain(word);
|
||||
}
|
||||
});
|
||||
|
||||
it("serializes to valid [Shot N] text with speaker-tagged dialogue", () => {
|
||||
const [clip] = buildClips(parseScript(SCRIPT), { speakers: SPEAKERS, targetSeconds: 60 });
|
||||
const text = detailedDescriptionFromShots(clip.shots);
|
||||
expect(text).toContain("[Shot 1] Establishing shot.");
|
||||
expect(text).toContain("(S1) <d>[English] Sit down. <exhale> We need to talk about the ledger.</d>");
|
||||
expect(text).toContain("(S2) <d>[English] Yes, sir.</d>");
|
||||
});
|
||||
|
||||
describe("speaker tags per clip", () => {
|
||||
it("numbers speakers by who speaks first in each clip, not across the whole script", () => {
|
||||
const turns = [
|
||||
{ speaker: "CAPTAIN", text: "Every one of these lines is long enough to stand as its own turn today." },
|
||||
{ speaker: "CLERK", text: "And every reply here is long enough to stand as its own turn as well." },
|
||||
{ speaker: "CLERK", text: "The clerk speaks again first when the next clip begins, with a full sentence." },
|
||||
{ speaker: "CAPTAIN", text: "And then the captain answers the clerk with another complete sentence here." },
|
||||
];
|
||||
const clips = buildClips(turns, { speakers: SPEAKERS, targetSeconds: 8 });
|
||||
expect(clips.length).toBeGreaterThan(1);
|
||||
for (const c of clips) {
|
||||
const first = c.shots.flatMap((s) => shotLines(s))[0];
|
||||
expect(first.speaker).toBe("S1");
|
||||
}
|
||||
// a clip that opens on the clerk maps the clerk to S1 and the captain to S2
|
||||
const clerkFirst = clips.find((c) => shotLines(c.shots.find((s) => shotLines(s).length)!)[0].text.startsWith("The clerk"));
|
||||
const speakers = clerkFirst!.shots.flatMap((s) => shotLines(s).map((l) => l.speaker));
|
||||
expect(new Set(speakers).size).toBeLessThanOrEqual(2);
|
||||
});
|
||||
|
||||
it("leaves custom speaker tags alone", () => {
|
||||
const clips = buildClips(
|
||||
[{ speaker: "CLERK", text: "A line for the clerk that is long enough to stand alone here." }],
|
||||
{ speakers: { CLERK: { subject: "<Subject 2>", tag: "CLERK" }, CAPTAIN: SPEAKERS.CAPTAIN }, targetSeconds: 11 },
|
||||
);
|
||||
expect(shotLines(clips[0].shots[1])[0].speaker).toBe("CLERK");
|
||||
});
|
||||
});
|
||||
|
||||
describe("clip boundaries", () => {
|
||||
const longSpeech = Array.from({ length: 12 }, (_, i) => `This is sentence number ${i + 1} of a long speech.`).join(" ");
|
||||
|
||||
it("keeps a long turn as one continuous shot when a clip can hold it", () => {
|
||||
const clips = buildClips([{ speaker: "CAPTAIN", text: longSpeech }], { speakers: SPEAKERS, targetSeconds: 60 });
|
||||
expect(clips).toHaveLength(1);
|
||||
const dialogueShots = clips[0].shots.filter((s) => shotLines(s).length > 0);
|
||||
expect(dialogueShots).toHaveLength(1);
|
||||
expect(shotLines(dialogueShots[0])[0].text).toBe(longSpeech);
|
||||
});
|
||||
|
||||
it("only splits inside a shot when the turn is longer than a whole clip, and marks the follow-on clips", () => {
|
||||
const clips = buildClips([{ speaker: "CAPTAIN", text: longSpeech }], { speakers: SPEAKERS, targetSeconds: 11 });
|
||||
expect(clips.length).toBeGreaterThan(1);
|
||||
expect(clips[0].continuation).toBe(false);
|
||||
for (const c of clips.slice(1)) {
|
||||
expect(c.continuation).toBe(true);
|
||||
expect(shotProse(c.shots[0])).toContain("Continuing the same shot");
|
||||
expect(c.shots[0].type).toBe("close-up");
|
||||
}
|
||||
const spoken = clips.flatMap((c) => c.shots.flatMap((s) => shotLines(s).map((l) => l.text))).join(" ");
|
||||
expect(spoken).toBe(longSpeech);
|
||||
});
|
||||
|
||||
it("marks speech that runs past the end of a clip with <cutoff>, and only there", () => {
|
||||
const clips = buildClips([{ speaker: "CAPTAIN", text: longSpeech }], { speakers: SPEAKERS, targetSeconds: 11 });
|
||||
const endings = clips.map((c) => detailedDescriptionFromShots(c.shots));
|
||||
for (const text of endings.slice(0, -1)) expect(text).toMatch(/<\/d> <cutoff>$/);
|
||||
expect(endings[endings.length - 1]).not.toContain("<cutoff>");
|
||||
// the marker is outside the spoken text, so it never leaks into the dialogue itself
|
||||
const spoken = clips.flatMap((c) => c.shots.flatMap((s) => shotLines(s).map((l) => l.text))).join(" ");
|
||||
expect(spoken).not.toContain("cutoff");
|
||||
});
|
||||
|
||||
it("breaks between separate turns at a cut, not as a continuation", () => {
|
||||
const turns = Array.from({ length: 6 }, () => ({
|
||||
speaker: "CAPTAIN",
|
||||
text: "Every one of these turns is a complete thought that takes a while to say aloud.",
|
||||
}));
|
||||
const clips = buildClips(turns, { speakers: SPEAKERS, targetSeconds: 11 });
|
||||
expect(clips.length).toBeGreaterThan(1);
|
||||
expect(clips.every((c) => !c.continuation)).toBe(true);
|
||||
});
|
||||
|
||||
it("re-wraps a whispered line on every piece so no tag is left unmatched", () => {
|
||||
const clips = buildClips([{ speaker: "CAPTAIN", text: `(WHISPERING) ${longSpeech}` }], {
|
||||
speakers: SPEAKERS,
|
||||
targetSeconds: 11,
|
||||
});
|
||||
const texts = clips.flatMap((c) => c.shots.flatMap((s) => shotLines(s).map((l) => l.text)));
|
||||
expect(texts.length).toBeGreaterThan(1);
|
||||
for (const t of texts) {
|
||||
expect(t.startsWith("<whisper>")).toBe(true);
|
||||
expect(t.endsWith("</whisper>")).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("puts a trailing reaction with the last piece of a split speech", () => {
|
||||
const clips = buildClips(
|
||||
[
|
||||
{ speaker: "CAPTAIN", text: longSpeech },
|
||||
{ speaker: "CLERK", text: "Understood." },
|
||||
],
|
||||
{ speakers: SPEAKERS, targetSeconds: 11 },
|
||||
);
|
||||
const lastShot = clips[clips.length - 1].shots.at(-1)!;
|
||||
expect(shotLines(lastShot).map((l) => l.speaker)).toEqual(["S1", "S2"]);
|
||||
const earlier = clips.slice(0, -1).flatMap((c) => c.shots.flatMap((s) => shotLines(s).map((l) => l.speaker)));
|
||||
expect(earlier.every((sp) => sp === "S1")).toBe(true);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,328 @@
|
||||
/**
|
||||
* Turns a plain dialogue script ("NAME: line" turns) into clip-sized shot
|
||||
* lists for the shot editor. Mechanical work only -- speakers, dialogue,
|
||||
* stage-direction-to-tag mapping, grouping turns into shots, and splitting
|
||||
* the whole scene into clips that fit one generation. Camera choices are
|
||||
* defaults for the user to refine; a script says nothing about them.
|
||||
*/
|
||||
|
||||
import { SPEECH_RATES, estimateDuration } from "@/lib/h3DialogueEstimate";
|
||||
import { estimateSpokenSeconds, makeShot, newLine, newTextPart, shotLines, type Shot } from "@/lib/h3Shots";
|
||||
import { H3_TAGS } from "@/lib/h3Tags";
|
||||
|
||||
export interface ScriptTurn {
|
||||
speaker: string;
|
||||
text: string;
|
||||
}
|
||||
|
||||
// "MARK: ..." -- 1-3 capitalized words, so prose like "note: ..." or scene
|
||||
// headings without a colon are never mistaken for a speaker.
|
||||
const SPEAKER_LINE_RE = /^([A-Z][A-Za-z0-9.'_-]*(?: [A-Z][A-Za-z0-9.'_-]*){0,2}):\s*(.*)$/;
|
||||
|
||||
function stripMarkdown(line: string): string {
|
||||
return line
|
||||
.replace(/^\s*(?:[>*-]\s+)+/, "")
|
||||
.replace(/\*\*|__/g, "")
|
||||
.trim();
|
||||
}
|
||||
|
||||
/**
|
||||
* A speaker line starts a turn; non-blank lines directly after it continue
|
||||
* that turn. A blank line ends the turn, so scene headings and action lines
|
||||
* between turns are ignored rather than glued onto someone's dialogue.
|
||||
*/
|
||||
export function parseScript(script: string): ScriptTurn[] {
|
||||
const turns: ScriptTurn[] = [];
|
||||
let continuing = false;
|
||||
for (const raw of script.split(/\r?\n/)) {
|
||||
const line = stripMarkdown(raw);
|
||||
if (!line) {
|
||||
continuing = false;
|
||||
continue;
|
||||
}
|
||||
const m = line.match(SPEAKER_LINE_RE);
|
||||
if (m) {
|
||||
turns.push({ speaker: m[1], text: m[2].trim() });
|
||||
continuing = true;
|
||||
} else if (continuing && turns.length > 0) {
|
||||
const last = turns[turns.length - 1];
|
||||
last.text = `${last.text} ${line}`.trim();
|
||||
}
|
||||
}
|
||||
return turns.filter((t) => t.text);
|
||||
}
|
||||
|
||||
/** Speaker names in order of first appearance. */
|
||||
export function scriptSpeakers(turns: ScriptTurn[]): string[] {
|
||||
const seen: string[] = [];
|
||||
for (const t of turns) if (!seen.includes(t.speaker)) seen.push(t.speaker);
|
||||
return seen;
|
||||
}
|
||||
|
||||
/* ---------------------------------------------------------------- */
|
||||
/* Stage directions -> speech tags */
|
||||
/* ---------------------------------------------------------------- */
|
||||
|
||||
const TAG_BY_ID = Object.fromEntries(H3_TAGS.map((t) => [t.id, t]));
|
||||
|
||||
// Order matters: the more specific phrase must be tried before its substring.
|
||||
const DIRECTION_TAGS: [RegExp, string][] = [
|
||||
[/\blong pause\b/, "long-pause"],
|
||||
[/\bcatch(?:es)? (?:his |her |their )?breath\b/, "catches-breath"],
|
||||
[/\bexhal/, "exhale"],
|
||||
[/\binhal/, "inhale"],
|
||||
[/\bsigh/, "sighs"],
|
||||
[/\blaugh/, "laughs"],
|
||||
[/\bchuckl/, "chuckle"],
|
||||
[/\bgasp/, "gasp"],
|
||||
[/\bcough/, "coughs"],
|
||||
[/\bclears? (?:his |her |their )?throat\b/, "clears-throat"],
|
||||
[/\bsniff/, "sniff"],
|
||||
[/\b(?:pause|beat)\b/, "pause"],
|
||||
[/\bstutter|\bstammer/, "stutter"],
|
||||
[/\bbreath/, "breath"],
|
||||
];
|
||||
|
||||
export interface ConvertedLine {
|
||||
text: string;
|
||||
/** parentheticals with no matching tag, kept as stage directions for the shot description */
|
||||
directions: string[];
|
||||
}
|
||||
|
||||
/** Rewrites "(EXHALES)"-style parentheticals as speech tags; "(WHISPERING)" wraps the whole line. */
|
||||
export function convertDirections(text: string): ConvertedLine {
|
||||
const directions: string[] = [];
|
||||
let wrapWhisper = false;
|
||||
|
||||
let out = text.replace(/\(([^)]*)\)/g, (_m, inner: string) => {
|
||||
const key = inner.trim().toLowerCase();
|
||||
if (/\bwhisper/.test(key)) {
|
||||
wrapWhisper = true;
|
||||
return " ";
|
||||
}
|
||||
for (const [re, id] of DIRECTION_TAGS) {
|
||||
const tag = TAG_BY_ID[id];
|
||||
if (tag && tag.kind === "point" && re.test(key)) return ` ${tag.open} `;
|
||||
}
|
||||
if (key) directions.push(key);
|
||||
return " ";
|
||||
});
|
||||
|
||||
out = out
|
||||
.replace(/\s+/g, " ")
|
||||
.replace(/\s+([.,!?…])/g, "$1")
|
||||
.trim();
|
||||
|
||||
const whisper = TAG_BY_ID.whisper;
|
||||
if (wrapWhisper && whisper?.close && out) out = `${whisper.open}${out}${whisper.close}`;
|
||||
return { text: out, directions };
|
||||
}
|
||||
|
||||
/* ---------------------------------------------------------------- */
|
||||
/* Shots and clips */
|
||||
/* ---------------------------------------------------------------- */
|
||||
|
||||
export interface ImportSpeaker {
|
||||
/** <Subject N> label this speaker is, or null when they aren't a tracked subject */
|
||||
subject: string | null;
|
||||
/** speaker tag written as "(S1)" */
|
||||
tag: string;
|
||||
}
|
||||
|
||||
export interface ImportOptions {
|
||||
speakers: Record<string, ImportSpeaker>;
|
||||
/** rough length one clip should fit, in seconds */
|
||||
targetSeconds: number;
|
||||
}
|
||||
|
||||
export interface ImportClip {
|
||||
shots: Shot[];
|
||||
/**
|
||||
* True when this clip picks up in the middle of the previous clip's last
|
||||
* shot (a speech too long for one generation). Its first shot continues
|
||||
* that one instead of cutting to something new, so it should be generated
|
||||
* from the previous clip's last frame. False means the boundary is a cut.
|
||||
*/
|
||||
continuation: boolean;
|
||||
}
|
||||
|
||||
const ESTABLISHING_SECONDS = 1.5;
|
||||
const REACTION_MAX_WORDS = 4;
|
||||
// Space left in a clip is only worth starting a forced mid-shot split in if
|
||||
// it can hold at least this much of the speech.
|
||||
const MIN_PART_SECONDS = 3;
|
||||
|
||||
function spokenSeconds(text: string): number {
|
||||
return estimateDuration(text, SPEECH_RATES.Natural, 1).total;
|
||||
}
|
||||
|
||||
function wordCount(text: string): number {
|
||||
return (text.replace(/<[^>]*>/g, " ").match(/[\p{L}\p{N}'’-]+/gu) ?? []).length;
|
||||
}
|
||||
|
||||
/**
|
||||
* Split one over-long line at sentence boundaries into parts that fit the
|
||||
* given budgets (first part, then every later part). An outer wrapping tag
|
||||
* like <whisper>...</whisper> is re-applied to each part so none is left
|
||||
* with an unmatched tag.
|
||||
*/
|
||||
function splitToBudgets(text: string, firstBudget: number, laterBudget: number): string[] {
|
||||
const wrap = text.match(/^<([a-z ]+)>([\s\S]*)<\/\1>$/);
|
||||
const inner = wrap ? wrap[2] : text;
|
||||
const sentences = inner.match(/[^.!?…]+(?:[.!?…]+|$)\s*/g) ?? [inner];
|
||||
const parts: string[] = [];
|
||||
let current = "";
|
||||
let budget = firstBudget;
|
||||
for (const sentence of sentences) {
|
||||
const candidate = `${current}${sentence}`;
|
||||
if (current && spokenSeconds(candidate) > budget) {
|
||||
parts.push(current.trim());
|
||||
current = sentence;
|
||||
budget = laterBudget;
|
||||
} else {
|
||||
current = candidate;
|
||||
}
|
||||
}
|
||||
if (current.trim()) parts.push(current.trim());
|
||||
return wrap ? parts.map((p) => `<${wrap[1]}>${p}</${wrap[1]}>`) : parts;
|
||||
}
|
||||
|
||||
interface Beat {
|
||||
/** the first line is the turn the shot is about; later lines are short reactions */
|
||||
lines: { speaker: string; text: string }[];
|
||||
directions: string[];
|
||||
}
|
||||
|
||||
function beatSeconds(beat: Beat): number {
|
||||
return estimateSpokenSeconds(beat.lines.map((l) => l.text));
|
||||
}
|
||||
|
||||
function titleCase(name: string): string {
|
||||
return name.toLowerCase().replace(/(^|[\s'-])(\p{L})/gu, (_m, sep: string, ch: string) => sep + ch.toUpperCase());
|
||||
}
|
||||
|
||||
function joinNatural(items: string[]): string {
|
||||
return items.length <= 2 ? items.join(" and ") : `${items.slice(0, -1).join(", ")}, and ${items[items.length - 1]}`;
|
||||
}
|
||||
|
||||
function shotFromBeat(beat: Beat, opts: ImportOptions, continued = false, truncated = false): Shot {
|
||||
const speakers = [...new Set(beat.lines.map((l) => l.speaker))];
|
||||
const subjectIds = [
|
||||
...new Set(speakers.map((s) => opts.speakers[s]?.subject).filter((s): s is string => Boolean(s))),
|
||||
];
|
||||
const named = speakers.map((s) => opts.speakers[s]?.subject ?? titleCase(s));
|
||||
|
||||
let description: string;
|
||||
if (continued) {
|
||||
description = `Continuing the same shot on ${joinNatural(named)}, still speaking.`;
|
||||
} else {
|
||||
description =
|
||||
speakers.length > 1
|
||||
? `${joinNatural(named)} in frame, exchanging lines.`
|
||||
: `Close on ${named[0]}, speaking.`;
|
||||
}
|
||||
if (beat.directions.length) description += ` Stage direction: ${beat.directions.join("; ")}.`;
|
||||
|
||||
const shot = makeShot({
|
||||
type: speakers.length > 1 ? "medium" : "close-up",
|
||||
// A static camera: added movement (a dolly on every long shot) showed up as
|
||||
// unwanted panning. Movement can still be set per shot in the shot editor.
|
||||
movement: "static",
|
||||
subjectIds,
|
||||
description,
|
||||
lines: beat.lines.map((l) => newLine({ speaker: opts.speakers[l.speaker]?.tag ?? "", text: l.text })),
|
||||
});
|
||||
// The guide's marker for speech truncated by the video ending: this clip
|
||||
// stops mid-speech and the next one picks it up.
|
||||
if (truncated) shot.body.push(newTextPart("<cutoff>"));
|
||||
return shot;
|
||||
}
|
||||
|
||||
/** One shot per speaking turn; a short reaction ("Yeah.") rides along in the shot it reacts to. */
|
||||
function groupBeats(turns: ScriptTurn[]): Beat[] {
|
||||
const beats: Beat[] = [];
|
||||
for (const turn of turns) {
|
||||
const { text, directions } = convertDirections(turn.text);
|
||||
if (!text) continue;
|
||||
const last = beats[beats.length - 1];
|
||||
if (last && wordCount(text) <= REACTION_MAX_WORDS) {
|
||||
last.lines.push({ speaker: turn.speaker, text });
|
||||
last.directions.push(...directions);
|
||||
} else {
|
||||
beats.push({ lines: [{ speaker: turn.speaker, text }], directions: [...directions] });
|
||||
}
|
||||
}
|
||||
return beats;
|
||||
}
|
||||
|
||||
/**
|
||||
* The guide assigns speaker IDs by who first speaks in *each video*, so a
|
||||
* clip that opens on the second character must call them S1. Renumber a
|
||||
* clip's default S1/S2/... tags in order of first appearance; custom tags
|
||||
* (anything not of the form S<n>) are left alone.
|
||||
*/
|
||||
function renumberSpeakerTags(clip: ImportClip): void {
|
||||
const lines = clip.shots.flatMap((s) => shotLines(s));
|
||||
const seen: string[] = [];
|
||||
for (const l of lines) if (l.speaker && !seen.includes(l.speaker)) seen.push(l.speaker);
|
||||
if (seen.length === 0 || !seen.every((t) => /^S\d+$/.test(t))) return;
|
||||
const renamed = new Map(seen.map((t, i) => [t, `S${i + 1}`]));
|
||||
for (const l of lines) if (l.speaker) l.speaker = renamed.get(l.speaker) ?? l.speaker;
|
||||
}
|
||||
|
||||
/**
|
||||
* Pack shots into clips that fit `targetSeconds`. Clips break only at cuts
|
||||
* (a cut should bring new information), so a turn is never divided just to
|
||||
* balance lengths. The one exception is a single turn longer than a whole
|
||||
* clip: it is split at sentence breaks, and each later piece starts a
|
||||
* `continuation` clip. The first clip opens with an establishing shot.
|
||||
*/
|
||||
export function buildClips(turns: ScriptTurn[], opts: ImportOptions): ImportClip[] {
|
||||
const beats = groupBeats(turns);
|
||||
|
||||
const allSubjects = [
|
||||
...new Set(Object.values(opts.speakers).map((s) => s.subject).filter((s): s is string => Boolean(s))),
|
||||
];
|
||||
const establishing = makeShot({
|
||||
type: "establishing",
|
||||
subjectIds: allSubjects,
|
||||
description: `Establishing view of the setting${allSubjects.length ? ` with ${joinNatural(allSubjects)} in frame` : ""}.`,
|
||||
});
|
||||
|
||||
const clips: ImportClip[] = [];
|
||||
let current: ImportClip = { shots: [establishing], continuation: false };
|
||||
let seconds = ESTABLISHING_SECONDS;
|
||||
const hasDialogue = () => current.shots.some((s) => shotLines(s).length > 0);
|
||||
const close = (continuation: boolean) => {
|
||||
clips.push(current);
|
||||
current = { shots: [], continuation };
|
||||
seconds = 0;
|
||||
};
|
||||
|
||||
for (const beat of beats) {
|
||||
const total = beatSeconds(beat);
|
||||
|
||||
if (total <= opts.targetSeconds) {
|
||||
if (hasDialogue() && seconds + total > opts.targetSeconds) close(false);
|
||||
current.shots.push(shotFromBeat(beat, opts));
|
||||
seconds += total;
|
||||
continue;
|
||||
}
|
||||
|
||||
// Longer than any clip can hold: the split has to fall inside this shot.
|
||||
if (current.shots.length > 0 && opts.targetSeconds - seconds < MIN_PART_SECONDS) close(false);
|
||||
const [main, ...reactions] = beat.lines;
|
||||
const parts = splitToBudgets(main.text, opts.targetSeconds - seconds, opts.targetSeconds);
|
||||
parts.forEach((part, k) => {
|
||||
if (k > 0) close(true);
|
||||
const lines = [{ ...main, text: part }, ...(k === parts.length - 1 ? reactions : [])];
|
||||
const pieceBeat: Beat = { lines, directions: k === 0 ? beat.directions : [] };
|
||||
current.shots.push(shotFromBeat(pieceBeat, opts, k > 0, k < parts.length - 1));
|
||||
seconds += beatSeconds(pieceBeat);
|
||||
});
|
||||
}
|
||||
|
||||
if (current.shots.length > 0 && (clips.length === 0 || hasDialogue())) clips.push(current);
|
||||
for (const clip of clips) renumberSpeakerTags(clip);
|
||||
return clips;
|
||||
}
|
||||
@@ -0,0 +1,336 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
detailedDescriptionFromShots,
|
||||
formatCutTime,
|
||||
makeShot,
|
||||
newLine,
|
||||
parseCutTime,
|
||||
parseDetailedDescription,
|
||||
parseSubjectLabels,
|
||||
resolveShotStarts,
|
||||
retentionAnalysisFromShots,
|
||||
shotLines,
|
||||
shotProse,
|
||||
shotsFromDetailedDescription,
|
||||
type Shot,
|
||||
type ShotInit,
|
||||
} from "@/lib/h3Shots";
|
||||
|
||||
function shot(init: ShotInit): Shot {
|
||||
return makeShot(init);
|
||||
}
|
||||
|
||||
describe("parseSubjectLabels", () => {
|
||||
it("returns ordered, de-duplicated <Subject N> labels", () => {
|
||||
const text = "<Subject 1> is the dog.\n<Subject 2> is the cat. <Subject 1> again.";
|
||||
expect(parseSubjectLabels(text)).toEqual(["<Subject 1>", "<Subject 2>"]);
|
||||
});
|
||||
|
||||
it("returns an empty list when there are none", () => {
|
||||
expect(parseSubjectLabels("no subjects here")).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("detailedDescriptionFromShots", () => {
|
||||
it("serializes shot type, movement, description, and dialogue", () => {
|
||||
const shots = [
|
||||
shot({
|
||||
type: "close-up",
|
||||
movement: "handheld",
|
||||
description: "She looks up.",
|
||||
lines: [newLine({ speaker: "S1", text: "I found it." })],
|
||||
}),
|
||||
];
|
||||
expect(detailedDescriptionFromShots(shots)).toBe(
|
||||
"[Shot 1] Close-up shot, handheld. She looks up. (S1) <d>[English] I found it.</d>",
|
||||
);
|
||||
});
|
||||
|
||||
it("omits the movement clause for a static camera", () => {
|
||||
const shots = [shot({ type: "wide", movement: "static", description: "An empty street." })];
|
||||
expect(detailedDescriptionFromShots(shots)).toBe("[Shot 1] Wide shot. An empty street.");
|
||||
});
|
||||
|
||||
it("numbers shots in order, one per line, and marks later shots with a cut time", () => {
|
||||
const shots = [shot({ description: "First." }), shot({ description: "Second." })];
|
||||
const out = detailedDescriptionFromShots(shots);
|
||||
expect(out).toBe(
|
||||
"[Shot 1] Medium shot. First.\n[Shot 2] At 00:02.000, the shot cuts to a medium shot. Second.",
|
||||
);
|
||||
});
|
||||
|
||||
it("uses the right article and movement in the cut wording", () => {
|
||||
const shots = [
|
||||
shot({ type: "wide" }),
|
||||
shot({ type: "establishing", startSeconds: 3 }),
|
||||
shot({ type: "extreme-close-up", movement: "dolly", startSeconds: 5.25 }),
|
||||
shot({ type: "pov", startSeconds: 7 }),
|
||||
];
|
||||
const out = detailedDescriptionFromShots(shots).split("\n");
|
||||
expect(out[1]).toBe("[Shot 2] At 00:03.000, the shot cuts to an establishing shot.");
|
||||
expect(out[2]).toBe("[Shot 3] At 00:05.250, the shot cuts to an extreme close-up shot, dolly.");
|
||||
expect(out[3]).toBe("[Shot 4] At 00:07.000, the shot cuts to a POV shot.");
|
||||
});
|
||||
|
||||
it("writes the preamble before the first shot", () => {
|
||||
const out = detailedDescriptionFromShots([shot({ description: "A." })], "Keep the film-noir look.");
|
||||
expect(out).toBe("Keep the film-noir look.\n[Shot 1] Medium shot. A.");
|
||||
});
|
||||
});
|
||||
|
||||
describe("cut times", () => {
|
||||
it("formats and parses MM:SS.mmm", () => {
|
||||
expect(formatCutTime(3.5)).toBe("00:03.500");
|
||||
expect(formatCutTime(75.04)).toBe("01:15.040");
|
||||
expect(parseCutTime("00:03.500")).toBe(3.5);
|
||||
expect(parseCutTime("0:03.5")).toBe(3.5);
|
||||
expect(parseCutTime("3.5")).toBe(3.5);
|
||||
expect(parseCutTime("1:15")).toBe(75);
|
||||
expect(parseCutTime("soon")).toBeNull();
|
||||
});
|
||||
|
||||
it("starts the opening shot at 0 and follows estimated shot lengths", () => {
|
||||
const shots = [
|
||||
shot({ lines: [newLine({ text: "A short line." })] }),
|
||||
shot({}),
|
||||
shot({}),
|
||||
];
|
||||
const starts = resolveShotStarts(shots);
|
||||
expect(starts[0]).toBe(0);
|
||||
expect(starts[1]).toBeGreaterThanOrEqual(2);
|
||||
expect(starts[2]).toBeCloseTo(starts[1] + 2, 3);
|
||||
});
|
||||
|
||||
it("honors an explicit time and keeps times strictly increasing", () => {
|
||||
const shots = [shot({}), shot({ startSeconds: 4 }), shot({ startSeconds: 1 })];
|
||||
const starts = resolveShotStarts(shots);
|
||||
expect(starts[1]).toBe(4);
|
||||
expect(starts[2]).toBeGreaterThan(starts[1]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("shotsFromDetailedDescription", () => {
|
||||
it("round-trips type, movement, dialogue, speaker, and subjects", () => {
|
||||
const original = [
|
||||
shot({
|
||||
type: "extreme-close-up",
|
||||
movement: "dolly",
|
||||
description: "Her hand trembles.",
|
||||
lines: [newLine({ speaker: "S1", text: "Wait." })],
|
||||
subjectIds: ["<Subject 1>"],
|
||||
}),
|
||||
];
|
||||
const serialized = detailedDescriptionFromShots(original);
|
||||
const withSubject = serialized.replace("Her hand trembles.", "Her hand trembles, <Subject 1> hesitates.");
|
||||
const parsed = shotsFromDetailedDescription(withSubject, ["<Subject 1>", "<Subject 2>"]);
|
||||
|
||||
expect(parsed).toHaveLength(1);
|
||||
expect(parsed[0].type).toBe("extreme-close-up");
|
||||
expect(parsed[0].movement).toBe("dolly");
|
||||
expect(shotLines(parsed[0])).toHaveLength(1);
|
||||
expect(shotLines(parsed[0])[0].speaker).toBe("S1");
|
||||
expect(shotLines(parsed[0])[0].text).toBe("Wait.");
|
||||
expect(parsed[0].subjectIds).toEqual(["<Subject 1>"]);
|
||||
});
|
||||
|
||||
it("keeps every dialogue line of an exchange, in order", () => {
|
||||
const original = [
|
||||
shot({
|
||||
description: "They talk.",
|
||||
lines: [
|
||||
newLine({ speaker: "S1", text: "Ready?" }),
|
||||
newLine({ speaker: "S2", text: "<pause> Always." }),
|
||||
newLine({ speaker: "S1", text: "Then go." }),
|
||||
],
|
||||
}),
|
||||
];
|
||||
const serialized = detailedDescriptionFromShots(original);
|
||||
expect(serialized).toBe(
|
||||
"[Shot 1] Medium shot. They talk. (S1) <d>[English] Ready?</d> (S2) <d>[English] <pause> Always.</d> (S1) <d>[English] Then go.</d>",
|
||||
);
|
||||
|
||||
const parsed = shotsFromDetailedDescription(serialized, []);
|
||||
expect(shotProse(parsed[0])).toBe("They talk.");
|
||||
expect(shotLines(parsed[0]).map((l) => [l.speaker, l.text])).toEqual([
|
||||
["S1", "Ready?"],
|
||||
["S2", "<pause> Always."],
|
||||
["S1", "Then go."],
|
||||
]);
|
||||
});
|
||||
|
||||
it("round-trips a non-English language marker instead of relabelling it English", () => {
|
||||
const original = [shot({ lines: [newLine({ speaker: "S1", text: "你好。", language: "Chinese" })] })];
|
||||
const serialized = detailedDescriptionFromShots(original);
|
||||
expect(serialized).toContain("<d>[Chinese] 你好。</d>");
|
||||
expect(shotLines(shotsFromDetailedDescription(serialized, [])[0])[0].language).toBe("Chinese");
|
||||
});
|
||||
|
||||
it("round-trips a multi-shot list without pinning the automatic cut times", () => {
|
||||
const original = [
|
||||
shot({ type: "establishing", description: "A rooftop." }),
|
||||
shot({
|
||||
type: "close-up",
|
||||
movement: "handheld",
|
||||
description: "He turns.",
|
||||
lines: [newLine({ speaker: "S2", text: "I had no choice." })],
|
||||
}),
|
||||
shot({ type: "wide", movement: "dolly", description: "They stand apart." }),
|
||||
];
|
||||
const serialized = detailedDescriptionFromShots(original);
|
||||
const parsed = shotsFromDetailedDescription(serialized, []);
|
||||
|
||||
expect(parsed.map((p) => [p.type, p.movement, p.freeLead])).toEqual([
|
||||
["establishing", "static", false],
|
||||
["close-up", "handheld", false],
|
||||
["wide", "dolly", false],
|
||||
]);
|
||||
expect(parsed.every((p) => p.startSeconds === null)).toBe(true);
|
||||
expect(detailedDescriptionFromShots(parsed)).toBe(serialized);
|
||||
});
|
||||
|
||||
it("pins a cut time that differs from the automatic one", () => {
|
||||
const parsed = shotsFromDetailedDescription(
|
||||
"[Shot 1] Wide shot. A dock.\n[Shot 2] At 00:07.400, the shot cuts to a close-up shot. A face.",
|
||||
[],
|
||||
);
|
||||
expect(parsed[1].startSeconds).toBe(7.4);
|
||||
});
|
||||
|
||||
it("keeps hand-written shots verbatim, with the preamble and cut times", () => {
|
||||
const handWritten = [
|
||||
"The target video keeps the live-action style of <Video 1> unchanged.",
|
||||
"[Shot 1] A handheld medium shot holds <Subject 1> at frame left, half-occluded by plastic sheeting.",
|
||||
"[Shot 2] At 00:03.800, the shot cuts to a low tracking shot behind the boots of <Subject 1> as he sprints.",
|
||||
"[Shot 3] At 00:07.400, the camera cuts to a close front-on tracking shot. He (S1) says: <d>[English] Go.</d> <cutoff>",
|
||||
].join("\n");
|
||||
const { preamble, shots } = parseDetailedDescription(handWritten, ["<Subject 1>"]);
|
||||
|
||||
expect(preamble).toBe("The target video keeps the live-action style of <Video 1> unchanged.");
|
||||
expect(shots).toHaveLength(3);
|
||||
expect(shots.every((s) => s.freeLead)).toBe(true);
|
||||
expect(shots.map((s) => s.startSeconds)).toEqual([null, 3.8, 7.4]);
|
||||
expect(shots[0].subjectIds).toEqual(["<Subject 1>"]);
|
||||
// Dialogue in a hand-written shot is pulled out into an editable line...
|
||||
expect(shotLines(shots[2]).map((l) => [l.speaker, l.text])).toEqual([["", "Go."]]);
|
||||
expect(detailedDescriptionFromShots(shots, preamble)).toBe(handWritten);
|
||||
});
|
||||
|
||||
it("keeps a trailing tag after the dialogue it modifies", () => {
|
||||
const text =
|
||||
"[Shot 1] He (S1) says: <d>[English] <breath> I think you're... <i>sorry</i>.</d> <cutoff>";
|
||||
const [shot1] = shotsFromDetailedDescription(text, []);
|
||||
expect(shot1.freeLead).toBe(true);
|
||||
expect(shot1.body.map((p) => p.kind)).toEqual(["text", "line", "text"]);
|
||||
expect(shotLines(shot1)[0].text).toBe("<breath> I think you're... <i>sorry</i>.");
|
||||
expect(detailedDescriptionFromShots([shot1])).toBe(text);
|
||||
});
|
||||
|
||||
it("keeps interleaved dialogue in place, whitespace and all, in hand-written shots", () => {
|
||||
const text = "[Shot 1] She waits.\n (S1) <d>[English] Ready?</d>\nHe nods. (S2) <d>[English] Ready.</d> The door opens.";
|
||||
const [s1] = shotsFromDetailedDescription(text, []);
|
||||
expect(s1.body.map((p) => p.kind)).toEqual(["text", "line", "text", "line", "text"]);
|
||||
expect(detailedDescriptionFromShots([s1])).toBe(text);
|
||||
});
|
||||
|
||||
it("keeps dialogue between prose in shots this editor wrote too", () => {
|
||||
const text = "[Shot 1] Wide shot. He looks up. (S1) <d>[English] Hello.</d> Then he leaves.";
|
||||
const [s1] = shotsFromDetailedDescription(text, []);
|
||||
expect(s1.freeLead).toBe(false);
|
||||
expect(s1.body.map((p) => p.kind)).toEqual(["text", "line", "text"]);
|
||||
expect(detailedDescriptionFromShots([s1])).toBe(text);
|
||||
});
|
||||
|
||||
it("edits a line without moving the text or lines around it", () => {
|
||||
const text = "[Shot 1] Wide shot. A. (S1) <d>[English] One.</d> B. (S2) <d>[English] Two.</d> C.";
|
||||
const [s1] = shotsFromDetailedDescription(text, []);
|
||||
const second = shotLines(s1)[1];
|
||||
second.text = "Changed.";
|
||||
expect(detailedDescriptionFromShots([s1])).toBe(
|
||||
"[Shot 1] Wide shot. A. (S1) <d>[English] One.</d> B. (S2) <d>[English] Changed.</d> C.",
|
||||
);
|
||||
});
|
||||
|
||||
it("puts a newly added line at the end and separates it from the prose", () => {
|
||||
const text = "[Shot 1] Wide shot. He speaks. (S1) <d>[English] First.</d> He pauses.";
|
||||
const [s1] = shotsFromDetailedDescription(text, []);
|
||||
s1.body.push({ kind: "line", line: newLine({ speaker: "S1", text: "Later." }) });
|
||||
expect(detailedDescriptionFromShots([s1])).toBe(
|
||||
"[Shot 1] Wide shot. He speaks. (S1) <d>[English] First.</d> He pauses. (S1) <d>[English] Later.</d>",
|
||||
);
|
||||
});
|
||||
|
||||
it("reordering blocks changes the written order", () => {
|
||||
const [s1] = shotsFromDetailedDescription(
|
||||
"[Shot 1] Wide shot. (S1) <d>[English] One.</d> (S2) <d>[English] Two.</d>",
|
||||
[],
|
||||
);
|
||||
const lineBlocks = s1.body.filter((p) => p.kind === "line");
|
||||
s1.body = [...s1.body.filter((p) => p.kind !== "line"), lineBlocks[1], lineBlocks[0]];
|
||||
expect(detailedDescriptionFromShots([s1])).toBe(
|
||||
"[Shot 1] Wide shot. (S2) <d>[English] Two.</d> (S1) <d>[English] One.</d>",
|
||||
);
|
||||
});
|
||||
|
||||
it("recovers type and movement from the guide's cut wording", () => {
|
||||
const parsed = shotsFromDetailedDescription(
|
||||
"[Shot 1] Wide shot.\n[Shot 2] At 00:04.000, the camera cuts to an over-the-shoulder shot, handheld. Two people talk.",
|
||||
[],
|
||||
);
|
||||
expect(parsed[1].type).toBe("over-the-shoulder");
|
||||
expect(parsed[1].movement).toBe("handheld");
|
||||
expect(shotProse(parsed[1])).toBe("Two people talk.");
|
||||
expect(parsed[1].freeLead).toBe(false);
|
||||
});
|
||||
|
||||
it("falls back to defaults for arbitrary prose, keeping it as the description", () => {
|
||||
const parsed = shotsFromDetailedDescription("A toy car drives into a plush dog.", []);
|
||||
expect(parsed).toHaveLength(1);
|
||||
expect(parsed[0].type).toBe("medium");
|
||||
expect(parsed[0].movement).toBe("static");
|
||||
expect(shotProse(parsed[0])).toBe("A toy car drives into a plush dog.");
|
||||
});
|
||||
|
||||
it("returns an empty list for blank text", () => {
|
||||
expect(shotsFromDetailedDescription("", [])).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("retentionAnalysisFromShots", () => {
|
||||
it("generates an appears-in line per subject with shots", () => {
|
||||
const shots = [
|
||||
shot({ subjectIds: ["<Subject 1>"] }),
|
||||
shot({ subjectIds: ["<Subject 1>", "<Subject 2>"] }),
|
||||
];
|
||||
const out = retentionAnalysisFromShots(shots, ["<Subject 1>", "<Subject 2>"], "");
|
||||
expect(out).toBe(
|
||||
[
|
||||
"<Subject 1> (appears in [Shot 1], [Shot 2]): fully_preserved - what is retained.",
|
||||
"<Subject 2> (appears in [Shot 2]): fully_preserved - what is retained.",
|
||||
].join("\n"),
|
||||
);
|
||||
});
|
||||
|
||||
it("preserves an existing retention note instead of overwriting it", () => {
|
||||
const shots = [shot({ subjectIds: ["<Subject 1>"] })];
|
||||
const previous = "<Subject 1> (appears in [Shot 1]): fully_preserved - fur and collar retained.";
|
||||
const out = retentionAnalysisFromShots(shots, ["<Subject 1>"], previous);
|
||||
expect(out).toBe("<Subject 1> (appears in [Shot 1]): fully_preserved - fur and collar retained.");
|
||||
});
|
||||
|
||||
it("passes through lines it doesn't own, like audio references", () => {
|
||||
const shots = [shot({ subjectIds: ["<Subject 1>"] })];
|
||||
const previous = [
|
||||
"<Subject 1> (appears in [Shot 3]): fully_preserved - old note.",
|
||||
"<Audio 1>: reference - how it guides the audio.",
|
||||
].join("\n");
|
||||
const out = retentionAnalysisFromShots(shots, ["<Subject 1>"], previous);
|
||||
expect(out).toContain("<Subject 1> (appears in [Shot 1]): fully_preserved - old note.");
|
||||
expect(out).toContain("<Audio 1>: reference - how it guides the audio.");
|
||||
});
|
||||
|
||||
it("omits a subject with no shots rather than emitting an empty appears-in", () => {
|
||||
const shots = [shot({ subjectIds: [] })];
|
||||
const out = retentionAnalysisFromShots(shots, ["<Subject 1>"], "");
|
||||
expect(out).toBe("");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,478 @@
|
||||
/**
|
||||
* MiniMax-H3 shot list: a structured camera-direction editor that serializes
|
||||
* into `detailed_description`'s `[Shot N]` blocks and keeps
|
||||
* `retention_analysis`'s "appears in [...]" lines in sync, per the model's
|
||||
* reference prompt guide (see h3Prompt.ts / h3References.ts for the six
|
||||
* guided sections this plugs into).
|
||||
*/
|
||||
|
||||
import { SPEECH_RATES, estimateDuration } from "@/lib/h3DialogueEstimate";
|
||||
|
||||
export type ShotType =
|
||||
| "establishing"
|
||||
| "wide"
|
||||
| "medium"
|
||||
| "close-up"
|
||||
| "extreme-close-up"
|
||||
| "over-the-shoulder"
|
||||
| "pov"
|
||||
| "insert";
|
||||
|
||||
export type CameraMovement =
|
||||
| "static"
|
||||
| "pan"
|
||||
| "tilt"
|
||||
| "dolly"
|
||||
| "zoom"
|
||||
| "handheld"
|
||||
| "tracking"
|
||||
| "crane";
|
||||
|
||||
export interface DialogueLine {
|
||||
/** stable key for React lists -- not part of the serialized prompt */
|
||||
id: string;
|
||||
/** speaker tag without parens, e.g. "S1" -- blank omits the "(S1)" prefix */
|
||||
speaker: string;
|
||||
/** spoken text only (speech tags allowed); wrapped in <d>[language] ...</d> on serialize */
|
||||
text: string;
|
||||
/** language marker written inside <d>; recovered on parse so non-English dialogue round-trips */
|
||||
language: string;
|
||||
}
|
||||
|
||||
/** A run of prose inside a shot (composition, action, environment, delivery notes). */
|
||||
export interface TextPart {
|
||||
kind: "text";
|
||||
/** stable key for React lists -- not part of the serialized prompt */
|
||||
id: string;
|
||||
text: string;
|
||||
}
|
||||
|
||||
/** One spoken line, written as `(S1) <d>[English] ...</d>`. Its position among the parts is where it plays. */
|
||||
export interface LinePart {
|
||||
kind: "line";
|
||||
line: DialogueLine;
|
||||
}
|
||||
|
||||
export type ShotPart = TextPart | LinePart;
|
||||
|
||||
export interface Shot {
|
||||
/** stable key for React lists -- not part of the serialized prompt */
|
||||
id: string;
|
||||
type: ShotType;
|
||||
movement: CameraMovement;
|
||||
/** <Subject N> labels (from subject_definitions) that appear in this shot */
|
||||
subjectIds: string[];
|
||||
/**
|
||||
* The shot's content in the order it reads: prose and spoken lines
|
||||
* interleaved. This one list is the only record of where each line falls,
|
||||
* so a `<d>` block stays mid-sentence if that is where it was written.
|
||||
*/
|
||||
body: ShotPart[];
|
||||
/** cut-in time in seconds; null = computed from the preceding shots. Ignored for the opening shot. */
|
||||
startSeconds: number | null;
|
||||
/**
|
||||
* The description carries its own lead-in (text opened from a hand-written
|
||||
* prompt) so type/movement are not written and the wording is kept verbatim.
|
||||
*/
|
||||
freeLead: boolean;
|
||||
}
|
||||
|
||||
export const SHOT_TYPES: { id: ShotType; label: string }[] = [
|
||||
{ id: "establishing", label: "Establishing" },
|
||||
{ id: "wide", label: "Wide" },
|
||||
{ id: "medium", label: "Medium" },
|
||||
{ id: "close-up", label: "Close-up" },
|
||||
{ id: "extreme-close-up", label: "Extreme close-up" },
|
||||
{ id: "over-the-shoulder", label: "Over-the-shoulder" },
|
||||
{ id: "pov", label: "POV" },
|
||||
{ id: "insert", label: "Insert" },
|
||||
];
|
||||
|
||||
export const CAMERA_MOVEMENTS: { id: CameraMovement; label: string }[] = [
|
||||
{ id: "static", label: "Static" },
|
||||
{ id: "pan", label: "Pan" },
|
||||
{ id: "tilt", label: "Tilt" },
|
||||
{ id: "dolly", label: "Dolly" },
|
||||
{ id: "zoom", label: "Zoom" },
|
||||
{ id: "handheld", label: "Handheld" },
|
||||
{ id: "tracking", label: "Tracking" },
|
||||
{ id: "crane", label: "Crane" },
|
||||
];
|
||||
|
||||
const SHOT_TYPE_LABEL = Object.fromEntries(SHOT_TYPES.map((t) => [t.id, t.label])) as Record<ShotType, string>;
|
||||
const MOVEMENT_LABEL = Object.fromEntries(CAMERA_MOVEMENTS.map((m) => [m.id, m.label])) as Record<
|
||||
CameraMovement,
|
||||
string
|
||||
>;
|
||||
const SHOT_TYPE_BY_LABEL = Object.fromEntries(SHOT_TYPES.map((t) => [t.label.toLowerCase(), t.id])) as Record<
|
||||
string,
|
||||
ShotType
|
||||
>;
|
||||
const MOVEMENT_BY_LABEL = Object.fromEntries(
|
||||
CAMERA_MOVEMENTS.map((m) => [m.label.toLowerCase(), m.id]),
|
||||
) as Record<string, CameraMovement>;
|
||||
|
||||
let shotSeq = 0;
|
||||
export function newShot(): Shot {
|
||||
shotSeq += 1;
|
||||
return {
|
||||
id: `shot-${Date.now()}-${shotSeq}`,
|
||||
type: "medium",
|
||||
movement: "static",
|
||||
subjectIds: [],
|
||||
body: [newTextPart()],
|
||||
startSeconds: null,
|
||||
freeLead: false,
|
||||
};
|
||||
}
|
||||
|
||||
let partSeq = 0;
|
||||
export function newTextPart(text = ""): TextPart {
|
||||
partSeq += 1;
|
||||
return { kind: "text", id: `text-${Date.now()}-${partSeq}`, text };
|
||||
}
|
||||
|
||||
export function linePart(line: DialogueLine): LinePart {
|
||||
return { kind: "line", line };
|
||||
}
|
||||
|
||||
/** Every spoken line of the shot, in playback order. */
|
||||
export function shotLines(shot: Shot): DialogueLine[] {
|
||||
return shot.body.flatMap((p) => (p.kind === "line" ? [p.line] : []));
|
||||
}
|
||||
|
||||
/** The shot's prose with the dialogue left out. */
|
||||
export function shotProse(shot: Shot): string {
|
||||
return shot.body
|
||||
.flatMap((p) => (p.kind === "text" ? [p.text.trim()] : []))
|
||||
.filter(Boolean)
|
||||
.join(" ");
|
||||
}
|
||||
|
||||
/** A body of one prose part followed by the given lines -- the common "description, then dialogue" shape. */
|
||||
export function makeBody(description = "", lines: DialogueLine[] = []): ShotPart[] {
|
||||
return [newTextPart(description), ...lines.map(linePart)];
|
||||
}
|
||||
|
||||
export interface ShotInit extends Partial<Omit<Shot, "id" | "body">> {
|
||||
description?: string;
|
||||
lines?: DialogueLine[];
|
||||
body?: ShotPart[];
|
||||
}
|
||||
|
||||
/** Build a shot from a description and lines (or an explicit body). */
|
||||
export function makeShot(init: ShotInit = {}): Shot {
|
||||
const { description, lines, body, ...rest } = init;
|
||||
return { ...newShot(), ...rest, body: body ?? makeBody(description ?? "", lines ?? []) };
|
||||
}
|
||||
|
||||
let lineSeq = 0;
|
||||
export function newLine(patch: Partial<Omit<DialogueLine, "id">> = {}): DialogueLine {
|
||||
lineSeq += 1;
|
||||
return { id: `line-${Date.now()}-${lineSeq}`, speaker: "", text: "", language: "English", ...patch };
|
||||
}
|
||||
|
||||
/** Copy shots with brand-new ids so components keyed by id (which own their own text state) remount instead of showing stale edits. */
|
||||
export function withFreshIds(shots: Shot[]): Shot[] {
|
||||
return shots.map((s) => ({
|
||||
...s,
|
||||
id: newShot().id,
|
||||
subjectIds: [...s.subjectIds],
|
||||
body: s.body.map((p) =>
|
||||
p.kind === "text" ? { ...p, id: newTextPart().id } : { kind: "line" as const, line: { ...p.line, id: newLine().id } },
|
||||
),
|
||||
}));
|
||||
}
|
||||
|
||||
/** Ordered, de-duplicated <Subject N> labels mentioned in subject_definitions. */
|
||||
export function parseSubjectLabels(subjectDefinitions: string): string[] {
|
||||
const seen: string[] = [];
|
||||
const re = /<Subject\s+\d+>/g;
|
||||
let m: RegExpExecArray | null;
|
||||
while ((m = re.exec(subjectDefinitions))) {
|
||||
if (!seen.includes(m[0])) seen.push(m[0]);
|
||||
}
|
||||
return seen;
|
||||
}
|
||||
|
||||
/* ---------------------------------------------------------------- */
|
||||
/* Cut times */
|
||||
/* ---------------------------------------------------------------- */
|
||||
|
||||
const MIN_SHOT_SECONDS = 2;
|
||||
const LINE_GAP_SECONDS = 0.5;
|
||||
|
||||
/** Ballpark seconds a shot with these spoken lines runs; drives auto cut times. */
|
||||
export function estimateSpokenSeconds(texts: string[]): number {
|
||||
const spoken = texts.filter((t) => t.trim());
|
||||
const speech = spoken.reduce((n, t) => n + estimateDuration(t, SPEECH_RATES.Natural, 1).total, 0);
|
||||
return Math.max(MIN_SHOT_SECONDS, speech + (spoken.length ? LINE_GAP_SECONDS : 0));
|
||||
}
|
||||
|
||||
export function estimateShotSeconds(shot: Shot): number {
|
||||
return estimateSpokenSeconds(shotLines(shot).map((l) => l.text));
|
||||
}
|
||||
|
||||
/**
|
||||
* Cut-in time of every shot: the opening shot is 0, later ones use their
|
||||
* explicit time or follow the previous shot's estimated length. Times are
|
||||
* kept strictly increasing, as the guide requires.
|
||||
*/
|
||||
export function resolveShotStarts(shots: Shot[]): number[] {
|
||||
const starts: number[] = [];
|
||||
shots.forEach((shot, i) => {
|
||||
if (i === 0) {
|
||||
starts.push(0);
|
||||
return;
|
||||
}
|
||||
const prev = starts[i - 1];
|
||||
let start = shot.startSeconds ?? prev + estimateShotSeconds(shots[i - 1]);
|
||||
if (start <= prev) start = prev + 0.1;
|
||||
starts.push(Math.round(start * 1000) / 1000);
|
||||
});
|
||||
return starts;
|
||||
}
|
||||
|
||||
/** MM:SS.mmm, the format the guide uses for cut times ("At 00:03.500, ..."). */
|
||||
export function formatCutTime(seconds: number): string {
|
||||
const totalMs = Math.max(0, Math.round(seconds * 1000));
|
||||
const pad = (n: number, w: number) => String(n).padStart(w, "0");
|
||||
const totalSec = Math.floor(totalMs / 1000);
|
||||
return `${pad(Math.floor(totalSec / 60), 2)}:${pad(totalSec % 60, 2)}.${pad(totalMs % 1000, 3)}`;
|
||||
}
|
||||
|
||||
/** Accepts "3.5", "0:03.5", or "00:03.500"; returns seconds, or null if unreadable. */
|
||||
export function parseCutTime(text: string): number | null {
|
||||
const m = text.trim().match(/^(?:(\d+):)?(\d+(?:\.\d+)?)$/);
|
||||
if (!m) return null;
|
||||
return (m[1] ? Number(m[1]) * 60 : 0) + Number(m[2]);
|
||||
}
|
||||
|
||||
/* ---------------------------------------------------------------- */
|
||||
/* Serialize */
|
||||
/* ---------------------------------------------------------------- */
|
||||
|
||||
/** `raw` keeps a line's text exactly as stored (hand-written shots); otherwise it is trimmed. */
|
||||
function partText(part: ShotPart, raw = false): string {
|
||||
if (part.kind === "text") return part.text;
|
||||
const { speaker, language, text } = part.line;
|
||||
if (!text.trim()) return "";
|
||||
return `${speaker.trim() ? `(${speaker.trim()}) ` : ""}<d>[${language.trim() || "English"}] ${raw ? text : text.trim()}</d>`;
|
||||
}
|
||||
|
||||
/** Concatenate pieces, adding one space only where neither side already has whitespace. */
|
||||
function joinPieces(pieces: string[]): string {
|
||||
let out = "";
|
||||
for (const piece of pieces) {
|
||||
if (!piece) continue;
|
||||
if (out && !/\s$/.test(out) && !/^\s/.test(piece)) out += " ";
|
||||
out += piece;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function serializeShot(shot: Shot, n: number, start: number): string {
|
||||
// [Shot 1] has no timestamp; later shots mark their cut time.
|
||||
const cut = n > 1 ? `At ${formatCutTime(start)}, ` : "";
|
||||
|
||||
if (shot.freeLead) {
|
||||
// Text parts keep their exact whitespace, so hand-written wording round-trips byte for byte.
|
||||
return `[Shot ${n}] ${cut}${joinPieces(shot.body.map((p) => partText(p, true)))}`.trim();
|
||||
}
|
||||
|
||||
const body = shot.body
|
||||
.map((p) => partText(p).trim())
|
||||
.filter(Boolean)
|
||||
.join(" ");
|
||||
const label = SHOT_TYPE_LABEL[shot.type];
|
||||
const movementPart = shot.movement === "static" ? "" : `, ${MOVEMENT_LABEL[shot.movement].toLowerCase()}`;
|
||||
const lead =
|
||||
n > 1
|
||||
? `the shot cuts to ${/^[aeiou]/i.test(label) ? "an" : "a"} ${label === "POV" ? label : label.toLowerCase()} shot${movementPart}.`
|
||||
: `${label} shot${movementPart}.`;
|
||||
return [`[Shot ${n}] ${cut}${lead}`, body].filter(Boolean).join(" ");
|
||||
}
|
||||
|
||||
/**
|
||||
* Serialize the shot list into detailed_description's [Shot N] format:
|
||||
* `[Shot 1] ...` then `[Shot N] At MM:SS.mmm, the shot cuts to ...`. Optional
|
||||
* `preamble` (style notes that come before the first shot) is written first.
|
||||
*/
|
||||
export function detailedDescriptionFromShots(shots: Shot[], preamble = ""): string {
|
||||
const starts = resolveShotStarts(shots);
|
||||
return [preamble.trim(), ...shots.map((s, i) => serializeShot(s, i + 1, starts[i]))]
|
||||
.filter(Boolean)
|
||||
.join("\n");
|
||||
}
|
||||
|
||||
/* ---------------------------------------------------------------- */
|
||||
/* Parse */
|
||||
/* ---------------------------------------------------------------- */
|
||||
|
||||
const CUT_TIME_RE = /^At (\d+):(\d{2})(?:\.(\d{1,3}))?,\s*/i;
|
||||
// "[the shot cuts to] [a] <type> shot[, <movement>]." -- the lead-in this
|
||||
// editor writes, recovered on parse so type/movement survive a reopen.
|
||||
const LEAD_RE =
|
||||
/^(?:(?:the (?:shot|camera) (?:cuts|transitions|changes|switches) to |cuts to )?(?:an? )?)([a-zA-Z -]+?) shot(?:,\s*([a-zA-Z]+))?\.\s*/i;
|
||||
const DIALOGUE_RE = /(?:\(([^)]+)\)\s*)?<d>(?:\[([^\]]*)\]\s*)?([\s\S]*?)<\/d>/g;
|
||||
|
||||
/**
|
||||
* Split shot text at its `<d>` blocks into an ordered body. `exact` keeps the
|
||||
* surrounding text untouched (whitespace included) so it can be rebuilt
|
||||
* byte for byte; otherwise prose is trimmed and empty runs dropped.
|
||||
*/
|
||||
function splitBody(text: string, exact: boolean): ShotPart[] {
|
||||
const parts: ShotPart[] = [];
|
||||
const pushText = (raw: string) => {
|
||||
const t = exact ? raw : raw.trim();
|
||||
if (t) parts.push(newTextPart(t));
|
||||
};
|
||||
let last = 0;
|
||||
for (const m of text.matchAll(DIALOGUE_RE)) {
|
||||
const start = m.index ?? 0;
|
||||
pushText(text.slice(last, start));
|
||||
parts.push(
|
||||
linePart(
|
||||
newLine({
|
||||
speaker: m[1]?.trim() ?? "",
|
||||
language: m[2]?.trim() || "English",
|
||||
text: exact ? (m[3] ?? "") : (m[3] ?? "").trim(),
|
||||
}),
|
||||
),
|
||||
);
|
||||
last = start + m[0].length;
|
||||
}
|
||||
pushText(text.slice(last));
|
||||
return parts;
|
||||
}
|
||||
|
||||
function parseShotBlock(block: string, subjectLabels: string[]): { shot: Shot; time: number | null } {
|
||||
const shot = newShot();
|
||||
shot.subjectIds = subjectLabels.filter((label) => block.includes(label));
|
||||
let rest = block;
|
||||
|
||||
let time: number | null = null;
|
||||
const t = rest.match(CUT_TIME_RE);
|
||||
if (t) {
|
||||
time = Number(t[1]) * 60 + Number(t[2]) + (t[3] ? Number(`0.${t[3]}`) : 0);
|
||||
rest = rest.slice(t[0].length);
|
||||
}
|
||||
|
||||
const lead = rest.match(LEAD_RE);
|
||||
const typeId = lead ? SHOT_TYPE_BY_LABEL[lead[1].trim().toLowerCase()] : undefined;
|
||||
if (lead && typeId) {
|
||||
shot.type = typeId;
|
||||
const moveLabel = lead[2]?.trim().toLowerCase();
|
||||
if (moveLabel && MOVEMENT_BY_LABEL[moveLabel]) shot.movement = MOVEMENT_BY_LABEL[moveLabel];
|
||||
rest = rest.slice(lead[0].length);
|
||||
shot.body = splitBody(rest, false);
|
||||
} else {
|
||||
// Hand-written wording: keep it exactly as written. Dialogue is still
|
||||
// pulled out into editable lines, but each stays where it was written.
|
||||
shot.freeLead = true;
|
||||
shot.body = splitBody(rest, true);
|
||||
}
|
||||
if (shot.body.length === 0) shot.body = [newTextPart()];
|
||||
return { shot, time };
|
||||
}
|
||||
|
||||
/**
|
||||
* Split an existing detailed_description back into shots so opening the
|
||||
* builder on a job someone already wrote doesn't discard it. Shots are found
|
||||
* by their line-start [Shot N] markers; anything before the first is kept as
|
||||
* the preamble. Only wording this editor generated is decomposed into
|
||||
* type/movement/dialogue -- anything else is kept verbatim.
|
||||
*/
|
||||
export function parseDetailedDescription(
|
||||
text: string,
|
||||
subjectLabels: string[],
|
||||
): { preamble: string; shots: Shot[] } {
|
||||
const markers = [...text.matchAll(/^\[Shot\s+\d+\][ \t]*/gm)];
|
||||
if (markers.length === 0) {
|
||||
const body = text.trim();
|
||||
if (!body) return { preamble: "", shots: [] };
|
||||
const { shot } = parseShotBlock(body, subjectLabels);
|
||||
shot.freeLead = true;
|
||||
shot.body = splitBody(body, true);
|
||||
return { preamble: "", shots: [shot] };
|
||||
}
|
||||
|
||||
const preamble = text.slice(0, markers[0].index).trim();
|
||||
const parsed = markers.map((m, i) => {
|
||||
const from = (m.index ?? 0) + m[0].length;
|
||||
const to = i + 1 < markers.length ? (markers[i + 1].index ?? text.length) : text.length;
|
||||
return parseShotBlock(text.slice(from, to).trim(), subjectLabels);
|
||||
});
|
||||
const shots = parsed.map((p) => p.shot);
|
||||
|
||||
// Only pin a cut time when it differs from what the editor would compute,
|
||||
// so text this editor wrote keeps flowing as earlier shots change.
|
||||
const starts: number[] = [0];
|
||||
shots.forEach((shot, i) => {
|
||||
if (i === 0) return;
|
||||
const auto = starts[i - 1] + estimateShotSeconds(shots[i - 1]);
|
||||
const time = parsed[i].time;
|
||||
if (time == null) {
|
||||
starts.push(auto);
|
||||
return;
|
||||
}
|
||||
starts.push(time);
|
||||
if (Math.abs(time - auto) > 0.001) shot.startSeconds = time;
|
||||
});
|
||||
return { preamble, shots };
|
||||
}
|
||||
|
||||
export function shotsFromDetailedDescription(text: string, subjectLabels: string[]): Shot[] {
|
||||
return parseDetailedDescription(text, subjectLabels).shots;
|
||||
}
|
||||
|
||||
const RETENTION_LINE_RE = /^(<[^>]+>)\s*\(appears in [^)]*\):\s*(.*)$/gm;
|
||||
const DEFAULT_RETENTION_NOTE = "fully_preserved - what is retained.";
|
||||
|
||||
/** label -> free-text retention note (everything after the colon), for lines this parser recognizes. */
|
||||
function parseRetentionNotes(retentionAnalysis: string): Map<string, string> {
|
||||
const notes = new Map<string, string>();
|
||||
let m: RegExpExecArray | null;
|
||||
RETENTION_LINE_RE.lastIndex = 0;
|
||||
while ((m = RETENTION_LINE_RE.exec(retentionAnalysis))) notes.set(m[1], m[2].trim());
|
||||
return notes;
|
||||
}
|
||||
|
||||
/**
|
||||
* Rebuild retention_analysis's shot-tracked lines from the shot list. Each
|
||||
* subject's existing retention note (the free text after the colon) is kept
|
||||
* untouched -- only the "(appears in [...])" shot list is regenerated, using
|
||||
* a sensible default note for a subject that's newly shot-tracked. Anything
|
||||
* this shot list doesn't own (untracked subjects, audio-reference lines,
|
||||
* free text) passes through unchanged rather than being discarded.
|
||||
*/
|
||||
export function retentionAnalysisFromShots(
|
||||
shots: Shot[],
|
||||
subjectLabels: string[],
|
||||
previousRetentionAnalysis: string,
|
||||
): string {
|
||||
const existingNotes = parseRetentionNotes(previousRetentionAnalysis);
|
||||
const regenerated = new Set<string>();
|
||||
const shotLines: string[] = [];
|
||||
|
||||
for (const label of subjectLabels) {
|
||||
const shotNums = shots
|
||||
.map((s, i) => (s.subjectIds.includes(label) ? i + 1 : null))
|
||||
.filter((n): n is number => n !== null);
|
||||
if (shotNums.length === 0) continue;
|
||||
regenerated.add(label);
|
||||
const note = existingNotes.get(label) ?? DEFAULT_RETENTION_NOTE;
|
||||
shotLines.push(`${label} (appears in ${shotNums.map((n) => `[Shot ${n}]`).join(", ")}): ${note}`);
|
||||
}
|
||||
|
||||
const passthroughLines = previousRetentionAnalysis
|
||||
.split("\n")
|
||||
.map((l) => l.trim())
|
||||
.filter((line) => {
|
||||
if (!line) return false;
|
||||
const m = line.match(/^(<[^>]+>)\s*\(appears in/);
|
||||
return !(m && regenerated.has(m[1]));
|
||||
});
|
||||
|
||||
return [...shotLines, ...passthroughLines].join("\n");
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
applyH3Preset,
|
||||
applyH3Tag,
|
||||
H3_PRESETS,
|
||||
H3_TAG_CATEGORIES,
|
||||
H3_TAGS,
|
||||
type H3Preset,
|
||||
type H3Tag,
|
||||
} from "@/lib/h3Tags";
|
||||
|
||||
const pointTag = H3_TAGS.find((t) => t.id === "pause") as H3Tag;
|
||||
const wrapTag = H3_TAGS.find((t) => t.id === "emphasis") as H3Tag;
|
||||
const emptyWrapTag = H3_TAGS.find((t) => t.id === "smacks-lips") as H3Tag;
|
||||
|
||||
describe("H3 tag catalogue", () => {
|
||||
it("gives every tag a unique id", () => {
|
||||
const ids = H3_TAGS.map((t) => t.id);
|
||||
expect(new Set(ids).size).toBe(ids.length);
|
||||
});
|
||||
|
||||
it("gives wrap tags a close tag and point tags none", () => {
|
||||
for (const tag of H3_TAGS) {
|
||||
if (tag.kind === "wrap") expect(tag.close).toBeTruthy();
|
||||
else expect(tag.close).toBeUndefined();
|
||||
}
|
||||
});
|
||||
|
||||
it("flattens to the same tags the categories declare", () => {
|
||||
const flattened = H3_TAG_CATEGORIES.flatMap((c) => c.tags);
|
||||
expect(H3_TAGS).toEqual(flattened);
|
||||
});
|
||||
});
|
||||
|
||||
describe("applyH3Tag: point tags", () => {
|
||||
it("inserts at the cursor with padding on both sides", () => {
|
||||
const result = applyH3Tag("Okay, so. This is me.", 9, 9, pointTag);
|
||||
expect(result.value).toBe("Okay, so. <pause> This is me.");
|
||||
// cursor collapses to just past the inserted token
|
||||
expect(result.value.slice(result.start, result.end)).toBe("");
|
||||
expect(result.value.slice(0, result.start)).toBe("Okay, so. <pause> ".slice(0, result.start));
|
||||
});
|
||||
|
||||
it("does not double up whitespace already present", () => {
|
||||
const result = applyH3Tag("Okay, so. This is me.", 10, 10, pointTag);
|
||||
expect(result.value).toBe("Okay, so. <pause> This is me.");
|
||||
});
|
||||
|
||||
it("inserts cleanly at the very start and end of the text", () => {
|
||||
expect(applyH3Tag("Hello.", 0, 0, pointTag).value).toBe("<pause> Hello.");
|
||||
expect(applyH3Tag("Hello.", 6, 6, pointTag).value).toBe("Hello. <pause>");
|
||||
});
|
||||
|
||||
it("inserts before a selection, keeping the selected text", () => {
|
||||
const result = applyH3Tag("Hello there.", 6, 11, pointTag);
|
||||
expect(result.value).toBe("Hello <pause> there.");
|
||||
expect(result.value.slice(result.start, result.end)).toBe("there");
|
||||
});
|
||||
});
|
||||
|
||||
describe("applyH3Tag: wrap tags", () => {
|
||||
it("wraps a selection and collapses the cursor after the close tag", () => {
|
||||
const result = applyH3Tag("I was not expecting that.", 6, 9, wrapTag);
|
||||
expect(result.value).toBe("I was <i>not</i> expecting that.");
|
||||
expect(result.start).toBe(result.end);
|
||||
expect(result.value.slice(0, result.start)).toBe("I was <i>not</i>");
|
||||
});
|
||||
|
||||
it("inserts an empty open+close pair with the cursor between them when nothing is selected", () => {
|
||||
const result = applyH3Tag("Don't tell anyone.", 0, 0, wrapTag);
|
||||
expect(result.value).toBe("<i></i>Don't tell anyone.");
|
||||
expect(result.start).toBe("<i>".length);
|
||||
expect(result.end).toBe(result.start);
|
||||
});
|
||||
|
||||
it("supports an intentionally empty pair, as used for a sentence-final effect", () => {
|
||||
const result = applyH3Tag("Okay.", 0, 0, emptyWrapTag);
|
||||
expect(result.value).toBe("<smacks lips></smacks lips>Okay.");
|
||||
});
|
||||
});
|
||||
|
||||
describe("applyH3Tag: out-of-range selection", () => {
|
||||
it("clamps a selection beyond the text length", () => {
|
||||
const result = applyH3Tag("Hi.", 10, 20, pointTag);
|
||||
expect(result.value).toBe("Hi. <pause>");
|
||||
});
|
||||
});
|
||||
|
||||
describe("applyH3Preset", () => {
|
||||
const insertPreset = H3_PRESETS.find((p) => p.label === "Nervous") as H3Preset;
|
||||
const wrapPreset = H3_PRESETS.find((p) => p.label === "Intimate") as H3Preset;
|
||||
|
||||
it("inserts with a leading space when the cursor follows text", () => {
|
||||
const r = applyH3Preset("Well", 4, 4, insertPreset);
|
||||
expect(r.value).toBe("Well <uh> <stutter> ");
|
||||
expect(r.start).toBe(r.value.length);
|
||||
expect(r.end).toBe(r.value.length);
|
||||
});
|
||||
|
||||
it("adds no leading space at the start or after whitespace", () => {
|
||||
expect(applyH3Preset("", 0, 0, insertPreset).value).toBe("<uh> <stutter> ");
|
||||
expect(applyH3Preset("Well ", 5, 5, insertPreset).value).toBe("Well <uh> <stutter> ");
|
||||
});
|
||||
|
||||
it("wraps a selection and keeps it selected", () => {
|
||||
const r = applyH3Preset("say it softly now", 4, 13, wrapPreset);
|
||||
expect(r.value).toBe("say <whisper>it softly</whisper> now");
|
||||
expect(r.value.slice(r.start, r.end)).toBe("it softly");
|
||||
});
|
||||
|
||||
it("wraps an empty selection with the caret between the tags", () => {
|
||||
const r = applyH3Preset("hi ", 3, 3, wrapPreset);
|
||||
expect(r.value).toBe("hi <whisper></whisper>");
|
||||
expect(r.start).toBe(r.end);
|
||||
expect(r.value.slice(0, r.start)).toBe("hi <whisper>");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,409 @@
|
||||
/**
|
||||
* MiniMax-H3 speech tags: the inline markers the model recognizes inside
|
||||
* `<d>[English] dialogue</d>` blocks (and elsewhere in a prompt) to shape
|
||||
* delivery -- pauses, breath, emphasis, and non-verbal sounds. `applyH3Tag`
|
||||
* is the pure insertion logic; the tag/category tables drive the console UI.
|
||||
*/
|
||||
|
||||
export type H3TagKind = "point" | "wrap";
|
||||
|
||||
export interface H3Tag {
|
||||
/** stable key for React lists and lookups */
|
||||
id: string;
|
||||
/** short button label, usually the bare tag name */
|
||||
label: string;
|
||||
description: string;
|
||||
/** short usage example shown as a tooltip */
|
||||
example: string;
|
||||
kind: H3TagKind;
|
||||
/** point: the literal token inserted, e.g. "<pause>". wrap: the opening tag. */
|
||||
open: string;
|
||||
/** wrap only: the closing tag, e.g. "</i>". */
|
||||
close?: string;
|
||||
/**
|
||||
* Rough seconds this tag adds to spoken duration, used only for the
|
||||
* dialogue editor's length estimate -- ballpark figures, not measured.
|
||||
* Delivery-only tags (emphasis, whisper, softer) add ~0.
|
||||
*/
|
||||
seconds?: number;
|
||||
}
|
||||
|
||||
export interface H3TagCategory {
|
||||
id: string;
|
||||
label: string;
|
||||
tags: H3Tag[];
|
||||
}
|
||||
|
||||
export const H3_TAG_CATEGORIES: H3TagCategory[] = [
|
||||
{
|
||||
id: "pace",
|
||||
label: "Pace & breath",
|
||||
tags: [
|
||||
{
|
||||
id: "pause",
|
||||
label: "<pause>",
|
||||
description: "Short pause",
|
||||
example: "Okay, so. <pause> This is just me talking.",
|
||||
kind: "point",
|
||||
open: "<pause>",
|
||||
seconds: 0.45,
|
||||
},
|
||||
{
|
||||
id: "long-pause",
|
||||
label: "<long pause>",
|
||||
description: "Longer pause",
|
||||
example: "I mean... <long pause> I don't even know.",
|
||||
kind: "point",
|
||||
open: "<long pause>",
|
||||
seconds: 1.2,
|
||||
},
|
||||
{
|
||||
id: "breath",
|
||||
label: "<breath>",
|
||||
description: "Breathing sound",
|
||||
example: "And then... <breath> it just happened.",
|
||||
kind: "point",
|
||||
open: "<breath>",
|
||||
seconds: 0.55,
|
||||
},
|
||||
{
|
||||
id: "inhale",
|
||||
label: "<inhale>",
|
||||
description: "In-breath",
|
||||
example: "<inhale> Alright, let's do this.",
|
||||
kind: "point",
|
||||
open: "<inhale>",
|
||||
seconds: 0.5,
|
||||
},
|
||||
{
|
||||
id: "exhale",
|
||||
label: "<exhale>",
|
||||
description: "Out-breath",
|
||||
example: "<exhale> Okay then.",
|
||||
kind: "point",
|
||||
open: "<exhale>",
|
||||
seconds: 0.6,
|
||||
},
|
||||
{
|
||||
id: "catches-breath",
|
||||
label: "<catches breath>",
|
||||
description: "Out of breath",
|
||||
example: "Wait... <catches breath> hold on a sec.",
|
||||
kind: "point",
|
||||
open: "<catches breath>",
|
||||
seconds: 0.8,
|
||||
},
|
||||
{
|
||||
id: "deep-breath",
|
||||
label: "<deep breath>",
|
||||
description: "Calming down",
|
||||
example: "<deep breath> Okay. I can do this.",
|
||||
kind: "point",
|
||||
open: "<deep breath>",
|
||||
seconds: 1.2,
|
||||
},
|
||||
{
|
||||
id: "pant",
|
||||
label: "<pant>",
|
||||
description: "Panting (single)",
|
||||
example: "Run... <pant> run now!",
|
||||
kind: "point",
|
||||
open: "<pant>",
|
||||
seconds: 0.7,
|
||||
},
|
||||
{
|
||||
id: "pants",
|
||||
label: "<pants>",
|
||||
description: "Panting (continued)",
|
||||
example: "Run... <pants> run now!",
|
||||
kind: "point",
|
||||
open: "<pants>",
|
||||
seconds: 0.7,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "delivery",
|
||||
label: "Emphasis & delivery",
|
||||
tags: [
|
||||
{
|
||||
id: "emphasis",
|
||||
label: "<i>…</i>",
|
||||
description: "Emphasize the wrapped words (1-4 words)",
|
||||
example: "I was <i>not</i> expecting that.",
|
||||
kind: "wrap",
|
||||
open: "<i>",
|
||||
close: "</i>",
|
||||
seconds: 0,
|
||||
},
|
||||
{
|
||||
id: "whisper",
|
||||
label: "<whisper>…</whisper>",
|
||||
description: "Whisper delivery",
|
||||
example: "<whisper>Don't tell anyone this.</whisper>",
|
||||
kind: "wrap",
|
||||
open: "<whisper>",
|
||||
close: "</whisper>",
|
||||
seconds: 0,
|
||||
},
|
||||
{
|
||||
id: "softer",
|
||||
label: "<softer>",
|
||||
description: "Quieter delivery",
|
||||
example: "<softer> I don't think I can say it.",
|
||||
kind: "point",
|
||||
open: "<softer>",
|
||||
seconds: 0,
|
||||
},
|
||||
{
|
||||
id: "humming",
|
||||
label: "<humming>…</humming>",
|
||||
description: "Humming a tune",
|
||||
example: "<humming>da-da-da-beautiful-day.</humming>",
|
||||
kind: "wrap",
|
||||
open: "<humming>",
|
||||
close: "</humming>",
|
||||
seconds: 2,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "vocal",
|
||||
label: "Vocalizations",
|
||||
tags: [
|
||||
{
|
||||
id: "laughs",
|
||||
label: "<laughs>",
|
||||
description: "Laughing",
|
||||
example: "That's... <laughs> that's actually funny.",
|
||||
kind: "point",
|
||||
open: "<laughs>",
|
||||
seconds: 1.1,
|
||||
},
|
||||
{
|
||||
id: "chuckle",
|
||||
label: "<chuckle>",
|
||||
description: "Laughing (light)",
|
||||
example: "That's... <chuckle> that's actually funny.",
|
||||
kind: "point",
|
||||
open: "<chuckle>",
|
||||
seconds: 0.7,
|
||||
},
|
||||
{
|
||||
id: "sighs",
|
||||
label: "<sighs>",
|
||||
description: "Sigh",
|
||||
example: "<sighs> I really tried.",
|
||||
kind: "point",
|
||||
open: "<sighs>",
|
||||
seconds: 0.85,
|
||||
},
|
||||
{
|
||||
id: "gasp",
|
||||
label: "<gasp>",
|
||||
description: "Sharp intake",
|
||||
example: "<gasp> Oh my God.",
|
||||
kind: "point",
|
||||
open: "<gasp>",
|
||||
seconds: 0.5,
|
||||
},
|
||||
{
|
||||
id: "phew",
|
||||
label: "<phew>",
|
||||
description: "Relief",
|
||||
example: "<phew> That was close.",
|
||||
kind: "point",
|
||||
open: "<phew>",
|
||||
seconds: 0.6,
|
||||
},
|
||||
{
|
||||
id: "mhm",
|
||||
label: "<mhm>",
|
||||
description: "Agreement sound",
|
||||
example: "Yeah, <mhm> exactly.",
|
||||
kind: "point",
|
||||
open: "<mhm>",
|
||||
seconds: 0.4,
|
||||
},
|
||||
{
|
||||
id: "uh",
|
||||
label: "<uh>",
|
||||
description: "Filler / hesitation",
|
||||
example: "So, like... <uh> what was I saying?",
|
||||
kind: "point",
|
||||
open: "<uh>",
|
||||
seconds: 0.35,
|
||||
},
|
||||
],
|
||||
},
|
||||
{
|
||||
id: "body",
|
||||
label: "Disfluency & body",
|
||||
tags: [
|
||||
{
|
||||
id: "stutter",
|
||||
label: "<stutter>",
|
||||
description: "Stutters the words that follow",
|
||||
example: "<stutter> I ca can't believe that.",
|
||||
kind: "point",
|
||||
open: "<stutter>",
|
||||
seconds: 0.45,
|
||||
},
|
||||
{
|
||||
id: "coughs",
|
||||
label: "<coughs>",
|
||||
description: "Cough",
|
||||
example: "<coughs> Sorry, one sec.",
|
||||
kind: "point",
|
||||
open: "<coughs>",
|
||||
seconds: 0.6,
|
||||
},
|
||||
{
|
||||
id: "clears-throat",
|
||||
label: "<clears throat>",
|
||||
description: "Throat clear",
|
||||
example: "<clears throat> So anyway...",
|
||||
kind: "point",
|
||||
open: "<clears throat>",
|
||||
seconds: 0.8,
|
||||
},
|
||||
{
|
||||
id: "sniff",
|
||||
label: "<sniff>",
|
||||
description: "Sniffing",
|
||||
example: "<sniff> It's just... really sad.",
|
||||
kind: "point",
|
||||
open: "<sniff>",
|
||||
seconds: 0.45,
|
||||
},
|
||||
{
|
||||
id: "smacks-lips",
|
||||
label: "<smacks lips>",
|
||||
description:
|
||||
"Lip smack; leave empty to trigger at the end of the sentence, or wrap words to place it there",
|
||||
example: "<smacks lips></smacks lips> Okay.",
|
||||
kind: "wrap",
|
||||
open: "<smacks lips>",
|
||||
close: "</smacks lips>",
|
||||
seconds: 0.3,
|
||||
},
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
export const H3_TAGS: H3Tag[] = H3_TAG_CATEGORIES.flatMap((c) => c.tags);
|
||||
|
||||
export interface H3TagApplyResult {
|
||||
value: string;
|
||||
/** selection to restore focus to after the insertion (collapsed unless text was wrapped). */
|
||||
start: number;
|
||||
end: number;
|
||||
}
|
||||
|
||||
function needsLeadingSpace(before: string): boolean {
|
||||
return before.length > 0 && !/\s$/.test(before);
|
||||
}
|
||||
|
||||
function needsTrailingSpace(after: string): boolean {
|
||||
return after.length > 0 && !/^\s/.test(after);
|
||||
}
|
||||
|
||||
/**
|
||||
* Insert `tag` into `value` at the given selection.
|
||||
*
|
||||
* Point tags are inserted at the selection start, padded with a space on
|
||||
* either side where the surrounding text doesn't already have one, and the
|
||||
* cursor collapses to just past the inserted token.
|
||||
*
|
||||
* Wrap tags wrap the current selection (cursor lands just past the closing
|
||||
* tag), or insert an empty `open+close` pair with the cursor left between
|
||||
* them when nothing is selected, ready to type into.
|
||||
*/
|
||||
export function applyH3Tag(
|
||||
value: string,
|
||||
selectionStart: number,
|
||||
selectionEnd: number,
|
||||
tag: H3Tag,
|
||||
): H3TagApplyResult {
|
||||
const start = Math.max(0, Math.min(selectionStart, value.length));
|
||||
const end = Math.max(start, Math.min(selectionEnd, value.length));
|
||||
const before = value.slice(0, start);
|
||||
const selected = value.slice(start, end);
|
||||
const after = value.slice(end);
|
||||
|
||||
if (tag.kind === "point") {
|
||||
const lead = needsLeadingSpace(before) ? " " : "";
|
||||
const trailingContext = selected.length > 0 ? selected : after;
|
||||
const trail = needsTrailingSpace(trailingContext) ? " " : "";
|
||||
const token = `${lead}${tag.open}${trail}`;
|
||||
const cursor = before.length + token.length;
|
||||
return {
|
||||
value: before + token + selected + after,
|
||||
start: cursor,
|
||||
end: cursor + selected.length,
|
||||
};
|
||||
}
|
||||
|
||||
const close = tag.close ?? "";
|
||||
if (selected.length > 0) {
|
||||
const cursor = before.length + tag.open.length + selected.length + close.length;
|
||||
return {
|
||||
value: before + tag.open + selected + close + after,
|
||||
start: cursor,
|
||||
end: cursor,
|
||||
};
|
||||
}
|
||||
const cursor = before.length + tag.open.length;
|
||||
return {
|
||||
value: before + tag.open + close + after,
|
||||
start: cursor,
|
||||
end: cursor,
|
||||
};
|
||||
}
|
||||
|
||||
/** One-click bundles of tags for common delivery beats (dialogue editor + shot list). */
|
||||
export interface H3Preset {
|
||||
label: string;
|
||||
/** literal text inserted at the cursor (a leading space is added when needed). */
|
||||
insert?: string;
|
||||
/** wrap the selection between these two tokens instead of inserting. */
|
||||
wrap?: [string, string];
|
||||
}
|
||||
|
||||
export const H3_PRESETS: H3Preset[] = [
|
||||
{ label: "Cracking", insert: "<catches breath> <stutter> " },
|
||||
{ label: "Nervous", insert: "<uh> <stutter> " },
|
||||
{ label: "Relief", insert: "<phew> <exhale> " },
|
||||
{ label: "Cold turn", insert: "<long pause> <softer> " },
|
||||
{ label: "Intimate", wrap: ["<whisper>", "</whisper>"] },
|
||||
{ label: "Amused", insert: "<chuckle> " },
|
||||
];
|
||||
|
||||
/** Pure counterpart of `applyH3Tag` for presets: same {value, selection} contract. */
|
||||
export function applyH3Preset(
|
||||
value: string,
|
||||
selectionStart: number,
|
||||
selectionEnd: number,
|
||||
preset: H3Preset,
|
||||
): H3TagApplyResult {
|
||||
const start = Math.max(0, Math.min(selectionStart, value.length));
|
||||
const end = Math.max(start, Math.min(selectionEnd, value.length));
|
||||
const before = value.slice(0, start);
|
||||
const mid = value.slice(start, end);
|
||||
const after = value.slice(end);
|
||||
|
||||
if (preset.wrap) {
|
||||
const [open, close] = preset.wrap;
|
||||
const innerStart = before.length + open.length;
|
||||
return {
|
||||
value: before + open + mid + close + after,
|
||||
start: innerStart,
|
||||
end: innerStart + mid.length,
|
||||
};
|
||||
}
|
||||
|
||||
const lead = before.length && !/\s$/.test(before) ? " " : "";
|
||||
const chunk = lead + (preset.insert ?? "");
|
||||
const caret = before.length + chunk.length;
|
||||
return { value: before + chunk + after, start: caret, end: caret };
|
||||
}
|
||||
@@ -0,0 +1,78 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { parseJobJson } from "@/lib/jobJson";
|
||||
|
||||
const ok = (text: string) => {
|
||||
const r = parseJobJson(text);
|
||||
if (!r.ok) throw new Error(r.error);
|
||||
return r;
|
||||
};
|
||||
|
||||
describe("parseJobJson", () => {
|
||||
it("joins a prompt given as lines and carries the settings over", () => {
|
||||
const r = ok(
|
||||
JSON.stringify({
|
||||
model_id: "MiniMaxAI/MiniMax-H3",
|
||||
name: "demo",
|
||||
workload_type: "i2v",
|
||||
prompt: ["summary:", "Hello.", "", "detailed_description:", "[Shot 1] A."],
|
||||
references: [{ source: "/data/a.png", media_type: "image" }],
|
||||
guidance_scale: 1,
|
||||
num_frames: 243,
|
||||
fps: 24,
|
||||
num_gpus: 4,
|
||||
use_fsdp_inference: true,
|
||||
}),
|
||||
);
|
||||
expect(r.workloadType).toBe("i2v");
|
||||
expect(r.job.prompt).toBe("summary:\nHello.\n\ndetailed_description:\n[Shot 1] A.");
|
||||
expect(r.job).toMatchObject({
|
||||
model_id: "MiniMaxAI/MiniMax-H3",
|
||||
name: "demo",
|
||||
guidance_scale: 1,
|
||||
num_frames: 243,
|
||||
num_gpus: 4,
|
||||
use_fsdp_inference: true,
|
||||
references: [{ source: "/data/a.png", media_type: "image" }],
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts a plain string prompt", () => {
|
||||
expect(ok(JSON.stringify({ model_id: "m", prompt: "one line" })).job.prompt).toBe("one line");
|
||||
});
|
||||
|
||||
it("infers the workload when it isn't declared", () => {
|
||||
expect(ok(JSON.stringify({ model_id: "m", prompt: "p" })).workloadType).toBe("t2v");
|
||||
expect(
|
||||
ok(JSON.stringify({ model_id: "m", prompt: "p", references: [{ source: "/a/b.mp4" }] })).workloadType,
|
||||
).toBe("i2v");
|
||||
expect(ok(JSON.stringify({ model_id: "m", prompt: "p", image_path: "/a.png" })).workloadType).toBe("i2v");
|
||||
});
|
||||
|
||||
it("infers a reference's media type from its extension", () => {
|
||||
const r = ok(
|
||||
JSON.stringify({
|
||||
model_id: "m",
|
||||
prompt: "p",
|
||||
references: [{ source: "/a/b.mp4" }, { source: "/a/c.wav" }, { source: "/a/d.png" }],
|
||||
}),
|
||||
);
|
||||
expect(r.job.references?.map((x) => x.media_type)).toEqual(["video", "audio", "image"]);
|
||||
});
|
||||
|
||||
it("ignores settings of the wrong type instead of passing them through", () => {
|
||||
const r = ok(JSON.stringify({ model_id: "m", prompt: "p", num_frames: "many", use_fsdp_inference: "yes", extra: 1 }));
|
||||
expect(r.job.num_frames).toBeUndefined();
|
||||
expect(r.job.use_fsdp_inference).toBeUndefined();
|
||||
expect(r.job).not.toHaveProperty("extra");
|
||||
});
|
||||
|
||||
it.each([
|
||||
["not json", "That file isn't valid JSON."],
|
||||
["[1,2]", "Expected a JSON object describing one job."],
|
||||
[JSON.stringify({ prompt: "p" }), "The job file has no model_id."],
|
||||
[JSON.stringify({ model_id: "m", prompt: 5 }), "prompt must be text or a list of text lines."],
|
||||
[JSON.stringify({ model_id: "m", prompt: "p", references: [{}] }), "references[0] needs a source path."],
|
||||
])("rejects %s", (text, error) => {
|
||||
expect(parseJobJson(text)).toEqual({ ok: false, error });
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,102 @@
|
||||
/**
|
||||
* Read a job file (the `ref2va_*.json` shape) into the create-job form's
|
||||
* job shape. Keys match the API's CreateJobRequest; `prompt` may be a string
|
||||
* or a list of lines (joined with newlines), since job files keep it as lines
|
||||
* for readability.
|
||||
*/
|
||||
import type { JobLike } from "@/lib/jobToFields";
|
||||
|
||||
export type JobJsonResult =
|
||||
| { ok: true; job: JobLike; workloadType: "t2v" | "i2v" | "t2i" }
|
||||
| { ok: false; error: string };
|
||||
|
||||
const WORKLOADS = ["t2v", "i2v", "t2i"] as const;
|
||||
const MEDIA_TYPES = ["image", "video", "audio"] as const;
|
||||
const VIDEO_EXT = /\.(mp4|mov|mkv|webm|avi)$/i;
|
||||
const AUDIO_EXT = /\.(wav|mp3|flac|m4a|ogg)$/i;
|
||||
|
||||
const STRING_KEYS = ["name", "image_path", "last_image_path", "negative_prompt"] as const;
|
||||
const NUMBER_KEYS = [
|
||||
"num_inference_steps",
|
||||
"num_frames",
|
||||
"height",
|
||||
"width",
|
||||
"guidance_scale",
|
||||
"guidance_rescale",
|
||||
"fps",
|
||||
"seed",
|
||||
"num_gpus",
|
||||
"vsa_sparsity",
|
||||
"tp_size",
|
||||
"sp_size",
|
||||
] as const;
|
||||
const BOOLEAN_KEYS = [
|
||||
"dit_cpu_offload",
|
||||
"dit_layerwise_offload",
|
||||
"text_encoder_cpu_offload",
|
||||
"vae_cpu_offload",
|
||||
"image_encoder_cpu_offload",
|
||||
"use_fsdp_inference",
|
||||
"enable_torch_compile",
|
||||
] as const;
|
||||
|
||||
function inferMediaType(source: string): (typeof MEDIA_TYPES)[number] {
|
||||
if (VIDEO_EXT.test(source)) return "video";
|
||||
if (AUDIO_EXT.test(source)) return "audio";
|
||||
return "image";
|
||||
}
|
||||
|
||||
export function parseJobJson(text: string): JobJsonResult {
|
||||
let raw: unknown;
|
||||
try {
|
||||
raw = JSON.parse(text);
|
||||
} catch {
|
||||
return { ok: false, error: "That file isn't valid JSON." };
|
||||
}
|
||||
if (typeof raw !== "object" || raw === null || Array.isArray(raw)) {
|
||||
return { ok: false, error: "Expected a JSON object describing one job." };
|
||||
}
|
||||
const data = raw as Record<string, unknown>;
|
||||
|
||||
if (typeof data.model_id !== "string" || !data.model_id.trim()) {
|
||||
return { ok: false, error: "The job file has no model_id." };
|
||||
}
|
||||
|
||||
let prompt: string;
|
||||
if (typeof data.prompt === "string") prompt = data.prompt;
|
||||
else if (Array.isArray(data.prompt) && data.prompt.every((l) => typeof l === "string")) {
|
||||
prompt = (data.prompt as string[]).join("\n");
|
||||
} else {
|
||||
return { ok: false, error: "prompt must be text or a list of text lines." };
|
||||
}
|
||||
|
||||
const references: { source: string; media_type: string }[] = [];
|
||||
if (data.references !== undefined && data.references !== null) {
|
||||
if (!Array.isArray(data.references)) return { ok: false, error: "references must be a list." };
|
||||
for (const [i, ref] of data.references.entries()) {
|
||||
const r = ref as Record<string, unknown> | null;
|
||||
if (!r || typeof r.source !== "string" || !r.source) {
|
||||
return { ok: false, error: `references[${i}] needs a source path.` };
|
||||
}
|
||||
const mediaType =
|
||||
typeof r.media_type === "string" && (MEDIA_TYPES as readonly string[]).includes(r.media_type)
|
||||
? r.media_type
|
||||
: inferMediaType(r.source);
|
||||
references.push({ source: r.source, media_type: mediaType });
|
||||
}
|
||||
}
|
||||
|
||||
const job: JobLike = { id: `imported-${Date.now()}`, model_id: data.model_id.trim(), prompt, references };
|
||||
const target = job as unknown as Record<string, unknown>;
|
||||
for (const key of STRING_KEYS) if (typeof data[key] === "string") target[key] = data[key];
|
||||
for (const key of NUMBER_KEYS) if (typeof data[key] === "number" && Number.isFinite(data[key])) target[key] = data[key];
|
||||
for (const key of BOOLEAN_KEYS) if (typeof data[key] === "boolean") target[key] = data[key];
|
||||
|
||||
const declared = (WORKLOADS as readonly string[]).includes(data.workload_type as string)
|
||||
? (data.workload_type as (typeof WORKLOADS)[number])
|
||||
: null;
|
||||
const workloadType = declared ?? (references.length > 0 || job.image_path ? "i2v" : "t2v");
|
||||
job.workload_type = workloadType;
|
||||
|
||||
return { ok: true, job, workloadType };
|
||||
}
|
||||
@@ -71,4 +71,8 @@ describe("referenceFileName", () => {
|
||||
expect(referenceFileName("/a/b/c.mp4")).toBe("c.mp4");
|
||||
expect(referenceFileName("c.mp4")).toBe("c.mp4");
|
||||
});
|
||||
|
||||
it("names a last-frame reference by the job it comes from, not its internal id", () => {
|
||||
expect(referenceFileName("job-last-frame:0a1b2c3d-1111-2222-3333-444455556666")).toBe("Last frame of job 0a1b2c3d");
|
||||
});
|
||||
});
|
||||
|
||||
@@ -72,6 +72,7 @@ export interface JobFormFields {
|
||||
|
||||
/** Uploads keep their original basename, so this is the display name. */
|
||||
export function referenceFileName(source: string): string {
|
||||
if (source.startsWith("job-last-frame:")) return `Last frame of job ${source.slice(15, 23)}`;
|
||||
return source.split("/").filter(Boolean).pop() ?? source;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,107 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
|
||||
import { deferredLastFrameSource } from '@/lib/continuation';
|
||||
import { planSceneQueue } from '@/lib/sceneQueue';
|
||||
import { buildScenes, type SceneClip } from '@/lib/scenes';
|
||||
import { makeJob } from '@/test/factories';
|
||||
|
||||
const SIX_SECTION = [
|
||||
'subject_definitions:',
|
||||
'<Subject 1> is a man.',
|
||||
'',
|
||||
'summary:',
|
||||
'[reference generation] A man.',
|
||||
'',
|
||||
'retention_analysis:',
|
||||
'<Subject 1> (appears in [Shot 1]): fully_preserved - suit.',
|
||||
'',
|
||||
'detailed_description:',
|
||||
'[Shot 1] Wide shot. A man.',
|
||||
'',
|
||||
'overall_soundscape:',
|
||||
'Quiet.',
|
||||
'',
|
||||
'non_diegetic_music:',
|
||||
'N/A',
|
||||
].join('\n');
|
||||
|
||||
const clips = (...overrides: Parameters<typeof makeJob>[0][]): SceneClip[] => {
|
||||
const jobs = overrides.map((over, i) =>
|
||||
makeJob({
|
||||
id: `c${i + 1}`,
|
||||
name: `s-clip-${i + 1}`,
|
||||
created_at: i + 1,
|
||||
status: 'pending',
|
||||
prompt: SIX_SECTION,
|
||||
...over,
|
||||
}),
|
||||
);
|
||||
return buildScenes(jobs)[0].clips;
|
||||
};
|
||||
|
||||
const ids = (list: { job: { id: string } }[]) => list.map((c) => c.job.id);
|
||||
|
||||
describe('planSceneQueue', () => {
|
||||
it('queues every clip that can run, in scene order', () => {
|
||||
const plan = planSceneQueue(clips({}, {}, {}), { chain: false });
|
||||
expect(ids(plan.queue)).toEqual(['c1', 'c2', 'c3']);
|
||||
expect(plan.skipped).toEqual([]);
|
||||
});
|
||||
|
||||
it.each([
|
||||
['completed', 'already finished'],
|
||||
['running', 'already running'],
|
||||
['queued', 'already queued'],
|
||||
])('leaves %s clips alone', (status, reason) => {
|
||||
const plan = planSceneQueue(clips({}, { status }, {}), { chain: false });
|
||||
expect(ids(plan.queue)).toEqual(['c1', 'c3']);
|
||||
expect(plan.skipped.map((s) => [s.clip.job.id, s.reason])).toEqual([['c2', reason]]);
|
||||
});
|
||||
|
||||
it('requeues failed and stopped clips', () => {
|
||||
const plan = planSceneQueue(clips({ status: 'failed' }, { status: 'stopped' }), { chain: false });
|
||||
expect(ids(plan.queue)).toEqual(['c1', 'c2']);
|
||||
});
|
||||
|
||||
describe('chaining clips through last frames', () => {
|
||||
it('links each clip after the first to the one before it', () => {
|
||||
const plan = planSceneQueue(clips({}, {}, {}), { chain: true });
|
||||
expect(plan.chain.map((c) => [c.clip.job.id, c.previous.job.id])).toEqual([
|
||||
['c2', 'c1'],
|
||||
['c3', 'c2'],
|
||||
]);
|
||||
});
|
||||
|
||||
it('links nothing when not asked to', () => {
|
||||
expect(planSceneQueue(clips({}, {}), { chain: false }).chain).toEqual([]);
|
||||
});
|
||||
|
||||
it('links a clip to a previous clip that has already finished', () => {
|
||||
const plan = planSceneQueue(clips({ status: 'completed' }, {}), { chain: true });
|
||||
expect(plan.chain.map((c) => c.previous.job.id)).toEqual(['c1']);
|
||||
expect(ids(plan.queue)).toEqual(['c2']);
|
||||
});
|
||||
|
||||
it('does not link clips that are not being queued', () => {
|
||||
const plan = planSceneQueue(clips({}, { status: 'completed' }, {}), { chain: true });
|
||||
expect(plan.chain.map((c) => c.clip.job.id)).toEqual(['c3']);
|
||||
});
|
||||
|
||||
it('keeps a frame the clip already opens on', () => {
|
||||
const own = [{ source: deferredLastFrameSource('c1'), media_type: 'image' }];
|
||||
const stale = [{ source: '/uploads/last_frames/last_frame_old-take.png', media_type: 'image' }];
|
||||
const plan = planSceneQueue(clips({}, { references: own }, { references: stale }), { chain: true });
|
||||
expect(plan.chain).toEqual([]);
|
||||
expect(plan.cannotChain).toEqual([]);
|
||||
});
|
||||
|
||||
it("reports clips whose prompt can't be rewritten instead of linking them", () => {
|
||||
const plan = planSceneQueue(clips({}, { prompt: 'A plain prompt.' }), { chain: true });
|
||||
expect(plan.chain).toEqual([]);
|
||||
expect(plan.cannotChain).toHaveLength(1);
|
||||
expect(plan.cannotChain[0].clip.job.id).toBe('c2');
|
||||
expect(plan.cannotChain[0].reason).toMatch(/six-section/);
|
||||
expect(ids(plan.queue)).toEqual(['c1', 'c2']); // still queued, just not linked
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,52 @@
|
||||
/**
|
||||
* Work out what "queue this scene" would do, so the user can confirm it first.
|
||||
*
|
||||
* Clips are queued in scene order. The server runs them one after another and
|
||||
* holds a clip that starts from another clip's last frame until that clip has
|
||||
* finished -- so a whole scene can be queued at once and each clip still opens
|
||||
* on the frame the one before it ended on. This decides which clips are queued,
|
||||
* and (optionally) which get that link attached first.
|
||||
*/
|
||||
import { attachedLastFrames, whyCannotAttach } from "@/lib/continuation";
|
||||
import type { SceneClip } from "@/lib/scenes";
|
||||
|
||||
/** Statuses a job can be queued from (the same ones it can be started or edited in). */
|
||||
export const QUEUEABLE_STATUSES = ["pending", "failed", "stopped"];
|
||||
|
||||
export interface SceneQueuePlan {
|
||||
/** clips to queue, in order */
|
||||
queue: SceneClip[];
|
||||
/** clips that will get the previous clip's last frame attached first */
|
||||
chain: { clip: SceneClip; previous: SceneClip }[];
|
||||
/** clips that would need the link but can't have it attached automatically */
|
||||
cannotChain: { clip: SceneClip; reason: string }[];
|
||||
/** clips left alone, and why */
|
||||
skipped: { clip: SceneClip; reason: string }[];
|
||||
}
|
||||
|
||||
function skipReason(status: string): string {
|
||||
if (status === "completed") return "already finished";
|
||||
if (status === "running") return "already running";
|
||||
if (status === "queued") return "already queued";
|
||||
return `is ${status}`;
|
||||
}
|
||||
|
||||
export function planSceneQueue(clips: SceneClip[], { chain: wantChain }: { chain: boolean }): SceneQueuePlan {
|
||||
const plan: SceneQueuePlan = { queue: [], chain: [], cannotChain: [], skipped: [] };
|
||||
clips.forEach((clip, i) => {
|
||||
if (!QUEUEABLE_STATUSES.includes(clip.job.status)) {
|
||||
plan.skipped.push({ clip, reason: skipReason(clip.job.status) });
|
||||
return;
|
||||
}
|
||||
plan.queue.push(clip);
|
||||
|
||||
const previous = clips[i - 1];
|
||||
if (!wantChain || !previous) return;
|
||||
// A clip that already opens on a frame (of this or another take) keeps it.
|
||||
if (attachedLastFrames(clip.job.references).length > 0) return;
|
||||
const blocked = whyCannotAttach(clip.job.prompt, clip.job.references);
|
||||
if (blocked) plan.cannotChain.push({ clip, reason: blocked });
|
||||
else plan.chain.push({ clip, previous });
|
||||
});
|
||||
return plan;
|
||||
}
|
||||
@@ -0,0 +1,250 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { UNNAMED_SCENE, buildScenes, clipSeconds, formatClock, parseSceneName, shotsOfJob } from "@/lib/scenes";
|
||||
import { shotLines, shotProse } from "@/lib/h3Shots";
|
||||
import type { Job } from "@/lib/types";
|
||||
|
||||
function job(over: Partial<Job> & { id: string }): Job {
|
||||
return {
|
||||
model_id: "m",
|
||||
prompt: "",
|
||||
status: "pending",
|
||||
created_at: 0,
|
||||
started_at: null,
|
||||
finished_at: null,
|
||||
error: null,
|
||||
output_path: null,
|
||||
log_file_path: null,
|
||||
num_inference_steps: 50,
|
||||
num_frames: 240,
|
||||
fps: 24,
|
||||
height: 768,
|
||||
width: 1344,
|
||||
guidance_scale: 1,
|
||||
seed: 0,
|
||||
num_gpus: 1,
|
||||
progress: 0,
|
||||
progress_msg: "",
|
||||
phase: "",
|
||||
...over,
|
||||
};
|
||||
}
|
||||
|
||||
const SECTIONS = (detailed: string, subjects = "<Subject 1> is a man.") =>
|
||||
["subject_definitions:", subjects, "", "detailed_description:", detailed].join("\n");
|
||||
|
||||
describe("parseSceneName", () => {
|
||||
it.each([
|
||||
["wolf-lunch-clip-01", "wolf-lunch", 1],
|
||||
["wolf-lunch-clip-15", "wolf-lunch", 15],
|
||||
["closeup-fictional-actor-monologue-shot2", "closeup-fictional-actor-monologue", 2],
|
||||
["Rooftop Standoff part 3", "Rooftop Standoff", 3],
|
||||
["scene_a_07", "scene_a", 7],
|
||||
])("%s -> stem %s, clip %i", (name, title, order) => {
|
||||
expect(parseSceneName(name)).toEqual({ key: title.toLowerCase(), title, order });
|
||||
});
|
||||
|
||||
it("treats a name with no trailing number as its own scene", () => {
|
||||
expect(parseSceneName("Rooftop Standoff")).toEqual({ key: "rooftop standoff", title: "Rooftop Standoff", order: null });
|
||||
});
|
||||
|
||||
it("keeps the word scene as part of the name", () => {
|
||||
expect(parseSceneName("rooftop-scene-2")).toEqual({ key: "rooftop-scene", title: "rooftop-scene", order: 2 });
|
||||
});
|
||||
|
||||
it("does not treat a bare number or number-only suffix as a stem", () => {
|
||||
expect(parseSceneName("42").order).toBeNull();
|
||||
expect(parseSceneName("clip-3").order).toBeNull();
|
||||
});
|
||||
|
||||
it("puts blank or missing names in the unnamed group", () => {
|
||||
expect(parseSceneName(undefined).key).toBe(UNNAMED_SCENE);
|
||||
expect(parseSceneName(" ").key).toBe(UNNAMED_SCENE);
|
||||
});
|
||||
|
||||
it("groups names that differ only in case", () => {
|
||||
expect(parseSceneName("Wolf-Lunch-Clip-1").key).toBe(parseSceneName("wolf-lunch-clip-2").key);
|
||||
});
|
||||
});
|
||||
|
||||
describe("shotsOfJob", () => {
|
||||
it("reads shots and dialogue out of a six-section prompt", () => {
|
||||
const shots = shotsOfJob({
|
||||
prompt: SECTIONS(
|
||||
"[Shot 1] Wide shot. A dock.\n[Shot 2] At 00:02.000, the shot cuts to a close-up shot. (S1) <d>[English] Hi.</d>",
|
||||
),
|
||||
});
|
||||
expect(shots).toHaveLength(2);
|
||||
expect(shots[1].type).toBe("close-up");
|
||||
expect(shotLines(shots[1])[0].text).toBe("Hi.");
|
||||
expect(shots[0].subjectIds).toEqual([]);
|
||||
});
|
||||
|
||||
it("reads shots out of a base-format integrated_multimodal_description", () => {
|
||||
const shots = shotsOfJob({
|
||||
prompt: "For the target video, <Picture 1> is used.\n\nintegrated_multimodal_description: [Shot 1] He (S1) says: <d>[English] Sorry.</d> <cutoff>\n\noverall_soundscape: Quiet.",
|
||||
});
|
||||
expect(shots).toHaveLength(1);
|
||||
expect(shotLines(shots[0]).map((l) => l.text)).toEqual(["Sorry."]);
|
||||
});
|
||||
|
||||
it("shows a plain prompt as a single shot", () => {
|
||||
const shots = shotsOfJob({ prompt: "A toy car drives into a plush dog." });
|
||||
expect(shots).toHaveLength(1);
|
||||
expect(shotProse(shots[0])).toBe("A toy car drives into a plush dog.");
|
||||
});
|
||||
|
||||
it("returns no shots for an empty prompt", () => {
|
||||
expect(shotsOfJob({ prompt: "" })).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("buildScenes", () => {
|
||||
const three = [
|
||||
job({ id: "c", name: "wolf-lunch-clip-03", created_at: 3, status: "pending", prompt: "[Shot 1] Wide shot. C." }),
|
||||
job({ id: "a", name: "wolf-lunch-clip-01", created_at: 1, status: "completed", num_frames: 240, prompt: "[Shot 1] Wide shot. A.\n[Shot 2] At 00:02.000, the shot cuts to a close-up shot. A2." }),
|
||||
job({ id: "b", name: "wolf-lunch-clip-02", created_at: 2, status: "completed", num_frames: 120, fps: 24, prompt: "[Shot 1] Close-up shot. B." }),
|
||||
];
|
||||
|
||||
it("groups by stem and orders clips by their number, not creation order", () => {
|
||||
const [scene] = buildScenes(three);
|
||||
expect(scene.title).toBe("wolf-lunch");
|
||||
expect(scene.clips.map((c) => c.job.id)).toEqual(["a", "b", "c"]);
|
||||
expect(scene.clips.map((c) => c.position)).toEqual([1, 2, 3]);
|
||||
});
|
||||
|
||||
it("lays every shot on one timeline using each clip's frames and fps", () => {
|
||||
const [scene] = buildScenes(three);
|
||||
expect(scene.clips.map((c) => c.startSeconds)).toEqual([0, 10, 15]);
|
||||
expect(scene.totalSeconds).toBe(25); // 240/24 + 120/24 + 240/24 = 10 + 5 + 10
|
||||
});
|
||||
|
||||
it("counts shots and statuses across the scene", () => {
|
||||
const [scene] = buildScenes(three);
|
||||
expect(scene.shotCount).toBe(4);
|
||||
expect(scene.statusCounts).toEqual({ completed: 2, pending: 1 });
|
||||
});
|
||||
|
||||
it("offsets each shot's cut time by the clip's start", () => {
|
||||
const [scene] = buildScenes(three);
|
||||
const second = scene.clips[1];
|
||||
expect(second.shotStarts).toEqual([10]);
|
||||
const first = scene.clips[0];
|
||||
expect(first.shotStarts[0]).toBe(0);
|
||||
expect(first.shotStarts[1]).toBe(2);
|
||||
});
|
||||
|
||||
it("keeps separate scenes apart, newest first, with unnamed jobs last", () => {
|
||||
const scenes = buildScenes([
|
||||
job({ id: "1", name: "old-scene-1", created_at: 5 }),
|
||||
job({ id: "2", name: "new-scene-1", created_at: 50 }),
|
||||
job({ id: "3", name: "", created_at: 100 }),
|
||||
]);
|
||||
expect(scenes.map((s) => s.title)).toEqual(["new-scene", "old-scene", "Unnamed jobs"]);
|
||||
});
|
||||
|
||||
it("puts clips without a number after numbered ones, oldest first", () => {
|
||||
const [scene] = buildScenes([
|
||||
job({ id: "x", name: "wolf-1", created_at: 9 }),
|
||||
job({ id: "y", name: "wolf", created_at: 1 }),
|
||||
job({ id: "z", name: "wolf-2", created_at: 2 }),
|
||||
]);
|
||||
// "wolf" shares the stem with wolf-1 / wolf-2, so it joins them -- last, having no number
|
||||
expect(scene.clips.map((c) => c.job.id)).toEqual(["x", "z", "y"]);
|
||||
});
|
||||
|
||||
it("flags a clip that continues the previous shot", () => {
|
||||
const [scene] = buildScenes([
|
||||
job({ id: "a", name: "s-1", prompt: "[Shot 1] Close-up shot. A speech. <cutoff>" }),
|
||||
job({ id: "b", name: "s-2", prompt: "[Shot 1] Close-up shot, dolly. Continuing the same shot on <Subject 1>, still speaking." }),
|
||||
job({ id: "c", name: "s-3", prompt: "summary:\n[reference generation] It continues the previous shot without a cut.\n\ndetailed_description:\n[Shot 1] Wide shot. X." }),
|
||||
job({ id: "d", name: "s-4", prompt: "[Shot 1] Wide shot. Fresh." }),
|
||||
]);
|
||||
expect(scene.clips.map((c) => c.continuation)).toEqual([false, true, true, false]);
|
||||
});
|
||||
|
||||
it("falls back to 24 fps when a job has none", () => {
|
||||
expect(clipSeconds({ num_frames: 240, fps: undefined })).toBe(10);
|
||||
expect(clipSeconds({ num_frames: 240, fps: 0 })).toBe(10);
|
||||
expect(clipSeconds({ num_frames: 240, fps: 30 })).toBe(8);
|
||||
});
|
||||
|
||||
it("returns no scenes for no jobs", () => {
|
||||
expect(buildScenes([])).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("formatClock", () => {
|
||||
it("formats m:ss and h:mm:ss", () => {
|
||||
expect(formatClock(0)).toBe("0:00");
|
||||
expect(formatClock(9.6)).toBe("0:10");
|
||||
expect(formatClock(65)).toBe("1:05");
|
||||
expect(formatClock(3725)).toBe("1:02:05");
|
||||
expect(formatClock(-4)).toBe("0:00");
|
||||
});
|
||||
});
|
||||
|
||||
describe("takes of the same clip", () => {
|
||||
const take = (id: string, n: number, over: Partial<Job> = {}) =>
|
||||
job({ id, name: `wolf-lunch-clip-0${n}`, prompt: `[Shot 1] Wide shot. ${id}.`, num_frames: 240, ...over });
|
||||
|
||||
it("counts re-runs of a clip number once, not as extra clips", () => {
|
||||
const [scene] = buildScenes([
|
||||
take("a1", 1, { created_at: 1, status: "completed" }),
|
||||
take("a2", 1, { created_at: 2, status: "completed" }),
|
||||
take("a3", 1, { created_at: 3, status: "completed" }),
|
||||
take("b1", 2, { created_at: 4, status: "completed" }),
|
||||
]);
|
||||
expect(scene.clips).toHaveLength(2);
|
||||
expect(scene.totalSeconds).toBe(20);
|
||||
expect(scene.shotCount).toBe(2);
|
||||
expect(scene.statusCounts).toEqual({ completed: 2 });
|
||||
expect(scene.clips[0].takes.map((j) => j.id)).toEqual(["a1", "a2", "a3"]);
|
||||
});
|
||||
|
||||
it("shows the most recent completed take, even over a newer failed one", () => {
|
||||
const [scene] = buildScenes([
|
||||
take("old", 1, { created_at: 1, status: "completed", finished_at: 10 }),
|
||||
take("good", 1, { created_at: 2, status: "completed", finished_at: 20 }),
|
||||
take("bad", 1, { created_at: 3, status: "failed", finished_at: 30 }),
|
||||
]);
|
||||
expect(scene.clips[0].job.id).toBe("good");
|
||||
});
|
||||
|
||||
it("falls back to the newest take when none completed", () => {
|
||||
const [scene] = buildScenes([
|
||||
take("p1", 1, { created_at: 1, status: "failed" }),
|
||||
take("p2", 1, { created_at: 2, status: "pending" }),
|
||||
]);
|
||||
expect(scene.clips[0].job.id).toBe("p2");
|
||||
});
|
||||
|
||||
it("shows the take the user chose, and lays the timeline out from it", () => {
|
||||
const jobs = [
|
||||
take("short", 1, { created_at: 1, status: "completed", num_frames: 120 }),
|
||||
take("long", 1, { created_at: 2, status: "completed", num_frames: 480 }),
|
||||
take("next", 2, { created_at: 3 }),
|
||||
];
|
||||
const [auto] = buildScenes(jobs);
|
||||
expect(auto.clips[0].job.id).toBe("long");
|
||||
expect(auto.clips[1].startSeconds).toBe(20);
|
||||
|
||||
const [chosen] = buildScenes(jobs, { [auto.clips[0].takeKey as string]: "short" });
|
||||
expect(chosen.clips[0].job.id).toBe("short");
|
||||
expect(chosen.clips[1].startSeconds).toBe(5);
|
||||
});
|
||||
|
||||
it("ignores a choice that isn't one of the takes", () => {
|
||||
const jobs = [take("a", 1, { status: "completed" })];
|
||||
const [scene] = buildScenes(jobs, { "wolf-lunch#1": "no-such-job" });
|
||||
expect(scene.clips[0].job.id).toBe("a");
|
||||
});
|
||||
|
||||
it("keeps each unnumbered job as its own clip", () => {
|
||||
const [scene] = buildScenes([
|
||||
job({ id: "u1", name: "solo", created_at: 1 }),
|
||||
job({ id: "u2", name: "solo", created_at: 2 }),
|
||||
]);
|
||||
expect(scene.clips.map((c) => c.job.id)).toEqual(["u1", "u2"]);
|
||||
expect(scene.clips.every((c) => c.takeKey === null)).toBe(true);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,182 @@
|
||||
/**
|
||||
* Group jobs into scenes and lay their shots out on one timeline.
|
||||
*
|
||||
* A long scene is generated as many short clips, one job each. Nothing on the
|
||||
* server links them, so the link is the job name: `wolf-lunch-clip-01`,
|
||||
* `wolf-lunch-clip-02`, ... share the stem `wolf-lunch` and are ordered by the
|
||||
* trailing number. Everything else -- shots, dialogue, subjects -- is read back
|
||||
* out of each job's own prompt, so no backend support is needed.
|
||||
*/
|
||||
import { readField, shotTarget } from "@/lib/h3PromptFields";
|
||||
import {
|
||||
parseDetailedDescription,
|
||||
parseSubjectLabels,
|
||||
resolveShotStarts,
|
||||
shotProse,
|
||||
type Shot,
|
||||
} from "@/lib/h3Shots";
|
||||
import type { Job } from "@/lib/types";
|
||||
|
||||
export const UNNAMED_SCENE = "__unnamed__";
|
||||
export const DEFAULT_FPS = 24;
|
||||
|
||||
// "<stem>[-_. ][clip|part|pt|shot|take][-_. ]<number>". "scene" is deliberately not
|
||||
// a clip word: it is far more likely part of the scene's own name ("rooftop-scene-2").
|
||||
const CLIP_SUFFIX_RE = /^(.*?)[\s._-]*(?:clip|part|pt|shot|take)?[\s._-]*(\d+)$/i;
|
||||
|
||||
export interface SceneName {
|
||||
key: string;
|
||||
title: string;
|
||||
/** the clip number from the end of the name, or null when there isn't one */
|
||||
order: number | null;
|
||||
}
|
||||
|
||||
export function parseSceneName(name: string | undefined | null): SceneName {
|
||||
const trimmed = (name ?? "").trim();
|
||||
if (!trimmed) return { key: UNNAMED_SCENE, title: "Unnamed jobs", order: null };
|
||||
const m = trimmed.match(CLIP_SUFFIX_RE);
|
||||
const stem = m ? m[1].replace(/[\s._-]+$/, "") : "";
|
||||
if (m && stem) return { key: stem.toLowerCase(), title: stem, order: Number(m[2]) };
|
||||
return { key: trimmed.toLowerCase(), title: trimmed, order: null };
|
||||
}
|
||||
|
||||
/** The shots a job's prompt describes, wherever they live in it (base, reference, or plain prompt). */
|
||||
export function shotsOfJob(job: Pick<Job, "prompt">): Shot[] {
|
||||
const prompt = job.prompt ?? "";
|
||||
const labels = parseSubjectLabels(readField(prompt, "subject_definitions") ?? "");
|
||||
return parseDetailedDescription(shotTarget(prompt).body, labels).shots;
|
||||
}
|
||||
|
||||
export interface SceneClip {
|
||||
/** the take shown for this clip */
|
||||
job: Job;
|
||||
/**
|
||||
* Every job that claims this clip number, oldest first. Re-running a clip
|
||||
* gives the same name again, so these are takes of one clip, not extra clips.
|
||||
*/
|
||||
takes: Job[];
|
||||
/** identifies this clip for choosing a take; null for a clip with no number */
|
||||
takeKey: string | null;
|
||||
/** 1-based place in the scene */
|
||||
position: number;
|
||||
shots: Shot[];
|
||||
/** seconds from the start of the scene to the start of this clip */
|
||||
startSeconds: number;
|
||||
durationSeconds: number;
|
||||
/** each shot's cut-in time measured from the start of the scene */
|
||||
shotStarts: number[];
|
||||
/** this clip picks up in the middle of the previous clip's last shot */
|
||||
continuation: boolean;
|
||||
}
|
||||
|
||||
export interface Scene {
|
||||
key: string;
|
||||
title: string;
|
||||
clips: SceneClip[];
|
||||
totalSeconds: number;
|
||||
shotCount: number;
|
||||
statusCounts: Record<string, number>;
|
||||
/** most recent created_at among the clips, for ordering scenes */
|
||||
latest: number;
|
||||
}
|
||||
|
||||
export function clipSeconds(job: Pick<Job, "num_frames" | "fps">): number {
|
||||
const fps = job.fps && job.fps > 0 ? job.fps : DEFAULT_FPS;
|
||||
return Math.max(0, (job.num_frames ?? 0) / fps);
|
||||
}
|
||||
|
||||
function isContinuation(job: Job, shots: Shot[]): boolean {
|
||||
if (/continues the previous shot/i.test(job.prompt ?? "")) return true;
|
||||
return shots.length > 0 && /^Continuing the same shot/i.test(shotProse(shots[0]));
|
||||
}
|
||||
|
||||
function createdAt(job: Job): number {
|
||||
return job.created_at ?? 0;
|
||||
}
|
||||
|
||||
/** The take to show: the one the user chose, else the most recent completed one, else the newest. */
|
||||
function pickTake(takes: Job[], choice?: string): Job {
|
||||
const chosen = choice ? takes.find((j) => j.id === choice) : undefined;
|
||||
if (chosen) return chosen;
|
||||
const recent = (a: Job, b: Job) => (b.finished_at ?? createdAt(b)) - (a.finished_at ?? createdAt(a));
|
||||
const completed = takes.filter((j) => j.status === "completed").sort(recent);
|
||||
return completed[0] ?? [...takes].sort((a, b) => createdAt(b) - createdAt(a))[0];
|
||||
}
|
||||
|
||||
/**
|
||||
* Group inference jobs into scenes, each with its clips in order and a
|
||||
* scene-wide timeline. `choices` maps a clip's `takeKey` to the job id of the
|
||||
* take to show in place of the default.
|
||||
*/
|
||||
export function buildScenes(jobs: Job[], choices: Record<string, string> = {}): Scene[] {
|
||||
const groups = new Map<string, { title: string; entries: { job: Job; order: number | null }[] }>();
|
||||
for (const job of jobs) {
|
||||
const { key, title, order } = parseSceneName(job.name);
|
||||
const group = groups.get(key) ?? { title, entries: [] };
|
||||
group.entries.push({ job, order });
|
||||
groups.set(key, group);
|
||||
}
|
||||
|
||||
const scenes: Scene[] = [];
|
||||
for (const [key, { title, entries }] of groups) {
|
||||
// One clip per clip number (its takes together); each unnumbered job stands alone, after the numbered ones.
|
||||
const byNumber = new Map<number, Job[]>();
|
||||
const unnumbered: Job[][] = [];
|
||||
for (const { job, order } of entries) {
|
||||
if (order === null) unnumbered.push([job]);
|
||||
else byNumber.set(order, [...(byNumber.get(order) ?? []), job]);
|
||||
}
|
||||
const slots: { takes: Job[]; takeKey: string | null }[] = [
|
||||
...[...byNumber.entries()].sort(([a], [b]) => a - b).map(([order, takes]) => ({ takes, takeKey: `${key}#${order}` })),
|
||||
...unnumbered.sort((a, b) => createdAt(a[0]) - createdAt(b[0])).map((takes) => ({ takes, takeKey: null })),
|
||||
];
|
||||
|
||||
let elapsed = 0;
|
||||
const statusCounts: Record<string, number> = {};
|
||||
const clips: SceneClip[] = slots.map(({ takes, takeKey }, i) => {
|
||||
const ordered = [...takes].sort((a, b) => createdAt(a) - createdAt(b));
|
||||
const job = pickTake(ordered, takeKey ? choices[takeKey] : undefined);
|
||||
const shots = shotsOfJob(job);
|
||||
const durationSeconds = clipSeconds(job);
|
||||
const clip: SceneClip = {
|
||||
job,
|
||||
takes: ordered,
|
||||
takeKey,
|
||||
position: i + 1,
|
||||
shots,
|
||||
startSeconds: elapsed,
|
||||
durationSeconds,
|
||||
shotStarts: resolveShotStarts(shots).map((t) => elapsed + t),
|
||||
continuation: isContinuation(job, shots),
|
||||
};
|
||||
elapsed += durationSeconds;
|
||||
statusCounts[job.status] = (statusCounts[job.status] ?? 0) + 1;
|
||||
return clip;
|
||||
});
|
||||
|
||||
scenes.push({
|
||||
key,
|
||||
title,
|
||||
clips,
|
||||
totalSeconds: elapsed,
|
||||
shotCount: clips.reduce((n, c) => n + c.shots.length, 0),
|
||||
statusCounts,
|
||||
latest: Math.max(0, ...entries.map((e) => createdAt(e.job))),
|
||||
});
|
||||
}
|
||||
|
||||
// Newest scene first; the catch-all group of unnamed jobs goes last.
|
||||
return scenes.sort((a, b) => {
|
||||
if ((a.key === UNNAMED_SCENE) !== (b.key === UNNAMED_SCENE)) return a.key === UNNAMED_SCENE ? 1 : -1;
|
||||
return b.latest - a.latest;
|
||||
});
|
||||
}
|
||||
|
||||
/** m:ss, or h:mm:ss past an hour -- for clip ranges and runtimes. */
|
||||
export function formatClock(seconds: number): string {
|
||||
const total = Math.max(0, Math.round(seconds));
|
||||
const h = Math.floor(total / 3600);
|
||||
const m = Math.floor((total % 3600) / 60);
|
||||
const sec = String(total % 60).padStart(2, "0");
|
||||
return h > 0 ? `${h}:${String(m).padStart(2, "0")}:${sec}` : `${m}:${sec}`;
|
||||
}
|
||||
@@ -2,6 +2,19 @@
|
||||
|
||||
export type JobType = "inference" | "finetuning" | "distillation";
|
||||
|
||||
/**
|
||||
* One razor-cut section of a clip's kept (already start/end-trimmed) range,
|
||||
* and its own color grade. `end_seconds` is seconds into that kept range, not
|
||||
* the original file's timeline; only the last section in a list may leave it
|
||||
* null, meaning "to the end". Sections are contiguous, in order, no gaps.
|
||||
*/
|
||||
export interface GradeSegment {
|
||||
end_seconds: number | null;
|
||||
brightness: number;
|
||||
contrast: number;
|
||||
saturation: number;
|
||||
}
|
||||
|
||||
export interface Job {
|
||||
id: string;
|
||||
model_id: string;
|
||||
@@ -11,13 +24,23 @@ export interface Job {
|
||||
workload_type?: string;
|
||||
status: string;
|
||||
created_at: number;
|
||||
/** when the job joined the queue; null unless it is queued */
|
||||
queued_at?: number | null;
|
||||
started_at: number | null;
|
||||
finished_at: number | null;
|
||||
error: string | null;
|
||||
output_path: string | null;
|
||||
log_file_path: string | null;
|
||||
/** the range/sections last applied via the trim API; neutral/full-range for a never-edited job */
|
||||
edit_start_seconds?: number;
|
||||
edit_end_seconds?: number | null;
|
||||
edit_segments?: GradeSegment[];
|
||||
num_inference_steps: number;
|
||||
num_frames: number;
|
||||
/** frames per second; the API sends it, older cached objects may not have it */
|
||||
fps?: number;
|
||||
/** reference media (image/video/audio) attached to the job, as saved on the server */
|
||||
references?: { source: string; media_type: string }[] | null;
|
||||
height: number;
|
||||
width: number;
|
||||
guidance_scale: number;
|
||||
|
||||
@@ -16,6 +16,23 @@ if (!globalThis.crypto?.randomUUID) {
|
||||
});
|
||||
}
|
||||
|
||||
// jsdom's Blob/File predate Blob.text(); real browsers all have it.
|
||||
if (!Blob.prototype.text) {
|
||||
Blob.prototype.text = function text(this: Blob) {
|
||||
return new Promise<string>((resolve, reject) => {
|
||||
const reader = new FileReader();
|
||||
reader.onload = () => resolve(String(reader.result));
|
||||
reader.onerror = () => reject(reader.error);
|
||||
reader.readAsText(this);
|
||||
});
|
||||
};
|
||||
}
|
||||
|
||||
// jsdom doesn't implement scrollIntoView.
|
||||
if (!Element.prototype.scrollIntoView) {
|
||||
Element.prototype.scrollIntoView = vi.fn();
|
||||
}
|
||||
|
||||
if (!URL.createObjectURL) {
|
||||
URL.createObjectURL = vi.fn(() => 'blob:mock-url');
|
||||
}
|
||||
|
||||
@@ -0,0 +1,278 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Queued jobs run in order, one at a time, and a clip waits for the clip whose last frame it uses."""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from fastvideo_studio.job_queue import (
|
||||
circular_dependency,
|
||||
deferred_last_clip_source,
|
||||
deferred_last_frame_source,
|
||||
dependencies,
|
||||
dependency_of,
|
||||
plan_dispatch,
|
||||
resolve_references,
|
||||
UnresolvedReference,
|
||||
)
|
||||
|
||||
|
||||
@dataclass
|
||||
class J:
|
||||
id: str
|
||||
status: str = "pending"
|
||||
queued_at: float | None = None
|
||||
references: list[dict[str, Any]] = field(default_factory=list)
|
||||
name: str = ""
|
||||
|
||||
|
||||
def after(job_id: str) -> list[dict[str, str]]:
|
||||
return [{"source": deferred_last_frame_source(job_id), "media_type": "image"}]
|
||||
|
||||
|
||||
def queue(*jobs: J) -> dict[str, J]:
|
||||
for i, job in enumerate(jobs):
|
||||
if job.status == "queued" and job.queued_at is None:
|
||||
job.queued_at = float(i)
|
||||
return {j.id: j for j in jobs}
|
||||
|
||||
|
||||
def test_deferred_source_round_trips():
|
||||
assert dependency_of({"source": deferred_last_frame_source("abc")}) == "abc"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("ref", [None, {}, {"source": "/tmp/a.png"}, {"source": "job-last-frame:"}, {"source": 3}])
|
||||
def test_ordinary_references_have_no_dependency(ref):
|
||||
assert dependency_of(ref) is None
|
||||
|
||||
|
||||
def test_dependencies_keep_order_and_drop_repeats():
|
||||
refs = [*after("b"), {"source": "/x.png"}, *after("a"), *after("b")]
|
||||
assert dependencies(refs) == ["b", "a"]
|
||||
|
||||
|
||||
class TestDispatchOrder:
|
||||
def test_starts_the_oldest_queued_job_first(self):
|
||||
jobs = queue(J("late", "queued", 5.0), J("early", "queued", 1.0))
|
||||
assert plan_dispatch(jobs, 1).start == ["early"]
|
||||
|
||||
def test_runs_only_max_concurrent_including_running_jobs(self):
|
||||
jobs = queue(J("r", "running"), J("a", "queued"), J("b", "queued"))
|
||||
assert plan_dispatch(jobs, 1).start == []
|
||||
assert plan_dispatch(jobs, 2).start == ["a"]
|
||||
assert plan_dispatch(jobs, 3).start == ["a", "b"]
|
||||
|
||||
def test_ignores_jobs_that_are_not_queued(self):
|
||||
jobs = queue(J("p"), J("c", "completed"), J("f", "failed"), J("s", "stopped"))
|
||||
assert plan_dispatch(jobs, 4).start == []
|
||||
|
||||
def test_a_limit_below_one_still_runs_one_job(self):
|
||||
assert plan_dispatch(queue(J("a", "queued")), 0).start == ["a"]
|
||||
|
||||
def test_ties_are_broken_by_id_for_a_stable_order(self):
|
||||
jobs = {"b": J("b", "queued", 1.0), "a": J("a", "queued", 1.0)}
|
||||
assert plan_dispatch(jobs, 2).start == ["a", "b"]
|
||||
|
||||
|
||||
class TestWaitingOnAFrame:
|
||||
def test_waits_while_the_source_clip_is_running_or_queued(self):
|
||||
for source_status in ("running", "queued"):
|
||||
jobs = queue(J("one", source_status), J("two", "queued", references=after("one")))
|
||||
plan = plan_dispatch(jobs, 2)
|
||||
assert "two" not in plan.start and not plan.fail
|
||||
|
||||
def test_starts_once_the_source_clip_completed(self):
|
||||
jobs = queue(J("one", "completed"), J("two", "queued", references=after("one")))
|
||||
assert plan_dispatch(jobs, 1).start == ["two"]
|
||||
|
||||
def test_waiting_does_not_hold_up_later_jobs(self):
|
||||
jobs = queue(
|
||||
J("one", "running"),
|
||||
J("two", "queued", references=after("one")),
|
||||
J("free", "queued"),
|
||||
)
|
||||
assert plan_dispatch(jobs, 2).start == ["free"]
|
||||
|
||||
def test_a_source_that_is_only_pending_keeps_waiting(self):
|
||||
# The user may still queue it.
|
||||
jobs = queue(J("one", "pending"), J("two", "queued", references=after("one")))
|
||||
plan = plan_dispatch(jobs, 1)
|
||||
assert plan.start == [] and plan.fail == {}
|
||||
|
||||
def test_needs_every_source(self):
|
||||
refs = [*after("a"), *after("b")]
|
||||
jobs = queue(J("a", "completed"), J("b", "running"), J("c", "queued", references=refs))
|
||||
assert plan_dispatch(jobs, 2).start == []
|
||||
jobs["b"].status = "completed"
|
||||
assert plan_dispatch(jobs, 2).start == ["c"]
|
||||
|
||||
def test_a_chain_advances_one_clip_at_a_time(self):
|
||||
jobs = queue(
|
||||
J("c1", "queued"),
|
||||
J("c2", "queued", references=after("c1")),
|
||||
J("c3", "queued", references=after("c2")),
|
||||
)
|
||||
assert plan_dispatch(jobs, 5).start == ["c1"]
|
||||
jobs["c1"].status = "completed"
|
||||
assert plan_dispatch(jobs, 5).start == ["c2"]
|
||||
jobs["c2"].status = "completed"
|
||||
assert plan_dispatch(jobs, 5).start == ["c3"]
|
||||
|
||||
|
||||
class TestImpossibleJobs:
|
||||
@pytest.mark.parametrize(("status", "word"), [("failed", "failed"), ("stopped", "stopped")])
|
||||
def test_fails_when_the_source_failed_or_was_stopped(self, status, word):
|
||||
jobs = queue(J("one", status, name="Clip 1"), J("two", "queued", references=after("one")))
|
||||
plan = plan_dispatch(jobs, 1)
|
||||
assert plan.start == []
|
||||
assert "Clip 1" in plan.fail["two"] and word in plan.fail["two"]
|
||||
|
||||
def test_fails_when_the_source_was_deleted(self):
|
||||
jobs = queue(J("two", "queued", references=after("gone")))
|
||||
assert "deleted" in plan_dispatch(jobs, 1).fail["two"]
|
||||
|
||||
def test_failure_cascades_down_the_chain(self):
|
||||
jobs = queue(
|
||||
J("c1", "failed"),
|
||||
J("c2", "queued", references=after("c1")),
|
||||
J("c3", "queued", references=after("c2")),
|
||||
J("c4", "queued", references=after("c3")),
|
||||
J("other", "queued"),
|
||||
)
|
||||
plan = plan_dispatch(jobs, 3)
|
||||
assert set(plan.fail) == {"c2", "c3", "c4"}
|
||||
assert plan.start == ["other"]
|
||||
|
||||
def test_a_failed_job_frees_its_slot_for_the_next(self):
|
||||
jobs = queue(
|
||||
J("c1", "failed"),
|
||||
J("c2", "queued", references=after("c1")),
|
||||
J("next", "queued"),
|
||||
)
|
||||
assert plan_dispatch(jobs, 1).start == ["next"]
|
||||
|
||||
def test_planning_does_not_change_the_jobs(self):
|
||||
jobs = queue(J("c1", "failed"), J("c2", "queued", references=after("c1")))
|
||||
plan_dispatch(jobs, 1)
|
||||
assert jobs["c2"].status == "queued"
|
||||
|
||||
|
||||
class TestCircularDependency:
|
||||
def test_none_for_a_plain_chain(self):
|
||||
jobs = queue(J("a"), J("b", references=after("a")), J("c", references=after("b")))
|
||||
assert circular_dependency(jobs, "c") is None
|
||||
|
||||
def test_finds_a_loop_through_the_job(self):
|
||||
jobs = queue(J("a", references=after("c")), J("b", references=after("a")), J("c", references=after("b")))
|
||||
assert circular_dependency(jobs, "a") is not None
|
||||
|
||||
def test_a_job_that_waits_on_itself(self):
|
||||
assert circular_dependency(queue(J("a", references=after("a"))), "a") is not None
|
||||
|
||||
def test_a_loop_elsewhere_is_not_this_jobs_problem(self):
|
||||
jobs = queue(
|
||||
J("a", references=after("b")),
|
||||
J("b", references=after("a")),
|
||||
J("c", references=after("a")),
|
||||
)
|
||||
assert circular_dependency(jobs, "c") is None
|
||||
|
||||
|
||||
class TestResolveReferences:
|
||||
def _jobs(self, **statuses: str) -> dict[str, J]:
|
||||
return {k: J(k, v, name=f"Clip {k}") for k, v in statuses.items()}
|
||||
|
||||
def test_replaces_a_deferred_reference_with_the_extracted_frame(self):
|
||||
jobs = self._jobs(one="completed")
|
||||
stored = [{"source": "/ref.png", "media_type": "image"}, *after("one")]
|
||||
resolved = resolve_references(stored, jobs.get, lambda job: f"/frames/{job.id}.png")
|
||||
assert resolved == [
|
||||
{"source": "/ref.png", "media_type": "image"},
|
||||
{"source": "/frames/one.png", "media_type": "image"},
|
||||
]
|
||||
|
||||
def test_leaves_the_stored_references_untouched(self):
|
||||
jobs = self._jobs(one="completed")
|
||||
stored = after("one")
|
||||
resolve_references(stored, jobs.get, lambda job: "/f.png")
|
||||
assert stored == after("one")
|
||||
|
||||
def test_no_references(self):
|
||||
assert resolve_references(None, {}.get, lambda job: "") == []
|
||||
|
||||
def test_refuses_an_unfinished_source(self):
|
||||
jobs = self._jobs(one="running")
|
||||
with pytest.raises(UnresolvedReference, match="hasn't finished.*running"):
|
||||
resolve_references(after("one"), jobs.get, lambda job: "/f.png")
|
||||
|
||||
def test_refuses_a_deleted_source(self):
|
||||
with pytest.raises(UnresolvedReference, match="no longer exists"):
|
||||
resolve_references(after("gone"), {}.get, lambda job: "/f.png")
|
||||
|
||||
def test_passes_on_the_extraction_error(self):
|
||||
class Boom(Exception):
|
||||
detail = "The video has no frames."
|
||||
|
||||
def fail(job):
|
||||
raise Boom()
|
||||
|
||||
with pytest.raises(UnresolvedReference, match="no frames"):
|
||||
resolve_references(after("one"), self._jobs(one="completed").get, fail)
|
||||
|
||||
|
||||
def after_clip(job_id: str) -> list[dict[str, str]]:
|
||||
return [{"source": deferred_last_clip_source(job_id), "media_type": "video"}]
|
||||
|
||||
|
||||
class TestResolveClipReferences:
|
||||
"""The same resolution, but for a trailing-clip (job-last-clip:) reference."""
|
||||
|
||||
def _jobs(self, **statuses: str) -> dict[str, J]:
|
||||
return {k: J(k, v, name=f"Clip {k}") for k, v in statuses.items()}
|
||||
|
||||
def test_uses_the_clip_resolver_not_the_frame_one(self):
|
||||
jobs = self._jobs(one="completed")
|
||||
stored = after_clip("one")
|
||||
resolved = resolve_references(
|
||||
stored, jobs.get,
|
||||
last_frame=lambda job: f"/frames/{job.id}.png",
|
||||
last_clip=lambda job: f"/clips/{job.id}.mp4",
|
||||
)
|
||||
assert resolved == [{"source": "/clips/one.mp4", "media_type": "video"}]
|
||||
|
||||
def test_a_mix_of_frame_and_clip_references_each_use_their_own_resolver(self):
|
||||
jobs = self._jobs(a="completed", b="completed")
|
||||
stored = [*after("a"), *after_clip("b"), {"source": "/x.png", "media_type": "image"}]
|
||||
resolved = resolve_references(
|
||||
stored, jobs.get,
|
||||
last_frame=lambda job: f"/frames/{job.id}.png",
|
||||
last_clip=lambda job: f"/clips/{job.id}.mp4",
|
||||
)
|
||||
assert resolved == [
|
||||
{"source": "/frames/a.png", "media_type": "image"},
|
||||
{"source": "/clips/b.mp4", "media_type": "video"},
|
||||
{"source": "/x.png", "media_type": "image"},
|
||||
]
|
||||
|
||||
def test_refuses_an_unfinished_source(self):
|
||||
jobs = self._jobs(one="running")
|
||||
with pytest.raises(UnresolvedReference, match="hasn't finished.*running"):
|
||||
resolve_references(after_clip("one"), jobs.get, last_frame=lambda j: "", last_clip=lambda j: "/c.mp4")
|
||||
|
||||
def test_refuses_a_deleted_source(self):
|
||||
with pytest.raises(UnresolvedReference, match="no longer exists"):
|
||||
resolve_references(after_clip("gone"), {}.get, last_frame=lambda j: "", last_clip=lambda j: "/c.mp4")
|
||||
|
||||
def test_without_a_clip_resolver_a_clip_reference_is_refused_not_ignored(self):
|
||||
jobs = self._jobs(one="completed")
|
||||
with pytest.raises(UnresolvedReference, match="can't resolve"):
|
||||
resolve_references(after_clip("one"), jobs.get, last_frame=lambda j: "/f.png")
|
||||
|
||||
def test_dependencies_and_circular_checks_see_clip_references_too(self):
|
||||
jobs = self._jobs(a="pending")
|
||||
jobs["a"].references = after_clip("a")
|
||||
assert dependency_of(after_clip("x")[0]) == "x"
|
||||
assert dependencies(after_clip("a") + after("a")) == ["a"]
|
||||
assert circular_dependency(jobs, "a") is not None
|
||||
@@ -0,0 +1,768 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""JobRunner's queue: order, concurrency, last-frame dependencies, and surviving a restart.
|
||||
|
||||
Runs without a GPU: `fastvideo` is stubbed (importing it needs a working triton
|
||||
driver) and jobs "run" by waiting on an event instead of generating video.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import collections
|
||||
import importlib.util
|
||||
import sys
|
||||
import threading
|
||||
import time
|
||||
import types
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from fastvideo_studio.database import Database
|
||||
from fastvideo_studio.frames import FrameError
|
||||
from fastvideo_studio.job_queue import deferred_last_frame_source
|
||||
|
||||
RUNNER_PATH = Path(__file__).resolve().parents[1] / "job_runner.py"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def job_runner_module(monkeypatch):
|
||||
class Manager:
|
||||
def shutdown(self):
|
||||
pass
|
||||
|
||||
stub_utils = types.ModuleType("fastvideo.utils")
|
||||
stub_utils.get_mp_context = lambda: types.SimpleNamespace(Manager=Manager)
|
||||
stub_pkg = types.ModuleType("fastvideo")
|
||||
stub_pkg.utils = stub_utils
|
||||
monkeypatch.setitem(sys.modules, "fastvideo", stub_pkg)
|
||||
monkeypatch.setitem(sys.modules, "fastvideo.utils", stub_utils)
|
||||
|
||||
spec = importlib.util.spec_from_file_location("job_runner_under_test", RUNNER_PATH)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
monkeypatch.setitem(sys.modules, spec.name, module) # @dataclass looks its module up here
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
class Harness:
|
||||
"""Builds runners whose jobs finish only when the test says so."""
|
||||
|
||||
def __init__(self, module, tmp_path, monkeypatch):
|
||||
self.module = module
|
||||
self.tmp_path = tmp_path
|
||||
self.gates: dict[str, threading.Event] = collections.defaultdict(threading.Event)
|
||||
self.outcomes: dict[str, object] = {}
|
||||
self.ran: list[str] = []
|
||||
self.runners = []
|
||||
self.thread_of: dict[str, threading.Thread] = {}
|
||||
self.closed = False
|
||||
|
||||
harness = self
|
||||
|
||||
def fake_run(runner_self, job):
|
||||
harness.thread_of[job.id] = threading.current_thread()
|
||||
harness.ran.append(job.id)
|
||||
if harness.closed:
|
||||
harness.gates[job.id].set()
|
||||
harness.gates[job.id].wait(10)
|
||||
job.status = harness.outcomes.get(job.id, module.JobStatus.COMPLETED)
|
||||
if job.status == module.JobStatus.FAILED:
|
||||
job.error = "boom"
|
||||
job.finished_at = time.time()
|
||||
runner_self._save_job(job)
|
||||
|
||||
monkeypatch.setattr(module.JobRunner, "_run_job", fake_run)
|
||||
|
||||
def runner(self, **kwargs):
|
||||
runner = self.module.JobRunner(
|
||||
output_dir=str(self.tmp_path / "out"),
|
||||
log_dir=str(self.tmp_path / "logs"),
|
||||
database=Database(self.tmp_path / "t.db"),
|
||||
upload_dir=str(self.tmp_path / "uploads"),
|
||||
**kwargs,
|
||||
)
|
||||
self.runners.append(runner)
|
||||
return runner
|
||||
|
||||
def make(self, runner, name, **over):
|
||||
kwargs = dict(job_id=str(uuid.uuid4()), model_id="m", name=name, prompt="p")
|
||||
kwargs.update(over)
|
||||
return runner.create_job(**kwargs)
|
||||
|
||||
def finish(self, job, status=None):
|
||||
if status is not None:
|
||||
self.outcomes[job.id] = status
|
||||
self.gates[job.id].set()
|
||||
|
||||
def release_all(self):
|
||||
for gate in self.gates.values():
|
||||
gate.set()
|
||||
|
||||
|
||||
def wait_for(condition, timeout=5.0):
|
||||
deadline = time.time() + timeout
|
||||
while time.time() < deadline:
|
||||
if condition():
|
||||
return True
|
||||
time.sleep(0.01)
|
||||
return False
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def h(job_runner_module, tmp_path, monkeypatch):
|
||||
harness = Harness(job_runner_module, tmp_path, monkeypatch)
|
||||
yield harness
|
||||
# Let every job (and the queue's follow-up launches) finish while `_run_job` is
|
||||
# still patched, so no stray thread starts a real job after the test is over.
|
||||
harness.closed = True
|
||||
active = ("queued", "running")
|
||||
for runner in harness.runners:
|
||||
for _ in range(100):
|
||||
harness.release_all()
|
||||
if not any(j.status.value in active for j in runner.list_jobs()):
|
||||
break
|
||||
time.sleep(0.05)
|
||||
runner._shutdown() # stops the worker threads
|
||||
|
||||
|
||||
def statuses(runner, *jobs):
|
||||
return [runner.get_job(j.id).status.value for j in jobs]
|
||||
|
||||
|
||||
def frame_of(job):
|
||||
return [{"source": deferred_last_frame_source(job.id), "media_type": "image"}]
|
||||
|
||||
|
||||
class TestOrderAndConcurrency:
|
||||
def test_one_job_runs_at_a_time_in_the_order_queued(self, h):
|
||||
r = h.runner()
|
||||
a, b, c = (h.make(r, n) for n in "abc")
|
||||
r.enqueue_jobs([b.id, c.id, a.id])
|
||||
|
||||
assert wait_for(lambda: statuses(r, a, b, c) == ["queued", "running", "queued"])
|
||||
h.finish(b)
|
||||
assert wait_for(lambda: statuses(r, a, b, c) == ["queued", "completed", "running"])
|
||||
h.finish(c)
|
||||
assert wait_for(lambda: statuses(r, a, b, c) == ["running", "completed", "completed"])
|
||||
h.finish(a)
|
||||
assert wait_for(lambda: statuses(r, a, b, c) == ["completed"] * 3)
|
||||
assert h.ran == [b.id, c.id, a.id]
|
||||
|
||||
def test_runs_as_many_at_once_as_allowed(self, h):
|
||||
r = h.runner(max_concurrent_jobs=2)
|
||||
a, b, c = (h.make(r, n) for n in "abc")
|
||||
r.enqueue_jobs([a.id, b.id, c.id])
|
||||
assert wait_for(lambda: statuses(r, a, b, c) == ["running", "running", "queued"])
|
||||
h.finish(a)
|
||||
assert wait_for(lambda: statuses(r, a, b, c) == ["completed", "running", "running"])
|
||||
|
||||
def test_a_failed_job_does_not_stop_the_queue(self, h):
|
||||
r = h.runner()
|
||||
a, b = h.make(r, "a"), h.make(r, "b")
|
||||
r.enqueue_jobs([a.id, b.id])
|
||||
h.finish(a, h.module.JobStatus.FAILED)
|
||||
assert wait_for(lambda: statuses(r, a, b) == ["failed", "running"])
|
||||
|
||||
def test_queued_jobs_have_a_queue_time_and_running_ones_do_not(self, h):
|
||||
r = h.runner()
|
||||
a, b = h.make(r, "a"), h.make(r, "b")
|
||||
r.enqueue_jobs([a.id, b.id])
|
||||
assert wait_for(lambda: r.get_job(a.id).status.value == "running")
|
||||
assert r.get_job(a.id).queued_at is None
|
||||
assert r.get_job(b.id).queued_at is not None
|
||||
assert r.get_job(b.id).to_dict()["status"] == "queued"
|
||||
|
||||
|
||||
class TestLongLivedWorkers:
|
||||
"""Model workers die with the thread that started them, so inference jobs share long-lived threads."""
|
||||
|
||||
def _run_all(self, h, r, n):
|
||||
jobs = [h.make(r, f"j{i}") for i in range(n)]
|
||||
for job in jobs:
|
||||
h.gates[job.id].set() # each finishes as soon as it starts
|
||||
r.enqueue_jobs([j.id for j in jobs])
|
||||
assert wait_for(lambda: statuses(r, *jobs) == ["completed"] * n)
|
||||
return jobs
|
||||
|
||||
def test_consecutive_jobs_run_on_the_same_thread_and_it_stays_alive(self, h):
|
||||
r = h.runner()
|
||||
jobs = self._run_all(h, r, 4)
|
||||
threads = {h.thread_of[j.id] for j in jobs}
|
||||
assert len(threads) == 1
|
||||
assert threads.pop().is_alive()
|
||||
|
||||
def test_the_thread_is_not_one_made_for_the_job(self, h):
|
||||
r = h.runner()
|
||||
(job,) = self._run_all(h, r, 1)
|
||||
assert h.thread_of[job.id].name.startswith("inference-worker")
|
||||
|
||||
def test_as_many_workers_as_jobs_allowed_at_once(self, h):
|
||||
r = h.runner(max_concurrent_jobs=2)
|
||||
jobs = self._run_all(h, r, 6)
|
||||
assert len({h.thread_of[j.id] for j in jobs}) <= 2
|
||||
|
||||
def test_two_jobs_at_once_use_two_different_workers(self, h):
|
||||
r = h.runner(max_concurrent_jobs=2)
|
||||
a, b = h.make(r, "a"), h.make(r, "b")
|
||||
r.enqueue_jobs([a.id, b.id])
|
||||
assert wait_for(lambda: statuses(r, a, b) == ["running", "running"])
|
||||
assert wait_for(lambda: len(h.thread_of) == 2)
|
||||
assert h.thread_of[a.id] is not h.thread_of[b.id]
|
||||
|
||||
def test_a_job_that_blows_up_does_not_kill_its_worker(self, h):
|
||||
r = h.runner()
|
||||
bad, good = h.make(r, "bad"), h.make(r, "good")
|
||||
original = h.module.JobRunner._run_job
|
||||
|
||||
def explode_once(self, job):
|
||||
if job.id == bad.id:
|
||||
raise RuntimeError("boom")
|
||||
return original(self, job)
|
||||
|
||||
h.module.JobRunner._run_job = explode_once
|
||||
h.gates[good.id].set()
|
||||
r.enqueue_jobs([bad.id, good.id])
|
||||
assert wait_for(lambda: statuses(r, bad, good) == ["failed", "completed"])
|
||||
assert "boom" in r.get_job(bad.id).error
|
||||
|
||||
def test_a_direct_start_is_refused_while_the_workers_are_busy(self, h):
|
||||
r = h.runner()
|
||||
first, second = h.make(r, "first"), h.make(r, "second")
|
||||
r.start_job(first.id)
|
||||
with pytest.raises(ValueError, match="already running.*Queue this job"):
|
||||
r.start_job(second.id)
|
||||
assert r.get_job(second.id).status.value == "pending"
|
||||
|
||||
def test_a_chain_runs_clean_when_every_clip_finishes_instantly(self, h):
|
||||
r = h.runner()
|
||||
clips = []
|
||||
for i in range(6):
|
||||
clips.append(h.make(r, f"clip {i}", references=frame_of(clips[-1]) if clips else []))
|
||||
for clip in clips:
|
||||
h.gates[clip.id].set()
|
||||
r.enqueue_jobs([c.id for c in clips])
|
||||
assert wait_for(lambda: statuses(r, *clips) == ["completed"] * 6)
|
||||
assert h.ran == [c.id for c in clips]
|
||||
|
||||
|
||||
class TestGeneratorCache:
|
||||
@pytest.fixture
|
||||
def generators(self, job_runner_module, monkeypatch):
|
||||
made = []
|
||||
|
||||
class FakeGenerator:
|
||||
def __init__(self, model_id):
|
||||
self.model_id = model_id
|
||||
self.shut_down = False
|
||||
made.append(self)
|
||||
|
||||
def shutdown(self):
|
||||
self.shut_down = True
|
||||
|
||||
stub = types.SimpleNamespace(from_pretrained=lambda model_id, *a, **k: FakeGenerator(model_id))
|
||||
monkeypatch.setattr(sys.modules["fastvideo"], "VideoGenerator", stub, raising=False)
|
||||
return made
|
||||
|
||||
def _get(self, runner, model="m"):
|
||||
return runner._get_or_create_generator(model, "i2v", 1)
|
||||
|
||||
def _on_thread(self, fn):
|
||||
box = []
|
||||
t = threading.Thread(target=lambda: box.append(fn()))
|
||||
t.start()
|
||||
t.join()
|
||||
return box[0]
|
||||
|
||||
def test_the_thread_that_built_it_keeps_reusing_it(self, h, generators):
|
||||
r = h.runner()
|
||||
first = self._get(r)
|
||||
assert self._get(r) is first
|
||||
assert len(generators) == 1
|
||||
|
||||
def test_a_generator_from_an_exited_thread_is_dropped_and_replaced(self, h, generators):
|
||||
r = h.runner()
|
||||
old = self._on_thread(lambda: self._get(r)) # that thread has exited: its workers are gone
|
||||
new = self._get(r)
|
||||
assert new is not old
|
||||
assert old.shut_down
|
||||
assert len(generators) == 2
|
||||
|
||||
def test_threads_do_not_share_generators(self, h, generators):
|
||||
# Sharing would break the second the moment the first one's thread ends.
|
||||
r = h.runner()
|
||||
mine = self._get(r)
|
||||
other = self._on_thread(lambda: self._get(r))
|
||||
assert other is not mine
|
||||
assert not mine.shut_down # still owned by a live thread
|
||||
|
||||
def test_a_different_model_replaces_the_loaded_one_instead_of_sitting_beside_it(self, h, generators):
|
||||
r = h.runner()
|
||||
first = self._get(r, "model-a")
|
||||
second = self._get(r, "model-b")
|
||||
assert first.shut_down # its GPU memory is freed before the next load
|
||||
assert not second.shut_down
|
||||
assert self._get(r, "model-b") is second
|
||||
|
||||
def test_switching_back_reloads(self, h, generators):
|
||||
r = h.runner()
|
||||
a1 = self._get(r, "model-a")
|
||||
self._get(r, "model-b")
|
||||
a2 = self._get(r, "model-a")
|
||||
assert a2 is not a1
|
||||
|
||||
def test_discarding_unloads_this_threads_models_only(self, h, generators):
|
||||
r = h.runner()
|
||||
mine = self._get(r)
|
||||
gate, done = threading.Event(), threading.Event()
|
||||
theirs = []
|
||||
|
||||
def other_thread():
|
||||
theirs.append(self._get(r, "other"))
|
||||
done.set()
|
||||
gate.wait(5)
|
||||
|
||||
t = threading.Thread(target=other_thread)
|
||||
t.start()
|
||||
done.wait(5)
|
||||
r._discard_generators()
|
||||
gate.set()
|
||||
t.join()
|
||||
assert mine.shut_down and not theirs[0].shut_down
|
||||
assert self._get(r) is not mine # reloads fresh next time
|
||||
|
||||
|
||||
class TestLastFrameChains:
|
||||
def _chain(self, h, r, n=3):
|
||||
clips = []
|
||||
for i in range(n):
|
||||
refs = frame_of(clips[-1]) if clips else []
|
||||
clips.append(h.make(r, f"clip {i + 1}", references=refs))
|
||||
return clips
|
||||
|
||||
def test_a_scene_runs_clip_by_clip_however_many_slots_there_are(self, h):
|
||||
r = h.runner(max_concurrent_jobs=4)
|
||||
c1, c2, c3 = self._chain(h, r)
|
||||
r.enqueue_jobs([c1.id, c2.id, c3.id])
|
||||
|
||||
assert wait_for(lambda: statuses(r, c1, c2, c3) == ["running", "queued", "queued"])
|
||||
time.sleep(0.05)
|
||||
assert statuses(r, c1, c2, c3) == ["running", "queued", "queued"] # spare slots stay unused
|
||||
h.finish(c1)
|
||||
assert wait_for(lambda: statuses(r, c1, c2, c3) == ["completed", "running", "queued"])
|
||||
h.finish(c2)
|
||||
assert wait_for(lambda: statuses(r, c1, c2, c3) == ["completed", "completed", "running"])
|
||||
|
||||
def test_a_clip_does_not_block_unrelated_jobs_behind_it(self, h):
|
||||
r = h.runner(max_concurrent_jobs=2)
|
||||
c1, c2 = self._chain(h, r, 2)
|
||||
other = h.make(r, "other")
|
||||
r.enqueue_jobs([c1.id, c2.id, other.id])
|
||||
assert wait_for(lambda: statuses(r, c1, c2, other) == ["running", "queued", "running"])
|
||||
|
||||
def test_a_failed_clip_fails_the_clips_after_it_and_says_why(self, h):
|
||||
r = h.runner()
|
||||
c1, c2, c3 = self._chain(h, r)
|
||||
r.enqueue_jobs([c1.id, c2.id, c3.id])
|
||||
h.finish(c1, h.module.JobStatus.FAILED)
|
||||
|
||||
assert wait_for(lambda: statuses(r, c1, c2, c3) == ["failed"] * 3)
|
||||
assert "clip 1" in r.get_job(c2.id).error and "failed" in r.get_job(c2.id).error
|
||||
assert "clip 2" in r.get_job(c3.id).error
|
||||
assert h.ran == [c1.id]
|
||||
|
||||
def test_a_stopped_clip_fails_the_clips_after_it(self, h):
|
||||
r = h.runner()
|
||||
c1, c2 = self._chain(h, r, 2)
|
||||
r.enqueue_jobs([c1.id, c2.id])
|
||||
h.finish(c1, h.module.JobStatus.STOPPED)
|
||||
assert wait_for(lambda: statuses(r, c1, c2) == ["stopped", "failed"])
|
||||
assert "stopped" in r.get_job(c2.id).error
|
||||
|
||||
def test_deleting_a_queued_clip_fails_the_clips_after_it(self, h):
|
||||
r = h.runner()
|
||||
blocker = h.make(r, "blocker")
|
||||
c1, c2 = self._chain(h, r, 2)
|
||||
r.enqueue_jobs([blocker.id, c1.id, c2.id])
|
||||
assert wait_for(lambda: statuses(r, blocker, c1, c2) == ["running", "queued", "queued"])
|
||||
assert r.delete_job(c1.id)
|
||||
assert wait_for(lambda: r.get_job(c2.id).status.value == "failed")
|
||||
assert "deleted" in r.get_job(c2.id).error
|
||||
|
||||
def test_a_failed_clip_can_be_fixed_and_the_scene_requeued(self, h):
|
||||
r = h.runner()
|
||||
c1, c2 = self._chain(h, r, 2)
|
||||
r.enqueue_jobs([c1.id, c2.id])
|
||||
h.finish(c1, h.module.JobStatus.FAILED)
|
||||
assert wait_for(lambda: statuses(r, c1, c2) == ["failed", "failed"])
|
||||
|
||||
h.outcomes.pop(c1.id)
|
||||
h.gates[c1.id].clear()
|
||||
r.enqueue_jobs([c1.id, c2.id])
|
||||
assert wait_for(lambda: statuses(r, c1, c2) == ["running", "queued"])
|
||||
h.finish(c1)
|
||||
assert wait_for(lambda: statuses(r, c1, c2) == ["completed", "running"])
|
||||
|
||||
def test_the_stored_reference_stays_deferred(self, h):
|
||||
r = h.runner()
|
||||
c1, c2 = self._chain(h, r, 2)
|
||||
r.enqueue_jobs([c1.id, c2.id])
|
||||
h.finish(c1)
|
||||
assert wait_for(lambda: r.get_job(c2.id).status.value == "running")
|
||||
assert r.get_job(c2.id).references == frame_of(c1)
|
||||
|
||||
|
||||
class TestStartingDirectly:
|
||||
def test_refuses_while_the_frame_source_is_unfinished(self, h):
|
||||
r = h.runner()
|
||||
c1, c2 = h.make(r, "clip 1"), None
|
||||
c2 = h.make(r, "clip 2", references=frame_of(c1))
|
||||
with pytest.raises(ValueError, match="clip 1.*hasn't finished.*pending"):
|
||||
r.start_job(c2.id)
|
||||
assert r.get_job(c2.id).status.value == "pending"
|
||||
assert h.ran == []
|
||||
|
||||
def test_starts_once_the_frame_source_completed(self, h):
|
||||
r = h.runner()
|
||||
c1 = h.make(r, "clip 1")
|
||||
c1.status = h.module.JobStatus.COMPLETED
|
||||
c2 = h.make(r, "clip 2", references=frame_of(c1))
|
||||
r.start_job(c2.id)
|
||||
assert r.get_job(c2.id).status.value == "running"
|
||||
|
||||
def test_refuses_when_the_frame_source_was_deleted(self, h):
|
||||
r = h.runner()
|
||||
c1 = h.make(r, "clip 1")
|
||||
c2 = h.make(r, "clip 2", references=frame_of(c1))
|
||||
r.delete_job(c1.id)
|
||||
with pytest.raises(ValueError, match="no longer exists"):
|
||||
r.start_job(c2.id)
|
||||
|
||||
def test_a_direct_start_takes_a_queue_slot(self, h):
|
||||
r = h.runner()
|
||||
running, queued = h.make(r, "running"), h.make(r, "queued")
|
||||
r.start_job(running.id)
|
||||
r.enqueue_job(queued.id)
|
||||
time.sleep(0.05)
|
||||
assert statuses(r, running, queued) == ["running", "queued"]
|
||||
h.finish(running)
|
||||
assert wait_for(lambda: r.get_job(queued.id).status.value == "running")
|
||||
|
||||
def test_a_queued_job_cannot_also_be_started(self, h):
|
||||
r = h.runner()
|
||||
blocker, waiting = h.make(r, "blocker"), h.make(r, "waiting")
|
||||
r.enqueue_jobs([blocker.id, waiting.id])
|
||||
with pytest.raises(ValueError, match="already queued"):
|
||||
r.start_job(waiting.id)
|
||||
|
||||
|
||||
class TestQueueManagement:
|
||||
def test_dequeue_returns_a_job_to_pending_and_lets_others_go(self, h):
|
||||
r = h.runner()
|
||||
a, b, c = (h.make(r, n) for n in "abc")
|
||||
r.enqueue_jobs([a.id, b.id, c.id])
|
||||
assert wait_for(lambda: r.get_job(a.id).status.value == "running")
|
||||
r.dequeue_job(b.id)
|
||||
assert r.get_job(b.id).status.value == "pending"
|
||||
assert r.get_job(b.id).queued_at is None
|
||||
h.finish(a)
|
||||
assert wait_for(lambda: statuses(r, a, b, c) == ["completed", "pending", "running"])
|
||||
|
||||
def test_stopping_a_queued_job_dequeues_it(self, h):
|
||||
r = h.runner()
|
||||
a, b = h.make(r, "a"), h.make(r, "b")
|
||||
r.enqueue_jobs([a.id, b.id])
|
||||
assert wait_for(lambda: r.get_job(a.id).status.value == "running")
|
||||
assert r.stop_job(b.id).status.value == "pending"
|
||||
|
||||
def test_dequeue_of_a_job_not_queued_is_refused(self, h):
|
||||
r = h.runner()
|
||||
a = h.make(r, "a")
|
||||
with pytest.raises(ValueError, match="not queued"):
|
||||
r.dequeue_job(a.id)
|
||||
with pytest.raises(ValueError, match="not found"):
|
||||
r.dequeue_job("nope")
|
||||
|
||||
def test_enqueue_is_all_or_nothing(self, h):
|
||||
r = h.runner()
|
||||
a, done = h.make(r, "a"), h.make(r, "done")
|
||||
done.status = h.module.JobStatus.COMPLETED
|
||||
with pytest.raises(ValueError, match="already completed"):
|
||||
r.enqueue_jobs([a.id, done.id])
|
||||
assert r.get_job(a.id).status.value == "pending"
|
||||
with pytest.raises(ValueError, match="not found"):
|
||||
r.enqueue_jobs([a.id, "nope"])
|
||||
assert h.ran == []
|
||||
|
||||
def test_a_job_cannot_be_queued_twice(self, h):
|
||||
r = h.runner()
|
||||
blocker, a = h.make(r, "blocker"), h.make(r, "a")
|
||||
r.enqueue_jobs([blocker.id, a.id])
|
||||
with pytest.raises(ValueError, match="already queued"):
|
||||
r.enqueue_job(a.id)
|
||||
with pytest.raises(ValueError, match="more than once"):
|
||||
r.enqueue_jobs([a.id, a.id])
|
||||
|
||||
def test_loops_of_last_frames_are_refused(self, h):
|
||||
r = h.runner()
|
||||
a = h.make(r, "a")
|
||||
b = h.make(r, "b", references=frame_of(a))
|
||||
r.update_job_config(a.id, {"references": frame_of(b)})
|
||||
with pytest.raises(ValueError, match="loop"):
|
||||
r.enqueue_jobs([a.id, b.id])
|
||||
assert statuses(r, a, b) == ["pending", "pending"]
|
||||
|
||||
def test_queued_jobs_cannot_be_edited(self, h):
|
||||
r = h.runner()
|
||||
blocker, a = h.make(r, "blocker"), h.make(r, "a")
|
||||
r.enqueue_jobs([blocker.id, a.id])
|
||||
with pytest.raises(ValueError, match="queued"):
|
||||
r.update_job_config(a.id, {"prompt": "changed"})
|
||||
|
||||
def test_requeueing_clears_the_previous_error(self, h):
|
||||
r = h.runner()
|
||||
a = h.make(r, "a")
|
||||
h.finish(a, h.module.JobStatus.FAILED)
|
||||
r.enqueue_job(a.id)
|
||||
assert wait_for(lambda: r.get_job(a.id).status.value == "failed")
|
||||
h.outcomes.pop(a.id)
|
||||
h.gates[a.id].clear()
|
||||
r.enqueue_job(a.id)
|
||||
assert wait_for(lambda: r.get_job(a.id).status.value == "running")
|
||||
assert r.get_job(a.id).error is None
|
||||
|
||||
|
||||
class TestRestart:
|
||||
def test_queued_jobs_resume_in_order_after_a_restart(self, h, job_runner_module):
|
||||
first = h.runner()
|
||||
blocker = h.make(first, "blocker")
|
||||
a, b = h.make(first, "a"), h.make(first, "b")
|
||||
first.enqueue_jobs([blocker.id, b.id, a.id])
|
||||
assert wait_for(lambda: h.ran == [blocker.id]) # actually started, not just handed to a worker
|
||||
|
||||
h.ran.clear()
|
||||
second = h.runner() # same database
|
||||
# The job that was running is failed, exactly as before queues existed...
|
||||
assert second.get_job(blocker.id).status.value == "failed"
|
||||
assert "restarted" in second.get_job(blocker.id).error
|
||||
# ...and the queue carries on where it was.
|
||||
assert wait_for(lambda: h.ran[:1] == [b.id])
|
||||
assert second.get_job(a.id).status.value == "queued"
|
||||
|
||||
def test_edits_survive_a_restart(self, h):
|
||||
first = h.runner()
|
||||
job = h.make(first, "a", fps=24, negative_prompt="neg")
|
||||
first.update_job_config(job.id, {"prompt": "edited", "references": frame_of(job), "fps": 12})
|
||||
|
||||
second = h.runner()
|
||||
restored = second.get_job(job.id)
|
||||
assert restored.prompt == "edited"
|
||||
assert restored.references == frame_of(job)
|
||||
assert restored.fps == 12
|
||||
assert restored.negative_prompt == "neg"
|
||||
|
||||
|
||||
def seg(brightness=0.0, contrast=1.0, saturation=1.0, end_seconds=None):
|
||||
"""One razor-cut section dict, as trim_job()/the API expects."""
|
||||
return {"end_seconds": end_seconds, "brightness": brightness, "contrast": contrast, "saturation": saturation}
|
||||
|
||||
|
||||
class TestTrimming:
|
||||
"""trim_job/restore_job_video against a real (small, synthetic) video file."""
|
||||
|
||||
FPS = 24
|
||||
|
||||
def _write_video(self, path, n=48):
|
||||
import av
|
||||
import numpy as np
|
||||
|
||||
Path(path).parent.mkdir(parents=True, exist_ok=True)
|
||||
container = av.open(path, mode="w")
|
||||
try:
|
||||
stream = container.add_stream("h264", rate=self.FPS)
|
||||
stream.width, stream.height, stream.pix_fmt = 64, 48, "yuv420p"
|
||||
for i in range(n):
|
||||
array = np.full((48, 64, 3), min(i * 5, 255), dtype=np.uint8)
|
||||
for packet in stream.encode(av.VideoFrame.from_ndarray(array, format="rgb24")):
|
||||
container.mux(packet)
|
||||
for packet in stream.encode():
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
def _completed(self, h, r, n=48):
|
||||
job = h.make(r, "clip")
|
||||
r.start_job(job.id)
|
||||
h.finish(job)
|
||||
assert wait_for(lambda: r.get_job(job.id).status.value == "completed")
|
||||
live = r.get_job(job.id)
|
||||
path = str(h.tmp_path / "videos" / f"{job.id}.mp4")
|
||||
self._write_video(path, n)
|
||||
live.output_path = path
|
||||
live.num_frames = n
|
||||
return live
|
||||
|
||||
def test_trim_updates_output_path_and_frame_count(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r) # 2s @ 24fps
|
||||
original_path = job.output_path
|
||||
trimmed = r.trim_job(job.id, start_seconds=0.5, end_seconds=1.5)
|
||||
assert trimmed.output_path != original_path
|
||||
assert trimmed.output_path.endswith("edited.mp4")
|
||||
assert trimmed.num_frames == self.FPS # 1s kept
|
||||
|
||||
def test_grading_updates_output_path_without_changing_frame_count(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
graded = r.trim_job(job.id, start_seconds=0.0, end_seconds=None, segments=[seg(brightness=40.0)])
|
||||
assert graded.output_path.endswith("edited.mp4")
|
||||
assert graded.num_frames == self.FPS * 2 # full length kept, only color changed
|
||||
|
||||
def test_grading_and_a_trim_requested_together_both_take_effect(self, h):
|
||||
import av
|
||||
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
edited = r.trim_job(job.id, start_seconds=0.5, end_seconds=1.5, segments=[seg(brightness=80.0)])
|
||||
assert edited.num_frames == self.FPS # the range applied
|
||||
with av.open(edited.output_path) as c:
|
||||
frame = next(c.decode(c.streams.video[0]))
|
||||
# Source frame at 0.5s (index 12) is level 60; +80 brightness should clearly lighten it.
|
||||
assert int(frame.to_ndarray(format="rgb24")[0, 0, 0]) > 60 + 40
|
||||
|
||||
def test_two_sections_grade_independently(self, h):
|
||||
import av
|
||||
|
||||
r = h.runner()
|
||||
job = self._completed(h, r) # 48 frames @ 24fps = 2s
|
||||
edited = r.trim_job(
|
||||
job.id, start_seconds=0.0, end_seconds=None,
|
||||
segments=[seg(brightness=80.0, end_seconds=1.0), seg()],
|
||||
)
|
||||
with av.open(edited.output_path) as c:
|
||||
frames = list(c.decode(c.streams.video[0]))
|
||||
first_half_level = int(frames[0].to_ndarray(format="rgb24")[0, 0, 0])
|
||||
second_half_level = int(frames[-1].to_ndarray(format="rgb24")[0, 0, 0])
|
||||
assert first_half_level > 0 + 40 # source level 0, graded section
|
||||
assert second_half_level == min(47 * 5, 255) # source's actual last-frame level, untouched
|
||||
|
||||
def test_trim_persists_across_a_restart(self, h):
|
||||
first = h.runner()
|
||||
job = self._completed(h, first)
|
||||
trimmed = first.trim_job(job.id, start_seconds=0.5, end_seconds=1.5)
|
||||
|
||||
second = h.runner()
|
||||
restored = second.get_job(job.id)
|
||||
assert restored.output_path == trimmed.output_path
|
||||
assert restored.num_frames == trimmed.num_frames
|
||||
|
||||
def test_trim_records_the_applied_values_on_the_job(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
edited = r.trim_job(
|
||||
job.id, start_seconds=0.5, end_seconds=1.5,
|
||||
segments=[seg(brightness=20.0, contrast=1.4, saturation=0.5)],
|
||||
)
|
||||
assert edited.edit_start_seconds == 0.5
|
||||
assert edited.edit_end_seconds == 1.5
|
||||
assert edited.edit_segments == [seg(brightness=20.0, contrast=1.4, saturation=0.5)]
|
||||
|
||||
def test_a_second_trim_overwrites_the_recorded_values_of_the_first(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
r.trim_job(job.id, start_seconds=0.5, end_seconds=1.5, segments=[seg(brightness=20.0)])
|
||||
again = r.trim_job(job.id, start_seconds=0.0, end_seconds=None, segments=[seg(contrast=1.4)])
|
||||
assert again.edit_start_seconds == 0.0
|
||||
assert again.edit_end_seconds is None
|
||||
assert again.edit_segments == [seg(contrast=1.4)] # not carried over from the first trim
|
||||
|
||||
def test_the_recorded_edit_values_persist_across_a_restart(self, h):
|
||||
first = h.runner()
|
||||
job = self._completed(h, first)
|
||||
first.trim_job(
|
||||
job.id, start_seconds=0.5, end_seconds=1.5,
|
||||
segments=[seg(brightness=20.0, contrast=1.4, saturation=0.5)],
|
||||
)
|
||||
|
||||
second = h.runner()
|
||||
restored = second.get_job(job.id)
|
||||
assert restored.edit_start_seconds == 0.5
|
||||
assert restored.edit_end_seconds == 1.5
|
||||
assert restored.edit_segments == [seg(brightness=20.0, contrast=1.4, saturation=0.5)]
|
||||
|
||||
def test_a_multi_section_edit_persists_across_a_restart(self, h):
|
||||
first = h.runner()
|
||||
job = self._completed(h, first)
|
||||
sections = [seg(brightness=20.0, end_seconds=0.5), seg(contrast=1.4)]
|
||||
first.trim_job(job.id, start_seconds=0.0, end_seconds=None, segments=sections)
|
||||
|
||||
second = h.runner()
|
||||
restored = second.get_job(job.id)
|
||||
assert restored.edit_segments == sections
|
||||
|
||||
def test_restore_resets_the_recorded_edit_values_too(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
r.trim_job(
|
||||
job.id, start_seconds=0.5, end_seconds=1.5,
|
||||
segments=[seg(brightness=20.0, contrast=1.4, saturation=0.5)],
|
||||
)
|
||||
restored = r.restore_job_video(job.id)
|
||||
assert restored.edit_start_seconds == 0.0
|
||||
assert restored.edit_end_seconds is None
|
||||
assert restored.edit_segments == []
|
||||
|
||||
def test_a_never_edited_job_has_neutral_recorded_values(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
assert job.edit_start_seconds == 0.0
|
||||
assert job.edit_end_seconds is None
|
||||
assert job.edit_segments == []
|
||||
|
||||
def test_trimming_again_is_relative_to_the_original_not_the_last_trim(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
r.trim_job(job.id, start_seconds=0.5, end_seconds=1.5)
|
||||
again = r.trim_job(job.id, start_seconds=0.0, end_seconds=2.0)
|
||||
assert again.num_frames == self.FPS * 2
|
||||
|
||||
def test_restore_undoes_a_trim(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
r.trim_job(job.id, start_seconds=0.5, end_seconds=1.5)
|
||||
restored = r.restore_job_video(job.id)
|
||||
# Points at the untouched backup, not literally the job's original path --
|
||||
# trim_job() never overwrites or renames what H3 actually generated.
|
||||
assert restored.output_path.endswith("original.mp4")
|
||||
assert restored.num_frames == self.FPS * 2
|
||||
|
||||
def test_restore_refuses_a_job_never_edited(self, h):
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
with pytest.raises(FrameError, match="not been edited"):
|
||||
r.restore_job_video(job.id)
|
||||
|
||||
def test_restore_undoes_a_grade_too(self, h):
|
||||
import av
|
||||
|
||||
r = h.runner()
|
||||
job = self._completed(h, r)
|
||||
graded = r.trim_job(job.id, start_seconds=0.0, end_seconds=None, segments=[seg(brightness=80.0)])
|
||||
with av.open(graded.output_path) as c:
|
||||
graded_level = int(next(c.decode(c.streams.video[0])).to_ndarray(format="rgb24")[0, 0, 0])
|
||||
assert graded_level > 60 # the grade visibly lightened the first frame (source level 0)
|
||||
|
||||
restored = r.restore_job_video(job.id)
|
||||
with av.open(restored.output_path) as c:
|
||||
restored_level = int(next(c.decode(c.streams.video[0])).to_ndarray(format="rgb24")[0, 0, 0])
|
||||
assert restored_level == 0 # back to the source's actual first-frame level
|
||||
|
||||
def test_trim_refuses_an_unknown_job(self, h):
|
||||
r = h.runner()
|
||||
with pytest.raises(ValueError, match="not found"):
|
||||
r.trim_job("nope", 0.0, 1.0)
|
||||
|
||||
def test_trim_refuses_a_pending_job(self, h):
|
||||
r = h.runner()
|
||||
job = h.make(r, "pending")
|
||||
with pytest.raises(FrameError, match="No output"):
|
||||
r.trim_job(job.id, 0.0, 1.0)
|
||||
@@ -0,0 +1,209 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""A finished clip's trailing seconds are saved as a short video the next clip can continue from."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
from enum import Enum
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from fastvideo_studio import frames
|
||||
from fastvideo_studio.frames import FrameError, last_clip_for_job, save_last_clip
|
||||
|
||||
WIDTH, HEIGHT, FPS, FRAMES = 128, 96, 24, 48 # 2 seconds
|
||||
|
||||
|
||||
class Status(Enum): # mirrors JobStatus without importing job_runner (which needs fastvideo)
|
||||
PENDING = "pending"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
|
||||
|
||||
def _write_video(path: str, n: int = FRAMES, fps: int = FPS) -> None:
|
||||
"""A clip whose frames are numbered by brightness (frame i is roughly i*5 grey), so the tail is identifiable."""
|
||||
import av
|
||||
|
||||
container = av.open(path, mode="w")
|
||||
try:
|
||||
stream = container.add_stream("h264", rate=fps)
|
||||
stream.width, stream.height, stream.pix_fmt = WIDTH, HEIGHT, "yuv420p"
|
||||
for i in range(n):
|
||||
level = min(i * 5, 255)
|
||||
array = np.full((HEIGHT, WIDTH, 3), level, dtype=np.uint8)
|
||||
frame = av.VideoFrame.from_ndarray(array, format="rgb24")
|
||||
for packet in stream.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in stream.encode():
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
|
||||
def _read_levels(path: str) -> list[int]:
|
||||
import av
|
||||
|
||||
with av.open(path) as container:
|
||||
return [int(f.to_ndarray(format="rgb24")[0, 0, 0]) for f in container.decode(container.streams.video[0])]
|
||||
|
||||
|
||||
def _job(job_id="job-1", status=Status.COMPLETED, output_path=None):
|
||||
return SimpleNamespace(id=job_id, status=status, output_path=output_path)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def video(tmp_path):
|
||||
path = str(tmp_path / "clip.mp4")
|
||||
_write_video(path)
|
||||
return path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def uploads(tmp_path):
|
||||
return str(tmp_path / "uploads")
|
||||
|
||||
|
||||
class TestSaveLastClip:
|
||||
def test_keeps_roughly_the_requested_seconds(self, video, tmp_path):
|
||||
out = str(tmp_path / "tail.mp4")
|
||||
save_last_clip(video, out, seconds=1.0)
|
||||
assert os.path.isfile(out)
|
||||
levels = _read_levels(out)
|
||||
assert FPS - 2 <= len(levels) <= FPS + 2 # ~1s at 24fps
|
||||
|
||||
def test_the_muxed_frame_rate_is_correct(self, video, tmp_path):
|
||||
# Regression: every frame stamped with the same pts (e.g. all left at 0,
|
||||
# or all set to `None` and left uncorrected) still decodes back the right
|
||||
# *number* of frames, so a frame-count-only check doesn't catch it -- but
|
||||
# it corrupts the container's average-frame-rate metadata, which is what
|
||||
# H3's own reference pipeline reads to resample the clip. A wrong rate
|
||||
# there silently collapses a short reference clip to as little as one
|
||||
# usable frame.
|
||||
out = str(tmp_path / "tail.mp4")
|
||||
save_last_clip(video, out, seconds=1.0)
|
||||
import av
|
||||
|
||||
with av.open(out) as container:
|
||||
rate = float(container.streams.video[0].average_rate)
|
||||
assert FPS - 1 <= rate <= FPS + 1
|
||||
|
||||
def test_frame_timestamps_are_distinct_and_increasing(self, video, tmp_path):
|
||||
out = str(tmp_path / "tail.mp4")
|
||||
save_last_clip(video, out, seconds=1.0)
|
||||
import av
|
||||
|
||||
with av.open(out) as container:
|
||||
times = [f.time for f in container.decode(container.streams.video[0])]
|
||||
assert len(set(times)) == len(times)
|
||||
assert times == sorted(times)
|
||||
|
||||
def test_keeps_the_end_of_the_source_not_the_start(self, video, tmp_path):
|
||||
out = str(tmp_path / "tail.mp4")
|
||||
save_last_clip(video, out, seconds=1.0)
|
||||
levels = _read_levels(out)
|
||||
full = _read_levels(video)
|
||||
assert levels[-1] == full[-1] # the very last frame is preserved
|
||||
assert levels[0] > full[0] # it does not start from the source's beginning
|
||||
|
||||
def test_asking_for_the_whole_clip_or_more_keeps_every_frame(self, video, tmp_path):
|
||||
out = str(tmp_path / "tail.mp4")
|
||||
save_last_clip(video, out, seconds=10.0)
|
||||
assert len(_read_levels(out)) == FRAMES
|
||||
|
||||
def test_a_clip_with_no_frames(self, video, tmp_path, monkeypatch):
|
||||
# A container that opens fine but yields zero frames (e.g. a truncated encode).
|
||||
import av
|
||||
|
||||
class EmptyContainer:
|
||||
class _Streams:
|
||||
video = [SimpleNamespace(average_rate=24, guessed_rate=24)]
|
||||
|
||||
streams = _Streams()
|
||||
|
||||
def decode(self, *_a):
|
||||
return iter(())
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *_a):
|
||||
return False
|
||||
|
||||
monkeypatch.setattr(av, "open", lambda *_a, **_k: EmptyContainer())
|
||||
with pytest.raises(ValueError, match="no frames"):
|
||||
save_last_clip(video, str(tmp_path / "out.mp4"))
|
||||
|
||||
def test_a_nonpositive_duration_is_refused(self, video, tmp_path):
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
save_last_clip(video, str(tmp_path / "out.mp4"), seconds=0)
|
||||
|
||||
def test_writes_no_temp_file_leftovers(self, video, tmp_path):
|
||||
out = str(tmp_path / "tail.mp4")
|
||||
save_last_clip(video, out, seconds=1.0)
|
||||
assert os.listdir(tmp_path) == ["clip.mp4", "tail.mp4"]
|
||||
|
||||
|
||||
class TestLastClipForJob:
|
||||
def test_extracts_and_returns_a_path(self, video, uploads):
|
||||
job = _job(output_path=video)
|
||||
path = last_clip_for_job(job, uploads)
|
||||
assert os.path.isfile(path)
|
||||
assert path.endswith("last_clip_job-1.mp4")
|
||||
assert "last_clips" in path
|
||||
|
||||
def test_reuses_an_existing_extraction(self, video, uploads, monkeypatch):
|
||||
job = _job(output_path=video)
|
||||
first = last_clip_for_job(job, uploads)
|
||||
mtime = os.path.getmtime(first)
|
||||
calls = []
|
||||
monkeypatch.setattr(frames, "save_last_clip", lambda *a, **k: calls.append(a))
|
||||
second = last_clip_for_job(job, uploads)
|
||||
assert second == first
|
||||
assert os.path.getmtime(second) == mtime
|
||||
assert calls == []
|
||||
|
||||
def test_re_extracts_when_the_video_changed_since(self, video, uploads, monkeypatch):
|
||||
job = _job(output_path=video)
|
||||
last_clip_for_job(job, uploads)
|
||||
_write_video(video, n=FRAMES + 24) # a longer re-render
|
||||
# Force the source to read as newer, regardless of filesystem mtime resolution.
|
||||
monkeypatch.setattr(os.path, "getmtime", lambda p: (time.time() + 1) if p == video else os.stat(p).st_mtime)
|
||||
calls = []
|
||||
real_save = frames.save_last_clip
|
||||
monkeypatch.setattr(frames, "save_last_clip", lambda *a, **k: (calls.append(a), real_save(*a, **k))[1])
|
||||
last_clip_for_job(job, uploads)
|
||||
assert len(calls) == 1
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("job", "match"),
|
||||
[
|
||||
(_job(status=Status.PENDING), "No output"),
|
||||
(_job(status=Status.COMPLETED, output_path=None), "No output"),
|
||||
(_job(status=Status.COMPLETED, output_path="/nope.mp4"), "not found"),
|
||||
],
|
||||
)
|
||||
def test_refusals(self, job, match, uploads):
|
||||
with pytest.raises(FrameError, match=match):
|
||||
last_clip_for_job(job, uploads)
|
||||
|
||||
def test_refuses_a_non_video_output(self, uploads, tmp_path):
|
||||
img = tmp_path / "out.png"
|
||||
img.write_bytes(b"x")
|
||||
job = _job(output_path=str(img))
|
||||
with pytest.raises(FrameError, match="not a video"):
|
||||
last_clip_for_job(job, uploads)
|
||||
|
||||
def test_refuses_without_an_upload_dir(self, video):
|
||||
job = _job(output_path=video)
|
||||
with pytest.raises(FrameError, match="Upload directory"):
|
||||
last_clip_for_job(job, "")
|
||||
|
||||
def test_a_decode_failure_is_reported_as_a_500(self, uploads, tmp_path):
|
||||
bad = tmp_path / "bad.mp4"
|
||||
bad.write_bytes(b"not a video")
|
||||
job = _job(output_path=str(bad))
|
||||
with pytest.raises(FrameError, match="Could not read the end of the clip") as e:
|
||||
last_clip_for_job(job, uploads)
|
||||
assert e.value.status_code == 500
|
||||
@@ -0,0 +1,155 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""A finished clip's final frame is saved as an image the next clip can use as a reference."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from enum import Enum
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
from PIL import Image
|
||||
|
||||
from fastvideo_studio import frames
|
||||
from fastvideo_studio.frames import FrameError, last_frame_for_job
|
||||
|
||||
WIDTH, HEIGHT, FRAMES = 128, 96, 12
|
||||
|
||||
|
||||
class Status(Enum): # mirrors JobStatus without importing job_runner (which needs fastvideo)
|
||||
PENDING = "pending"
|
||||
RUNNING = "running"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
STOPPED = "stopped"
|
||||
|
||||
|
||||
def _write_video(path: str, *, last_rgb=(0, 0, 255), other_rgb=(255, 0, 0)) -> None:
|
||||
"""A clip whose frames are all `other_rgb` except the last, which is `last_rgb`."""
|
||||
import av
|
||||
|
||||
container = av.open(path, mode="w")
|
||||
try:
|
||||
stream = container.add_stream("h264", rate=24)
|
||||
stream.width, stream.height, stream.pix_fmt = WIDTH, HEIGHT, "yuv420p"
|
||||
for i in range(FRAMES):
|
||||
rgb = last_rgb if i == FRAMES - 1 else other_rgb
|
||||
array = np.full((HEIGHT, WIDTH, 3), rgb, dtype=np.uint8)
|
||||
frame = av.VideoFrame.from_ndarray(array, format="rgb24")
|
||||
for packet in stream.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in stream.encode():
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
|
||||
def _job(job_id="job-1", status=Status.COMPLETED, output_path=None):
|
||||
return SimpleNamespace(id=job_id, status=status, output_path=output_path)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def video(tmp_path):
|
||||
path = str(tmp_path / "clip.mp4")
|
||||
_write_video(path)
|
||||
return path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def uploads(tmp_path):
|
||||
return str(tmp_path / "uploads")
|
||||
|
||||
|
||||
def test_saves_the_final_frame_not_the_first(video, uploads, tmp_path):
|
||||
path = last_frame_for_job(_job(output_path=video), uploads)
|
||||
|
||||
assert path == str(tmp_path / "uploads" / "last_frames" / "last_frame_job-1.png")
|
||||
assert os.path.isabs(path)
|
||||
image = Image.open(path).convert("RGB")
|
||||
assert image.size == (WIDTH, HEIGHT)
|
||||
r, _g, b = image.getpixel((WIDTH // 2, HEIGHT // 2))
|
||||
# the last frame is blue and every other frame is red -- a wrong frame would be red
|
||||
assert b > 150 and r < 100, (r, b)
|
||||
|
||||
|
||||
def test_accepts_a_plain_string_status(video, uploads):
|
||||
assert last_frame_for_job(_job(status="completed", output_path=video), uploads).endswith(".png")
|
||||
|
||||
|
||||
def test_extraction_is_reused_until_the_video_changes(video, uploads, monkeypatch):
|
||||
calls = []
|
||||
real = frames.save_last_frame
|
||||
monkeypatch.setattr(frames, "save_last_frame", lambda v, o: (calls.append(o), real(v, o))[1])
|
||||
|
||||
first = last_frame_for_job(_job(output_path=video), uploads)
|
||||
second = last_frame_for_job(_job(output_path=video), uploads)
|
||||
assert first == second
|
||||
assert len(calls) == 1
|
||||
|
||||
# the clip is regenerated -> the saved frame is stale and gets rebuilt
|
||||
later = os.path.getmtime(first) + 10
|
||||
os.utime(video, (later, later))
|
||||
last_frame_for_job(_job(output_path=video), uploads)
|
||||
assert len(calls) == 2
|
||||
|
||||
|
||||
def test_each_job_gets_its_own_file(video, uploads):
|
||||
a = last_frame_for_job(_job("a", output_path=video), uploads)
|
||||
b = last_frame_for_job(_job("b", output_path=video), uploads)
|
||||
assert a != b and os.path.isfile(a) and os.path.isfile(b)
|
||||
|
||||
|
||||
def test_leaves_no_temp_file_behind(video, uploads):
|
||||
last_frame_for_job(_job(output_path=video), uploads)
|
||||
assert os.listdir(os.path.join(uploads, "last_frames")) == ["last_frame_job-1.png"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("status", [Status.PENDING, Status.RUNNING, Status.FAILED, Status.STOPPED])
|
||||
def test_a_job_that_has_not_finished_is_404(video, uploads, status):
|
||||
with pytest.raises(FrameError) as e:
|
||||
last_frame_for_job(_job(status=status, output_path=video), uploads)
|
||||
assert e.value.status_code == 404
|
||||
assert "No output" in e.value.detail
|
||||
|
||||
|
||||
def test_a_completed_job_with_no_output_path_is_404(uploads):
|
||||
with pytest.raises(FrameError) as e:
|
||||
last_frame_for_job(_job(output_path=None), uploads)
|
||||
assert e.value.status_code == 404
|
||||
|
||||
|
||||
def test_missing_output_file_is_404(uploads, tmp_path):
|
||||
with pytest.raises(FrameError) as e:
|
||||
last_frame_for_job(_job(output_path=str(tmp_path / "gone.mp4")), uploads)
|
||||
assert (e.value.status_code, "not found on disk" in e.value.detail) == (404, True)
|
||||
|
||||
|
||||
def test_an_image_output_is_rejected(uploads, tmp_path):
|
||||
png = tmp_path / "out.png"
|
||||
Image.new("RGB", (8, 8)).save(png)
|
||||
with pytest.raises(FrameError) as e:
|
||||
last_frame_for_job(_job(output_path=str(png)), uploads)
|
||||
assert e.value.status_code == 400
|
||||
|
||||
|
||||
def test_an_unreadable_video_is_a_500_with_a_reason(uploads, tmp_path):
|
||||
bad = tmp_path / "bad.mp4"
|
||||
bad.write_bytes(b"not a video")
|
||||
with pytest.raises(FrameError) as e:
|
||||
last_frame_for_job(_job(output_path=str(bad)), uploads)
|
||||
assert e.value.status_code == 500
|
||||
assert "Could not read the last frame" in e.value.detail
|
||||
|
||||
|
||||
def test_a_failed_extraction_does_not_leave_a_partial_file(uploads, tmp_path):
|
||||
bad = tmp_path / "bad.mp4"
|
||||
bad.write_bytes(b"not a video")
|
||||
with pytest.raises(FrameError):
|
||||
last_frame_for_job(_job(output_path=str(bad)), uploads)
|
||||
assert os.listdir(os.path.join(uploads, "last_frames")) == []
|
||||
|
||||
|
||||
def test_needs_an_upload_directory(video):
|
||||
with pytest.raises(FrameError) as e:
|
||||
last_frame_for_job(_job(output_path=video), "")
|
||||
assert e.value.status_code == 503
|
||||
@@ -0,0 +1,251 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""A scene's clips are joined, in order, into one video without re-encoding."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from enum import Enum
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from fastvideo_studio import merge
|
||||
from fastvideo_studio.merge import MergeError, find_ffmpeg, merge_clips, merge_jobs, merged_path, probe_clip
|
||||
|
||||
pytestmark = pytest.mark.skipif(find_ffmpeg() is None, reason="needs an ffmpeg binary")
|
||||
|
||||
FPS = 24
|
||||
COLORS = {"red": (255, 0, 0), "green": (0, 255, 0), "blue": (0, 0, 255)}
|
||||
|
||||
|
||||
class Status(Enum): # mirrors JobStatus without importing job_runner (which needs fastvideo)
|
||||
PENDING = "pending"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
|
||||
|
||||
def write_clip(path, *, frames=24, color="red", size=(128, 96), fps=FPS, audio=True, audio_extra=0.042):
|
||||
"""A clip of one flat colour. Like real H3 output, its audio runs a little past the video."""
|
||||
import av
|
||||
|
||||
container = av.open(path, mode="w")
|
||||
try:
|
||||
video = container.add_stream("h264", rate=fps)
|
||||
video.width, video.height, video.pix_fmt = size[0], size[1], "yuv420p"
|
||||
sound = None
|
||||
if audio:
|
||||
sound = container.add_stream("aac", rate=32000)
|
||||
sound.layout = "stereo"
|
||||
for _ in range(frames):
|
||||
array = np.full((size[1], size[0], 3), COLORS[color], dtype=np.uint8)
|
||||
for packet in video.encode(av.VideoFrame.from_ndarray(array, format="rgb24")):
|
||||
container.mux(packet)
|
||||
for packet in video.encode():
|
||||
container.mux(packet)
|
||||
if sound is not None:
|
||||
total = int((frames / fps + audio_extra) * 32000)
|
||||
tone = (np.sin(np.arange(total) * 2 * np.pi * 440 / 32000) * 0.2).astype(np.float32)
|
||||
pts = 0
|
||||
for start in range(0, total, 1024):
|
||||
chunk = tone[start:start + 1024]
|
||||
chunk = np.pad(chunk, (0, 1024 - len(chunk)))
|
||||
frame = av.AudioFrame.from_ndarray(np.stack([chunk, chunk]), format="fltp", layout="stereo")
|
||||
frame.sample_rate, frame.pts = 32000, pts
|
||||
pts += 1024
|
||||
for packet in sound.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in sound.encode():
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
|
||||
def read_back(path):
|
||||
"""(frame colours in order, audio seconds, video seconds) of a finished video."""
|
||||
import av
|
||||
|
||||
with av.open(path) as c:
|
||||
v = c.streams.video[0]
|
||||
colours = []
|
||||
for frame in c.decode(v):
|
||||
r, g, b = frame.to_ndarray(format="rgb24")[48, 64]
|
||||
colours.append("red" if r > 150 else "green" if g > 150 else "blue")
|
||||
audio = next((s for s in c.streams if s.type == "audio"), None)
|
||||
a_secs = float(audio.duration * audio.time_base) if audio and audio.duration else None
|
||||
return colours, a_secs, float(v.duration * v.time_base)
|
||||
|
||||
|
||||
def job(path, name="", status=Status.COMPLETED, job_id="j"):
|
||||
return SimpleNamespace(id=job_id, name=name, status=status, output_path=path)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clips(tmp_path):
|
||||
paths = []
|
||||
for i, (color, frames) in enumerate([("red", 24), ("green", 36), ("blue", 12)]):
|
||||
p = str(tmp_path / f"clip{i}.mp4")
|
||||
write_clip(p, frames=frames, color=color)
|
||||
paths.append(p)
|
||||
return paths
|
||||
|
||||
|
||||
class TestMergeClips:
|
||||
def test_joins_clips_in_the_order_given(self, clips, tmp_path):
|
||||
out = str(tmp_path / "out.mp4")
|
||||
merge_clips(clips, out)
|
||||
colours, _, _ = read_back(out)
|
||||
assert colours == ["red"] * 24 + ["green"] * 36 + ["blue"] * 12
|
||||
|
||||
def test_the_order_is_the_callers_not_the_filenames(self, clips, tmp_path):
|
||||
out = str(tmp_path / "out.mp4")
|
||||
merge_clips([clips[2], clips[0], clips[1]], out)
|
||||
colours, _, _ = read_back(out)
|
||||
assert colours == ["blue"] * 12 + ["red"] * 24 + ["green"] * 36
|
||||
|
||||
def test_keeps_every_frame_and_the_full_length(self, clips, tmp_path):
|
||||
out = str(tmp_path / "out.mp4")
|
||||
seconds = merge_clips(clips, out)
|
||||
colours, _, video_secs = read_back(out)
|
||||
assert len(colours) == 72
|
||||
assert seconds == pytest.approx(72 / FPS, abs=0.02)
|
||||
assert video_secs == pytest.approx(72 / FPS, abs=0.05)
|
||||
|
||||
def test_audio_stays_in_step_with_the_picture(self, clips, tmp_path):
|
||||
# Each clip's audio runs ~40 ms past its video; joined naively that adds up to a gap or drift.
|
||||
out = str(tmp_path / "out.mp4")
|
||||
merge_clips(clips, out)
|
||||
_, audio_secs, video_secs = read_back(out)
|
||||
assert audio_secs is not None
|
||||
assert abs(audio_secs - video_secs) < 0.08 # not 3 x 0.042 and growing
|
||||
|
||||
def test_a_single_clip_is_fine(self, clips, tmp_path):
|
||||
out = str(tmp_path / "out.mp4")
|
||||
merge_clips(clips[:1], out)
|
||||
assert len(read_back(out)[0]) == 24
|
||||
|
||||
def test_clips_without_audio_join_too(self, tmp_path):
|
||||
paths = []
|
||||
for i, color in enumerate(("red", "green")):
|
||||
p = str(tmp_path / f"s{i}.mp4")
|
||||
write_clip(p, frames=12, color=color, audio=False)
|
||||
paths.append(p)
|
||||
out = str(tmp_path / "out.mp4")
|
||||
merge_clips(paths, out)
|
||||
colours, audio_secs, _ = read_back(out)
|
||||
assert colours == ["red"] * 12 + ["green"] * 12 and audio_secs is None
|
||||
|
||||
def test_creates_the_output_folder_and_leaves_no_temp_files(self, clips, tmp_path):
|
||||
out = str(tmp_path / "new" / "dir" / "out.mp4")
|
||||
merge_clips(clips, out)
|
||||
assert os.listdir(os.path.dirname(out)) == ["out.mp4"]
|
||||
|
||||
def test_paths_with_quotes_and_spaces(self, tmp_path):
|
||||
paths = []
|
||||
for i, color in enumerate(("red", "blue")):
|
||||
p = str(tmp_path / f"it's clip {i}.mp4")
|
||||
write_clip(p, frames=6, color=color)
|
||||
paths.append(p)
|
||||
out = str(tmp_path / "out.mp4")
|
||||
merge_clips(paths, out)
|
||||
assert read_back(out)[0] == ["red"] * 6 + ["blue"] * 6
|
||||
|
||||
|
||||
class TestRefusals:
|
||||
def test_clips_of_different_sizes(self, clips, tmp_path):
|
||||
odd = str(tmp_path / "odd.mp4")
|
||||
write_clip(odd, size=(160, 96))
|
||||
with pytest.raises(MergeError, match=r"'clip 3' \(160x96.*doesn't match 'clip 1' \(128x96") as e:
|
||||
merge_clips([clips[0], clips[1], odd], str(tmp_path / "o.mp4"), labels=["'clip 1'", "'clip 2'", "'clip 3'"])
|
||||
assert e.value.status_code == 409
|
||||
assert not (tmp_path / "o.mp4").exists()
|
||||
|
||||
def test_clips_of_different_frame_rates(self, clips, tmp_path):
|
||||
odd = str(tmp_path / "odd.mp4")
|
||||
write_clip(odd, fps=30)
|
||||
with pytest.raises(MergeError, match="doesn't match"):
|
||||
merge_clips([clips[0], odd], str(tmp_path / "o.mp4"))
|
||||
|
||||
def test_one_clip_with_audio_and_one_without(self, clips, tmp_path):
|
||||
silent = str(tmp_path / "silent.mp4")
|
||||
write_clip(silent, audio=False)
|
||||
with pytest.raises(MergeError, match="no audio"):
|
||||
merge_clips([clips[0], silent], str(tmp_path / "o.mp4"))
|
||||
|
||||
def test_nothing_to_merge(self, tmp_path):
|
||||
with pytest.raises(MergeError) as e:
|
||||
merge_clips([], str(tmp_path / "o.mp4"))
|
||||
assert e.value.status_code == 400
|
||||
|
||||
def test_a_file_that_is_not_a_video(self, tmp_path):
|
||||
bad = tmp_path / "bad.mp4"
|
||||
bad.write_bytes(b"not a video")
|
||||
with pytest.raises(MergeError, match="Could not read bad.mp4"):
|
||||
merge_clips([str(bad)], str(tmp_path / "o.mp4"))
|
||||
|
||||
def test_no_ffmpeg(self, clips, tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(merge, "find_ffmpeg", lambda: None)
|
||||
with pytest.raises(MergeError) as e:
|
||||
merge_clips(clips, str(tmp_path / "o.mp4"))
|
||||
assert e.value.status_code == 503 and "ffmpeg" in e.value.detail
|
||||
|
||||
|
||||
class TestMergeJobs:
|
||||
def test_merges_the_jobs_videos_into_a_merged_folder(self, clips, tmp_path):
|
||||
jobs = [job(p, name=f"clip {i + 1}", job_id=f"j{i}") for i, p in enumerate(clips)]
|
||||
result = merge_jobs(jobs, str(tmp_path / "out"), name="Wolf Lunch")
|
||||
assert result.clips == 3 and result.seconds == pytest.approx(3.0, abs=0.02)
|
||||
assert result.filename.startswith("Wolf-Lunch-") and result.filename.endswith(".mp4")
|
||||
assert result.path == str(tmp_path / "out" / "merged" / result.filename)
|
||||
assert len(read_back(result.path)[0]) == 72
|
||||
|
||||
def test_a_name_cannot_escape_the_folder(self, clips, tmp_path):
|
||||
result = merge_jobs([job(clips[0])], str(tmp_path / "out"), name="../../etc/passwd")
|
||||
assert os.path.dirname(result.path) == str(tmp_path / "out" / "merged")
|
||||
assert "/" not in result.filename
|
||||
|
||||
def test_an_empty_name_still_gets_a_filename(self, clips, tmp_path):
|
||||
assert merge_jobs([job(clips[0])], str(tmp_path / "out"), name=" ").filename.startswith("scene-")
|
||||
|
||||
def test_refuses_a_clip_that_has_not_finished_and_names_it(self, clips, tmp_path):
|
||||
jobs = [job(clips[0], name="clip 1"), job(None, name="clip 2", status=Status.PENDING)]
|
||||
with pytest.raises(MergeError, match=r"'clip 2' hasn't finished \(pending\)") as e:
|
||||
merge_jobs(jobs, str(tmp_path / "out"))
|
||||
assert e.value.status_code == 409
|
||||
assert not (tmp_path / "out").exists() # nothing partial
|
||||
|
||||
def test_refuses_a_video_missing_from_disk(self, clips, tmp_path):
|
||||
jobs = [job(clips[0], name="a"), job(str(tmp_path / "gone.mp4"), name="b")]
|
||||
with pytest.raises(MergeError, match="'b'.*missing from disk"):
|
||||
merge_jobs(jobs, str(tmp_path / "out"))
|
||||
|
||||
def test_refuses_an_output_that_is_not_a_video(self, tmp_path):
|
||||
img = tmp_path / "a.png"
|
||||
img.write_bytes(b"x")
|
||||
with pytest.raises(MergeError, match="isn't a video"):
|
||||
merge_jobs([job(str(img), name="a")], str(tmp_path / "out"))
|
||||
|
||||
def test_refuses_no_jobs(self, tmp_path):
|
||||
with pytest.raises(MergeError) as e:
|
||||
merge_jobs([], str(tmp_path))
|
||||
assert e.value.status_code == 400
|
||||
|
||||
|
||||
class TestMergedPath:
|
||||
def test_finds_a_merged_video(self, clips, tmp_path):
|
||||
result = merge_jobs([job(clips[0])], str(tmp_path))
|
||||
assert merged_path(str(tmp_path), result.filename) == result.path
|
||||
|
||||
@pytest.mark.parametrize("name", ["../secret.mp4", "a/b.mp4", "..", "x.txt", "", "nope.mp4", "a b.mp4"])
|
||||
def test_refuses_anything_else(self, tmp_path, name):
|
||||
(tmp_path / "merged").mkdir()
|
||||
(tmp_path / "secret.mp4").write_bytes(b"x")
|
||||
with pytest.raises(MergeError) as e:
|
||||
merged_path(str(tmp_path), name)
|
||||
assert e.value.status_code == 404
|
||||
|
||||
|
||||
def test_probe_reads_real_clip_settings(clips):
|
||||
info = probe_clip(clips[1])
|
||||
assert (info.width, info.height, info.video_codec, info.pix_fmt) == (128, 96, "h264", "yuv420p")
|
||||
assert info.fps == pytest.approx(24) and info.video_seconds == pytest.approx(1.5, abs=0.03)
|
||||
assert info.audio == ("aac", 32000, 2)
|
||||
@@ -0,0 +1,428 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""The mock API's job edit and last-frame routes behave like the real server's."""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from fastvideo_studio import mock_server
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_state():
|
||||
mock_server._jobs.clear()
|
||||
yield
|
||||
mock_server._jobs.clear()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def client():
|
||||
return TestClient(mock_server.app)
|
||||
|
||||
|
||||
def _create(client, **over):
|
||||
body = {"model_id": "MiniMaxAI/MiniMax-H3", "prompt": "p", "workload_type": "i2v", **over}
|
||||
res = client.post("/api/jobs", json=body)
|
||||
assert res.status_code == 201
|
||||
return res.json()
|
||||
|
||||
|
||||
def test_edits_a_pending_jobs_prompt_and_references(client):
|
||||
job = _create(client, name="clip-2", references=[{"source": "/a.png", "media_type": "image"}])
|
||||
refs = job["references"] + [{"source": "/b.png", "media_type": "image"}]
|
||||
|
||||
res = client.patch(f"/api/jobs/{job['id']}", json={"prompt": "new prompt", "references": refs})
|
||||
|
||||
assert res.status_code == 200
|
||||
body = res.json()
|
||||
assert body["prompt"] == "new prompt"
|
||||
assert [r["source"] for r in body["references"]] == ["/a.png", "/b.png"]
|
||||
assert body["name"] == "clip-2" # untouched fields stay
|
||||
assert client.get(f"/api/jobs/{job['id']}").json()["prompt"] == "new prompt"
|
||||
|
||||
|
||||
def test_a_started_job_cannot_be_edited(client):
|
||||
job = _create(client)
|
||||
mock_server._jobs[job["id"]]["status"] = "completed"
|
||||
res = client.patch(f"/api/jobs/{job['id']}", json={"prompt": "x"})
|
||||
assert res.status_code == 400
|
||||
assert "only pending, failed, stopped jobs can be edited" in res.json()["detail"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("status", ["failed", "stopped"])
|
||||
def test_failed_and_stopped_jobs_can_be_edited(client, status):
|
||||
job = _create(client)
|
||||
mock_server._jobs[job["id"]]["status"] = status
|
||||
assert client.patch(f"/api/jobs/{job['id']}", json={"prompt": "again"}).status_code == 200
|
||||
|
||||
|
||||
def test_unknown_fields_are_rejected(client):
|
||||
job = _create(client)
|
||||
res = client.patch(f"/api/jobs/{job['id']}", json={"status": "completed", "nonsense": 1})
|
||||
assert res.status_code == 400
|
||||
assert "Not editable: nonsense, status" in res.json()["detail"]
|
||||
|
||||
|
||||
def test_editing_an_unknown_job_is_404(client):
|
||||
assert client.patch("/api/jobs/nope", json={"prompt": "x"}).status_code == 404
|
||||
|
||||
|
||||
def test_last_frame_of_a_completed_job(client):
|
||||
job = _create(client)
|
||||
mock_server._jobs[job["id"]].update(status="completed", output_path="/out/a.mp4")
|
||||
res = client.post(f"/api/jobs/{job['id']}/last-frame")
|
||||
assert res.status_code == 200
|
||||
assert res.json() == {"path": f"/mock/last_frames/last_frame_{job['id']}.png", "media_type": "image"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("status", ["pending", "failed", "stopped"])
|
||||
def test_last_frame_of_an_unfinished_job_is_404(client, status):
|
||||
job = _create(client)
|
||||
mock_server._jobs[job["id"]]["status"] = status
|
||||
res = client.post(f"/api/jobs/{job['id']}/last-frame")
|
||||
assert res.status_code == 404
|
||||
assert "No output" in res.json()["detail"]
|
||||
|
||||
|
||||
def test_last_frame_of_an_unknown_job_is_404(client):
|
||||
assert client.post("/api/jobs/nope/last-frame").status_code == 404
|
||||
|
||||
|
||||
# --- queue -------------------------------------------------------------------
|
||||
|
||||
|
||||
def _frame_of(job):
|
||||
return [{"source": f"job-last-frame:{job['id']}", "media_type": "image"}]
|
||||
|
||||
|
||||
def _statuses(client, *jobs):
|
||||
return [client.get(f"/api/jobs/{j['id']}").json()["status"] for j in jobs]
|
||||
|
||||
|
||||
def _age(job, seconds):
|
||||
"""Pretend `seconds` have passed since `job` started."""
|
||||
record = mock_server._jobs[job["id"]]
|
||||
record["started_at"] -= seconds
|
||||
|
||||
|
||||
def test_queued_jobs_run_one_at_a_time_in_order(client):
|
||||
a, b, c = (_create(client, name=n) for n in "abc")
|
||||
res = client.post("/api/jobs/queue", json={"job_ids": [b["id"], c["id"], a["id"]]})
|
||||
|
||||
assert res.status_code == 200
|
||||
assert [j["name"] for j in res.json()] == ["b", "c", "a"]
|
||||
assert _statuses(client, a, b, c) == ["queued", "running", "queued"]
|
||||
_age(b, 10)
|
||||
assert _statuses(client, a, b, c) == ["queued", "completed", "running"]
|
||||
|
||||
|
||||
def test_a_long_idle_queue_is_all_done_on_the_next_poll(client):
|
||||
clips = [_create(client, name=f"c{i}") for i in range(3)]
|
||||
client.post("/api/jobs/queue", json={"job_ids": [c["id"] for c in clips]})
|
||||
for record in mock_server._jobs.values(): # everything happened a minute ago
|
||||
if record.get("queued_at"):
|
||||
record["queued_at"] -= 60
|
||||
if record.get("started_at"):
|
||||
record["started_at"] -= 60
|
||||
assert _statuses(client, *clips) == ["completed"] * 3
|
||||
starts = [mock_server._jobs[c["id"]]["started_at"] for c in clips]
|
||||
assert starts == sorted(starts) # each took over when the previous finished
|
||||
|
||||
|
||||
def test_a_clip_waits_for_the_clip_whose_last_frame_it_uses(client):
|
||||
one = _create(client, name="one")
|
||||
two = _create(client, name="two", references=_frame_of(one))
|
||||
client.post("/api/jobs/queue", json={"job_ids": [one["id"], two["id"]]})
|
||||
assert _statuses(client, one, two) == ["running", "queued"]
|
||||
_age(one, 10)
|
||||
assert _statuses(client, one, two) == ["completed", "running"]
|
||||
|
||||
|
||||
def test_failure_of_a_frame_source_fails_the_clips_after_it(client):
|
||||
one = _create(client, name="Clip 1")
|
||||
two = _create(client, name="Clip 2", references=_frame_of(one))
|
||||
three = _create(client, name="Clip 3", references=_frame_of(two))
|
||||
mock_server._jobs[one["id"]]["status"] = "failed"
|
||||
client.post("/api/jobs/queue", json={"job_ids": [two["id"], three["id"]]})
|
||||
|
||||
assert _statuses(client, two, three) == ["failed", "failed"]
|
||||
error = client.get(f"/api/jobs/{two['id']}").json()["error"]
|
||||
assert "Clip 1" in error and "failed" in error
|
||||
|
||||
|
||||
def test_deleting_a_frame_source_fails_the_clips_waiting_on_it(client):
|
||||
blocker = _create(client)
|
||||
one = _create(client, name="one")
|
||||
two = _create(client, name="two", references=_frame_of(one))
|
||||
client.post("/api/jobs/queue", json={"job_ids": [blocker["id"], one["id"], two["id"]]})
|
||||
assert client.delete(f"/api/jobs/{one['id']}").status_code == 200
|
||||
assert _statuses(client, two) == ["failed"]
|
||||
assert "deleted" in client.get(f"/api/jobs/{two['id']}").json()["error"]
|
||||
|
||||
|
||||
def test_dequeue_and_stop_return_a_queued_job_to_pending(client):
|
||||
blocker, a, b = _create(client), _create(client), _create(client)
|
||||
client.post("/api/jobs/queue", json={"job_ids": [blocker["id"], a["id"], b["id"]]})
|
||||
|
||||
assert client.post(f"/api/jobs/{a['id']}/dequeue").json()["status"] == "pending"
|
||||
assert client.post(f"/api/jobs/{b['id']}/stop").json()["status"] == "pending"
|
||||
assert client.post(f"/api/jobs/{a['id']}/dequeue").status_code == 409
|
||||
assert client.post("/api/jobs/nope/dequeue").status_code == 404
|
||||
|
||||
|
||||
def test_queueing_is_all_or_nothing_and_refuses_started_jobs(client):
|
||||
a, done = _create(client), _create(client)
|
||||
mock_server._jobs[done["id"]]["status"] = "completed"
|
||||
res = client.post("/api/jobs/queue", json={"job_ids": [a["id"], done["id"]]})
|
||||
assert res.status_code == 409 and "already completed" in res.json()["detail"]
|
||||
assert _statuses(client, a) == ["pending"]
|
||||
assert client.post("/api/jobs/queue", json={"job_ids": [a["id"], "nope"]}).status_code == 404
|
||||
assert client.post("/api/jobs/queue", json={"job_ids": [a["id"], a["id"]]}).status_code == 409
|
||||
|
||||
assert client.post(f"/api/jobs/{a['id']}/queue").status_code == 200
|
||||
assert client.post(f"/api/jobs/{a['id']}/queue").status_code == 409 # running now
|
||||
|
||||
|
||||
def test_starting_a_clip_before_its_frame_source_finished_is_refused(client):
|
||||
one = _create(client, name="Clip 1")
|
||||
two = _create(client, references=_frame_of(one))
|
||||
res = client.post(f"/api/jobs/{two['id']}/start")
|
||||
assert res.status_code == 409
|
||||
assert "Clip 1" in res.json()["detail"] and "hasn't finished" in res.json()["detail"]
|
||||
|
||||
mock_server._jobs[one["id"]]["status"] = "completed"
|
||||
assert client.post(f"/api/jobs/{two['id']}/start").status_code == 200
|
||||
|
||||
|
||||
def test_queued_jobs_cannot_be_edited(client):
|
||||
blocker, a = _create(client), _create(client)
|
||||
client.post("/api/jobs/queue", json={"job_ids": [blocker["id"], a["id"]]})
|
||||
assert client.patch(f"/api/jobs/{a['id']}", json={"prompt": "x"}).status_code == 400
|
||||
|
||||
|
||||
# --- merging a scene -----------------------------------------------------------
|
||||
|
||||
|
||||
def _completed(client, name):
|
||||
job = _create(client, name=name)
|
||||
record = mock_server._jobs[job["id"]]
|
||||
record.update(status="completed", output_path=f"/mock/outputs/{job['id']}/output.mp4", finished_at=1.0)
|
||||
return job
|
||||
|
||||
|
||||
def test_merges_finished_clips_into_one_downloadable_video(client):
|
||||
clips = [_completed(client, f"wolf-{i}") for i in range(3)]
|
||||
res = client.post("/api/scenes/merge", json={"job_ids": [c["id"] for c in clips], "name": "wolf"})
|
||||
|
||||
assert res.status_code == 200
|
||||
body = res.json()
|
||||
assert body["clips"] == 3 and body["filename"].startswith("wolf-") and body["seconds"] == pytest.approx(3.0, abs=0.1)
|
||||
video = client.get(body["url"])
|
||||
assert video.status_code == 200 and video.headers["content-type"] == "video/mp4"
|
||||
assert len(video.content) > 1000
|
||||
|
||||
|
||||
def test_refuses_to_merge_a_scene_with_an_unfinished_clip(client):
|
||||
done, pending = _completed(client, "a"), _create(client, name="b")
|
||||
res = client.post("/api/scenes/merge", json={"job_ids": [done["id"], pending["id"]]})
|
||||
assert res.status_code == 409 and "'b' hasn't finished" in res.json()["detail"]
|
||||
|
||||
|
||||
def test_merge_of_an_unknown_job_and_of_nothing(client):
|
||||
assert client.post("/api/scenes/merge", json={"job_ids": ["nope"]}).status_code == 404
|
||||
assert client.post("/api/scenes/merge", json={"job_ids": []}).status_code == 400
|
||||
|
||||
|
||||
@pytest.mark.parametrize("filename", ["nope.mp4", "..%2Fetc%2Fpasswd", "x.txt"])
|
||||
def test_an_unknown_merged_file_is_not_found(client, filename):
|
||||
assert client.get(f"/api/merged/{filename}").status_code == 404
|
||||
|
||||
|
||||
# --- trimming ------------------------------------------------------------------
|
||||
|
||||
|
||||
def seg(brightness=0.0, contrast=1.0, saturation=1.0, end_seconds=None):
|
||||
"""One razor-cut section dict, as the trim API expects."""
|
||||
return {"end_seconds": end_seconds, "brightness": brightness, "contrast": contrast, "saturation": saturation}
|
||||
|
||||
|
||||
def test_trims_a_completed_jobs_video_by_resizing_its_frame_count(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240 # 10s @ 24fps
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 1.0, "end_seconds": 3.0})
|
||||
|
||||
assert res.status_code == 200
|
||||
assert res.json()["num_frames"] == 48 # 2s @ 24fps
|
||||
|
||||
|
||||
def test_an_open_end_keeps_to_the_original_end(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 8.0})
|
||||
assert res.json()["num_frames"] == 48 # 10s source, 8-10s kept = 2s remaining
|
||||
|
||||
|
||||
def test_trimming_again_is_relative_to_the_original_length_not_the_last_trim(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
|
||||
client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 1.0, "end_seconds": 3.0})
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 0.0, "end_seconds": 10.0})
|
||||
assert res.json()["num_frames"] == 240
|
||||
|
||||
|
||||
def test_restore_undoes_a_trim(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 1.0, "end_seconds": 3.0})
|
||||
|
||||
res = client.post(f"/api/jobs/{job['id']}/restore-video")
|
||||
|
||||
assert res.status_code == 200
|
||||
assert res.json()["num_frames"] == 240
|
||||
assert client.get(f"/api/jobs/{job['id']}").json()["num_frames"] == 240
|
||||
|
||||
|
||||
def test_restore_refuses_a_job_that_was_never_edited(client):
|
||||
job = _completed(client, "clip")
|
||||
res = client.post(f"/api/jobs/{job['id']}/restore-video")
|
||||
assert res.status_code == 404 and "not been edited" in res.json()["detail"]
|
||||
|
||||
|
||||
def test_trim_refuses_an_invalid_range(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
assert client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": -1.0}).status_code == 400
|
||||
assert client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 5.0, "end_seconds": 5.0}).status_code == 400
|
||||
|
||||
|
||||
def test_trim_accepts_color_parameters(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
res = client.post(
|
||||
f"/api/jobs/{job['id']}/trim",
|
||||
json={"start_seconds": 0.0, "end_seconds": 5.0, "segments": [seg(brightness=20.0, contrast=1.5, saturation=0.5)]},
|
||||
)
|
||||
assert res.status_code == 200
|
||||
|
||||
|
||||
def test_a_never_edited_job_reports_neutral_edit_values(client):
|
||||
job = _completed(client, "clip")
|
||||
body = client.get(f"/api/jobs/{job['id']}").json()
|
||||
assert body["edit_start_seconds"] == 0.0
|
||||
assert body["edit_end_seconds"] is None
|
||||
assert body["edit_segments"] == []
|
||||
|
||||
|
||||
def test_trim_response_records_the_applied_values(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
sections = [seg(brightness=20.0, contrast=1.5, saturation=0.5)]
|
||||
res = client.post(
|
||||
f"/api/jobs/{job['id']}/trim",
|
||||
json={"start_seconds": 1.0, "end_seconds": 3.0, "segments": sections},
|
||||
)
|
||||
body = res.json()
|
||||
assert body["edit_start_seconds"] == 1.0
|
||||
assert body["edit_end_seconds"] == 3.0
|
||||
assert body["edit_segments"] == sections
|
||||
# And getting the job again afterwards reflects the same recorded values.
|
||||
again = client.get(f"/api/jobs/{job['id']}").json()
|
||||
assert again["edit_segments"] == sections
|
||||
|
||||
|
||||
def test_trim_response_records_multiple_sections(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
sections = [seg(brightness=20.0, end_seconds=2.0), seg(contrast=1.5)]
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 0.0, "end_seconds": 5.0,
|
||||
"segments": sections})
|
||||
assert res.json()["edit_segments"] == sections
|
||||
|
||||
|
||||
def test_restore_resets_the_recorded_edit_values(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
client.post(
|
||||
f"/api/jobs/{job['id']}/trim",
|
||||
json={"start_seconds": 1.0, "end_seconds": 3.0, "segments": [seg(brightness=20.0, contrast=1.5, saturation=0.5)]},
|
||||
)
|
||||
res = client.post(f"/api/jobs/{job['id']}/restore-video")
|
||||
body = res.json()
|
||||
assert body["edit_start_seconds"] == 0.0
|
||||
assert body["edit_end_seconds"] is None
|
||||
assert body["edit_segments"] == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"segments",
|
||||
[
|
||||
[seg(brightness=200.0)],
|
||||
[seg(contrast=-1.0)],
|
||||
[seg(saturation=10.0)],
|
||||
],
|
||||
)
|
||||
def test_trim_refuses_an_out_of_range_color_value(client, segments):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
body = {"start_seconds": 0.0, "segments": segments}
|
||||
assert client.post(f"/api/jobs/{job['id']}/trim", json=body).status_code == 400
|
||||
|
||||
|
||||
def test_an_empty_section_list_is_not_an_error_matching_trim_py(client):
|
||||
# Same as sending no `segments` at all -- defaults to one neutral section.
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 0.0, "segments": []})
|
||||
assert res.status_code == 200
|
||||
|
||||
|
||||
def test_trim_refuses_a_non_last_open_ended_section(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
body = {"start_seconds": 0.0, "segments": [seg(), seg(end_seconds=2.0)]}
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json=body)
|
||||
assert res.status_code == 400 and "open-ended" in res.json()["detail"]
|
||||
|
||||
|
||||
def test_trim_refuses_sections_out_of_order(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
body = {"start_seconds": 0.0, "segments": [seg(end_seconds=3.0), seg(end_seconds=2.0), seg()]}
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json=body)
|
||||
assert res.status_code == 400 and "increasing order" in res.json()["detail"]
|
||||
|
||||
|
||||
def test_trim_refuses_an_unfinished_job(client):
|
||||
job = _create(client, name="pending")
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 0.0, "end_seconds": 1.0})
|
||||
assert res.status_code == 404
|
||||
|
||||
|
||||
def test_trim_and_restore_of_an_unknown_job(client):
|
||||
assert client.post("/api/jobs/nope/trim", json={"start_seconds": 0.0}).status_code == 404
|
||||
assert client.post("/api/jobs/nope/restore-video").status_code == 404
|
||||
|
||||
|
||||
def test_trimmed_frame_count_does_not_leak_the_internal_backup_key(client):
|
||||
job = _completed(client, "clip")
|
||||
mock_server._jobs[job["id"]]["num_frames"] = 240
|
||||
mock_server._jobs[job["id"]]["fps"] = 24
|
||||
res = client.post(f"/api/jobs/{job['id']}/trim", json={"start_seconds": 1.0, "end_seconds": 3.0})
|
||||
assert not any(k.startswith("_") for k in res.json())
|
||||
assert not any(k.startswith("_") for k in client.get(f"/api/jobs/{job['id']}").json())
|
||||
@@ -0,0 +1,499 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Cutting a finished clip to a chosen in/out range and adjusting its color, non-destructively."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from enum import Enum
|
||||
from types import SimpleNamespace
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
from fastvideo_studio.frames import FrameError
|
||||
from fastvideo_studio.trim import (
|
||||
BRIGHTNESS_RANGE,
|
||||
CONTRAST_RANGE,
|
||||
EDITED_FILENAME,
|
||||
ORIGINAL_FILENAME,
|
||||
SATURATION_RANGE,
|
||||
clip_duration_seconds,
|
||||
restore_original,
|
||||
trim_video,
|
||||
)
|
||||
|
||||
WIDTH, HEIGHT, FPS, FRAMES = 128, 96, 24, 48 # 2 seconds, one frame per level 0..47
|
||||
|
||||
|
||||
class Status(Enum): # mirrors JobStatus without importing job_runner (which needs fastvideo)
|
||||
PENDING = "pending"
|
||||
COMPLETED = "completed"
|
||||
FAILED = "failed"
|
||||
|
||||
|
||||
def _write_video(path: str, n: int = FRAMES, fps: int = FPS) -> None:
|
||||
"""A clip whose frames are numbered by brightness (frame i is roughly i*5 grey), so a range is identifiable."""
|
||||
import av
|
||||
|
||||
container = av.open(path, mode="w")
|
||||
try:
|
||||
stream = container.add_stream("h264", rate=fps)
|
||||
stream.width, stream.height, stream.pix_fmt = WIDTH, HEIGHT, "yuv420p"
|
||||
for i in range(n):
|
||||
level = min(i * 5, 255)
|
||||
frame = av.VideoFrame.from_ndarray(np.full((HEIGHT, WIDTH, 3), level, dtype=np.uint8), format="rgb24")
|
||||
for packet in stream.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in stream.encode():
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
|
||||
AUDIO_SAMPLE_RATE = 32000
|
||||
AUDIO_SAMPLES_PER_FRAME = 1024
|
||||
|
||||
|
||||
def _write_video_with_audio(path: str, n: int = FRAMES, fps: int = FPS) -> None:
|
||||
"""Same as ``_write_video``, plus a silent stereo AAC track, to test that edits keep audio."""
|
||||
import av
|
||||
from fractions import Fraction
|
||||
|
||||
container = av.open(path, mode="w")
|
||||
try:
|
||||
vstream = container.add_stream("h264", rate=fps)
|
||||
vstream.width, vstream.height, vstream.pix_fmt = WIDTH, HEIGHT, "yuv420p"
|
||||
astream = container.add_stream("aac", rate=AUDIO_SAMPLE_RATE)
|
||||
astream.layout = "stereo"
|
||||
|
||||
for i in range(n):
|
||||
level = min(i * 5, 255)
|
||||
frame = av.VideoFrame.from_ndarray(np.full((HEIGHT, WIDTH, 3), level, dtype=np.uint8), format="rgb24")
|
||||
for packet in vstream.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in vstream.encode():
|
||||
container.mux(packet)
|
||||
|
||||
audio_time_base = Fraction(1, AUDIO_SAMPLE_RATE)
|
||||
n_audio_frames = int(n / fps * AUDIO_SAMPLE_RATE / AUDIO_SAMPLES_PER_FRAME) + 2
|
||||
pts = 0
|
||||
for _ in range(n_audio_frames):
|
||||
arr = np.zeros((2, AUDIO_SAMPLES_PER_FRAME), dtype=np.float32)
|
||||
aframe = av.AudioFrame.from_ndarray(arr, format="fltp", layout="stereo")
|
||||
aframe.sample_rate = AUDIO_SAMPLE_RATE
|
||||
aframe.pts = pts
|
||||
aframe.time_base = audio_time_base
|
||||
pts += AUDIO_SAMPLES_PER_FRAME
|
||||
for packet in astream.encode(aframe):
|
||||
container.mux(packet)
|
||||
for packet in astream.encode(None):
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
|
||||
def _write_colored_video(path: str, rgb: tuple[int, int, int], n: int = 12, fps: int = FPS) -> None:
|
||||
"""A clip of one solid, non-gray color, so saturation changes are visible (unlike on a gray clip)."""
|
||||
import av
|
||||
|
||||
container = av.open(path, mode="w")
|
||||
try:
|
||||
stream = container.add_stream("h264", rate=fps)
|
||||
stream.width, stream.height, stream.pix_fmt = WIDTH, HEIGHT, "yuv420p"
|
||||
for _ in range(n):
|
||||
frame = av.VideoFrame.from_ndarray(np.full((HEIGHT, WIDTH, 3), rgb, dtype=np.uint8), format="rgb24")
|
||||
for packet in stream.encode(frame):
|
||||
container.mux(packet)
|
||||
for packet in stream.encode():
|
||||
container.mux(packet)
|
||||
finally:
|
||||
container.close()
|
||||
|
||||
|
||||
def _read_levels(path: str) -> list[int]:
|
||||
import av
|
||||
|
||||
with av.open(path) as container:
|
||||
return [int(f.to_ndarray(format="rgb24")[0, 0, 0]) for f in container.decode(container.streams.video[0])]
|
||||
|
||||
|
||||
def _read_mean_rgb(path: str, index: int = 0) -> np.ndarray:
|
||||
import av
|
||||
|
||||
with av.open(path) as container:
|
||||
frames = list(container.decode(container.streams.video[0]))
|
||||
return frames[index].to_ndarray(format="rgb24").astype(float).mean(axis=(0, 1))
|
||||
|
||||
|
||||
def _job(job_id="job-1", status=Status.COMPLETED, output_path=None):
|
||||
return SimpleNamespace(id=job_id, status=status, output_path=output_path)
|
||||
|
||||
|
||||
def _top_level_atom_offset(path: str, atom_type: bytes) -> int:
|
||||
"""Byte offset of a top-level MP4 box (e.g. b"moov", b"mdat"), or -1 if absent."""
|
||||
import struct
|
||||
|
||||
with open(path, "rb") as f:
|
||||
f.seek(0, 2)
|
||||
size = f.tell()
|
||||
f.seek(0)
|
||||
pos = 0
|
||||
while pos < size:
|
||||
f.seek(pos)
|
||||
header = f.read(8)
|
||||
if len(header) < 8:
|
||||
break
|
||||
atom_size, found_type = struct.unpack(">I4s", header)
|
||||
if found_type == atom_type:
|
||||
return pos
|
||||
if atom_size in (0, 1):
|
||||
break
|
||||
pos += atom_size
|
||||
return -1
|
||||
|
||||
|
||||
def _moov_offset(path: str) -> int:
|
||||
return _top_level_atom_offset(path, b"moov")
|
||||
|
||||
|
||||
def _mdat_offset(path: str) -> int:
|
||||
return _top_level_atom_offset(path, b"mdat")
|
||||
|
||||
|
||||
def seg(brightness=0.0, contrast=1.0, saturation=1.0, end_seconds=None):
|
||||
"""One razor-cut section dict, as the API/trim_video expects."""
|
||||
return {"end_seconds": end_seconds, "brightness": brightness, "contrast": contrast, "saturation": saturation}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def job_dir(tmp_path):
|
||||
d = tmp_path / "job-1"
|
||||
d.mkdir()
|
||||
return d
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def video(job_dir):
|
||||
path = str(job_dir / "wolf-lunch-clip-06.mp4")
|
||||
_write_video(path)
|
||||
return path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def colored_video(job_dir):
|
||||
path = str(job_dir / "wolf-lunch-clip-07.mp4")
|
||||
_write_colored_video(path, (200, 100, 50)) # a warm orange, clearly non-gray
|
||||
return path
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def video_with_audio(job_dir):
|
||||
path = str(job_dir / "wolf-lunch-clip-08.mp4")
|
||||
_write_video_with_audio(path)
|
||||
return path
|
||||
|
||||
|
||||
class TestTrimVideo:
|
||||
def test_keeps_only_the_requested_range(self, video, job_dir):
|
||||
path, kept = trim_video(_job(output_path=video), start_seconds=0.5, end_seconds=1.5)
|
||||
assert path == str(job_dir / EDITED_FILENAME)
|
||||
levels = _read_levels(path)
|
||||
assert kept == len(levels) == FPS # 1 second kept
|
||||
assert levels[0] == _read_levels(video)[12] # starts at 0.5s = frame 12
|
||||
assert levels[-1] == _read_levels(video)[35] # ends just before 1.5s = frame 36
|
||||
|
||||
def test_an_open_end_keeps_to_the_end_of_the_source(self, video):
|
||||
path, kept = trim_video(_job(output_path=video), start_seconds=1.0, end_seconds=None)
|
||||
assert kept == FRAMES - FPS
|
||||
assert _read_levels(path)[-1] == _read_levels(video)[-1]
|
||||
|
||||
def test_start_at_zero_keeps_from_the_beginning(self, video):
|
||||
path, _ = trim_video(_job(output_path=video), start_seconds=0.0, end_seconds=1.0)
|
||||
assert _read_levels(path)[0] == _read_levels(video)[0]
|
||||
|
||||
def test_the_original_is_preserved_byte_for_byte(self, video, job_dir):
|
||||
before = open(video, "rb").read()
|
||||
trim_video(_job(output_path=video), start_seconds=0.5, end_seconds=1.5)
|
||||
original = job_dir / ORIGINAL_FILENAME
|
||||
assert original.is_file()
|
||||
assert original.read_bytes() == before
|
||||
|
||||
def test_trimming_again_re_cuts_from_the_original_not_the_edited_file(self, video, job_dir):
|
||||
# If it re-cut the already-edited 0.5-1.5s file, asking for 0-2s again
|
||||
# could not recover the parts outside that first window.
|
||||
trim_video(_job(output_path=video), start_seconds=0.5, end_seconds=1.5)
|
||||
path, kept = trim_video(_job(output_path=video), start_seconds=0.0, end_seconds=2.0)
|
||||
assert kept == FRAMES
|
||||
assert _read_levels(path) == _read_levels(video)
|
||||
|
||||
def test_narrowing_the_range_further_still_re_cuts_from_the_original(self, video):
|
||||
trim_video(_job(output_path=video), start_seconds=0.0, end_seconds=2.0)
|
||||
path, kept = trim_video(_job(output_path=video), start_seconds=1.0, end_seconds=1.5)
|
||||
assert kept == FPS // 2
|
||||
assert _read_levels(path)[0] == _read_levels(video)[24]
|
||||
|
||||
def test_muxed_frame_rate_stays_correct(self, video):
|
||||
path, _ = trim_video(_job(output_path=video), start_seconds=0.0, end_seconds=1.0)
|
||||
import av
|
||||
|
||||
with av.open(path) as c:
|
||||
assert FPS - 1 <= float(c.streams.video[0].average_rate) <= FPS + 1
|
||||
|
||||
def test_output_is_faststart_moov_before_mdat(self, video):
|
||||
# Otherwise a browser has to reach nearly the end of the file just to read
|
||||
# its duration/seek table -- painfully slow over a network filesystem.
|
||||
path, _ = trim_video(_job(output_path=video), start_seconds=0.0, end_seconds=1.0)
|
||||
assert _moov_offset(path) < _mdat_offset(path)
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("start", "end", "match"),
|
||||
[
|
||||
(-1.0, None, "negative"),
|
||||
(1.0, 1.0, "after the start"),
|
||||
(1.0, 0.5, "after the start"),
|
||||
(100.0, None, "no frames"),
|
||||
],
|
||||
)
|
||||
def test_refuses_an_invalid_range(self, video, start, end, match):
|
||||
with pytest.raises(FrameError, match=match) as e:
|
||||
trim_video(_job(output_path=video), start_seconds=start, end_seconds=end)
|
||||
assert e.value.status_code == 400
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("job", "match"),
|
||||
[
|
||||
(_job(status=Status.PENDING), "No output"),
|
||||
(_job(status=Status.COMPLETED, output_path=None), "No output"),
|
||||
(_job(status=Status.COMPLETED, output_path="/nope.mp4"), "not found"),
|
||||
],
|
||||
)
|
||||
def test_refusals(self, job, match):
|
||||
with pytest.raises(FrameError, match=match):
|
||||
trim_video(job, 0.0, 1.0)
|
||||
|
||||
def test_refuses_a_non_video_output(self, tmp_path):
|
||||
img = tmp_path / "out.png"
|
||||
img.write_bytes(b"x")
|
||||
with pytest.raises(FrameError, match="not a video"):
|
||||
trim_video(_job(output_path=str(img)), 0.0, 1.0)
|
||||
|
||||
|
||||
class TestGrading:
|
||||
def test_neutral_values_do_not_alter_the_pixels(self, colored_video):
|
||||
path, _ = trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg()])
|
||||
got = _read_mean_rgb(path)
|
||||
want = _read_mean_rgb(colored_video)
|
||||
assert np.allclose(got, want, atol=2) # h264 is lossy even unchanged
|
||||
|
||||
def test_neutral_values_skip_the_per_frame_numpy_round_trip(self, colored_video, monkeypatch):
|
||||
import fastvideo_studio.trim as trim_module
|
||||
|
||||
calls = []
|
||||
monkeypatch.setattr(trim_module, "_apply_grade", lambda *a: calls.append(a) or a[0])
|
||||
trim_video(_job(output_path=colored_video), 0.0, None) # no segments -> the default single neutral one
|
||||
assert calls == []
|
||||
|
||||
def test_positive_brightness_lightens_every_channel(self, colored_video):
|
||||
path, _ = trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(brightness=50.0)])
|
||||
got = _read_mean_rgb(path)
|
||||
want = _read_mean_rgb(colored_video)
|
||||
assert (got > want + 30).all()
|
||||
|
||||
def test_negative_brightness_darkens(self, colored_video):
|
||||
path, _ = trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(brightness=-50.0)])
|
||||
got = _read_mean_rgb(path)
|
||||
want = _read_mean_rgb(colored_video)
|
||||
assert (got < want - 30).all()
|
||||
|
||||
def test_zero_saturation_makes_every_channel_equal(self, colored_video):
|
||||
path, _ = trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(saturation=0.0)])
|
||||
r, g, b = _read_mean_rgb(path)
|
||||
assert abs(r - g) < 2 and abs(g - b) < 2 # gray: channels converge on the frame's luma
|
||||
|
||||
def test_saturation_above_one_widens_the_channel_spread(self, colored_video):
|
||||
base_r, base_g, base_b = _read_mean_rgb(colored_video)
|
||||
path, _ = trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(saturation=2.0)])
|
||||
r, g, b = _read_mean_rgb(path)
|
||||
assert (r - b) > (base_r - base_b) # the warm/cool spread grows, not shrinks
|
||||
|
||||
def test_high_contrast_pushes_values_away_from_mid_gray(self, colored_video):
|
||||
path, _ = trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(contrast=2.0)])
|
||||
got = _read_mean_rgb(path)
|
||||
want = _read_mean_rgb(colored_video)
|
||||
# 128 = mid-gray. This clip's R (200) sits above it and should be pushed further up;
|
||||
# its G and B (100, 50) sit below and should be pushed further down -- the spread
|
||||
# around 128 widens either way.
|
||||
assert got[0] > want[0] # R: 200 -> further above 128
|
||||
assert got[1] < want[1] # G: 100 -> further below 128
|
||||
assert got[2] < want[2] # B: 50 -> further below 128
|
||||
|
||||
def test_grading_and_trimming_the_same_call_compose(self, video, colored_video):
|
||||
# A range trim and a color grade requested together both take effect --
|
||||
# neither is silently dropped by the other.
|
||||
path, kept = trim_video(
|
||||
_job(output_path=colored_video), start_seconds=0.2, end_seconds=0.3, segments=[seg(brightness=40.0)])
|
||||
assert kept == pytest.approx(FPS * 0.1, abs=1)
|
||||
base = _read_mean_rgb(colored_video)
|
||||
assert (_read_mean_rgb(path) > base + 20).all()
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("kwargs", "bounds"),
|
||||
[
|
||||
({"brightness": BRIGHTNESS_RANGE[1] + 1}, BRIGHTNESS_RANGE),
|
||||
({"brightness": BRIGHTNESS_RANGE[0] - 1}, BRIGHTNESS_RANGE),
|
||||
({"contrast": CONTRAST_RANGE[1] + 1}, CONTRAST_RANGE),
|
||||
({"contrast": CONTRAST_RANGE[0] - 1}, CONTRAST_RANGE),
|
||||
({"saturation": SATURATION_RANGE[1] + 1}, SATURATION_RANGE),
|
||||
({"saturation": SATURATION_RANGE[0] - 1}, SATURATION_RANGE),
|
||||
],
|
||||
)
|
||||
def test_refuses_an_out_of_range_value(self, colored_video, kwargs, bounds):
|
||||
with pytest.raises(FrameError, match="between") as e:
|
||||
trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(**kwargs)])
|
||||
assert e.value.status_code == 400
|
||||
|
||||
@pytest.mark.parametrize("kwargs", [{"brightness": BRIGHTNESS_RANGE[1]}, {"contrast": CONTRAST_RANGE[0]},
|
||||
{"saturation": SATURATION_RANGE[1]}])
|
||||
def test_the_edges_of_each_range_are_allowed(self, colored_video, kwargs):
|
||||
trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(**kwargs)]) # does not raise
|
||||
|
||||
|
||||
class TestSections:
|
||||
"""Razor-cut sections: an ordered, contiguous list of grades within the kept range."""
|
||||
|
||||
def test_two_sections_grade_independently(self, colored_video):
|
||||
path, kept = trim_video(
|
||||
_job(output_path=colored_video), 0.0, None,
|
||||
segments=[seg(brightness=80.0, end_seconds=0.25), seg()],
|
||||
)
|
||||
base = _read_mean_rgb(colored_video)
|
||||
assert (_read_mean_rgb(path, index=0) > base + 40).all() # first section, graded
|
||||
assert np.allclose(_read_mean_rgb(path, index=kept - 1), base, atol=2) # second section, untouched
|
||||
|
||||
def test_three_sections_each_get_their_own_grade(self, colored_video):
|
||||
path, kept = trim_video(
|
||||
_job(output_path=colored_video), 0.0, None,
|
||||
segments=[
|
||||
seg(saturation=0.0, end_seconds=4 / FPS),
|
||||
seg(brightness=80.0, end_seconds=8 / FPS),
|
||||
seg(),
|
||||
],
|
||||
)
|
||||
base = _read_mean_rgb(colored_video)
|
||||
r0, g0, b0 = _read_mean_rgb(path, index=0)
|
||||
assert abs(r0 - g0) < 2 and abs(g0 - b0) < 2 # first section desaturated
|
||||
assert (_read_mean_rgb(path, index=6) > base + 40).all() # second section brightened
|
||||
assert np.allclose(_read_mean_rgb(path, index=kept - 1), base, atol=2) # third section untouched
|
||||
|
||||
def test_sections_compose_with_the_range_trim(self, colored_video):
|
||||
# The cut point (0.1) is relative to the KEPT range, not the original file's timeline.
|
||||
path, kept = trim_video(
|
||||
_job(output_path=colored_video), start_seconds=0.1, end_seconds=0.4,
|
||||
segments=[seg(brightness=80.0, end_seconds=0.1), seg()],
|
||||
)
|
||||
base = _read_mean_rgb(colored_video)
|
||||
assert (_read_mean_rgb(path, index=0) > base + 40).all()
|
||||
assert np.allclose(_read_mean_rgb(path, index=kept - 1), base, atol=2)
|
||||
|
||||
def test_an_empty_section_list_defaults_to_one_neutral_section(self, colored_video):
|
||||
path, _ = trim_video(_job(output_path=colored_video), 0.0, None, segments=[])
|
||||
assert np.allclose(_read_mean_rgb(path), _read_mean_rgb(colored_video), atol=2)
|
||||
|
||||
def test_a_single_explicit_end_that_does_not_cover_the_clip_is_refused(self, colored_video):
|
||||
with pytest.raises(FrameError, match="cover the whole clip"):
|
||||
trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(end_seconds=0.1)])
|
||||
|
||||
def test_only_the_last_section_can_be_open_ended(self, colored_video):
|
||||
with pytest.raises(FrameError, match="open-ended"):
|
||||
trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(), seg(end_seconds=0.25)])
|
||||
|
||||
def test_sections_must_be_in_increasing_order(self, colored_video):
|
||||
with pytest.raises(FrameError, match="increasing order"):
|
||||
trim_video(
|
||||
_job(output_path=colored_video), 0.0, None,
|
||||
segments=[seg(end_seconds=0.3), seg(end_seconds=0.2), seg()],
|
||||
)
|
||||
|
||||
def test_a_section_too_short_to_keep_a_frame_is_refused(self, colored_video):
|
||||
# 0.001s at 24fps rounds to 0 frames -- passes the ordering check (> 0) but keeps nothing.
|
||||
with pytest.raises(FrameError, match="too short"):
|
||||
trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(end_seconds=0.001), seg()])
|
||||
|
||||
def test_a_section_with_an_out_of_range_grade_is_refused(self, colored_video):
|
||||
with pytest.raises(FrameError, match="between"):
|
||||
trim_video(
|
||||
_job(output_path=colored_video), 0.0, None,
|
||||
segments=[seg(end_seconds=0.25), seg(brightness=500.0)],
|
||||
)
|
||||
|
||||
def test_only_frames_in_a_graded_section_go_through_the_numpy_round_trip(self, colored_video, monkeypatch):
|
||||
import fastvideo_studio.trim as trim_module
|
||||
|
||||
calls = []
|
||||
original = trim_module._apply_grade
|
||||
monkeypatch.setattr(trim_module, "_apply_grade", lambda *a: calls.append(a) or original(*a))
|
||||
trim_video(
|
||||
_job(output_path=colored_video), 0.0, None,
|
||||
segments=[seg(end_seconds=0.25), seg(brightness=50.0)], # first neutral, second graded
|
||||
)
|
||||
assert 0 < len(calls) < 12 # some frames graded, not all -- the neutral section was skipped
|
||||
|
||||
|
||||
class TestAudio:
|
||||
def test_a_source_with_no_audio_stays_video_only(self, video):
|
||||
path, _ = trim_video(_job(output_path=video), start_seconds=0.5, end_seconds=1.5)
|
||||
import av
|
||||
|
||||
with av.open(path) as c:
|
||||
assert [s.type for s in c.streams] == ["video"]
|
||||
|
||||
def test_a_range_trim_keeps_the_audio_track(self, video_with_audio):
|
||||
path, _ = trim_video(_job(output_path=video_with_audio), start_seconds=0.5, end_seconds=1.5)
|
||||
import av
|
||||
|
||||
with av.open(path) as c:
|
||||
assert "audio" in [s.type for s in c.streams]
|
||||
a = c.streams.audio[0]
|
||||
assert a.codec_context.sample_rate == AUDIO_SAMPLE_RATE
|
||||
assert sum(1 for _ in c.decode(a)) > 0
|
||||
|
||||
def test_grading_alongside_a_trim_also_keeps_the_audio_track(self, video_with_audio):
|
||||
path, _ = trim_video(
|
||||
_job(output_path=video_with_audio), start_seconds=0.0, end_seconds=1.0, segments=[seg(brightness=40.0)])
|
||||
import av
|
||||
|
||||
with av.open(path) as c:
|
||||
assert "audio" in [s.type for s in c.streams]
|
||||
|
||||
def test_restore_original_keeps_the_audio_track(self, video_with_audio, job_dir):
|
||||
trim_video(_job(output_path=video_with_audio), start_seconds=0.5, end_seconds=1.5)
|
||||
path, _ = restore_original(_job(output_path=str(job_dir / EDITED_FILENAME)))
|
||||
import av
|
||||
|
||||
with av.open(path) as c:
|
||||
assert "audio" in [s.type for s in c.streams]
|
||||
|
||||
|
||||
class TestRestoreOriginal:
|
||||
def test_restores_the_original_video_and_its_frame_count(self, video, job_dir):
|
||||
trim_video(_job(output_path=video), start_seconds=0.5, end_seconds=1.5)
|
||||
path, n = restore_original(_job(output_path=str(job_dir / EDITED_FILENAME)))
|
||||
assert path == str(job_dir / ORIGINAL_FILENAME)
|
||||
assert n == FRAMES
|
||||
assert _read_levels(path) == _read_levels(video)
|
||||
|
||||
def test_restores_past_a_grade_too(self, colored_video, job_dir):
|
||||
trim_video(_job(output_path=colored_video), 0.0, None, segments=[seg(brightness=50.0, saturation=0.0)])
|
||||
path, _ = restore_original(_job(output_path=str(job_dir / EDITED_FILENAME)))
|
||||
assert np.allclose(_read_mean_rgb(path), _read_mean_rgb(colored_video), atol=2)
|
||||
|
||||
def test_refuses_a_job_that_was_never_edited(self, video):
|
||||
with pytest.raises(FrameError, match="not been edited"):
|
||||
restore_original(_job(output_path=video))
|
||||
|
||||
def test_refuses_a_job_with_no_output(self):
|
||||
with pytest.raises(FrameError, match="No output"):
|
||||
restore_original(_job(output_path=None))
|
||||
|
||||
|
||||
def test_clip_duration_seconds(video):
|
||||
assert clip_duration_seconds(video) == pytest.approx(FRAMES / FPS, abs=0.05)
|
||||
@@ -0,0 +1,310 @@
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
"""Cut a finished clip to a chosen in/out range and adjust its color, non-destructively.
|
||||
|
||||
Kept apart from ``frames.py`` (which saves the *end* of a clip for another job
|
||||
to continue from) because this edits a clip's own timeline, not a reference
|
||||
for someone else. Needs only PyAV and numpy, so it can be exercised without a GPU.
|
||||
|
||||
Trimming and grading share one pipeline rather than being two independent
|
||||
edits, on purpose: both always re-render from the untouched original in a
|
||||
single pass with the *current* full set of parameters (range and color
|
||||
together). Two separate "always start from the original" operations would
|
||||
each undo the other's work the moment they ran -- grading, then trimming,
|
||||
would silently discard the grade, because the trim would re-cut the original
|
||||
as if the grade had never happened. Rendering everything from one function
|
||||
each time is what lets adjusting either one leave the other in place.
|
||||
|
||||
Color grading is per razor-cut section, not the whole clip at once: the kept
|
||||
range is split at zero or more cut points into contiguous sections (no
|
||||
reordering, no gaps), each with its own brightness/contrast/saturation. A
|
||||
plain range trim with no cuts is just the one-section case. Audio is one
|
||||
continuous track regardless -- cuts only change which grade a video frame
|
||||
gets, they don't touch the audio.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shutil
|
||||
from fractions import Fraction
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
from fastvideo_studio.frames import VIDEO_EXTENSIONS, FrameError
|
||||
|
||||
ORIGINAL_FILENAME = "original.mp4"
|
||||
EDITED_FILENAME = "edited.mp4"
|
||||
|
||||
# Neutral (no-op) values for each color control, and the range each is clamped to.
|
||||
BRIGHTNESS_RANGE = (-100.0, 100.0) # added directly to 0-255 pixel values
|
||||
CONTRAST_RANGE = (0.0, 3.0) # 1.0 = unchanged
|
||||
SATURATION_RANGE = (0.0, 3.0) # 1.0 = unchanged, 0 = grayscale
|
||||
|
||||
# Perceptual luma weights, matching the ones used elsewhere in Studio for brightness checks.
|
||||
_LUMA_WEIGHTS = np.array([0.2126, 0.7152, 0.0722], dtype=np.float32)
|
||||
|
||||
|
||||
def clip_duration_seconds(video_path: str) -> float:
|
||||
"""Decode ``video_path`` and return its length in seconds."""
|
||||
import av
|
||||
|
||||
with av.open(video_path) as container:
|
||||
stream = container.streams.video[0]
|
||||
fps = float(stream.average_rate or stream.guessed_rate or 24)
|
||||
n = sum(1 for _ in container.decode(stream))
|
||||
if n == 0:
|
||||
raise ValueError("The video has no frames.")
|
||||
return n / fps
|
||||
|
||||
|
||||
def _clamp_range(name: str, value: float, bounds: tuple[float, float]) -> float:
|
||||
low, high = bounds
|
||||
if not (low <= value <= high):
|
||||
raise ValueError(f"{name} must be between {low:g} and {high:g}, got {value:g}.")
|
||||
return value
|
||||
|
||||
|
||||
def _apply_grade(array: np.ndarray, brightness: float, contrast: float, saturation: float) -> np.ndarray:
|
||||
"""Adjust one RGB uint8 frame. Neutral values (0, 1, 1) are a no-op, skipped by the caller."""
|
||||
pixels = array.astype(np.float32)
|
||||
if saturation != 1.0:
|
||||
luma = (pixels * _LUMA_WEIGHTS).sum(axis=-1, keepdims=True)
|
||||
pixels = luma + (pixels - luma) * saturation
|
||||
if contrast != 1.0:
|
||||
pixels = (pixels - 128.0) * contrast + 128.0
|
||||
if brightness != 0.0:
|
||||
pixels = pixels + brightness
|
||||
return np.clip(pixels, 0, 255).astype(np.uint8)
|
||||
|
||||
|
||||
#: A neutral single section covering the whole kept range -- what a never-edited
|
||||
#: clip (or a call that doesn't care about sections) gets by default.
|
||||
DEFAULT_SEGMENTS: list[dict[str, Any]] = [{"end_seconds": None, "brightness": 0.0, "contrast": 1.0, "saturation": 1.0}]
|
||||
|
||||
|
||||
class _Segment:
|
||||
"""One validated, frame-indexed section: [start_frame, end_frame) plus its grade."""
|
||||
|
||||
__slots__ = ("start_frame", "end_frame", "brightness", "contrast", "saturation")
|
||||
|
||||
def __init__(self, start_frame: int, end_frame: int, brightness: float, contrast: float, saturation: float):
|
||||
self.start_frame = start_frame
|
||||
self.end_frame = end_frame
|
||||
self.brightness = brightness
|
||||
self.contrast = contrast
|
||||
self.saturation = saturation
|
||||
|
||||
@property
|
||||
def graded(self) -> bool:
|
||||
return self.brightness != 0.0 or self.contrast != 1.0 or self.saturation != 1.0
|
||||
|
||||
|
||||
def _normalize_segments(segments: list[dict[str, Any]] | None, kept_frames: int, fps: float) -> list[_Segment]:
|
||||
"""Validate a razor-cut section list and convert each boundary to a frame index.
|
||||
|
||||
Sections are contiguous and cover the whole kept range in order -- there's
|
||||
no reordering or gaps, only where to cut and what to grade each side. Cut
|
||||
points are seconds into the *kept* (already start/end-trimmed) range, i.e.
|
||||
the same timeline the preview video itself plays, not the original file's.
|
||||
"""
|
||||
segments = segments or DEFAULT_SEGMENTS
|
||||
|
||||
out: list[_Segment] = []
|
||||
start_frame = 0
|
||||
prev_end_seconds = 0.0
|
||||
for i, seg in enumerate(segments):
|
||||
is_last = i == len(segments) - 1
|
||||
end_seconds = seg.get("end_seconds")
|
||||
brightness = _clamp_range("brightness", float(seg.get("brightness", 0.0)), BRIGHTNESS_RANGE)
|
||||
contrast = _clamp_range("contrast", float(seg.get("contrast", 1.0)), CONTRAST_RANGE)
|
||||
saturation = _clamp_range("saturation", float(seg.get("saturation", 1.0)), SATURATION_RANGE)
|
||||
if end_seconds is None:
|
||||
if not is_last:
|
||||
raise ValueError("Only the last section can be open-ended.")
|
||||
end_frame = kept_frames
|
||||
else:
|
||||
if end_seconds <= prev_end_seconds:
|
||||
raise ValueError("Sections must be in increasing order.")
|
||||
end_frame = min(kept_frames, round(end_seconds * fps))
|
||||
prev_end_seconds = end_seconds
|
||||
if end_frame <= start_frame:
|
||||
raise ValueError("A section is too short to keep any frames.")
|
||||
out.append(_Segment(start_frame, end_frame, brightness, contrast, saturation))
|
||||
start_frame = end_frame
|
||||
if start_frame < kept_frames:
|
||||
raise ValueError("The sections must cover the whole clip; the last one can't end early.")
|
||||
return out
|
||||
|
||||
|
||||
def _render_edit(
|
||||
video_path: str,
|
||||
out_path: str,
|
||||
start_seconds: float,
|
||||
end_seconds: float | None,
|
||||
segments: list[dict[str, Any]] | None,
|
||||
) -> int:
|
||||
"""Re-encode [``start_seconds``, ``end_seconds``) of ``video_path`` to ``out_path``, graded per section.
|
||||
|
||||
Re-encoded rather than stream-copied, for a frame-accurate cut regardless
|
||||
of where the source's keyframes happen to fall. Returns the frame count kept.
|
||||
"""
|
||||
import av
|
||||
|
||||
if start_seconds < 0:
|
||||
raise ValueError("The start of the range can't be negative.")
|
||||
if end_seconds is not None and end_seconds <= start_seconds:
|
||||
raise ValueError("The end of the range must come after the start.")
|
||||
|
||||
with av.open(video_path) as src:
|
||||
video_stream = src.streams.video[0]
|
||||
audio_stream = src.streams.audio[0] if src.streams.audio else None
|
||||
fps = float(video_stream.average_rate or video_stream.guessed_rate or 24)
|
||||
frames: list[Any] = []
|
||||
audio_frames: list[Any] = []
|
||||
# One decode pass covers both streams -- the container can't be
|
||||
# decoded a second time once its packets are exhausted.
|
||||
decode_streams = (video_stream, audio_stream) if audio_stream else (video_stream,)
|
||||
for frame in src.decode(*decode_streams):
|
||||
(audio_frames if isinstance(frame, av.AudioFrame) else frames).append(frame)
|
||||
if not frames:
|
||||
raise ValueError("The video has no frames.")
|
||||
|
||||
start_i = max(0, round(start_seconds * fps))
|
||||
end_i = len(frames) if end_seconds is None else min(len(frames), round(end_seconds * fps))
|
||||
kept = frames[start_i:end_i]
|
||||
if not kept:
|
||||
raise ValueError("That range keeps no frames.")
|
||||
# Audio is cut to the same frame-granular window as the video, not the raw
|
||||
# requested seconds, so the two tracks stay aligned. Razor cuts only affect
|
||||
# which grade a frame gets, not the audio, which stays one continuous track.
|
||||
kept_audio = [f for f in audio_frames if f.time is not None and start_i / fps <= f.time < end_i / fps]
|
||||
section_list = _normalize_segments(segments, len(kept), fps)
|
||||
|
||||
tmp_path = f"{out_path}.tmp.mp4"
|
||||
# movflags=faststart puts the moov atom (duration, seek table) at the front of
|
||||
# the file instead of the end -- without it, a browser has to reach nearly the
|
||||
# end of the file just to read the video's metadata, which is painfully slow
|
||||
# over a network filesystem.
|
||||
with av.open(tmp_path, mode="w", options={"movflags": "faststart"}) as out:
|
||||
out_stream = out.add_stream("h264", rate=round(fps))
|
||||
out_stream.width, out_stream.height = kept[0].width, kept[0].height
|
||||
out_stream.pix_fmt = "yuv420p"
|
||||
# Every output stream has to be added before the first packet is muxed
|
||||
# (that's what writes the container header), so the audio stream and
|
||||
# resampler are set up here even though they're only used after the
|
||||
# video loop below.
|
||||
out_audio_stream = None
|
||||
resampler = None
|
||||
if kept_audio:
|
||||
out_audio_stream = out.add_stream("aac", rate=audio_stream.codec_context.sample_rate)
|
||||
out_audio_stream.layout = audio_stream.codec_context.layout.name
|
||||
resampler = av.AudioResampler(
|
||||
format=out_audio_stream.format,
|
||||
layout=out_audio_stream.layout,
|
||||
rate=out_audio_stream.rate,
|
||||
)
|
||||
|
||||
# Each frame needs its own increasing pts at a known time base, or the
|
||||
# muxed container's frame-rate metadata comes out wrong (see frames.py's
|
||||
# save_last_clip, which hit exactly this).
|
||||
frame_time_base = Fraction(1, round(fps))
|
||||
section_i = 0
|
||||
for i, frame in enumerate(kept):
|
||||
while i >= section_list[section_i].end_frame:
|
||||
section_i += 1
|
||||
section = section_list[section_i]
|
||||
if section.graded:
|
||||
array = _apply_grade(frame.to_ndarray(format="rgb24"), section.brightness, section.contrast,
|
||||
section.saturation)
|
||||
frame = av.VideoFrame.from_ndarray(array, format="rgb24")
|
||||
frame.pts = i
|
||||
frame.time_base = frame_time_base
|
||||
for packet in out_stream.encode(frame):
|
||||
out.mux(packet)
|
||||
for packet in out_stream.encode():
|
||||
out.mux(packet)
|
||||
|
||||
if out_audio_stream is not None:
|
||||
for frame in kept_audio:
|
||||
frame.pts = None # let the resampler retime relative to the trimmed start
|
||||
for rframe in resampler.resample(frame):
|
||||
for packet in out_audio_stream.encode(rframe):
|
||||
out.mux(packet)
|
||||
for packet in out_audio_stream.encode(None):
|
||||
out.mux(packet)
|
||||
os.replace(tmp_path, out_path) # atomic, so a reader never sees a half-written file
|
||||
return len(kept)
|
||||
|
||||
|
||||
def _require_completed_video(job: Any) -> None:
|
||||
status = getattr(job.status, "value", job.status)
|
||||
if status != "completed" or not job.output_path:
|
||||
raise FrameError(404, "No output available for this job")
|
||||
if not os.path.isfile(job.output_path):
|
||||
raise FrameError(404, "Output file not found on disk")
|
||||
if Path(job.output_path).suffix.lower() not in VIDEO_EXTENSIONS:
|
||||
raise FrameError(400, "This job's output is not a video")
|
||||
|
||||
|
||||
def trim_video(
|
||||
job: Any,
|
||||
start_seconds: float = 0.0,
|
||||
end_seconds: float | None = None,
|
||||
segments: list[dict[str, Any]] | None = None,
|
||||
) -> tuple[str, int]:
|
||||
"""Cut ``job``'s video to [``start_seconds``, ``end_seconds``) and grade it section by section.
|
||||
|
||||
``segments`` is an ordered, contiguous list of razor-cut sections covering
|
||||
the whole kept range -- each a ``{"end_seconds", "brightness", "contrast",
|
||||
"saturation"}`` dict, ``end_seconds`` being seconds into the *kept* range
|
||||
(only the last section's may be ``None``, meaning to the end). Omitted or
|
||||
empty defaults to one neutral section covering everything, i.e. what a
|
||||
plain range trim with no grading looks like.
|
||||
|
||||
Always renders from the untouched original with this *full* set of
|
||||
parameters, so trimming and grading compose -- adjusting one doesn't
|
||||
discard the other, and neither ever re-encodes an already-edited file.
|
||||
Returns ``(new_output_path, frames_kept)``. Raises :class:`FrameError` with
|
||||
the status the API should report.
|
||||
"""
|
||||
_require_completed_video(job)
|
||||
job_dir = os.path.dirname(job.output_path)
|
||||
original_path = os.path.join(job_dir, ORIGINAL_FILENAME)
|
||||
if not os.path.isfile(original_path):
|
||||
# First edit: keep an untouched copy of whatever is there now before
|
||||
# changing anything, so every later edit (and a restore) is relative
|
||||
# to this, never to a file this function already produced.
|
||||
shutil.copy2(job.output_path, original_path)
|
||||
|
||||
edited_path = os.path.join(job_dir, EDITED_FILENAME)
|
||||
try:
|
||||
kept = _render_edit(original_path, edited_path, start_seconds, end_seconds, segments)
|
||||
except FrameError:
|
||||
raise
|
||||
except Exception as e:
|
||||
raise FrameError(400, str(e)) from e
|
||||
return edited_path, kept
|
||||
|
||||
|
||||
def restore_original(job: Any) -> tuple[str, int]:
|
||||
"""Undo every edit: the path of ``job``'s untouched original video, and its frame count.
|
||||
|
||||
Undoes trimming and grading together, since they share one edit state.
|
||||
Raises :class:`FrameError` if the job was never edited.
|
||||
"""
|
||||
if not job.output_path:
|
||||
raise FrameError(404, "No output available for this job")
|
||||
original_path = os.path.join(os.path.dirname(job.output_path), ORIGINAL_FILENAME)
|
||||
if not os.path.isfile(original_path):
|
||||
raise FrameError(404, "This job has not been edited.")
|
||||
try:
|
||||
import av
|
||||
|
||||
with av.open(original_path) as container:
|
||||
stream = container.streams.video[0]
|
||||
n = sum(1 for _ in container.decode(stream))
|
||||
except Exception as e:
|
||||
raise FrameError(500, f"Could not read the original video: {e}") from e
|
||||
return original_path, n
|
||||
@@ -128,6 +128,30 @@ def _resolve_output_size(
|
||||
return fallback
|
||||
|
||||
|
||||
def _find_ffmpeg() -> str | None:
|
||||
"""An ffmpeg to run: $FASTVIDEO_FFMPEG_BIN, then PATH, then imageio-ffmpeg's bundled one.
|
||||
|
||||
A plain PATH lookup misses environments (this one included) where ffmpeg
|
||||
is only present as the copy imageio-ffmpeg installs for its own use, never
|
||||
exposed on PATH -- without this fallback, every save silently drops to the
|
||||
PyAV path below, which has no '+faststart', producing MP4s slow to start
|
||||
playback over a network filesystem (the metadata ends up at the end of the
|
||||
file instead of the front).
|
||||
"""
|
||||
configured = os.getenv("FASTVIDEO_FFMPEG_BIN")
|
||||
if configured and (shutil.which(configured) or os.path.isfile(configured)):
|
||||
return shutil.which(configured) or configured
|
||||
on_path = shutil.which("ffmpeg")
|
||||
if on_path:
|
||||
return on_path
|
||||
try:
|
||||
import imageio_ffmpeg
|
||||
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _validate_request_stage_overrides(model_path: str, request: GenerationRequest) -> None:
|
||||
"""Validate typed stage overrides against the model's registered preset."""
|
||||
if not request.stage_overrides:
|
||||
@@ -1100,7 +1124,10 @@ class VideoGenerator:
|
||||
try:
|
||||
audio_int16, num_channels = cls._audio_to_int16(audio)
|
||||
layout = "stereo" if num_channels == 2 else "mono"
|
||||
output = av.open(output_path, mode="w")
|
||||
# movflags=faststart puts the moov atom (duration, seek table) at the
|
||||
# front of the file instead of the end -- this is the fallback path
|
||||
# (ffmpeg unavailable), so it's the one place nothing else adds it.
|
||||
output = av.open(output_path, mode="w", options={"movflags": "faststart"})
|
||||
video_stream = output.add_stream("libx264", rate=fps)
|
||||
video_stream.width = int(frames[0].shape[1])
|
||||
video_stream.height = int(frames[0].shape[0])
|
||||
@@ -1185,9 +1212,11 @@ class VideoGenerator:
|
||||
sample_rate: int,
|
||||
) -> bool:
|
||||
"""Encode video+audio using ffmpeg via rawvideo stdin + WAV input."""
|
||||
ffmpeg_bin = shutil.which(os.getenv("FASTVIDEO_FFMPEG_BIN", "ffmpeg"))
|
||||
ffmpeg_bin = _find_ffmpeg()
|
||||
if ffmpeg_bin is None:
|
||||
logger.warning("ffmpeg not found; cannot use ffmpeg pipe save.")
|
||||
logger.warning("ffmpeg not found (checked $FASTVIDEO_FFMPEG_BIN, PATH, and imageio-ffmpeg's bundled "
|
||||
"binary); cannot use ffmpeg pipe save. Falling back to a PyAV save that skips "
|
||||
"'+faststart', so saved clips may be slow to start playback over a network filesystem.")
|
||||
return False
|
||||
|
||||
if not frames:
|
||||
|
||||
Reference in New Issue
Block a user