Shot Planner / Shot Loop: split long videos into model-sized shots (1.48.0)
A video model only follows a guide inside the clip length it was trained on, so long sources are split into shots that fit and run once per shot. - BFS Shot Planner: interactive Vue timeline (thumbnails, cut-detector curve, draggable boundaries, split/merge, per-shot reference gallery with upload, per-shot prompt). Splits at camera cuts via PySceneDetect (adaptive/content, built-in fallback), fixed length, or by hand; lengths snap to the model grid (H3 17n+5, LTX/Wan 8n+1, 4n+1). Outputs the shots as a ComfyUI list so everything downstream runs once per shot. - BFS Shot Unpack: guide frames, references, prompt, length, audio of one shot. - BFS Shot H3 Conditioning: native MiniMax H3 conditioning per shot (aligned guide, native reference video, or both). - BFS Shot Join: concatenates in order, trims each shot to its true length, cross-fades soft joins with the real overlap frames, returns the soundtrack. - Queue loop mode: one shot per run stored on disk, auto re-queue, nodes after the join blocked until the last shot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
alisson-anjos
co-authored by
Claude Opus 5.5
Claude Opus 5.5
parent
f145527ea5
commit
bf473e1dbd
@@ -123,6 +123,24 @@ The offset is applied only to appearance/reference inputs. Guide videos, masks,
|
||||
and target video positions remain unchanged. In **LTX Multiple Controls**, the field is named
|
||||
`identity_temporal_offset_latents` and affects only `identity_image`.
|
||||
|
||||
## Shot Planner / Shot Loop (1.48.0)
|
||||
|
||||
Split a long video into shots a video model can follow (at camera cuts via PySceneDetect, fixed
|
||||
length, or by hand on a timeline), set a reference image and prompt per shot, run the rest of the
|
||||
workflow once per shot, and join the results back with the original timing and soundtrack.
|
||||
|
||||
- **BFS Shot Planner** — interactive timeline (thumbnails, cut detector curve, draggable
|
||||
boundaries, per-shot reference gallery and prompt). Outputs the shots as a ComfyUI list, so
|
||||
every node downstream runs once per shot.
|
||||
- **BFS Shot Unpack** — guide frames, references, prompt and length of one shot, for any model.
|
||||
- **BFS Shot H3 Conditioning** — MiniMax H3 conditioning for one shot with the native nodes
|
||||
(aligned guide and/or native reference video).
|
||||
- **BFS Shot Join** — concatenates, trims to the true lengths, cross-fades soft joins, returns audio.
|
||||
|
||||
Two run modes: *auto loop* (all shots in one run) and *queue loop* (one shot per run, stored on
|
||||
disk, re-queued automatically; nodes after the join only run on the last shot). See
|
||||
[`web/docs/BFSShotPlanner.md`](web/docs/BFSShotPlanner.md).
|
||||
|
||||
## Requirements
|
||||
|
||||
- ComfyUI
|
||||
|
||||
@@ -74,6 +74,12 @@ except Exception as _e: # noqa
|
||||
# gain came from the projector + LoRA, not the CAN). Kept the file but not registered.
|
||||
CAN_NODE_CLASS_MAPPINGS, CAN_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
|
||||
|
||||
try:
|
||||
from .bfs_shot_loop import NODE_CLASS_MAPPINGS as SHOT_NODE_CLASS_MAPPINGS
|
||||
from .bfs_shot_loop import NODE_DISPLAY_NAME_MAPPINGS as SHOT_NODE_DISPLAY_NAME_MAPPINGS
|
||||
except Exception as _e: # noqa
|
||||
print(f"[BFSNodes] Shot loop nodes not loaded: {_e!r}")
|
||||
SHOT_NODE_CLASS_MAPPINGS, SHOT_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
|
||||
try:
|
||||
from .minimax_h3_downscaled_guide import NODE_CLASS_MAPPINGS as H3GUIDE_NODE_CLASS_MAPPINGS
|
||||
from .minimax_h3_downscaled_guide import NODE_DISPLAY_NAME_MAPPINGS as H3GUIDE_NODE_DISPLAY_NAME_MAPPINGS
|
||||
@@ -97,6 +103,7 @@ except Exception as _e: # noqa
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
**H3GUIDE_NODE_CLASS_MAPPINGS,
|
||||
**SHOT_NODE_CLASS_MAPPINGS,
|
||||
**GV_NODE_CLASS_MAPPINGS,
|
||||
**CAN_NODE_CLASS_MAPPINGS,
|
||||
**BFS_NODE_CLASS_MAPPINGS,
|
||||
@@ -122,6 +129,7 @@ NODE_CLASS_MAPPINGS = {
|
||||
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
**H3GUIDE_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**SHOT_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**GV_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**CAN_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
**BFS_NODE_DISPLAY_NAME_MAPPINGS,
|
||||
|
||||
@@ -0,0 +1,783 @@
|
||||
"""BFS Shot Loop: plan a long video into model-sized shots, run any workflow once per shot, join.
|
||||
|
||||
A video model only follows a guide reliably inside the clip length it was trained on (an H3 LoRA
|
||||
trained on 107 frames loses the scene at 243). This splits the source into shots that fit, at
|
||||
camera cuts when there are any, and hands them to the rest of the graph as a ComfyUI *list*.
|
||||
ComfyUI then runs every node that receives a list once per item, pairing the lists by index, so
|
||||
shot ``i`` always gets guide ``i``, reference ``i`` and prompt ``i``. ``BFS Shot Join`` takes the
|
||||
decoded list back, trims each shot to its true length and concatenates them in order.
|
||||
|
||||
Every shot is generated at a length the model accepts (for example 17n+5 frames for MiniMax H3).
|
||||
When a shot is shorter than that, the extra guide frames are taken from the video that follows it,
|
||||
so the guide never repeats or stretches; the join then cuts the result back to the shot's own
|
||||
frames, which keeps the timing identical to the source. Those extra frames also give the join a
|
||||
real overlap to cross-fade across boundaries that are not camera cuts.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import json
|
||||
import math
|
||||
import os
|
||||
import subprocess
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
|
||||
import folder_paths
|
||||
|
||||
# ---------------------------------------------------------------------------- frame grids
|
||||
|
||||
GRIDS: dict[str, tuple[int, int]] = {
|
||||
"H3 (17n+5)": (17, 5),
|
||||
"LTX / Wan (8n+1)": (8, 1),
|
||||
"Wan (4n+1)": (4, 1),
|
||||
"any": (1, 0),
|
||||
}
|
||||
DEFAULT_GRID = "H3 (17n+5)"
|
||||
VIDEO_EXTS = (".mp4", ".mov", ".webm", ".mkv", ".avi", ".m4v", ".gif")
|
||||
IMAGE_EXTS = (".png", ".jpg", ".jpeg", ".webp", ".bmp")
|
||||
|
||||
|
||||
def snap_up(n: int, grid: str) -> int:
|
||||
"""Smallest valid clip length >= n."""
|
||||
step, off = GRIDS.get(grid, GRIDS[DEFAULT_GRID])
|
||||
n = max(1, int(n))
|
||||
if step == 1:
|
||||
return n
|
||||
if n <= off:
|
||||
return off
|
||||
return off + step * math.ceil((n - off) / step)
|
||||
|
||||
|
||||
def snap_down(n: int, grid: str) -> int:
|
||||
"""Largest valid clip length <= n (never below the grid's smallest length)."""
|
||||
step, off = GRIDS.get(grid, GRIDS[DEFAULT_GRID])
|
||||
n = max(1, int(n))
|
||||
if step == 1:
|
||||
return n
|
||||
if n <= off:
|
||||
return off
|
||||
return off + step * ((n - off) // step)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------- video io
|
||||
|
||||
def _input_path(name: str) -> str:
|
||||
path = folder_paths.get_annotated_filepath(name) if name else ""
|
||||
if not path or not os.path.isfile(path):
|
||||
raise FileNotFoundError(f"BFS Shot Loop: file not found in the input folder: {name!r}")
|
||||
return path
|
||||
|
||||
|
||||
def _probe(path: str) -> dict:
|
||||
import cv2
|
||||
cap = cv2.VideoCapture(path)
|
||||
if not cap.isOpened():
|
||||
raise ValueError(f"BFS Shot Loop: cannot open video {path}")
|
||||
fps = cap.get(cv2.CAP_PROP_FPS) or 0.0
|
||||
n = int(cap.get(cv2.CAP_PROP_FRAME_COUNT) or 0)
|
||||
w = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH) or 0)
|
||||
h = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT) or 0)
|
||||
cap.release()
|
||||
if fps <= 0 or n <= 0:
|
||||
raise ValueError(f"BFS Shot Loop: could not read fps/frame count from {path}")
|
||||
return {"fps_src": float(fps), "n_src": n, "width": w, "height": h, "duration": n / fps}
|
||||
|
||||
|
||||
def _timeline(n_src: int, fps_src: float, fps: float) -> np.ndarray:
|
||||
"""Source frame index for each frame of the resampled timeline (nearest frame, real time)."""
|
||||
n = max(1, int(math.floor(n_src / fps_src * fps + 1e-6)))
|
||||
idx = np.round(np.arange(n) * fps_src / fps).astype(np.int64)
|
||||
return np.clip(idx, 0, n_src - 1)
|
||||
|
||||
|
||||
def _read_frames(path: str, src_indices: np.ndarray, size: tuple[int, int] | None) -> list[np.ndarray]:
|
||||
"""Decode the requested source frames (sorted, may repeat) as RGB uint8, sequentially."""
|
||||
import cv2
|
||||
want = sorted(set(int(i) for i in src_indices))
|
||||
got: dict[int, np.ndarray] = {}
|
||||
cap = cv2.VideoCapture(path)
|
||||
i, k = 0, 0
|
||||
last = None
|
||||
while k < len(want):
|
||||
ok = cap.grab()
|
||||
if not ok:
|
||||
break
|
||||
if i == want[k]:
|
||||
ok, fr = cap.retrieve()
|
||||
if ok:
|
||||
fr = cv2.cvtColor(fr, cv2.COLOR_BGR2RGB)
|
||||
if size is not None:
|
||||
fr = _fit(fr, size)
|
||||
last = fr
|
||||
got[i] = fr
|
||||
while k < len(want) and want[k] == i:
|
||||
k += 1
|
||||
i += 1
|
||||
cap.release()
|
||||
if not got:
|
||||
raise ValueError(f"BFS Shot Loop: no frames decoded from {path}")
|
||||
out = []
|
||||
for s in src_indices:
|
||||
s = int(s)
|
||||
out.append(got.get(s, last if s > max(got) else got[min(got, key=lambda x: abs(x - s))]))
|
||||
return out
|
||||
|
||||
|
||||
def _fit(img: np.ndarray, size: tuple[int, int]) -> np.ndarray:
|
||||
"""Cover-resize then center-crop to (w, h)."""
|
||||
import cv2
|
||||
w, h = size
|
||||
ih, iw = img.shape[:2]
|
||||
s = max(w / iw, h / ih)
|
||||
r = cv2.resize(img, (max(w, round(iw * s)), max(h, round(ih * s))),
|
||||
interpolation=cv2.INTER_AREA if s < 1 else cv2.INTER_CUBIC)
|
||||
y, x = (r.shape[0] - h) // 2, (r.shape[1] - w) // 2
|
||||
return r[y:y + h, x:x + w]
|
||||
|
||||
|
||||
def generation_size(src_w: int, src_h: int, megapixels: float, multiple: int) -> tuple[int, int]:
|
||||
"""Aspect-preserving size with the given pixel area, both sides on the multiple grid."""
|
||||
multiple = max(8, int(multiple))
|
||||
area = max(0.05, float(megapixels)) * 1e6
|
||||
ar = src_w / max(1, src_h)
|
||||
h = math.sqrt(area / ar)
|
||||
w = h * ar
|
||||
w = max(multiple, int(round(w / multiple)) * multiple)
|
||||
h = max(multiple, int(round(h / multiple)) * multiple)
|
||||
return w, h
|
||||
|
||||
|
||||
def _read_audio(path: str, sample_rate: int = 44100) -> dict | None:
|
||||
"""Decode the whole soundtrack as float32 [1, C, T] with ffmpeg; None when silent."""
|
||||
try:
|
||||
probe = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "a:0", "-show_entries",
|
||||
"stream=channels", "-of", "csv=p=0", path],
|
||||
capture_output=True, text=True, timeout=60)
|
||||
ch = int((probe.stdout.strip() or "0").split(",")[0] or 0)
|
||||
if ch <= 0:
|
||||
return None
|
||||
ch = min(ch, 2)
|
||||
raw = subprocess.run(["ffmpeg", "-v", "error", "-i", path, "-vn", "-ac", str(ch), "-ar", str(sample_rate),
|
||||
"-f", "f32le", "-"], capture_output=True, timeout=600).stdout
|
||||
if not raw:
|
||||
return None
|
||||
a = np.frombuffer(raw, dtype=np.float32).reshape(-1, ch).T.copy()
|
||||
return {"waveform": torch.from_numpy(a)[None], "sample_rate": sample_rate}
|
||||
except Exception: # noqa: BLE001 - audio is optional
|
||||
return None
|
||||
|
||||
|
||||
def _slice_audio(audio: dict | None, start_s: float, dur_s: float) -> dict | None:
|
||||
if audio is None:
|
||||
return None
|
||||
sr = audio["sample_rate"]
|
||||
wf = audio["waveform"]
|
||||
a, n = int(round(start_s * sr)), int(round(dur_s * sr))
|
||||
seg = wf[..., a:a + n]
|
||||
if seg.shape[-1] < n:
|
||||
seg = torch.nn.functional.pad(seg, (0, n - seg.shape[-1]))
|
||||
return {"waveform": seg.contiguous(), "sample_rate": sr}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------- analysis
|
||||
|
||||
_ANALYSIS_CACHE: dict[tuple, dict] = {}
|
||||
|
||||
|
||||
def analyze(path: str, fps: float, thumbs: int = 120, thumb_h: int = 72) -> dict:
|
||||
"""Probe, per-frame cut score on the resampled timeline, and a strip of thumbnails."""
|
||||
import cv2
|
||||
key = (path, os.path.getmtime(path), round(float(fps), 4), thumbs, thumb_h)
|
||||
if key in _ANALYSIS_CACHE:
|
||||
return _ANALYSIS_CACHE[key]
|
||||
info = _probe(path)
|
||||
src = _timeline(info["n_src"], info["fps_src"], fps)
|
||||
n = len(src)
|
||||
small = []
|
||||
for fr in _read_frames(path, src, None):
|
||||
g = cv2.resize(fr, (64, 36), interpolation=cv2.INTER_AREA)
|
||||
small.append(g)
|
||||
hsv = [cv2.calcHist([cv2.cvtColor(s, cv2.COLOR_RGB2HSV)], [0, 1], None, [16, 8], [0, 180, 0, 256]) for s in small]
|
||||
hsv = [cv2.normalize(h, h).flatten() for h in hsv]
|
||||
raw = np.zeros(n, np.float32)
|
||||
for i in range(1, n):
|
||||
pix = np.abs(small[i].astype(np.float32) - small[i - 1].astype(np.float32)).mean() / 255.0
|
||||
hist = cv2.compareHist(hsv[i - 1], hsv[i], cv2.HISTCMP_BHATTACHARYYA)
|
||||
raw[i] = 0.5 * pix + 0.5 * float(hist)
|
||||
# local normalisation: a cut stands out from the motion around it
|
||||
score = np.zeros(n, np.float32)
|
||||
w = max(4, int(round(fps)))
|
||||
for i in range(1, n):
|
||||
lo, hi = max(1, i - w), min(n, i + w + 1)
|
||||
neigh = np.concatenate([raw[lo:i], raw[i + 1:hi]])
|
||||
base = float(np.median(neigh)) if len(neigh) else 0.0
|
||||
score[i] = raw[i] / (base + 0.02)
|
||||
k = max(1, n // max(1, thumbs))
|
||||
th = []
|
||||
tw = max(16, int(round(thumb_h * info["width"] / max(1, info["height"]))))
|
||||
frames_for_thumbs = _read_frames(path, src[::k], (tw, thumb_h))
|
||||
for j, fr in enumerate(frames_for_thumbs):
|
||||
ok, buf = cv2.imencode(".jpg", cv2.cvtColor(fr, cv2.COLOR_RGB2BGR), [cv2.IMWRITE_JPEG_QUALITY, 70])
|
||||
th.append({"f": int(j * k), "src": "data:image/jpeg;base64," + base64.b64encode(buf.tobytes()).decode()})
|
||||
out = dict(info, fps=float(fps), n=n, thumbs=th, thumb_w=tw, thumb_h=thumb_h,
|
||||
raw=[round(float(x), 4) for x in raw], score=[round(float(x), 3) for x in score])
|
||||
_ANALYSIS_CACHE[key] = out
|
||||
return out
|
||||
|
||||
|
||||
_PSD_CACHE: dict[tuple, list[float]] = {}
|
||||
|
||||
|
||||
def scenedetect_cut_times(path: str, detector: str, sensitivity: float) -> list[float] | None:
|
||||
"""Cut times in seconds from PySceneDetect, or None when the package is missing."""
|
||||
try:
|
||||
from scenedetect import SceneManager, open_video
|
||||
from scenedetect.detectors import AdaptiveDetector, ContentDetector
|
||||
except ImportError:
|
||||
return None
|
||||
sensitivity = float(min(1.0, max(0.0, sensitivity)))
|
||||
key = (path, os.path.getmtime(path), detector, round(sensitivity, 3))
|
||||
if key in _PSD_CACHE:
|
||||
return _PSD_CACHE[key]
|
||||
video = open_video(path)
|
||||
min_len = max(3, int(round(video.frame_rate * 0.25)))
|
||||
sm = SceneManager()
|
||||
if detector == "content":
|
||||
sm.add_detector(ContentDetector(threshold=45.0 - 33.0 * sensitivity, min_scene_len=min_len))
|
||||
else:
|
||||
sm.add_detector(AdaptiveDetector(adaptive_threshold=5.0 - 3.5 * sensitivity, min_scene_len=min_len))
|
||||
sm.detect_scenes(video, show_progress=False)
|
||||
times = [float(a.get_seconds()) for a, _ in sm.get_scene_list()[1:]]
|
||||
_PSD_CACHE[key] = times
|
||||
return times
|
||||
|
||||
|
||||
def find_cuts(path: str, analysis: dict, detector: str, sensitivity: float) -> tuple[list[int], str]:
|
||||
"""Cut frames on the resampled timeline and which detector produced them."""
|
||||
fps = float(analysis["fps"])
|
||||
if detector in ("adaptive", "content"):
|
||||
times = scenedetect_cut_times(path, detector, sensitivity)
|
||||
if times is not None:
|
||||
n = analysis["n"]
|
||||
return sorted({int(round(t * fps)) for t in times if 0 < round(t * fps) < n}), f"PySceneDetect {detector}"
|
||||
return detect_cuts(analysis["score"], analysis["raw"], sensitivity, fps), "builtin"
|
||||
|
||||
|
||||
def detect_cuts(score: list[float], raw: list[float], sensitivity: float, fps: float) -> list[int]:
|
||||
"""Frame indices that start a new shot (built-in detector). Higher sensitivity finds more cuts."""
|
||||
sensitivity = float(min(1.0, max(0.0, sensitivity)))
|
||||
thr = 12.0 - 10.0 * sensitivity # local ratio threshold: 12 (strict) .. 2 (loose)
|
||||
abs_min = 0.12 - 0.09 * sensitivity # ignore tiny global changes
|
||||
gap = max(3, int(round(fps * 0.25))) # no two cuts closer than a quarter second
|
||||
cand = [i for i in range(1, len(score)) if score[i] >= thr and raw[i] >= abs_min]
|
||||
cuts: list[int] = []
|
||||
for i in sorted(cand, key=lambda j: -score[j]):
|
||||
if all(abs(i - c) >= gap for c in cuts):
|
||||
cuts.append(i)
|
||||
return sorted(cuts)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------- planning
|
||||
|
||||
def plan_segments(n: int, cuts: list[int], mode: str, max_len: int, min_len: int,
|
||||
max_parts: int = 0, max_total: int = 0, manual: list[int] | None = None) -> list[dict]:
|
||||
"""Split [0, n) into shots. Returns [{start, end, cut_before}]. Lengths are in frames."""
|
||||
if max_total and max_total > 0:
|
||||
n = min(n, int(max_total))
|
||||
max_len = max(1, int(max_len))
|
||||
min_len = max(1, min(int(min_len), max_len))
|
||||
cuts = sorted(c for c in set(cuts or []) if 0 < c < n)
|
||||
cutset = set(cuts)
|
||||
if mode == "manual" and manual:
|
||||
bounds = sorted(set([0, n] + [b for b in manual if 0 < b < n]))
|
||||
elif mode == "fixed":
|
||||
k = max(1, math.ceil(n / max_len))
|
||||
bounds = sorted(set(round(n * j / k) for j in range(k + 1)))
|
||||
else: # shots
|
||||
bounds = [0] + cuts + [n]
|
||||
# merge shots that are too short into a neighbour, as long as the merge still fits
|
||||
changed = True
|
||||
while changed:
|
||||
changed = False
|
||||
for j in range(len(bounds) - 1):
|
||||
if bounds[j + 1] - bounds[j] >= min_len or len(bounds) <= 2:
|
||||
continue
|
||||
left = bounds[j + 1] - bounds[j - 1] if j > 0 else None
|
||||
right = bounds[j + 2] - bounds[j] if j + 2 < len(bounds) else None
|
||||
opts = [(v, side) for v, side in ((left, "l"), (right, "r")) if v is not None and v <= max_len]
|
||||
if not opts:
|
||||
continue
|
||||
side = min(opts)[1]
|
||||
bounds.pop(j if side == "l" else j + 1)
|
||||
changed = True
|
||||
break
|
||||
# split shots that are too long into equal parts
|
||||
out = [0]
|
||||
for a, b in zip(bounds[:-1], bounds[1:]):
|
||||
k = max(1, math.ceil((b - a) / max_len))
|
||||
out += [a + round((b - a) * j / k) for j in range(1, k + 1)]
|
||||
bounds = sorted(set(out))
|
||||
segs = [{"start": a, "end": b, "cut_before": a in cutset} for a, b in zip(bounds[:-1], bounds[1:]) if b > a]
|
||||
if max_parts and max_parts > 0:
|
||||
segs = segs[:int(max_parts)]
|
||||
return segs
|
||||
|
||||
|
||||
DEFAULT_PLAN = {
|
||||
"video": "", "fps": 24.0, "grid": DEFAULT_GRID, "mode": "shots", "max_s": 4.5, "min_s": 1.0,
|
||||
"sensitivity": 0.5, "max_parts": 0, "max_total_s": 0.0, "bounds": [], "segs": [],
|
||||
"global_ref": "", "global_ref2": "", "global_prompt": "", "megapixels": 0.15, "multiple": 32,
|
||||
"detector": "adaptive", "run": "auto",
|
||||
}
|
||||
|
||||
|
||||
def _load_plan(plan_json: str) -> dict:
|
||||
try:
|
||||
p = json.loads(plan_json or "{}")
|
||||
except json.JSONDecodeError as exc:
|
||||
raise ValueError(f"BFS Shot Planner: the plan is not valid JSON ({exc})") from exc
|
||||
out = dict(DEFAULT_PLAN)
|
||||
out.update({k: v for k, v in p.items() if v is not None})
|
||||
return out
|
||||
|
||||
|
||||
def resolve_plan(plan: dict, analysis: dict, path: str | None = None) -> list[dict]:
|
||||
"""Shots for this plan: the boundaries edited in the panel, or an automatic split."""
|
||||
fps = float(analysis["fps"])
|
||||
grid = plan["grid"]
|
||||
max_len = snap_down(int(round(float(plan["max_s"]) * fps)), grid)
|
||||
min_len = max(1, int(round(float(plan["min_s"]) * fps)))
|
||||
max_total = int(round(float(plan.get("max_total_s") or 0) * fps))
|
||||
if path:
|
||||
cuts, _ = find_cuts(path, analysis, plan.get("detector", "adaptive"), float(plan["sensitivity"]))
|
||||
else:
|
||||
cuts = detect_cuts(analysis["score"], analysis["raw"], float(plan["sensitivity"]), fps)
|
||||
manual = plan.get("bounds") or None
|
||||
mode = "manual" if manual else plan["mode"]
|
||||
segs = plan_segments(analysis["n"], cuts, mode, max_len, min_len,
|
||||
int(plan.get("max_parts") or 0), max_total, manual)
|
||||
meta = plan.get("segs") or []
|
||||
for i, s in enumerate(segs):
|
||||
m = meta[i] if i < len(meta) and isinstance(meta[i], dict) else {}
|
||||
s["enabled"] = bool(m.get("enabled", True))
|
||||
s["ref"] = m.get("ref") or ""
|
||||
s["ref2"] = m.get("ref2") or ""
|
||||
s["prompt"] = m.get("prompt") or ""
|
||||
s["cut_before"] = bool(s.get("cut_before")) or (s["start"] in cuts)
|
||||
s["gen_len"] = snap_up(s["end"] - s["start"], grid)
|
||||
return segs
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------- queue loop state
|
||||
|
||||
def run_id_for(plan_json: str, path: str) -> str:
|
||||
h = hashlib.sha1((plan_json + str(os.path.getmtime(path))).encode()).hexdigest()[:16]
|
||||
return h
|
||||
|
||||
|
||||
def run_dir(run_id: str) -> str:
|
||||
d = os.path.join(folder_paths.get_temp_directory(), "bfs_shotloop", run_id)
|
||||
os.makedirs(d, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def run_state(run_id: str) -> dict:
|
||||
f = os.path.join(run_dir(run_id), "state.json")
|
||||
if os.path.isfile(f):
|
||||
with open(f) as fh:
|
||||
return json.load(fh)
|
||||
return {"done": [], "count": 0}
|
||||
|
||||
|
||||
def save_state(run_id: str, st: dict) -> None:
|
||||
with open(os.path.join(run_dir(run_id), "state.json"), "w") as fh:
|
||||
json.dump(st, fh)
|
||||
|
||||
|
||||
def _notify(event: str, data: dict) -> None:
|
||||
try:
|
||||
from server import PromptServer
|
||||
PromptServer.instance.send_sync(event, data)
|
||||
except Exception: # noqa: BLE001 - headless runs have no UI to notify
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------- images
|
||||
|
||||
def _load_image(name: str) -> torch.Tensor | None:
|
||||
if not name:
|
||||
return None
|
||||
from PIL import Image, ImageOps
|
||||
img = ImageOps.exif_transpose(Image.open(_input_path(name))).convert("RGB")
|
||||
return torch.from_numpy(np.asarray(img).astype(np.float32) / 255.0)[None]
|
||||
|
||||
|
||||
def _grey(h: int = 64, w: int = 64) -> torch.Tensor:
|
||||
return torch.full((1, h, w, 3), 0.5)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------- nodes
|
||||
|
||||
class BFSShotPlanner:
|
||||
"""Split a video into shots that fit the model, with a reference and prompt per shot."""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"plan": ("STRING", {"default": json.dumps(DEFAULT_PLAN), "multiline": True,
|
||||
"tooltip": "The panel writes this. JSON with the video, split settings, "
|
||||
"boundaries and the per-shot reference and prompt."}),
|
||||
},
|
||||
"optional": {
|
||||
"ref_image": ("IMAGE", {"tooltip": "Default reference for every shot that has none of its own "
|
||||
"(overrides the panel's global reference)."}),
|
||||
"ref_image_2": ("IMAGE", {"tooltip": "Default second reference (e.g. a full-body photo)."}),
|
||||
"prompt": ("STRING", {"forceInput": True,
|
||||
"tooltip": "Default prompt for shots without their own "
|
||||
"(overrides the panel's global prompt)."}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("BFS_SHOT", "INT", "FLOAT", "INT", "INT", "AUDIO", "STRING")
|
||||
RETURN_NAMES = ("shots", "count", "fps", "width", "height", "audio", "summary")
|
||||
OUTPUT_IS_LIST = (True, False, False, False, False, False, False)
|
||||
OUTPUT_TOOLTIPS = (
|
||||
"One item per shot. Every node that receives this list runs once per shot; connect it to "
|
||||
"BFS Shot Unpack or BFS Shot H3 Conditioning, sample, decode, then BFS Shot Join.",
|
||||
"Number of shots that will run.", "Timeline frame rate.", "Generation width.",
|
||||
"Generation height.", "The whole soundtrack, trimmed to the planned duration.",
|
||||
"Human-readable plan.")
|
||||
FUNCTION = "plan_shots"
|
||||
CATEGORY = "BFS/shot loop"
|
||||
DESCRIPTION = ("Split a long video into model-sized shots (at camera cuts, fixed length, or by hand), "
|
||||
"give every shot its own reference image and prompt, and run the rest of the graph once "
|
||||
"per shot. Join the decoded shots with BFS Shot Join.")
|
||||
|
||||
@classmethod
|
||||
def IS_CHANGED(cls, plan, **kwargs):
|
||||
try:
|
||||
if _load_plan(plan).get("run") == "queue":
|
||||
return float("nan") # the next shot depends on the loop state on disk
|
||||
except ValueError:
|
||||
pass
|
||||
return plan
|
||||
|
||||
def plan_shots(self, plan, ref_image=None, ref_image_2=None, prompt=None):
|
||||
p = _load_plan(plan)
|
||||
path = _input_path(p["video"])
|
||||
fps = float(p["fps"])
|
||||
a = analyze(path, fps)
|
||||
segs = resolve_plan(p, a, path)
|
||||
segs = [s for s in segs if s["enabled"]]
|
||||
if not segs:
|
||||
raise ValueError("BFS Shot Planner: no shots are enabled.")
|
||||
W, H = generation_size(a["width"], a["height"], float(p["megapixels"]), int(p["multiple"]))
|
||||
src = _timeline(a["n_src"], a["fps_src"], fps)
|
||||
audio = _read_audio(path)
|
||||
g_ref = ref_image if ref_image is not None else _load_image(p.get("global_ref", ""))
|
||||
g_ref2 = ref_image_2 if ref_image_2 is not None else _load_image(p.get("global_ref2", ""))
|
||||
g_prompt = prompt if prompt is not None else p.get("global_prompt", "")
|
||||
queue = p.get("run") == "queue"
|
||||
rid = run_id_for(plan, path) if queue else ""
|
||||
todo = list(range(len(segs)))
|
||||
if queue:
|
||||
st = run_state(rid)
|
||||
st["count"] = len(segs)
|
||||
save_state(rid, st)
|
||||
pending = [i for i in todo if i not in st["done"]]
|
||||
todo = pending[:1] if pending else [len(segs) - 1]
|
||||
shots = []
|
||||
for i, s in enumerate(segs):
|
||||
if i not in todo:
|
||||
continue
|
||||
idx = np.arange(s["start"], s["start"] + s["gen_len"])
|
||||
idx = np.clip(idx, 0, a["n"] - 1) # past the end: hold the last frame
|
||||
frames = _read_frames(path, src[idx], (W, H))
|
||||
ft = torch.from_numpy(np.stack(frames).astype(np.float32) / 255.0)
|
||||
ref = _load_image(s["ref"]) if s["ref"] else g_ref
|
||||
ref2 = _load_image(s["ref2"]) if s["ref2"] else g_ref2
|
||||
shots.append({
|
||||
"index": i, "count": len(segs), "start": s["start"], "end": s["end"],
|
||||
"length": s["end"] - s["start"], "gen_length": s["gen_len"], "fps": fps,
|
||||
"cut_before": s["cut_before"], "width": W, "height": H, "frames": ft,
|
||||
"ref": ref, "ref2": ref2, "prompt": s["prompt"] or g_prompt or "",
|
||||
"audio": _slice_audio(audio, s["start"] / fps, s["gen_len"] / fps),
|
||||
"run_id": rid, "queue": queue,
|
||||
})
|
||||
total = sum(s["end"] - s["start"] for s in segs)
|
||||
full_audio = _slice_audio(audio, segs[0]["start"] / fps, total / fps) if audio else None
|
||||
lines = [f"{len(segs)} shots, {total} frames ({total / fps:.2f}s) at {fps:g} fps, {W}x{H}"
|
||||
+ (f" | queue loop: running shot {shots[0]['index'] + 1}/{len(segs)}" if queue else "")]
|
||||
for s in shots:
|
||||
lines.append(f"#{s['index'] + 1}: frames {s['start']}-{s['end'] - 1} ({s['length']} -> generate "
|
||||
f"{s['gen_length']}){' cut' if s['cut_before'] else ''}"
|
||||
f"{' ref' if s['ref'] is not None else ''}{' ref2' if s['ref2'] is not None else ''}")
|
||||
return (shots, len(segs), fps, W, H, full_audio, "\n".join(lines))
|
||||
|
||||
|
||||
class BFSShotUnpack:
|
||||
"""Open one shot into plain values for any workflow (runs once per shot)."""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {"required": {"shot": ("BFS_SHOT",)}}
|
||||
|
||||
RETURN_TYPES = ("IMAGE", "IMAGE", "IMAGE", "STRING", "INT", "IMAGE", "AUDIO", "INT", "INT", "INT")
|
||||
RETURN_NAMES = ("guide_frames", "ref_image", "ref_image_2", "prompt", "length", "first_frame",
|
||||
"audio", "width", "height", "index")
|
||||
OUTPUT_TOOLTIPS = (
|
||||
"The shot's guide frames, already at a length the model accepts.",
|
||||
"This shot's reference (a grey placeholder if none was set).",
|
||||
"This shot's second reference (a grey placeholder if none was set).",
|
||||
"This shot's prompt.", "Frames to generate (the grid-valid length).",
|
||||
"First guide frame of the shot.", "Soundtrack of the shot (silence if the video has none).",
|
||||
"Generation width.", "Generation height.", "Shot index (0-based).")
|
||||
FUNCTION = "unpack"
|
||||
CATEGORY = "BFS/shot loop"
|
||||
DESCRIPTION = "Split one shot into its guide frames, references, prompt and length."
|
||||
|
||||
def unpack(self, shot):
|
||||
audio = shot["audio"] or {"waveform": torch.zeros(1, 1, int(44100 * shot["gen_length"] / shot["fps"])),
|
||||
"sample_rate": 44100}
|
||||
return (shot["frames"], shot["ref"] if shot["ref"] is not None else _grey(),
|
||||
shot["ref2"] if shot["ref2"] is not None else _grey(), shot["prompt"], shot["gen_length"],
|
||||
shot["frames"][:1], audio, shot["width"], shot["height"], shot["index"])
|
||||
|
||||
|
||||
class BFSShotH3Conditioning:
|
||||
"""Native MiniMax H3 conditioning for one shot: references, prompt and the shot as a guide."""
|
||||
|
||||
GUIDE_MODES = ["aligned guide (Add Guide)", "native reference video (<Video 1>)", "both", "none"]
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"shot": ("BFS_SHOT",),
|
||||
"clip": ("CLIP",),
|
||||
"vae": ("VAE",),
|
||||
"guide_mode": (cls.GUIDE_MODES, {"default": cls.GUIDE_MODES[0], "tooltip":
|
||||
"aligned guide: the shot's frames sit on the generated frames (MiniMax H3 Add Guide), "
|
||||
"frame by frame. native reference video: the shot enters as <Video 1> like the reference "
|
||||
"video input. both: the two together."}),
|
||||
"use_ref_2": ("BOOLEAN", {"default": True, "tooltip": "Also pass the second reference."}),
|
||||
"first_frame": (["none", "shot's first frame"], {"default": "none", "tooltip":
|
||||
"Anchor the shot's own first frame as an extra aligned image guide."}),
|
||||
"ref_image_size": (["match", "max"], {"default": "match"}),
|
||||
},
|
||||
"optional": {
|
||||
"audio_vae": ("VAE",),
|
||||
"with_audio": ("BOOLEAN", {"default": False, "tooltip":
|
||||
"Attach the shot's soundtrack to the guide / reference video (needs audio_vae)."}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("CONDITIONING", "LATENT")
|
||||
RETURN_NAMES = ("positive", "latent")
|
||||
FUNCTION = "condition"
|
||||
CATEGORY = "BFS/shot loop"
|
||||
DESCRIPTION = ("Build MiniMax H3 conditioning for one shot with the native nodes: Reference to Video "
|
||||
"(prompt, references, length) plus the shot as an aligned guide and/or reference video.")
|
||||
|
||||
def condition(self, shot, clip, vae, guide_mode, use_ref_2, first_frame, ref_image_size,
|
||||
audio_vae=None, with_audio=False):
|
||||
from comfy_extras.nodes_minimax_h3 import MiniMaxH3AddGuide, MiniMaxH3ReferenceToVideo
|
||||
refs = {}
|
||||
if shot["ref"] is not None:
|
||||
refs["ref_image_0"] = shot["ref"]
|
||||
if use_ref_2 and shot["ref2"] is not None:
|
||||
refs[f"ref_image_{len(refs)}"] = shot["ref2"]
|
||||
audio = shot["audio"] if (with_audio and audio_vae is not None) else None
|
||||
native = guide_mode in (self.GUIDE_MODES[1], self.GUIDE_MODES[2])
|
||||
aligned = guide_mode in (self.GUIDE_MODES[0], self.GUIDE_MODES[2])
|
||||
kwargs = dict(clip=clip, prompt=shot["prompt"], width=shot["width"], height=shot["height"],
|
||||
length=shot["gen_length"], ref_image_size=ref_image_size, vae=vae,
|
||||
audio_vae=audio_vae, ref_images=refs or None)
|
||||
if native:
|
||||
kwargs["ref_videos"] = {"ref_video_1": shot["frames"]}
|
||||
if audio is not None:
|
||||
kwargs["ref_video_audios"] = {"ref_video_audio_1": audio}
|
||||
positive, latent = MiniMaxH3ReferenceToVideo.execute(**kwargs).args[:2]
|
||||
if aligned:
|
||||
positive = MiniMaxH3AddGuide.execute(positive=positive, latent=latent, frame_idx=0, vae=vae,
|
||||
audio_vae=audio_vae if audio is not None else None,
|
||||
image=shot["frames"], audio=audio).args[0]
|
||||
if first_frame != "none":
|
||||
positive = MiniMaxH3AddGuide.execute(positive=positive, latent=latent, frame_idx=0, vae=vae,
|
||||
image=shot["frames"][:1]).args[0]
|
||||
return (positive, latent)
|
||||
|
||||
|
||||
class BFSShotJoin:
|
||||
"""Put the decoded shots back together in order, trimmed to their true lengths."""
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"images": ("IMAGE", {"tooltip": "Decoded frames of every shot (the list from VAE Decode)."}),
|
||||
"shots": ("BFS_SHOT", {"tooltip": "The shots list from BFS Shot Planner."}),
|
||||
"crossfade": ("INT", {"default": 4, "min": 0, "max": 24, "tooltip":
|
||||
"Frames to blend across boundaries that are not camera cuts, using the overlap each "
|
||||
"shot generated past its end. 0 = hard joins everywhere."}),
|
||||
},
|
||||
"optional": {"audio": ("AUDIO", {"tooltip": "Soundtrack to return with the video "
|
||||
"(e.g. the planner's audio output)."})},
|
||||
}
|
||||
|
||||
INPUT_IS_LIST = True
|
||||
RETURN_TYPES = ("IMAGE", "AUDIO", "FLOAT")
|
||||
RETURN_NAMES = ("images", "audio", "fps")
|
||||
FUNCTION = "join"
|
||||
CATEGORY = "BFS/shot loop"
|
||||
DESCRIPTION = "Concatenate the generated shots in order, trim each to its length and cross-fade soft joins."
|
||||
|
||||
def join(self, images, shots, crossfade, audio=None):
|
||||
if shots and shots[0].get("queue"):
|
||||
return self._join_queue(images, shots, crossfade, audio)
|
||||
return self._join_all(images, shots, crossfade, audio)
|
||||
|
||||
def _join_queue(self, images, shots, crossfade, audio):
|
||||
from comfy_execution.graph_utils import ExecutionBlocker
|
||||
shot = shots[0]
|
||||
rid = shot["run_id"]
|
||||
d = run_dir(rid)
|
||||
img = images[0]
|
||||
torch.save({"frames": (img.clamp(0, 1) * 255).round().to(torch.uint8).cpu(),
|
||||
"shot": {k: v for k, v in shot.items() if k in ("index", "count", "start", "end", "length",
|
||||
"gen_length", "fps", "cut_before")}},
|
||||
os.path.join(d, f"shot_{shot['index']:04d}.pt"))
|
||||
st = run_state(rid)
|
||||
if shot["index"] not in st["done"]:
|
||||
st["done"].append(shot["index"])
|
||||
st["count"] = shot["count"]
|
||||
save_state(rid, st)
|
||||
done = len(set(st["done"]))
|
||||
_notify("bfs-shotloop-progress", {"run_id": rid, "done": done, "count": shot["count"]})
|
||||
if done < shot["count"]:
|
||||
_notify("bfs-shotloop-next", {"run_id": rid, "done": done, "count": shot["count"]})
|
||||
blocker = ExecutionBlocker(None)
|
||||
return (blocker, blocker, blocker)
|
||||
stored = [torch.load(os.path.join(d, f"shot_{i:04d}.pt")) for i in range(shot["count"])]
|
||||
imgs = [x["frames"].float() / 255.0 for x in stored]
|
||||
meta = [x["shot"] for x in stored]
|
||||
return self._join_all(imgs, meta, crossfade, audio)
|
||||
|
||||
def _join_all(self, images, shots, crossfade, audio=None):
|
||||
xf = int(crossfade[0]) if crossfade else 0
|
||||
if len(images) != len(shots):
|
||||
raise ValueError(f"BFS Shot Join: got {len(images)} image batches for {len(shots)} shots. "
|
||||
"Connect the decoded images of the same shot list.")
|
||||
order = sorted(range(len(shots)), key=lambda i: shots[i]["index"])
|
||||
H, W = images[order[0]].shape[1:3]
|
||||
out = []
|
||||
prev_tail = None # frames the previous shot generated past its end
|
||||
for j in order:
|
||||
img = images[j]
|
||||
if img.shape[1:3] != (H, W):
|
||||
img = torch.nn.functional.interpolate(img.movedim(-1, 1), size=(H, W), mode="bilinear",
|
||||
align_corners=False).movedim(1, -1)
|
||||
L = shots[j]["length"]
|
||||
body = img[:L]
|
||||
if body.shape[0] < L: # shorter than planned: hold the last frame
|
||||
body = torch.cat([body, body[-1:].expand(L - body.shape[0], -1, -1, -1)], 0)
|
||||
if xf > 0 and prev_tail is not None and not shots[j]["cut_before"]:
|
||||
n = min(xf, prev_tail.shape[0], body.shape[0])
|
||||
if n > 0:
|
||||
w = torch.linspace(0, 1, n + 2)[1:-1].view(-1, 1, 1, 1)
|
||||
body = body.clone()
|
||||
body[:n] = prev_tail[:n] * (1 - w) + body[:n] * w
|
||||
out.append(body)
|
||||
prev_tail = img[L:]
|
||||
video = torch.cat(out, 0)
|
||||
fps = float(shots[order[0]]["fps"])
|
||||
a = audio[0] if audio else None
|
||||
if a is not None:
|
||||
n = int(round(video.shape[0] / fps * a["sample_rate"]))
|
||||
wf = a["waveform"][..., :n]
|
||||
if wf.shape[-1] < n:
|
||||
wf = torch.nn.functional.pad(wf, (0, n - wf.shape[-1]))
|
||||
a = {"waveform": wf, "sample_rate": a["sample_rate"]}
|
||||
return (video, a, fps)
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"BFSShotPlanner": BFSShotPlanner,
|
||||
"BFSShotUnpack": BFSShotUnpack,
|
||||
"BFSShotH3Conditioning": BFSShotH3Conditioning,
|
||||
"BFSShotJoin": BFSShotJoin,
|
||||
}
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"BFSShotPlanner": "BFS Shot Planner",
|
||||
"BFSShotUnpack": "BFS Shot Unpack",
|
||||
"BFSShotH3Conditioning": "BFS Shot H3 Conditioning",
|
||||
"BFSShotJoin": "BFS Shot Join",
|
||||
}
|
||||
|
||||
# ---------------------------------------------------------------------------- http api
|
||||
|
||||
try:
|
||||
from aiohttp import web
|
||||
from server import PromptServer
|
||||
|
||||
def _list_input(exts):
|
||||
root = folder_paths.get_input_directory()
|
||||
out = []
|
||||
for dp, _, fs in os.walk(root):
|
||||
for f in fs:
|
||||
if f.lower().endswith(exts):
|
||||
rel = os.path.relpath(os.path.join(dp, f), root).replace(os.sep, "/")
|
||||
out.append(rel)
|
||||
return sorted(out, key=str.lower)
|
||||
|
||||
@PromptServer.instance.routes.get("/bfs/shotloop/files")
|
||||
async def _bfs_shot_files(request):
|
||||
return web.json_response({"videos": _list_input(VIDEO_EXTS), "images": _list_input(IMAGE_EXTS)})
|
||||
|
||||
@PromptServer.instance.routes.get("/bfs/shotloop/analyze")
|
||||
async def _bfs_shot_analyze(request):
|
||||
name = request.query.get("video", "")
|
||||
fps = float(request.query.get("fps", "24") or 24)
|
||||
try:
|
||||
a = analyze(_input_path(name), fps)
|
||||
except Exception as exc: # noqa: BLE001 - the panel shows the reason
|
||||
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=500)
|
||||
return web.json_response(a)
|
||||
|
||||
@PromptServer.instance.routes.post("/bfs/shotloop/plan")
|
||||
async def _bfs_shot_plan(request):
|
||||
body = await request.json()
|
||||
try:
|
||||
p = _load_plan(json.dumps(body.get("plan", {})))
|
||||
a = analyze(_input_path(p["video"]), float(p["fps"]))
|
||||
path = _input_path(p["video"])
|
||||
segs = resolve_plan(p, a, path)
|
||||
cuts, used = find_cuts(path, a, p.get("detector", "adaptive"), float(p["sensitivity"]))
|
||||
W, H = generation_size(a["width"], a["height"], float(p["megapixels"]), int(p["multiple"]))
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=500)
|
||||
return web.json_response({"segs": segs, "cuts": cuts, "width": W, "height": H, "detector": used,
|
||||
"max_len": snap_down(int(round(float(p["max_s"]) * float(p["fps"]))), p["grid"])})
|
||||
@PromptServer.instance.routes.post("/bfs/shotloop/progress")
|
||||
async def _bfs_shot_progress(request):
|
||||
body = await request.json()
|
||||
try:
|
||||
plan_json = json.dumps(body.get("plan", {}))
|
||||
p = _load_plan(plan_json)
|
||||
rid = run_id_for(body.get("plan_raw") or plan_json, _input_path(p["video"]))
|
||||
st = run_state(rid)
|
||||
if body.get("reset"):
|
||||
import shutil
|
||||
shutil.rmtree(run_dir(rid), ignore_errors=True)
|
||||
st = {"done": [], "count": st.get("count", 0)}
|
||||
except Exception as exc: # noqa: BLE001
|
||||
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=500)
|
||||
return web.json_response({"run_id": rid, "done": len(set(st["done"])), "count": st.get("count", 0)})
|
||||
except Exception as _exc: # noqa: BLE001 - nodes still work without the panel
|
||||
print(f"[BFSNodes] Shot loop HTTP routes not registered: {_exc!r}")
|
||||
+3
-2
@@ -1,7 +1,7 @@
|
||||
[project]
|
||||
name = "bfsnodes"
|
||||
description = "Custom ComfyUI nodes for reference-guided video: the BFS Sampler (guide-driven IC-LoRAs with auto-configured crop, mask inpainting and temporal chunking in one node), LTX Multiple Controls and Identity Transfer (IPT2V reference + source-phase RoPE), LTXV Edit Anything, Head-Swap/Anime2Real Bernini conditioning, the AMV Guide Builder, MiniMax-H3 aligned downscaled latent guides with training-matched posterior encoding and spatial coordinates, experimental overlap/sidecar source-phase RoPE and caption-only identity references, true single-frame/video targets, single-frame VAE loaders with tile control, and LoRA Surgery (read any LoRA's layers and blocks, then scale or drop them live to find which modules carry a defect, and save the result as a new LoRA)."
|
||||
version = "1.47.0"
|
||||
description = "Custom ComfyUI nodes for reference-guided video: the BFS Sampler (guide-driven IC-LoRAs with auto-configured crop, mask inpainting and temporal chunking in one node), LTX Multiple Controls and Identity Transfer (IPT2V reference + source-phase RoPE), LTXV Edit Anything, Head-Swap/Anime2Real Bernini conditioning, the AMV Guide Builder, MiniMax-H3 aligned downscaled latent guides with training-matched posterior encoding and spatial coordinates, experimental overlap/sidecar source-phase RoPE and caption-only identity references, true single-frame/video targets, single-frame VAE loaders with tile control, LoRA Surgery (read any LoRA's layers and blocks, then scale or drop them live to find which modules carry a defect, and save the result as a new LoRA), and the Shot Planner / Shot Loop (split long videos into model-sized shots at camera cuts with PySceneDetect on an interactive timeline, set a reference and prompt per shot, run any workflow once per shot in one run or as a queue loop, and join the results with the original timing and audio)."
|
||||
version = "1.48.0"
|
||||
license = {file = "LICENSE"}
|
||||
dependencies = [
|
||||
"torch",
|
||||
@@ -10,6 +10,7 @@ dependencies = [
|
||||
"safetensors",
|
||||
"librosa",
|
||||
"opencv-python",
|
||||
"scenedetect",
|
||||
"insightface==0.7.3",
|
||||
"onnxruntime",
|
||||
"transformers>=4.55.0",
|
||||
|
||||
@@ -6,3 +6,4 @@ librosa
|
||||
opencv-python
|
||||
insightface==0.7.3
|
||||
onnxruntime
|
||||
scenedetect
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import sys
|
||||
import tempfile
|
||||
import types
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
import torch
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
_TMP = tempfile.mkdtemp(prefix="bfs_shotloop_test_")
|
||||
|
||||
# The module only needs a few folder_paths calls; stub them so the tests run without ComfyUI.
|
||||
_fp = types.ModuleType("folder_paths")
|
||||
_fp.get_temp_directory = lambda: _TMP
|
||||
_fp.get_input_directory = lambda: _TMP
|
||||
_fp.get_annotated_filepath = lambda name: str(Path(_TMP) / name)
|
||||
sys.modules.setdefault("folder_paths", _fp)
|
||||
|
||||
|
||||
class _Blocker:
|
||||
def __init__(self, message):
|
||||
self.message = message
|
||||
|
||||
|
||||
_ce = types.ModuleType("comfy_execution")
|
||||
_gu = types.ModuleType("comfy_execution.graph_utils")
|
||||
_gu.ExecutionBlocker = _Blocker
|
||||
sys.modules.setdefault("comfy_execution", _ce)
|
||||
sys.modules.setdefault("comfy_execution.graph_utils", _gu)
|
||||
|
||||
SPEC = importlib.util.spec_from_file_location("bfs_shot_loop", ROOT / "bfs_shot_loop.py")
|
||||
assert SPEC is not None and SPEC.loader is not None
|
||||
SL = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(SL)
|
||||
H3 = "H3 (17n+5)"
|
||||
|
||||
|
||||
def _shot(i, start, end, cut=False, gen=None, count=1, queue=False, run_id=""):
|
||||
length = end - start
|
||||
return {"index": i, "count": count, "start": start, "end": end, "length": length,
|
||||
"gen_length": gen or SL.snap_up(length, H3), "fps": 24.0, "cut_before": cut,
|
||||
"queue": queue, "run_id": run_id}
|
||||
|
||||
|
||||
class GridTest(unittest.TestCase):
|
||||
def test_h3_grid(self):
|
||||
self.assertEqual([SL.snap_up(x, H3) for x in (1, 5, 6, 22, 23, 107, 108)], [5, 5, 22, 22, 39, 107, 124])
|
||||
self.assertEqual([SL.snap_down(x, H3) for x in (4, 5, 21, 22, 106, 107, 110)], [5, 5, 5, 22, 90, 107, 107])
|
||||
|
||||
def test_other_grids(self):
|
||||
self.assertEqual(SL.snap_up(50, "LTX / Wan (8n+1)"), 57)
|
||||
self.assertEqual(SL.snap_down(50, "LTX / Wan (8n+1)"), 49)
|
||||
self.assertEqual(SL.snap_up(6, "Wan (4n+1)"), 9)
|
||||
self.assertEqual(SL.snap_up(37, "any"), 37)
|
||||
|
||||
def test_generation_size_is_on_the_multiple(self):
|
||||
w, h = SL.generation_size(1280, 720, 0.15, 32)
|
||||
self.assertEqual((w % 32, h % 32), (0, 0))
|
||||
self.assertGreater(w, h)
|
||||
|
||||
|
||||
class PlanTest(unittest.TestCase):
|
||||
def test_fixed_mode_splits_evenly_under_the_limit(self):
|
||||
segs = SL.plan_segments(300, [], "fixed", 107, 24)
|
||||
self.assertEqual([s["end"] - s["start"] for s in segs], [100, 100, 100])
|
||||
|
||||
def test_shots_split_at_cuts_and_long_shots_are_divided(self):
|
||||
segs = SL.plan_segments(300, [50, 80], "shots", 107, 10)
|
||||
bounds = [(s["start"], s["end"]) for s in segs]
|
||||
self.assertEqual(bounds[:2], [(0, 50), (50, 80)])
|
||||
self.assertTrue(all(e - s <= 107 for s, e in bounds))
|
||||
self.assertEqual(bounds[-1][1], 300)
|
||||
self.assertTrue(segs[1]["cut_before"])
|
||||
|
||||
def test_short_shots_merge_into_a_neighbour_that_still_fits(self):
|
||||
segs = SL.plan_segments(200, [10, 100], "shots", 107, 24)
|
||||
self.assertEqual([(s["start"], s["end"]) for s in segs], [(0, 100), (100, 200)])
|
||||
|
||||
def test_limits_and_manual_bounds(self):
|
||||
self.assertEqual(len(SL.plan_segments(1000, [], "fixed", 100, 10, max_parts=3)), 3)
|
||||
self.assertEqual(SL.plan_segments(1000, [], "fixed", 100, 10, max_total=250)[-1]["end"], 250)
|
||||
manual = SL.plan_segments(100, [], "manual", 107, 10, manual=[30, 70])
|
||||
self.assertEqual([(s["start"], s["end"]) for s in manual], [(0, 30), (30, 70), (70, 100)])
|
||||
|
||||
def test_builtin_detector_finds_spikes_and_respects_the_gap(self):
|
||||
score = [1.0] * 100
|
||||
raw = [0.01] * 100
|
||||
for i in (20, 22, 60):
|
||||
score[i], raw[i] = 50.0, 0.5
|
||||
self.assertEqual(SL.detect_cuts(score, raw, 0.5, 24.0), [20, 60])
|
||||
|
||||
|
||||
class JoinTest(unittest.TestCase):
|
||||
def _frames(self, n, value):
|
||||
return torch.full((n, 8, 8, 3), float(value))
|
||||
|
||||
def test_trims_to_true_lengths_in_order(self):
|
||||
shots = [_shot(1, 30, 50, cut=True, gen=22), _shot(0, 0, 30, gen=39)]
|
||||
imgs = [self._frames(22, 0.8), self._frames(39, 0.2)]
|
||||
video, audio, fps = SL.BFSShotJoin()._join_all(imgs, shots, [0])
|
||||
self.assertEqual(video.shape[0], 50)
|
||||
self.assertAlmostEqual(float(video[0, 0, 0, 0]), 0.2)
|
||||
self.assertAlmostEqual(float(video[-1, 0, 0, 0]), 0.8)
|
||||
self.assertEqual(fps, 24.0)
|
||||
|
||||
def test_crossfade_only_across_soft_joins(self):
|
||||
imgs = [self._frames(39, 0.0), self._frames(39, 1.0)]
|
||||
soft = SL.BFSShotJoin()._join_all(imgs, [_shot(0, 0, 30, gen=39), _shot(1, 30, 60, gen=39)], [4])[0]
|
||||
hard = SL.BFSShotJoin()._join_all(imgs, [_shot(0, 0, 30, gen=39), _shot(1, 30, 60, cut=True, gen=39)], [4])[0]
|
||||
self.assertGreater(float(soft[31, 0, 0, 0]), 0.0)
|
||||
self.assertLess(float(soft[31, 0, 0, 0]), 1.0)
|
||||
self.assertEqual(float(hard[31, 0, 0, 0]), 1.0)
|
||||
|
||||
def test_audio_is_trimmed_to_the_video(self):
|
||||
audio = {"waveform": torch.zeros(1, 2, 48000 * 10), "sample_rate": 48000}
|
||||
imgs = [self._frames(39, 0.5)]
|
||||
_, a, _ = SL.BFSShotJoin()._join_all(imgs, [_shot(0, 0, 24, gen=39)], [0], [audio])
|
||||
self.assertEqual(a["waveform"].shape[-1], 48000)
|
||||
|
||||
def test_queue_mode_blocks_until_every_shot_is_stored(self):
|
||||
join = SL.BFSShotJoin()
|
||||
shots = [_shot(0, 0, 20, count=2, queue=True, run_id="t1"), _shot(1, 20, 45, cut=True, count=2, queue=True, run_id="t1")]
|
||||
first = join.join([self._frames(22, 0.1)], [shots[0]], [0])
|
||||
self.assertIsInstance(first[0], _Blocker)
|
||||
video, _, _ = join.join([self._frames(39, 0.9)], [shots[1]], [0])
|
||||
self.assertEqual(video.shape[0], 45)
|
||||
self.assertAlmostEqual(float(video[0, 0, 0, 0]), 0.1, places=2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,78 @@
|
||||
# BFS Shot Planner / Shot Loop
|
||||
|
||||
Turn a long video into shots a video model can actually follow, give every shot its own
|
||||
reference image and prompt, run the rest of the workflow once per shot, and join the results
|
||||
back into one video with the original timing and soundtrack.
|
||||
|
||||
## Why
|
||||
|
||||
A video model only follows a guide reliably inside the clip length it was trained on. An H3
|
||||
body-swap LoRA trained on 107-frame clips keeps the scene, framing and motion at 4.5 s and
|
||||
loses them at 10 s, where it starts summarising the clip instead of following it frame by
|
||||
frame. Splitting the source into model-sized shots, at camera cuts when there are any, keeps
|
||||
every shot inside that range.
|
||||
|
||||
## Nodes
|
||||
|
||||
| node | what it does |
|
||||
|---|---|
|
||||
| **BFS Shot Planner** | Pick a video, split it on a timeline, set a reference and prompt per shot. Outputs the shots as a ComfyUI **list**. |
|
||||
| **BFS Shot Unpack** | Opens one shot into plain values: guide frames, reference, second reference, prompt, length, first frame, audio, size, index. Use it with any model. |
|
||||
| **BFS Shot H3 Conditioning** | Ready-made MiniMax H3 conditioning for one shot, built with the native nodes (Reference to Video + Add Guide). |
|
||||
| **BFS Shot Join** | Concatenates the decoded shots in order, trims each to its true length, cross-fades soft joins and returns the soundtrack. |
|
||||
|
||||
## How the loop works
|
||||
|
||||
ComfyUI runs any node that receives a list once per item and pairs several lists by index. The
|
||||
planner's `shots` output is a list, so everything downstream of it (conditioning, sampler,
|
||||
decode) runs once per shot, and shot *i* always gets guide *i*, reference *i* and prompt *i*.
|
||||
Single values such as the model and VAE are shared by every run. The join takes the whole list
|
||||
back (it is an `INPUT_IS_LIST` node) and puts the video together.
|
||||
|
||||
Two run modes:
|
||||
|
||||
- **Auto loop (one run)**: every shot in one queue run. Simple; holds all decoded shots in memory.
|
||||
- **Queue loop (one shot per run)**: each run generates the next pending shot and stores it on
|
||||
disk. The join blocks every node after it until the last shot, then outputs the full video,
|
||||
so a Save Video node only fires once. With *Auto-queue the next shot* on, the panel re-queues
|
||||
the workflow after each shot. *Reset loop* starts over. Changing the plan starts a new run.
|
||||
|
||||
## Guide frames and lengths
|
||||
|
||||
Each shot is generated at a length the model accepts (frame grid: H3 `17n+5`, LTX/Wan `8n+1`,
|
||||
Wan `4n+1`, or any). When a shot is shorter than that length, the extra guide frames come from
|
||||
the video that follows it, so the guide never repeats or stretches. The join cuts each result
|
||||
back to the shot's own frames, which keeps the timing identical to the source. Those extra
|
||||
frames are also a real overlap, which the join uses to cross-fade boundaries that are not
|
||||
camera cuts.
|
||||
|
||||
## Splitting
|
||||
|
||||
- **Camera cuts** (default): cuts from **PySceneDetect** (`adaptive` or `content`; install
|
||||
`scenedetect`), or the built-in detector when the package is missing. Shots shorter than the
|
||||
minimum merge into a neighbour when the merge still fits; shots longer than the maximum split
|
||||
into equal parts.
|
||||
- **Fixed length**: equal parts no longer than the maximum.
|
||||
- **By hand**: drag the white handles on the timeline to move a boundary, double-click the
|
||||
shot bar to split, *Merge with next* to join. Your edits are what runs.
|
||||
|
||||
*Max seconds / shot* is converted to frames and snapped down to the grid (4.5 s at 24 fps =
|
||||
107 frames for H3). *Max shots* and *Max total seconds* cap a long source.
|
||||
|
||||
## References and prompts
|
||||
|
||||
Every shot can have its own reference, second reference and prompt; shots without them use the
|
||||
global ones. Connected `ref_image`, `ref_image_2` and `prompt` inputs override the panel's global
|
||||
values, so an edited first frame or a prompt from another node can drive the defaults.
|
||||
|
||||
## MiniMax H3 with an aligned guide (example)
|
||||
|
||||
```
|
||||
BFS Shot Planner ── shots ──► BFS Shot H3 Conditioning (clip, vae, guide_mode = aligned guide)
|
||||
└─ positive, latent ─► CFGGuider / sampler ─► VAE Decode ─┐
|
||||
BFS Shot Join ◄── images ─────────────────────────────────────────────────────────────────┘
|
||||
└─► Create Video (fps, audio) ─► Save Video
|
||||
```
|
||||
|
||||
For other models, use **BFS Shot Unpack** and wire `guide_frames`, `ref_image`, `prompt` and
|
||||
`length` into that model's own guide and reference nodes; the list mapping works the same way.
|
||||
@@ -0,0 +1,470 @@
|
||||
/**
|
||||
* BFS Shot Planner: split a long video into model-sized shots on a timeline, give each shot its own
|
||||
* reference image and prompt, and run the rest of the graph once per shot.
|
||||
*
|
||||
* All state lives in the hidden `plan` STRING widget (JSON) so a workflow saves, reloads and shares
|
||||
* exactly what you set up. The panel is a Vue app mounted into a DOM widget; the server does the video
|
||||
* work (probe, thumbnails, PySceneDetect cuts, split) through /bfs/shotloop/* routes.
|
||||
*/
|
||||
import { app } from "../../scripts/app.js";
|
||||
import { api } from "../../scripts/api.js";
|
||||
import { createApp, ref, reactive, computed, onMounted, onBeforeUnmount, h } from "./vendor/vue.esm-browser.prod.mjs";
|
||||
|
||||
const GRIDS = { "H3 (17n+5)": [17, 5], "LTX / Wan (8n+1)": [8, 1], "Wan (4n+1)": [4, 1], "any": [1, 0] };
|
||||
const DEFAULTS = {
|
||||
video: "", fps: 24, grid: "H3 (17n+5)", mode: "shots", max_s: 4.5, min_s: 1.0, sensitivity: 0.5,
|
||||
max_parts: 0, max_total_s: 0, bounds: [], segs: [], global_ref: "", global_ref2: "", global_prompt: "",
|
||||
megapixels: 0.15, multiple: 32, detector: "adaptive", run: "auto", auto_continue: true,
|
||||
};
|
||||
|
||||
const snapUp = (n, grid) => {
|
||||
const [s, o] = GRIDS[grid] || GRIDS["H3 (17n+5)"]; n = Math.max(1, n | 0);
|
||||
if (s === 1) return n; if (n <= o) return o; return o + s * Math.ceil((n - o) / s);
|
||||
};
|
||||
const snapDown = (n, grid) => {
|
||||
const [s, o] = GRIDS[grid] || GRIDS["H3 (17n+5)"]; n = Math.max(1, n | 0);
|
||||
if (s === 1) return n; if (n <= o) return o; return o + s * Math.floor((n - o) / s);
|
||||
};
|
||||
const hue = i => `hsl(${(i * 47 + 200) % 360} 55% 46%)`;
|
||||
const viewUrl = name => {
|
||||
if (!name) return "";
|
||||
const i = name.lastIndexOf("/");
|
||||
const sub = i >= 0 ? name.slice(0, i) : "", file = i >= 0 ? name.slice(i + 1) : name;
|
||||
return api.apiURL(`/view?filename=${encodeURIComponent(file)}&subfolder=${encodeURIComponent(sub)}&type=input`);
|
||||
};
|
||||
const fmtT = (f, fps) => { const s = f / fps; return `${Math.floor(s / 60)}:${(s % 60).toFixed(2).padStart(5, "0")}`; };
|
||||
|
||||
function styles() {
|
||||
if (document.getElementById("bfs-shotloop-css")) return;
|
||||
const el = document.createElement("style");
|
||||
el.id = "bfs-shotloop-css";
|
||||
el.textContent = `
|
||||
.bsl{font:12px/1.45 var(--font-family,system-ui,sans-serif);color:#c9c9cf;background:#17171b;border-radius:10px;
|
||||
height:100%;overflow:auto;box-sizing:border-box;padding:10px;position:relative}
|
||||
.bsl *{box-sizing:border-box}
|
||||
.bsl .hdr{display:flex;align-items:center;gap:8px;margin-bottom:8px}
|
||||
.bsl .ttl{font-weight:600;color:#ececf1;font-size:13px;letter-spacing:.01em}
|
||||
.bsl .pill{font-size:10px;padding:1px 7px;border-radius:999px;background:#25252d;color:#a9a9b4;border:1px solid #33333d}
|
||||
.bsl .pill.ok{background:#173527;color:#7fe0a8;border-color:#245c40}
|
||||
.bsl .pill.warn{background:#3a2a12;color:#ffc46b;border-color:#6a4a16}
|
||||
.bsl .grow{flex:1}
|
||||
.bsl select,.bsl input[type=number],.bsl input[type=text],.bsl textarea{background:#101014;border:1px solid #33333d;color:#e6e6ea;
|
||||
border-radius:6px;padding:4px 7px;font-size:11px;font-family:inherit;width:100%}
|
||||
.bsl textarea{resize:vertical;min-height:54px;font-family:ui-monospace,monospace}
|
||||
.bsl input[type=range]{width:100%;accent-color:#5b8cff}
|
||||
.bsl button{background:#26262e;border:1px solid #393945;color:#dcdce2;border-radius:6px;padding:4px 9px;font-size:11px;cursor:pointer;white-space:nowrap}
|
||||
.bsl button:hover{background:#30303a}
|
||||
.bsl button.pri{background:#3a63e0;border-color:#4b74f0;color:#fff}
|
||||
.bsl button.pri:hover{background:#4672ee}
|
||||
.bsl button.dng{background:#3a1f22;border-color:#6a2c33;color:#ffb3b3}
|
||||
.bsl button:disabled{opacity:.45;cursor:default}
|
||||
.bsl .card{background:#1d1d23;border:1px solid #2c2c35;border-radius:9px;padding:8px;margin-bottom:8px}
|
||||
.bsl .card h5{margin:0 0 6px;font-size:11px;font-weight:600;color:#b9b9c4;text-transform:uppercase;letter-spacing:.06em;display:flex;gap:6px;align-items:center}
|
||||
.bsl .grid{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:6px 8px}
|
||||
.bsl .fld label{display:block;font-size:10px;color:#8c8c97;margin-bottom:2px}
|
||||
.bsl .hint{font-size:10px;color:#7d7d88}
|
||||
.bsl .err{color:#ff8f8f;background:#2a1517;border:1px solid #5a2228;border-radius:6px;padding:5px 8px;margin-bottom:8px}
|
||||
.bsl .tl{position:relative;overflow-x:auto;overflow-y:hidden;border-radius:7px;background:#111115;border:1px solid #2a2a33}
|
||||
.bsl .tlin{position:relative}
|
||||
.bsl .strip{display:flex;height:64px;overflow:hidden}
|
||||
.bsl .strip img{height:64px;object-fit:cover;flex:none;opacity:.92}
|
||||
.bsl .spark{display:block;height:26px;width:100%}
|
||||
.bsl .segs{position:relative;height:30px;margin-top:2px;cursor:crosshair}
|
||||
.bsl .seg{position:absolute;top:2px;bottom:2px;border-radius:5px;display:flex;align-items:center;justify-content:center;
|
||||
font-size:10px;color:#fff;font-weight:600;overflow:hidden;cursor:pointer;border:2px solid transparent;user-select:none}
|
||||
.bsl .seg.sel{border-color:#fff;box-shadow:0 0 0 2px #5b8cff66}
|
||||
.bsl .seg.off{opacity:.35;background-image:repeating-linear-gradient(45deg,#0004 0 6px,#0000 6px 12px)}
|
||||
.bsl .seg.long{outline:2px solid #ff5d5d;outline-offset:-2px}
|
||||
.bsl .hdl{position:absolute;top:-70px;bottom:0;width:9px;margin-left:-4px;cursor:ew-resize;z-index:3}
|
||||
.bsl .hdl::after{content:"";position:absolute;left:3px;top:0;bottom:0;width:3px;background:#fff;opacity:.85;border-radius:2px;box-shadow:0 0 4px #000}
|
||||
.bsl .hdl:hover::after{background:#ffd34d}
|
||||
.bsl .cut{position:absolute;top:0;height:64px;width:2px;background:#ff4d6d;opacity:.9;pointer-events:none}
|
||||
.bsl .ph{position:absolute;top:0;bottom:0;width:1px;background:#ffd34d;pointer-events:none;z-index:4}
|
||||
.bsl .tip{position:absolute;z-index:6;pointer-events:none;background:#0d0d10ee;border:1px solid #3a3a44;border-radius:6px;padding:3px;font-size:10px;color:#ddd}
|
||||
.bsl .tip img{display:block;height:84px;border-radius:4px}
|
||||
.bsl .shots{display:grid;grid-template-columns:repeat(auto-fill,minmax(150px,1fr));gap:6px}
|
||||
.bsl .sc{background:#202027;border:1px solid #30303a;border-radius:8px;padding:6px;cursor:pointer;position:relative}
|
||||
.bsl .sc:hover{border-color:#4a4a58}
|
||||
.bsl .sc.sel{border-color:#5b8cff;box-shadow:0 0 0 1px #5b8cff}
|
||||
.bsl .sc .bar{height:3px;border-radius:2px;margin-bottom:5px}
|
||||
.bsl .sc .t{font-size:10px;color:#9a9aa6}
|
||||
.bsl .sc .p{font-size:10px;color:#c9c9d3;margin-top:3px;height:28px;overflow:hidden}
|
||||
.bsl .thumbs{display:flex;gap:4px;margin-top:4px}
|
||||
.bsl .rt{width:34px;height:34px;border-radius:5px;object-fit:cover;background:#2a2a33;border:1px solid #3a3a45}
|
||||
.bsl .rt.ph2{display:flex;align-items:center;justify-content:center;color:#666;font-size:9px}
|
||||
.bsl .refbox{display:flex;gap:8px;align-items:center}
|
||||
.bsl .refslot{width:78px;height:78px;border-radius:8px;border:1px dashed #44444f;background:#141418;display:flex;align-items:center;
|
||||
justify-content:center;cursor:pointer;overflow:hidden;color:#6f6f7a;font-size:10px;text-align:center;flex:none}
|
||||
.bsl .refslot img{width:100%;height:100%;object-fit:cover}
|
||||
.bsl .refslot:hover{border-color:#5b8cff}
|
||||
.bsl .modal{position:absolute;inset:0;background:#0b0b0ecc;z-index:20;display:flex;align-items:center;justify-content:center;padding:14px}
|
||||
.bsl .mbox{background:#1b1b21;border:1px solid #3a3a45;border-radius:10px;width:100%;max-height:100%;display:flex;flex-direction:column}
|
||||
.bsl .mhd{display:flex;gap:6px;align-items:center;padding:8px;border-bottom:1px solid #2c2c35}
|
||||
.bsl .mgrid{display:grid;grid-template-columns:repeat(auto-fill,minmax(84px,1fr));gap:6px;padding:8px;overflow:auto}
|
||||
.bsl .mi{border:2px solid transparent;border-radius:7px;overflow:hidden;cursor:pointer;background:#141418}
|
||||
.bsl .mi:hover{border-color:#5b8cff}
|
||||
.bsl .mi img{width:100%;height:84px;object-fit:cover;display:block}
|
||||
.bsl .mi div{font-size:9px;padding:2px 4px;color:#9a9aa6;white-space:nowrap;overflow:hidden;text-overflow:ellipsis}
|
||||
.bsl .prog{height:7px;border-radius:4px;background:#26262e;overflow:hidden}
|
||||
.bsl .prog>div{height:100%;background:linear-gradient(90deg,#3a63e0,#7fe0a8)}
|
||||
.bsl .row{display:flex;gap:6px;align-items:center;flex-wrap:wrap}
|
||||
.bsl details>summary{cursor:pointer;list-style:none;user-select:none}
|
||||
.bsl details>summary::-webkit-details-marker{display:none}
|
||||
`;
|
||||
document.head.appendChild(el);
|
||||
}
|
||||
|
||||
function Panel(io) {
|
||||
const plan = reactive({ ...DEFAULTS });
|
||||
const files = reactive({ videos: [], images: [] });
|
||||
const an = ref(null); // analysis from the server
|
||||
const cuts = ref([]);
|
||||
const detectorUsed = ref("");
|
||||
const size = reactive({ w: 0, h: 0 });
|
||||
const busy = ref(""); const error = ref("");
|
||||
const sel = ref(0); const zoom = ref(1);
|
||||
const hover = ref(null); // {frame, x}
|
||||
const gallery = ref(null); // {target: index|'global', field: 'ref'|'ref2'}
|
||||
const gq = ref("");
|
||||
const prog = reactive({ done: 0, count: 0 });
|
||||
const tlEl = ref(null);
|
||||
|
||||
const load = () => {
|
||||
let p = {};
|
||||
try { p = JSON.parse(io.getPlan() || "{}"); } catch { p = {}; }
|
||||
Object.assign(plan, DEFAULTS, p);
|
||||
if (!Array.isArray(plan.bounds)) plan.bounds = [];
|
||||
if (!Array.isArray(plan.segs)) plan.segs = [];
|
||||
};
|
||||
const save = () => io.setPlan(JSON.stringify(plan));
|
||||
|
||||
const n = computed(() => {
|
||||
if (!an.value) return 0;
|
||||
const cap = plan.max_total_s > 0 ? Math.round(plan.max_total_s * plan.fps) : Infinity;
|
||||
return Math.min(an.value.n, cap);
|
||||
});
|
||||
const maxLen = computed(() => snapDown(Math.round(plan.max_s * plan.fps), plan.grid));
|
||||
const segs = computed(() => {
|
||||
const N = n.value; if (!N) return [];
|
||||
const b = [0, ...plan.bounds.filter(x => x > 0 && x < N), N].sort((a, c) => a - c);
|
||||
const cs = new Set(cuts.value);
|
||||
let out = [];
|
||||
for (let i = 0; i < b.length - 1; i++) {
|
||||
if (b[i + 1] <= b[i]) continue;
|
||||
const m = plan.segs[out.length] || {};
|
||||
out.push({ start: b[i], end: b[i + 1], len: b[i + 1] - b[i], gen: snapUp(b[i + 1] - b[i], plan.grid),
|
||||
cut: cs.has(b[i]), enabled: m.enabled !== false, ref: m.ref || "", ref2: m.ref2 || "", prompt: m.prompt || "" });
|
||||
}
|
||||
if (plan.max_parts > 0) out = out.slice(0, plan.max_parts);
|
||||
return out;
|
||||
});
|
||||
const active = computed(() => segs.value.filter(s => s.enabled));
|
||||
const pxPerFrame = computed(() => {
|
||||
const w = (tlEl.value?.clientWidth || 600) - 2;
|
||||
return Math.max(0.2, (w / Math.max(1, n.value)) * zoom.value);
|
||||
});
|
||||
const tlWidth = computed(() => Math.max(1, Math.round(n.value * pxPerFrame.value)));
|
||||
|
||||
const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return plan.segs[i]; };
|
||||
|
||||
async function refreshFiles() {
|
||||
try { const r = await api.fetchApi("/bfs/shotloop/files"); Object.assign(files, await r.json()); } catch (e) { /* ignore */ }
|
||||
}
|
||||
async function analyze() {
|
||||
an.value = null; error.value = "";
|
||||
if (!plan.video) return;
|
||||
busy.value = "Analysing video…";
|
||||
try {
|
||||
const r = await api.fetchApi(`/bfs/shotloop/analyze?video=${encodeURIComponent(plan.video)}&fps=${plan.fps}`);
|
||||
const j = await r.json();
|
||||
if (j.error) throw new Error(j.error);
|
||||
an.value = j;
|
||||
await autoSplit(plan.bounds.length === 0);
|
||||
} catch (e) { error.value = String(e.message || e); }
|
||||
busy.value = "";
|
||||
}
|
||||
async function autoSplit(apply = true) {
|
||||
if (!plan.video) return;
|
||||
busy.value = "Detecting cuts…"; error.value = "";
|
||||
try {
|
||||
const body = { plan: { ...plan, bounds: apply ? [] : plan.bounds } };
|
||||
const r = await api.fetchApi("/bfs/shotloop/plan", { method: "POST", body: JSON.stringify(body) });
|
||||
const j = await r.json();
|
||||
if (j.error) throw new Error(j.error);
|
||||
cuts.value = j.cuts || []; detectorUsed.value = j.detector || ""; size.w = j.width; size.h = j.height;
|
||||
if (apply) {
|
||||
plan.bounds = j.segs.slice(1).map(s => s.start);
|
||||
const keep = plan.segs; plan.segs = j.segs.map((_, i) => ({ enabled: true, ref: keep[i]?.ref || "", ref2: keep[i]?.ref2 || "", prompt: keep[i]?.prompt || "" }));
|
||||
sel.value = 0; save();
|
||||
}
|
||||
} catch (e) { error.value = String(e.message || e); }
|
||||
busy.value = "";
|
||||
}
|
||||
async function upload(file, cb) {
|
||||
const fd = new FormData(); fd.append("image", file); fd.append("type", "input"); fd.append("overwrite", "true");
|
||||
busy.value = `Uploading ${file.name}…`;
|
||||
try {
|
||||
const r = await api.fetchApi("/upload/image", { method: "POST", body: fd });
|
||||
const j = await r.json();
|
||||
await refreshFiles();
|
||||
cb(j.subfolder ? `${j.subfolder}/${j.name}` : j.name);
|
||||
} catch (e) { error.value = `Upload failed: ${e}`; }
|
||||
busy.value = "";
|
||||
}
|
||||
const pickFile = (accept, cb) => {
|
||||
const inp = document.createElement("input"); inp.type = "file"; inp.accept = accept;
|
||||
inp.onchange = () => inp.files[0] && upload(inp.files[0], cb); inp.click();
|
||||
};
|
||||
async function progress(reset = false) {
|
||||
if (!plan.video || plan.run !== "queue") return;
|
||||
try {
|
||||
const r = await api.fetchApi("/bfs/shotloop/progress", { method: "POST", body: JSON.stringify({ plan: { ...plan }, plan_raw: io.getPlan(), reset }) });
|
||||
const j = await r.json(); if (!j.error) { prog.done = j.done; prog.count = j.count || active.value.length; }
|
||||
} catch { /* ignore */ }
|
||||
}
|
||||
|
||||
// ---- editing
|
||||
const setVideo = v => { plan.video = v; plan.bounds = []; plan.segs = []; save(); analyze(); };
|
||||
const splitAt = f => {
|
||||
const N = n.value; f = Math.round(f);
|
||||
if (f <= 0 || f >= N || plan.bounds.includes(f)) return;
|
||||
const i = segs.value.findIndex(s => f > s.start && f < s.end);
|
||||
plan.bounds = [...plan.bounds, f].sort((a, c) => a - c);
|
||||
plan.segs.splice(i + 1, 0, { ...(plan.segs[i] || {}) });
|
||||
sel.value = i + 1; save();
|
||||
};
|
||||
const mergeNext = i => {
|
||||
const s = segs.value[i]; if (!s || i >= segs.value.length - 1) return;
|
||||
plan.bounds = plan.bounds.filter(b => b !== s.end); plan.segs.splice(i + 1, 1); save();
|
||||
};
|
||||
const setMeta = (i, k, v) => { meta(i)[k] = v; save(); };
|
||||
const applyAll = (k) => { const v = meta(sel.value)[k] || ""; segs.value.forEach((_, i) => { meta(i)[k] = v; }); save(); };
|
||||
const drag = (k, ev) => {
|
||||
ev.preventDefault(); ev.stopPropagation();
|
||||
const box = tlEl.value.getBoundingClientRect();
|
||||
const move = e => {
|
||||
const x = e.clientX - box.left + tlEl.value.scrollLeft;
|
||||
const f = Math.round(x / pxPerFrame.value);
|
||||
const lo = (plan.bounds[k - 1] ?? 0) + 1, hi = (plan.bounds[k + 1] ?? n.value) - 1;
|
||||
plan.bounds[k] = Math.max(lo, Math.min(hi, f));
|
||||
hover.value = { frame: plan.bounds[k], x };
|
||||
};
|
||||
const up = () => { window.removeEventListener("pointermove", move); window.removeEventListener("pointerup", up); save(); };
|
||||
window.addEventListener("pointermove", move); window.addEventListener("pointerup", up);
|
||||
};
|
||||
const frameAt = e => { const box = tlEl.value.getBoundingClientRect(); const x = e.clientX - box.left + tlEl.value.scrollLeft; return { frame: Math.max(0, Math.min(n.value - 1, Math.round(x / pxPerFrame.value))), x }; };
|
||||
const thumbFor = f => { const t = an.value?.thumbs; if (!t?.length) return ""; let b = t[0]; for (const x of t) { if (x.f <= f) b = x; else break; } return b.src; };
|
||||
|
||||
// ---- queue loop events
|
||||
const onProg = e => { prog.done = e.detail.done; prog.count = e.detail.count; };
|
||||
const onNext = e => { onProg(e); if (plan.run === "queue" && plan.auto_continue) setTimeout(() => app.queuePrompt(0, 1), 300); };
|
||||
onMounted(() => {
|
||||
load(); refreshFiles(); analyze(); progress();
|
||||
api.addEventListener("bfs-shotloop-progress", onProg); api.addEventListener("bfs-shotloop-next", onNext);
|
||||
});
|
||||
onBeforeUnmount(() => { api.removeEventListener("bfs-shotloop-progress", onProg); api.removeEventListener("bfs-shotloop-next", onNext); });
|
||||
io.expose({ reload: () => { load(); analyze(); progress(); } });
|
||||
|
||||
// ---- view helpers
|
||||
const fld = (label, input, hint) => h("div", { class: "fld" }, [h("label", label), input, hint ? h("div", { class: "hint" }, hint) : null]);
|
||||
const num = (k, step = 0.1, min = 0) => h("input", { type: "number", step, min, value: plan[k], onChange: e => { plan[k] = parseFloat(e.target.value) || 0; save(); } });
|
||||
const sel_ = (k, opts, after) => h("select", { value: plan[k], onChange: e => { plan[k] = e.target.value; save(); after && after(); } },
|
||||
opts.map(o => h("option", { value: Array.isArray(o) ? o[0] : o }, Array.isArray(o) ? o[1] : o)));
|
||||
const refSlot = (name, label, onClick, onClear) => h("div", { class: "refslot", title: name || label, onClick },
|
||||
name ? [h("img", { src: viewUrl(name) })] : [label]);
|
||||
|
||||
return () => {
|
||||
const S = segs.value, cur = S[sel.value], N = n.value, ppf = pxPerFrame.value, fps = plan.fps;
|
||||
const tooLong = S.filter(s => s.len > maxLen.value).length;
|
||||
const totalGen = active.value.reduce((a, s) => a + s.gen, 0);
|
||||
|
||||
const header = h("div", { class: "hdr" }, [
|
||||
h("span", { class: "ttl" }, "🎬 Shot Planner"),
|
||||
busy.value ? h("span", { class: "pill warn" }, busy.value) : (an.value ? h("span", { class: "pill ok" }, `${active.value.length} shots · ${(N / fps).toFixed(1)}s`) : null),
|
||||
h("span", { class: "grow" }),
|
||||
plan.run === "queue" ? h("span", { class: "pill" }, "queue loop") : h("span", { class: "pill" }, "auto loop"),
|
||||
]);
|
||||
|
||||
const source = h("div", { class: "card" }, [
|
||||
h("h5", ["Source video", detectorUsed.value ? h("span", { class: "pill" }, detectorUsed.value) : null]),
|
||||
h("div", { class: "row" }, [
|
||||
h("div", { style: "flex:1;min-width:180px" }, [h("select", { value: plan.video, onChange: e => setVideo(e.target.value) },
|
||||
[h("option", { value: "" }, "— choose a video —"), ...files.videos.map(v => h("option", { value: v }, v))])]),
|
||||
h("button", { onClick: () => pickFile("video/*", setVideo) }, "⬆ Upload"),
|
||||
h("button", { onClick: refreshFiles, title: "refresh the input folder" }, "↻"),
|
||||
]),
|
||||
an.value ? h("div", { class: "hint", style: "margin-top:4px" },
|
||||
`${an.value.width}×${an.value.height} · ${an.value.fps_src.toFixed(2)} fps · ${an.value.duration.toFixed(2)}s → timeline ${fps} fps, ${an.value.n} frames · generate at ${size.w}×${size.h}`) : null,
|
||||
]);
|
||||
|
||||
const settings = h("details", { class: "card", open: true }, [
|
||||
h("summary", h("h5", ["Split settings", h("span", { class: "hint", style: "text-transform:none;letter-spacing:0" }, `max ${maxLen.value} frames per shot (${(maxLen.value / fps).toFixed(2)}s)`)])),
|
||||
h("div", { class: "grid" }, [
|
||||
fld("Mode", sel_("mode", [["shots", "Camera cuts"], ["fixed", "Fixed length"]])),
|
||||
fld("Detector", sel_("detector", [["adaptive", "PySceneDetect adaptive"], ["content", "PySceneDetect content"], ["builtin", "Built-in"]])),
|
||||
fld(`Sensitivity ${Number(plan.sensitivity).toFixed(2)}`, h("input", { type: "range", min: 0, max: 1, step: 0.05, value: plan.sensitivity, onInput: e => { plan.sensitivity = parseFloat(e.target.value); }, onChange: save })),
|
||||
fld("Frame grid", sel_("grid", Object.keys(GRIDS))),
|
||||
fld("Max seconds / shot", num("max_s", 0.1, 0.2)),
|
||||
fld("Min seconds / shot", num("min_s", 0.1, 0)),
|
||||
fld("Timeline fps", num("fps", 1, 1), "re-analyses"),
|
||||
fld("Max shots (0 = all)", num("max_parts", 1, 0)),
|
||||
fld("Max total seconds (0 = all)", num("max_total_s", 0.5, 0)),
|
||||
fld("Megapixels", num("megapixels", 0.01, 0.02)),
|
||||
fld("Size multiple", num("multiple", 8, 8)),
|
||||
fld("Run", sel_("run", [["auto", "Auto loop (one run)"], ["queue", "Queue loop (one shot per run)"]], progress)),
|
||||
]),
|
||||
h("div", { class: "row", style: "margin-top:8px" }, [
|
||||
h("button", { class: "pri", disabled: !plan.video || !!busy.value, onClick: () => autoSplit(true) }, "✂ Auto split"),
|
||||
h("button", { disabled: !plan.video, onClick: analyze }, "↻ Re-analyse"),
|
||||
h("span", { class: "hint" }, "Drag the white handles to move a boundary · double-click the shot bar to split there"),
|
||||
]),
|
||||
]);
|
||||
|
||||
// timeline
|
||||
const thumbs = an.value?.thumbs || [];
|
||||
const thumbW = Math.max(8, (an.value?.thumb_w || 96) * 0 + (thumbs.length ? tlWidth.value / thumbs.length : 96));
|
||||
const score = an.value?.score || [];
|
||||
const maxS = Math.max(4, ...score.slice(0, N));
|
||||
const sparkPts = score.slice(0, N).map((v, i) => `${(i * ppf).toFixed(1)},${(24 - Math.min(1, v / maxS) * 22).toFixed(1)}`).join(" ");
|
||||
const timeline = an.value ? h("div", { class: "card" }, [
|
||||
h("h5", ["Timeline", h("span", { class: "grow" }), h("span", { class: "hint", style: "text-transform:none" }, "zoom"),
|
||||
h("input", { type: "range", min: 1, max: 8, step: 0.5, value: zoom.value, style: "width:110px", onInput: e => { zoom.value = parseFloat(e.target.value); } })]),
|
||||
h("div", { class: "tl", ref: tlEl,
|
||||
onPointermove: e => { if (!e.buttons) hover.value = frameAt(e); }, onPointerleave: () => { hover.value = null; } }, [
|
||||
h("div", { class: "tlin", style: `width:${tlWidth.value}px` }, [
|
||||
h("div", { class: "strip" }, thumbs.map(t => h("img", { src: t.src, style: `width:${thumbW}px` }))),
|
||||
...cuts.value.filter(c => c < N).map(c => h("div", { class: "cut", style: `left:${c * ppf}px`, title: `cut @ ${c}` })),
|
||||
h("svg", { class: "spark", viewBox: `0 0 ${tlWidth.value} 26`, preserveAspectRatio: "none" },
|
||||
[h("polyline", { points: sparkPts, fill: "none", stroke: "#ff7a90", "stroke-width": 1, "vector-effect": "non-scaling-stroke" })]),
|
||||
h("div", { class: "segs", onDblclick: e => splitAt(frameAt(e).frame) }, [
|
||||
...S.map((s, i) => h("div", {
|
||||
class: ["seg", i === sel.value && "sel", !s.enabled && "off", s.len > maxLen.value && "long"],
|
||||
style: `left:${s.start * ppf}px;width:${Math.max(2, s.len * ppf - 1)}px;background:${hue(i)}`,
|
||||
title: `#${i + 1} · frames ${s.start}-${s.end - 1} · ${s.len} → ${s.gen}`, onClick: () => { sel.value = i; },
|
||||
}, s.len * ppf > 26 ? `${i + 1}` : "")),
|
||||
...plan.bounds.filter(b => b < N).map((b, k) => h("div", { class: "hdl", style: `left:${b * ppf}px`, title: `boundary @ ${b}`, onPointerdown: e => drag(k, e) })),
|
||||
]),
|
||||
hover.value ? h("div", { class: "ph", style: `left:${hover.value.x}px` }) : null,
|
||||
]),
|
||||
hover.value ? h("div", { class: "tip", style: `left:${Math.min(tlWidth.value - 160, Math.max(0, hover.value.x - (tlEl.value?.scrollLeft || 0) - 70))}px;top:2px` },
|
||||
[h("img", { src: thumbFor(hover.value.frame) }), h("div", `frame ${hover.value.frame} · ${fmtT(hover.value.frame, fps)}`)]) : null,
|
||||
]),
|
||||
h("div", { class: "row", style: "margin-top:6px" }, [
|
||||
h("span", { class: "pill" }, `${S.length} shots`), h("span", { class: "pill" }, `${cuts.value.length} cuts`),
|
||||
h("span", { class: "pill" }, `generate ${totalGen} frames`),
|
||||
tooLong ? h("span", { class: "pill warn" }, `${tooLong} shot(s) longer than ${maxLen.value} frames`) : null,
|
||||
]),
|
||||
]) : null;
|
||||
|
||||
// shot cards
|
||||
const cards = S.length ? h("div", { class: "card" }, [
|
||||
h("h5", "Shots"),
|
||||
h("div", { class: "shots" }, S.map((s, i) => h("div", { class: ["sc", i === sel.value && "sel"], onClick: () => { sel.value = i; } }, [
|
||||
h("div", { class: "bar", style: `background:${hue(i)}` }),
|
||||
h("div", { class: "row" }, [h("b", `#${i + 1}`), s.cut ? h("span", { class: "pill" }, "cut") : null, !s.enabled ? h("span", { class: "pill warn" }, "off") : null]),
|
||||
h("div", { class: "t" }, `${fmtT(s.start, fps)} → ${fmtT(s.end, fps)} · ${s.len}f → ${s.gen}f`),
|
||||
h("div", { class: "thumbs" }, [
|
||||
s.ref || plan.global_ref ? h("img", { class: "rt", src: viewUrl(s.ref || plan.global_ref), style: s.ref ? "" : "opacity:.45" }) : h("div", { class: "rt ph2" }, "ref"),
|
||||
s.ref2 || plan.global_ref2 ? h("img", { class: "rt", src: viewUrl(s.ref2 || plan.global_ref2), style: s.ref2 ? "" : "opacity:.45" }) : h("div", { class: "rt ph2" }, "ref2"),
|
||||
h("img", { class: "rt", src: thumbFor(s.start + Math.floor(s.len / 2)), style: "width:56px" }),
|
||||
]),
|
||||
h("div", { class: "p" }, s.prompt ? s.prompt : (plan.global_prompt ? "↳ global prompt" : "— no prompt —")),
|
||||
]))),
|
||||
]) : null;
|
||||
|
||||
// editor for the selected shot
|
||||
const editor = cur ? h("div", { class: "card" }, [
|
||||
h("h5", [`Shot #${sel.value + 1}`, h("span", { class: "hint", style: "text-transform:none" }, `frames ${cur.start}–${cur.end - 1} · ${cur.len} → generate ${cur.gen}`)]),
|
||||
h("div", { class: "row", style: "align-items:flex-start;gap:10px" }, [
|
||||
h("div", { class: "refbox" }, [
|
||||
refSlot(cur.ref, "＋ reference\n(uses global)", () => { gallery.value = { target: sel.value, field: "ref" }; }),
|
||||
refSlot(cur.ref2, "＋ ref 2\n(uses global)", () => { gallery.value = { target: sel.value, field: "ref2" }; }),
|
||||
]),
|
||||
h("div", { style: "flex:1;min-width:200px" }, [
|
||||
h("textarea", { placeholder: "Prompt for this shot (empty = global prompt)", value: cur.prompt, onChange: e => setMeta(sel.value, "prompt", e.target.value) }),
|
||||
]),
|
||||
]),
|
||||
h("div", { class: "row", style: "margin-top:6px" }, [
|
||||
h("button", { onClick: () => setMeta(sel.value, "enabled", !cur.enabled) }, cur.enabled ? "⏸ Disable" : "▶ Enable"),
|
||||
h("button", { onClick: () => splitAt(cur.start + Math.floor(cur.len / 2)) }, "✂ Split in half"),
|
||||
h("button", { disabled: sel.value >= S.length - 1, onClick: () => mergeNext(sel.value) }, "⇥ Merge with next"),
|
||||
h("button", { onClick: () => applyAll("ref") }, "Ref → all"), h("button", { onClick: () => applyAll("prompt") }, "Prompt → all"),
|
||||
h("button", { class: "dng", onClick: () => { ["ref", "ref2", "prompt"].forEach(k => { meta(sel.value)[k] = ""; }); save(); } }, "Use global"),
|
||||
]),
|
||||
]) : null;
|
||||
|
||||
const globals = h("div", { class: "card" }, [
|
||||
h("h5", "Global (used by shots without their own)"),
|
||||
h("div", { class: "row", style: "align-items:flex-start;gap:10px" }, [
|
||||
h("div", { class: "refbox" }, [
|
||||
refSlot(plan.global_ref, "＋ global\nreference", () => { gallery.value = { target: "global", field: "global_ref" }; }),
|
||||
refSlot(plan.global_ref2, "＋ global\nref 2", () => { gallery.value = { target: "global", field: "global_ref2" }; }),
|
||||
]),
|
||||
h("div", { style: "flex:1;min-width:200px" }, [h("textarea", { placeholder: "Global prompt (a connected `prompt` input overrides it)", value: plan.global_prompt, onChange: e => { plan.global_prompt = e.target.value; save(); } })]),
|
||||
]),
|
||||
h("div", { class: "hint", style: "margin-top:4px" }, "Connected ref_image / ref_image_2 / prompt inputs on the node override these defaults."),
|
||||
]);
|
||||
|
||||
const queue = plan.run === "queue" ? h("div", { class: "card" }, [
|
||||
h("h5", ["Queue loop", h("span", { class: "grow" }), h("span", { class: "hint", style: "text-transform:none" }, `${prog.done}/${prog.count || active.value.length} shots done`)]),
|
||||
h("div", { class: "prog" }, [h("div", { style: `width:${(100 * prog.done / Math.max(1, prog.count || active.value.length)).toFixed(1)}%` })]),
|
||||
h("div", { class: "row", style: "margin-top:6px" }, [
|
||||
h("label", { class: "row" }, [h("input", { type: "checkbox", checked: plan.auto_continue, onChange: e => { plan.auto_continue = e.target.checked; save(); } }), "Auto-queue the next shot"]),
|
||||
h("button", { onClick: () => progress(false) }, "↻ Status"), h("button", { class: "dng", onClick: () => progress(true) }, "⟲ Reset loop"),
|
||||
]),
|
||||
h("div", { class: "hint", style: "margin-top:4px" }, "Each run generates one shot and stores it. Nodes after BFS Shot Join only run on the last shot, with the full video."),
|
||||
]) : null;
|
||||
|
||||
const modal = gallery.value ? h("div", { class: "modal", onClick: e => { if (e.target === e.currentTarget) gallery.value = null; } }, [
|
||||
h("div", { class: "mbox" }, [
|
||||
h("div", { class: "mhd" }, [
|
||||
h("b", "Choose reference"), h("input", { type: "text", placeholder: "search…", value: gq.value, onInput: e => { gq.value = e.target.value; }, style: "flex:1" }),
|
||||
h("button", { onClick: () => pickFile("image/*", name => { choose(name); }) }, "⬆ Upload"),
|
||||
h("button", { onClick: () => choose("") }, "None"), h("button", { onClick: () => { gallery.value = null; } }, "✕"),
|
||||
]),
|
||||
h("div", { class: "mgrid" }, files.images.filter(f => f.toLowerCase().includes(gq.value.toLowerCase())).slice(0, 400).map(f =>
|
||||
h("div", { class: "mi", onClick: () => choose(f), title: f }, [h("img", { src: viewUrl(f), loading: "lazy" }), h("div", f)]))),
|
||||
]),
|
||||
]) : null;
|
||||
function choose(name) {
|
||||
const g = gallery.value; if (!g) return;
|
||||
if (g.target === "global") plan[g.field] = name; else meta(g.target)[g.field] = name;
|
||||
save(); gallery.value = null;
|
||||
}
|
||||
|
||||
return h("div", { class: "bsl" }, [header, error.value ? h("div", { class: "err" }, error.value) : null, source, settings, timeline, cards, editor, globals, queue, modal]);
|
||||
};
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "BFSNodes.ShotPlanner",
|
||||
async nodeCreated(node) {
|
||||
if (node.comfyClass !== "BFSShotPlanner") return;
|
||||
styles();
|
||||
const planWidget = node.widgets?.find(w => w.name === "plan");
|
||||
if (planWidget) { planWidget.type = "hidden"; planWidget.computeSize = () => [0, -4]; }
|
||||
const host = document.createElement("div");
|
||||
host.style.cssText = "width:100%;height:100%;min-height:520px";
|
||||
node.addDOMWidget("bfs_shot_planner", "div", host, { serialize: false, hideOnZoom: false });
|
||||
const handle = {};
|
||||
createApp({
|
||||
setup: () => Panel({
|
||||
getPlan: () => planWidget?.value ?? "{}",
|
||||
setPlan: v => { if (planWidget) planWidget.value = v; node.graph?.setDirtyCanvas(true); },
|
||||
expose: o => Object.assign(handle, o),
|
||||
}),
|
||||
}).mount(host);
|
||||
if (planWidget) {
|
||||
planWidget.options = planWidget.options || {};
|
||||
planWidget.options.serialize = true;
|
||||
planWidget.serializeValue = () => planWidget.value;
|
||||
}
|
||||
const prev = node.onConfigure;
|
||||
node.onConfigure = function () {
|
||||
const r = prev?.apply(this, arguments);
|
||||
setTimeout(() => handle.reload?.(), 0);
|
||||
return r;
|
||||
};
|
||||
node.size = [Math.max(node.size[0], 780), Math.max(node.size[1], 860)];
|
||||
},
|
||||
});
|
||||
Reference in New Issue
Block a user