diff --git a/README.md b/README.md index f71ff09..8b2efbd 100644 --- a/README.md +++ b/README.md @@ -8,15 +8,17 @@ A ComfyUI custom node that brings a **DAW-style interactive video timeline** dir ## Features -![Node Controls](examples/ComfyUI-Video-Frame-Extractor--Node.png) +![Node Controls](examples/ComfyUI-Video-Frame-Extractor--Node.v1.3.0.png) -- **Interactive timeline** — filmstrip thumbnails, brightness waveform, and a zoomable timecode ruler all rendered directly inside the node +- **Interactive timeline** — filmstrip thumbnails with correct aspect ratio (including portrait video), brightness waveform, and a zoomable timecode ruler all rendered directly inside the node - **Loop region selection** — drag the in/out handles or the region body to define start and end frames; the video preview seeks live as you drag - **Resizable video preview** — drag the resize handle to make the preview pane as large as you need - **Collapsible preview section** — hide the video preview to save canvas space while keeping the timeline accessible -- **Transport controls** — Play / Pause toggle, frame-step ◀ ▶ buttons that shift the entire loop window one frame at a time +- **Transport controls** — Play / Pause toggle, frame-step ◀ ▶ buttons, and ⏮ ⏭ Jump buttons that advance the loop window by its own span for stepping through non-overlapping clips +- **Live loading status** — progress bar, buffering percentage, elapsed time, and health indicators (stalled / error / timeout) shown while a video loads +- **Memory-safe execution** — live peak-memory estimate in the info bar updates as you adjust the loop region and `target_fps`; the backend refuses to run clips that would exceed available RAM, so ComfyUI never freezes on an oversized extraction - **Zoom and pan** — scroll wheel to zoom the timeline, Shift + scroll to pan horizontally -- **Full outputs** — forward frames, reversed frames, first and last frame as individual images, filename prefix, dimensions, FPS, and a custom-FPS resampled frame batch +- **Full outputs** — 10 output pins: forward frames, reversed frames, first and last frame, filename prefix, dimensions, original FPS, custom-FPS resampled frame batch, and target FPS passthrough --- @@ -49,13 +51,16 @@ Restart ComfyUI after installation. | opencv-python | ≥ 4.7 | | Pillow | ≥ 9.0 | | numpy | ≥ 1.22 | +| psutil | ≥ 5.9 (optional) | | torch | (provided by ComfyUI) | +`psutil` is optional — when installed, the memory-safety check auto-tunes its limit to 75 % of your system's available RAM. Without it, the limit falls back to a fixed 8 GB. + --- ## Usage -[![Watch the video](https://img.youtube.com/vi/vsk4-RGSdnA/maxresdefault.jpg)](https://youtu.be/vsk4-RGSdnA) +[![Watch the video](https://img.youtube.com/vi/vsk4-RGSdnA/maxresdefault.jpg)](https://youtu.be/TWVsyQUB39s) 1. Place a video file in `ComfyUI/input/` or upload one directly via the **Choose Video to Upload** button on the node 2. Add the **Video Frame Extractor 🎞️** node to your graph _(right-click canvas → Add Node → video)_ @@ -65,16 +70,17 @@ Restart ComfyUI after installation. ### Timeline Controls -| Interaction | Action | -| ---------------------- | ------------------------------------------- | -| Drag loop handle (◀ ▶) | Resize the loop window | -| Drag loop body | Slide the window without changing its span | -| ▶ / ⏸ button | Play or pause the preview within the loop | -| ◀ / ▶ step buttons | Shift the entire loop window one frame | -| Scroll wheel | Zoom the timeline centred on the cursor | -| Shift + scroll | Pan the timeline left / right | -| Drag resize bar | Adjust the height of the video preview | -| Click collapse toggle | Show / hide the video preview and transport | +| Interaction | Action | +| ---------------------- | ----------------------------------------------------- | +| Drag loop handle (◀ ▶) | Resize the loop window | +| Drag loop body | Slide the window without changing its span | +| ▶ / ⏸ button | Play or pause the preview within the loop | +| ◀ / ▶ step buttons | Shift the entire loop window one frame | +| ⏮ / ⏭ jump buttons | Jump the loop window backward/forward by its own span | +| Scroll wheel | Zoom the timeline centred on the cursor | +| Shift + scroll | Pan the timeline left / right | +| Drag resize bar | Adjust the height of the video preview | +| Click collapse toggle | Show / hide the video preview and transport | --- @@ -82,27 +88,28 @@ Restart ComfyUI after installation. ### Inputs -| Name | Type | Description | -| ------------- | ----------- | ------------------------------------------------------------------- | -| `video` | File picker | MP4, AVI, MOV, MKV, or WebM file | -| `start_frame` | INT | Loop region start (auto-set by timeline) | -| `end_frame` | INT | Loop region end (auto-set by timeline) | -| `num_frames` | INT | Frames to extract — **read-only, auto-computed** as end − start + 1 | -| `target_fps` | FLOAT | Target frame rate for the `frames_at_fps` output (default: 24.0) | +| Name | Type | Description | +| ------------- | ----------- | ------------------------------------------------------------------------------- | +| `video` | File picker | MP4, AVI, MOV, MKV, or WebM file — upload via **Choose Video to Upload** button | +| `start_frame` | INT | Loop region start (auto-set by timeline) | +| `end_frame` | INT | Loop region end (auto-set by timeline) | +| `num_frames` | INT | Auto-computed as end − start + 1; hidden on the node, synced internally | +| `target_fps` | FLOAT | Target frame rate for the `Clipped Frames at Target FPS` output (default: 24.0) | ### Outputs -| Name | Type | Description | -| ----------------- | ------ | ------------------------------------------------------------------------------------------------------------- | -| `frames` | IMAGE | Batch of extracted frames, forward order `(N, H, W, 3)` | -| `frames_reversed` | IMAGE | Same batch in reverse order — useful for ping-pong effects | -| `first_frame` | IMAGE | Single frame at the loop start `(1, H, W, 3)` | -| `last_frame` | IMAGE | Single frame at the loop end `(1, H, W, 3)` | -| `filename_prefix` | STRING | Video filename without extension, e.g. `my_clip` | -| `width` | INT | Video width in pixels | -| `height` | INT | Video height in pixels | -| `fps` | FLOAT | Source frame rate | -| `frames_at_fps` | IMAGE | Frame batch resampled to `target_fps` — frame count is computed automatically from clip duration × target FPS | +| Pin name | Type | Description | +| -------------------------------- | ------ | ---------------------------------------------------------------------------------------------------- | +| `Clipped Frames` | IMAGE | Batch of extracted frames, forward order `(N, H, W, 3)` | +| `Reversed Clipped Frames` | IMAGE | Same batch in reverse order — useful for ping-pong effects | +| `First Frame` | IMAGE | Single frame at the loop start `(1, H, W, 3)` | +| `Last Frame` | IMAGE | Single frame at the loop end `(1, H, W, 3)` | +| `Filename Prefix (no extension)` | STRING | Video filename without extension, e.g. `my_clip` | +| `Width` | INT | Video width in pixels | +| `Height` | INT | Video height in pixels | +| `Original FPS` | FLOAT | Source frame rate of the video file | +| `Clipped Frames at Target FPS` | IMAGE | Frame batch resampled to `target_fps` — count computed automatically from clip duration × target FPS | +| `Target FPS` | FLOAT | Passes through the `target_fps` input value for use downstream | --- @@ -174,6 +181,21 @@ Any format supported by your OpenCV build will also work. - Added `target_fps` input and `frames_at_fps` output — extracts a frame batch resampled to any target frame rate, independent of `num_frames` +### 1.3.0 + +- **Reliable video loading** — previous video is now fully stopped, unloaded, and its memory released before a new file begins loading; playing/looping videos no longer block new loads from completing +- **Loading progress bar** — the preview area shows a live progress bar (`Loading video… N%`) while the video buffers, an animated shimmer for indeterminate states (metadata fetch), and detailed status lines (elapsed time, buffered/total seconds, HTML5 `readyState`) +- **Load health indicators** — stalled downloads turn the bar amber with a "Stalled at N%" label; network errors, decode failures, and unsupported formats surface a red error overlay with the specific error reason; a 30-second watchdog flags videos that never become playable +- **Memory estimation & safety guard** — the info bar now shows a live peak-memory estimate (`✓ MEM est. X GB / Y GB limit`) that turns amber above 75% and red at the limit; the backend independently blocks execution with a clear `RuntimeError` before allocation to prevent ComfyUI from freezing on oversized clips. The limit adapts to 75 % of available system RAM when `psutil` is installed, falling back to 8 GB otherwise +- **Single-output execution fix** — the node is now marked `OUTPUT_NODE = True`, so workflows that only connect one pin (e.g. just `Clipped Frames at Target FPS`) no longer fail with "Prompt has no outputs" +- **Jump buttons** — ⏮ **JUMP BACK** and ⏭ **JUMP FWD** added to the transport bar; each press shifts the loop window by its own span so you can step through non-overlapping clips; buttons dim and disable when no full jump fits in that direction +- **Dynamic filmstrip height** — the filmstrip now scales its height based on the actual video aspect ratio so portrait footage is no longer squished; thumbnails also use the real aspect ratio instead of assuming 16:9 +- **Improved info bar** — loop summary text is now larger (bold 13 px), white, and easier to read at a glance; a second row shows the memory estimate and the `N+M fr @ WxH` frame breakdown +- **Cleaner output labels** — all 10 output pins renamed to plain English: `Clipped Frames`, `Reversed Clipped Frames`, `First Frame`, `Last Frame`, `Filename Prefix (no extension)`, `Width`, `Height`, `Original FPS`, `Clipped Frames at Target FPS`, `Target FPS` +- **Target FPS output** — added a 10th output pin that passes through the `target_fps` input value for use downstream +- **Upload button label** — renamed from "choose file to upload" to "Choose Video to Upload" +- **Hidden num_frames widget** — the auto-computed `num_frames` field is no longer shown on the node; its value is still synced internally so the backend receives the correct frame count + --- ## License diff --git a/examples/ComfyUI-Video-Frame-Extractor--Node.v1.3.0.png b/examples/ComfyUI-Video-Frame-Extractor--Node.v1.3.0.png new file mode 100644 index 0000000..4c0bebf Binary files /dev/null and b/examples/ComfyUI-Video-Frame-Extractor--Node.v1.3.0.png differ diff --git a/examples/ComfyUI-Video-Frame-Extractor.v1.3.0.gif b/examples/ComfyUI-Video-Frame-Extractor.v1.3.0.gif new file mode 100644 index 0000000..46ab664 Binary files /dev/null and b/examples/ComfyUI-Video-Frame-Extractor.v1.3.0.gif differ diff --git a/examples/README.md b/examples/README.md index d1454e2..307ec96 100644 --- a/examples/README.md +++ b/examples/README.md @@ -2,6 +2,6 @@ ## Node Controls -![Node Controls](ComfyUI-Video-Frame-Extractor--Node.png) +![Node Controls](ComfyUI-Video-Frame-Extractor--Node.v1.3.0.png) -![Video Timeline](ComfyUI-Video-Frame-Extractor--002.gif) +![Video Timeline](ComfyUI-Video-Frame-Extractor.v1.3.0.gif) diff --git a/node_list.json b/node_list.json index ecf5b3f..8811b5e 100644 --- a/node_list.json +++ b/node_list.json @@ -1,7 +1,7 @@ { "name": "ComfyUI-Video-Frame-Extractor", "description": "A ComfyUI custom node that brings a DAW-style interactive video timeline directly into the node graph. Upload any video, scrub through it in real time, drag a loop region to define your extraction window, and pipe the resulting frame batch into any downstream node.", - "version": "1.1.1", + "version": "1.3.0", "author": "comfyuiattic-989", "nodes": [ { diff --git a/pyproject.toml b/pyproject.toml index 1a9f5e4..5e04572 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "comfyui-video-frame-extractor" -version = "1.2.0" +version = "1.3.0" description = "ComfyUI custom node that brings a DAW-style interactive video timeline directly into the node graph. Upload any video, scrub through it in real time, drag a loop region to define your extraction window, and pipe the resulting frame batch into any downstream node." readme = "README.md" requires-python = ">=3.9" diff --git a/requirements.txt b/requirements.txt index 7873cbf..1ce2e1e 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,3 +1,4 @@ opencv-python>=4.7 Pillow>=9.0 numpy>=1.22 +# psutil>=5.9 # optional — enables RAM-aware memory safety limit \ No newline at end of file diff --git a/video_frame_extractor.py b/video_frame_extractor.py index 0030ccf..abab050 100644 --- a/video_frame_extractor.py +++ b/video_frame_extractor.py @@ -12,6 +12,63 @@ import cv2 # Helper utilities # --------------------------------------------------------------------------- +# Safety limits for memory estimation +DEFAULT_MAX_MEMORY_BYTES = 8 * 1024**3 # hard cap if psutil unavailable +SYSTEM_RAM_FRACTION_CAP = 0.75 # never exceed 75% of available RAM + + +def estimate_peak_memory(width: int, height: int, + num_frames: int, custom_num_frames: int) -> int: + """ + Estimate peak RAM in bytes used during frame extraction. + + Breakdown per call to extract_frames: + • uint8 staging buffer N × H × W × 3 × 1 + • float32 output tensor N × H × W × 3 × 4 + Both live in memory simultaneously at peak. + + Plus we keep around: + • frames (float32, N frames) + • frames_reversed (float32, same N — torch.flip allocates new storage) + • frames_at_fps (float32, custom_N frames) + """ + bpp_u8 = width * height * 3 # bytes per frame, uint8 + bpp_f32 = bpp_u8 * 4 # bytes per frame, float32 + + # Peak during the first extract_frames() call + first_call_peak = (num_frames * bpp_u8) + (num_frames * bpp_f32) + + # After first call completes, frames (f32) + frames_reversed (f32) are resident + resident_after_first = (num_frames * bpp_f32) * 2 + + # Peak during the second extract_frames() call (target-fps) + second_call_peak = ( + resident_after_first + + (custom_num_frames * bpp_u8) + + (custom_num_frames * bpp_f32) + ) + + return max(first_call_peak, second_call_peak) + + +def get_memory_limit() -> int: + """Return the max allowed peak memory in bytes.""" + try: + import psutil + available = psutil.virtual_memory().available + return int(available * SYSTEM_RAM_FRACTION_CAP) + except Exception: + return DEFAULT_MAX_MEMORY_BYTES + + +def format_bytes(n: int) -> str: + for unit in ("B", "KB", "MB", "GB", "TB"): + if abs(n) < 1024.0: + return f"{n:.1f} {unit}" + n /= 1024.0 + return f"{n:.1f} PB" + + def get_video_info(video_path: str) -> dict: """Return basic metadata for a video file.""" cap = cv2.VideoCapture(video_path) @@ -30,6 +87,8 @@ def get_video_info(video_path: str) -> dict: "width": width, "height": height, "duration_seconds": total / fps if fps > 0 else 0, + "bytes_per_frame_f32": width * height * 3 * 4, + "memory_limit_bytes": get_memory_limit(), } @@ -142,12 +201,13 @@ class VideoFrameExtractor: }, } - RETURN_TYPES = ("IMAGE", "IMAGE", "IMAGE", "IMAGE", "STRING", "INT", "INT", "FLOAT", "IMAGE") - RETURN_NAMES = ("frames", "frames_reversed", "first_frame", "last_frame", - "filename_prefix", "width", "height", "fps", "frames_at_fps") + RETURN_TYPES = ("IMAGE", "IMAGE", "IMAGE", "IMAGE", "STRING", "INT", "INT", "FLOAT", "IMAGE", "FLOAT") + RETURN_NAMES = ("Clipped Frames", "Reversed Clipped Frames", "First Frame", "Last Frame", + "Filename Prefix (no extension)", "Width", "Height", "Original FPS", + "Clipped Frames at Target FPS", "Target FPS") FUNCTION = "extract" CATEGORY = "video" - OUTPUT_NODE = False + OUTPUT_NODE = True @classmethod def IS_CHANGED(cls, video, start_frame, end_frame, num_frames, target_fps): @@ -180,6 +240,24 @@ class VideoFrameExtractor: if num_frames <= 0: num_frames = end_frame - start_frame + 1 + # ── Memory safety check ────────────────────────────────────────── + clip_duration = (end_frame - start_frame + 1) / fps if fps > 0 else 0 + custom_num_frames = max(1, round(clip_duration * target_fps)) + peak_bytes = estimate_peak_memory(width, height, num_frames, + custom_num_frames) + limit_bytes = get_memory_limit() + + if peak_bytes > limit_bytes: + raise RuntimeError( + f"Refusing to run: estimated peak memory " + f"{format_bytes(peak_bytes)} exceeds the safe limit of " + f"{format_bytes(limit_bytes)}.\n\n" + f"Clip: {num_frames} frames @ {width}x{height} " + f"(+{custom_num_frames} frames at {target_fps:.1f} fps).\n\n" + f"Reduce the loop region, shrink the video resolution, or " + f"lower target_fps. A shorter clip uses proportionally less RAM." + ) + # Extract the frame batch frames = extract_frames(video_path, start_frame, end_frame, num_frames) frames_reversed = torch.flip(frames, dims=[0]) @@ -192,12 +270,11 @@ class VideoFrameExtractor: filename_prefix = os.path.splitext(os.path.basename(video_path))[0] # Custom FPS batch: resample the loop region to target_fps - clip_duration = (end_frame - start_frame + 1) / fps if fps > 0 else 0 - custom_num_frames = max(1, round(clip_duration * target_fps)) + # (custom_num_frames was computed above for the memory check) frames_at_fps = extract_frames(video_path, start_frame, end_frame, custom_num_frames) return (frames, frames_reversed, first_frame, last_frame, - filename_prefix, width, height, fps, frames_at_fps) + filename_prefix, width, height, fps, frames_at_fps, float(target_fps)) # ─── API routes ────────────────────────────────────────────────────────────── diff --git a/web/js/video_frame_extractor.js b/web/js/video_frame_extractor.js index fb0e392..909d85e 100644 --- a/web/js/video_frame_extractor.js +++ b/web/js/video_frame_extractor.js @@ -18,64 +18,65 @@ try { } // ─── Layout ────────────────────────────────────────────────────────────────── -const TRANSPORT_H = 44; -const RULER_H = 18; -const STRIP_H = 48; -const ACTIVITY_H = 28; -const LOOP_H = 24; -const INFO_H = 18; -const PAD = 10; -const HANDLE_W = 10; -const COLLAPSE_H = 20; // preview collapse toggle height -const RESIZE_H = 14; // preview resize handle — tall enough to be obvious +const TRANSPORT_H = 44; +const RULER_H = 18; +const STRIP_H_LANDSCAPE = 48; +const STRIP_H_MAX = 140; +const ACTIVITY_H = 28; +const LOOP_H = 24; +const INFO_H = 40; +const PAD = 10; +const HANDLE_W = 10; +const COLLAPSE_H = 20; // preview collapse toggle height +const RESIZE_H = 14; // preview resize handle — tall enough to be obvious const PREVIEW_H_DEFAULT = 160; -const PREVIEW_H_MIN = 80; -const PREVIEW_H_MAX = 400; +const PREVIEW_H_MIN = 80; +const PREVIEW_H_MAX = 400; // WIDGET_H is dynamic — computed in widget.computeSize() -const THUMB_COUNT = 32; +const THUMB_COUNT = 32; // ─── Colours ───────────────────────────────────────────────────────────────── const C = { - surface: "#131318", - panel: "#0f0f14", - border: "#222228", - ruler: "#1a1a22", - rulerText: "#555568", - rulerMajor: "#44445a", - strip: "#0d0d12", - stripBorder: "#1e1e28", - activity: "#22d3ee", - activityFill: "rgba(34,211,238,0.18)", + surface: "#131318", + panel: "#0f0f14", + border: "#222228", + ruler: "#1a1a22", + rulerText: "#555568", + rulerMajor: "#44445a", + strip: "#0d0d12", + stripBorder: "#1e1e28", + activity: "#22d3ee", + activityFill: "rgba(34,211,238,0.18)", // Loop region - loopFill: "rgba(99,102,241,0.22)", - loopFillHov: "rgba(99,102,241,0.42)", // brighter when hovered - loopBorder: "#6366f1", + loopFill: "rgba(99,102,241,0.22)", + loopFillHov: "rgba(99,102,241,0.42)", // brighter when hovered + loopBorder: "#6366f1", loopBorderHov: "#a5b4fc", - loopHandle: "#818cf8", + loopHandle: "#818cf8", loopHandleHov: "#e0e7ff", - loopGlow: "rgba(99,102,241,0.55)", + loopGlow: "rgba(99,102,241,0.55)", // Transport - btnBg: "#1e1e2a", - btnBgHov: "#2d2d42", - btnBgActive: "#3730a3", - btnBorder: "#3f3f5a", - btnBorderHov: "#818cf8", - playIcon: "#a5f3fc", - pauseIcon: "#fde68a", - stopIcon: "#fca5a5", + btnBg: "#1e1e2a", + btnBgHov: "#2d2d42", + btnBgActive: "#3730a3", + btnBorder: "#3f3f5a", + btnBorderHov: "#818cf8", + playIcon: "#a5f3fc", + pauseIcon: "#fde68a", + stopIcon: "#fca5a5", // Playback position line (replaces playhead — simpler, no dragging) - posLine: "rgba(99,102,241,0.7)", - stepBg: "#0f2744", - stepBgHov: "#1a3a5c", - stepBorder: "#1e4976", + posLine: "rgba(99,102,241,0.7)", + stepBg: "#0f2744", + stepBgHov: "#1a3a5c", + stepBorder: "#1e4976", stepBorderHov: "#3b82f6", - stepIcon: "#93c5fd", + stepIcon: "#93c5fd", - text: "#e2e2ea", - textDim: "#55556a", + text: "#e2e2ea", + textDim: "#55556a", }; // ─── Util ───────────────────────────────────────────────────────────────────── @@ -86,34 +87,46 @@ function roundRect(ctx, x, y, w, h, r = 6) { r = Math.min(r, w / 2, h / 2); ctx.beginPath(); ctx.moveTo(x + r, y); - ctx.arcTo(x + w, y, x + w, y + h, r); - ctx.arcTo(x + w, y + h, x, y + h, r); - ctx.arcTo(x, y + h, x, y, r); - ctx.arcTo(x, y, x + w, y, r); + ctx.arcTo(x + w, y, x + w, y + h, r); + ctx.arcTo(x + w, y + h, x, y + h, r); + ctx.arcTo(x, y + h, x, y, r); + ctx.arcTo(x, y, x + w, y, r); ctx.closePath(); } // ─── Widget factory ─────────────────────────────────────────────────────────── function createVideoTimelineWidget(node, videoWidgetName) { - // Preview size / collapse state - let previewH = 160; - let collapsed = false; // preview section - let resizeDrag = false; + let previewH = 160; + let collapsed = false; // preview section + let resizeDrag = false; let resizeDragStartY = 0; let resizeDragStartH = 0; // Video state - let videoEl = null; + let videoEl = null; + let videoReady = false; + let videoLoadProgress = 0; + let videoError = null; // string message or null + let videoStalled = false; + let loadStartTime = 0; + let loadTimeoutId = null; + let loadStage = "idle"; // idle | fetching-info | loading-metadata | buffering | ready | error + let lastStageEvent = ""; + const LOAD_TIMEOUT_MS = 30000; let totalFrames = 0; - let fps = 30; - let vidW = 0, vidH = 0; + let fps = 30; + let vidW = 0, + vidH = 0; + let bytesPerFrameF32 = 0; + let memoryLimitBytes = 8 * 1024 * 1024 * 1024; // Loop region (frame units) - let loopIn = 0; + let loopIn = 0; let loopOut = 0; // View window (frame units) - let viewStart = 0, viewEnd = 0; + let viewStart = 0, + viewEnd = 0; const MIN_VIEW = 10; // Filmstrip / activity @@ -122,20 +135,31 @@ function createVideoTimelineWidget(node, videoWidgetName) { let activityData = []; // Interaction - let drag = null; + let drag = null; let layout = {}; + // Dynamic filmstrip height: tall for portrait so thumbs aren't squished + function getStripH() { + if (!vidW || !vidH) return STRIP_H_LANDSCAPE; + const aspect = vidW / vidH; + if (aspect >= 1) return STRIP_H_LANDSCAPE; + return Math.min(STRIP_H_MAX, Math.round(STRIP_H_LANDSCAPE / aspect)); + } + // Transport playback state // "stopped" | "playing" | "paused" - let playState = "stopped"; - let rafId = null; - let currentFrame = 0; // tracks where we are during playback (for pos line) + let playState = "stopped"; + let rafId = null; + let currentFrame = 0; // tracks where we are during playback (for pos line) // ─── Cursor ────────────────────────────────────────────────────────────── - let _canvasEl = null, _lastCursor = ""; + let _canvasEl = null, + _lastCursor = ""; function getLGCanvas() { if (_canvasEl?.isConnected) return _canvasEl; - _canvasEl = document.querySelector("canvas.litegraph") ?? document.querySelector("canvas"); + _canvasEl = + document.querySelector("canvas.litegraph") ?? + document.querySelector("canvas"); return _canvasEl; } function setCursor(c) { @@ -147,14 +171,21 @@ function createVideoTimelineWidget(node, videoWidgetName) { function cursorFor(type, dragging) { if (type === "loopIn" || type === "loopOut") return "ew-resize"; if (type === "loopBody") return dragging ? "grabbing" : "grab"; - if (type === "btnPlayPause" || type === "btnPrev" || type === "btnNext") return "pointer"; - if (type === "resize") return "ns-resize"; + if ( + type === "btnPlayPause" || + type === "btnPrev" || + type === "btnNext" || + type === "btnJumpBack" || + type === "btnJumpFwd" + ) + return "pointer"; + if (type === "resize") return "ns-resize"; if (type === "collapse") return "pointer"; return "default"; } // ─── Helpers ───────────────────────────────────────────────────────────── - const getWidget = name => node.widgets?.find(w => w.name === name); + const getWidget = (name) => node.widgets?.find((w) => w.name === name); function syncWidgets() { const sf = getWidget("start_frame"); @@ -170,7 +201,8 @@ function createVideoTimelineWidget(node, videoWidgetName) { } function frameToX(frame, tX, tW) { - const vS = viewStart, vE = effectiveViewEnd(); + const vS = viewStart, + vE = effectiveViewEnd(); if (vE <= vS) return tX; return tX + ((frame - vS) / (vE - vS)) * tW; } @@ -193,7 +225,7 @@ function createVideoTimelineWidget(node, videoWidgetName) { } playState = "playing"; videoEl.muted = false; - videoEl.play().catch(() => {}); // tell the browser element to actually play + videoEl.play().catch(() => {}); // tell the browser element to actually play scheduleFrame(); node.setDirtyCanvas(true, false); } @@ -203,7 +235,7 @@ function createVideoTimelineWidget(node, videoWidgetName) { playState = "paused"; cancelAnimationFrame(rafId); rafId = null; - if (videoEl) videoEl.pause(); // stop the browser video element itself + if (videoEl) videoEl.pause(); // stop the browser video element itself node.setDirtyCanvas(true, false); } @@ -222,7 +254,9 @@ function createVideoTimelineWidget(node, videoWidgetName) { } // Convenience predicate used by mouse handler (avoids closure-over-playState issues) - function isPlaying() { return playState === "playing"; } + function isPlaying() { + return playState === "playing"; + } // Shift the entire loop window by one frame (keeps span constant). // Pauses playback, seeks preview to new loopIn, syncs widgets. @@ -230,10 +264,10 @@ function createVideoTimelineWidget(node, videoWidgetName) { if (playState === "playing") pause(); const span = loopOut - loopIn; // Clamp so neither end goes out of bounds - const newIn = clamp(loopIn + delta, 0, Math.max(0, totalFrames - 1 - span)); + const newIn = clamp(loopIn + delta, 0, Math.max(0, totalFrames - 1 - span)); const newOut = newIn + span; - loopIn = newIn; - loopOut = newOut; + loopIn = newIn; + loopOut = newOut; currentFrame = loopIn; if (videoEl && fps > 0) videoEl.currentTime = loopIn / fps; syncWidgets(); @@ -242,10 +276,9 @@ function createVideoTimelineWidget(node, videoWidgetName) { function scheduleFrame() { rafId = requestAnimationFrame(() => { - if (playState !== "playing" || !videoEl) return; - // Derive currentFrame from actual video time + // Bail out if play was cancelled, video was swapped, or it isn't ready + if (playState !== "playing" || !videoEl || !videoReady) return; currentFrame = Math.round((videoEl.currentTime || 0) * fps); - // Loop back when we reach loopOut if (currentFrame >= loopOut) { currentFrame = loopIn; videoEl.currentTime = loopIn / fps; @@ -253,36 +286,70 @@ function createVideoTimelineWidget(node, videoWidgetName) { node.setDirtyCanvas(true, false); scheduleFrame(); }); - // Also actually advance the video - if (videoEl.paused) videoEl.play().catch(() => {}); + // Only nudge play if the current src matches what we intend to play + if (videoEl && videoReady && videoEl.paused) videoEl.play().catch(() => {}); } // ─── Fetch ─────────────────────────────────────────────────────────────── async function fetchVideoInfo(videoName) { if (!videoName) return; + // Halt any playback/looping on the previous video immediately — before + // we await anything — so the rAF loop can't race with the new src load. + cancelAnimationFrame(rafId); + rafId = null; + playState = "paused"; + if (videoEl) { + try { videoEl.pause(); } catch (_) {} + // Fully detach the old source so it stops buffering/decoding. + try { + videoEl.removeAttribute("src"); + videoEl.load(); + } catch (_) {} + videoEl.dataset.src = ""; + } + + // Clear UI state so the old frame vanishes before metadata arrives + videoReady = false; + videoLoadProgress = 0; + videoError = null; + videoStalled = false; + loadStage = "fetching-info"; + lastStageEvent = `Requesting info for ${videoName}`; + loadStartTime = performance.now(); + clearTimeout(loadTimeoutId); + node.setDirtyCanvas(true, false); try { - const r = await fetch(`/video_frame_extractor/info?video=${encodeURIComponent(videoName)}`); + const r = await fetch( + `/video_frame_extractor/info?video=${encodeURIComponent(videoName)}`, + ); const info = await r.json(); if (info.error) return; - totalFrames = info.total_frames || 1; - fps = info.fps || 30; - vidW = info.width || 0; - vidH = info.height || 0; - loopIn = 0; - loopOut = totalFrames - 1; + totalFrames = info.total_frames || 1; + fps = info.fps || 30; + vidW = info.width || 0; + vidH = info.height || 0; + bytesPerFrameF32 = info.bytes_per_frame_f32 || vidW * vidH * 3 * 4; + memoryLimitBytes = info.memory_limit_bytes || 8 * 1024 * 1024 * 1024; + loopIn = 0; + loopOut = totalFrames - 1; currentFrame = 0; - viewStart = 0; - viewEnd = totalFrames; + viewStart = 0; + viewEnd = totalFrames; activityData = []; thumbsQueued = false; thumbCache.clear(); + videoLoadProgress = 0; stop(); syncWidgets(); loadVideoEl(videoName); queueThumbs(videoName); node.setDirtyCanvas(true, true); } catch (e) { + videoError = `Info fetch failed: ${e.message || e}`; + loadStage = "error"; + lastStageEvent = "Failed to fetch video info"; + node.setDirtyCanvas(true, false); console.warn("[VideoFrameExtractor] fetchVideoInfo:", e); } } @@ -290,16 +357,127 @@ function createVideoTimelineWidget(node, videoWidgetName) { function loadVideoEl(videoName) { const url = `/view?filename=${encodeURIComponent(videoName)}&type=input`; if (!videoEl) { - videoEl = document.createElement("video"); - videoEl.muted = true; - videoEl.preload = "auto"; + videoEl = document.createElement("video"); + videoEl.muted = true; + videoEl.preload = "auto"; videoEl.playsInline = true; videoEl.crossOrigin = "anonymous"; + const errMsg = (code) => { + switch (code) { + case 1: return "Load aborted"; + case 2: return "Network error"; + case 3: return "Video decode error"; + case 4: return "Format not supported"; + default: return "Unknown video error"; + } + }; + videoEl.addEventListener("loadstart", () => { + loadStage = "loading-metadata"; + lastStageEvent = "Request sent, waiting for response"; + node.setDirtyCanvas(true, false); + }); + videoEl.addEventListener("loadedmetadata", () => { + loadStage = "buffering"; + lastStageEvent = `Metadata parsed (${videoEl.videoWidth}×${videoEl.videoHeight})`; + node.setDirtyCanvas(true, false); + }); + videoEl.addEventListener("progress", () => { + if (videoEl.buffered.length > 0 && videoEl.duration > 0) { + videoLoadProgress = videoEl.buffered.end(videoEl.buffered.length - 1) / videoEl.duration; + } + if (loadStage !== "ready") loadStage = "buffering"; + lastStageEvent = "Downloading video data"; + videoStalled = false; + node.setDirtyCanvas(true, false); + }); + videoEl.addEventListener("canplay", () => { + videoReady = true; + videoLoadProgress = 1; + videoStalled = false; + videoError = null; + loadStage = "ready"; + lastStageEvent = `Ready in ${((performance.now() - loadStartTime) / 1000).toFixed(1)}s`; + clearTimeout(loadTimeoutId); + node.setDirtyCanvas(true, false); + }); + videoEl.addEventListener("loadeddata", () => { + videoReady = true; + videoStalled = false; + if (loadStage !== "ready") { + loadStage = "ready"; + lastStageEvent = "First frame decoded"; + } + node.setDirtyCanvas(true, false); + }); + videoEl.addEventListener("error", () => { + const code = videoEl.error?.code; + videoError = errMsg(code); + videoReady = false; + loadStage = "error"; + lastStageEvent = `Error code ${code ?? "?"}`; + clearTimeout(loadTimeoutId); + console.warn("[VideoFrameExtractor] video load error", videoEl.error); + node.setDirtyCanvas(true, false); + }); + videoEl.addEventListener("stalled", () => { + if (!videoReady) { + videoStalled = true; + lastStageEvent = "Network stalled"; + node.setDirtyCanvas(true, false); + } + }); + videoEl.addEventListener("suspend", () => { + if (!videoReady && videoLoadProgress < 0.99) { + videoStalled = true; + lastStageEvent = "Browser suspended download"; + node.setDirtyCanvas(true, false); + } + }); + videoEl.addEventListener("waiting", () => { + if (playState === "playing") { + videoStalled = true; + lastStageEvent = "Buffer underrun during playback"; + node.setDirtyCanvas(true, false); + } + }); + videoEl.addEventListener("playing", () => { + videoStalled = false; + lastStageEvent = "Playing"; + node.setDirtyCanvas(true, false); + }); } if (videoEl.dataset.src !== url) { - videoEl.src = url; + // Immediately mark not ready so the old frame is cleared + videoReady = false; + videoLoadProgress = 0; + videoError = null; + videoStalled = false; + loadStartTime = performance.now(); + + // Release old video memory + videoEl.pause(); + videoEl.removeAttribute("src"); + videoEl.load(); + + videoEl.src = url; videoEl.dataset.src = url; videoEl.load(); + + // Watchdog — if loading never completes, flag it as a timeout + clearTimeout(loadTimeoutId); + loadTimeoutId = setTimeout(() => { + if (!videoReady) { + videoError = "Load timed out — video may be corrupt or unreadable"; + node.setDirtyCanvas(true, false); + } + }, LOAD_TIMEOUT_MS); + + node.setDirtyCanvas(true, false); + } else if (videoEl.readyState >= 2) { + // Same URL, already loaded — just mark ready again + videoReady = true; + videoLoadProgress = 1; + node.setDirtyCanvas(true, false); } } @@ -307,35 +485,46 @@ function createVideoTimelineWidget(node, videoWidgetName) { if (thumbsQueued || !totalFrames) return; thumbsQueued = true; const indices = Array.from({ length: THUMB_COUNT }, (_, i) => - Math.round(i * (totalFrames - 1) / (THUMB_COUNT - 1)) + Math.round((i * (totalFrames - 1)) / (THUMB_COUNT - 1)), ); const BATCH = 4; for (let b = 0; b < indices.length; b += BATCH) { - await Promise.all(indices.slice(b, b + BATCH).map(async (fi, bi) => { - try { - const url = `/video_frame_extractor/thumbnail?video=${encodeURIComponent(videoName)}&frame=${fi}`; - const img = new Image(); - await new Promise((res, rej) => { img.onload = res; img.onerror = rej; img.src = url; }); - thumbCache.set(fi, img); - activityData[b + bi] = computeActivity(img); - node.setDirtyCanvas(true, false); - } catch (_) { activityData[b + bi] = 0; } - })); + await Promise.all( + indices.slice(b, b + BATCH).map(async (fi, bi) => { + try { + const url = `/video_frame_extractor/thumbnail?video=${encodeURIComponent(videoName)}&frame=${fi}`; + const img = new Image(); + await new Promise((res, rej) => { + img.onload = res; + img.onerror = rej; + img.src = url; + }); + thumbCache.set(fi, img); + activityData[b + bi] = computeActivity(img); + node.setDirtyCanvas(true, false); + } catch (_) { + activityData[b + bi] = 0; + } + }), + ); } } function computeActivity(img) { try { const oc = document.createElement("canvas"); - oc.width = 16; oc.height = 9; + oc.width = 16; + oc.height = 9; const ox = oc.getContext("2d"); ox.drawImage(img, 0, 0, 16, 9); const px = ox.getImageData(0, 0, 16, 9).data; let sum = 0; for (let i = 0; i < px.length; i += 4) - sum += (px[i] * 0.299 + px[i+1] * 0.587 + px[i+2] * 0.114) / 255; + sum += (px[i] * 0.299 + px[i + 1] * 0.587 + px[i + 2] * 0.114) / 255; return sum / (16 * 9); - } catch (_) { return 0.5; } + } catch (_) { + return 0.5; + } } // ─── Draw helpers ───────────────────────────────────────────────────────── @@ -348,42 +537,134 @@ function createVideoTimelineWidget(node, videoWidgetName) { ctx.lineWidth = 1; ctx.stroke(); - if (videoEl && videoEl.readyState >= 2) { + if (videoEl && videoReady) { ctx.save(); roundRect(ctx, x, y, w, h, 5); ctx.clip(); const vAR = (videoEl.videoWidth || 16) / (videoEl.videoHeight || 9); const pAR = w / h; let dw, dh, dx, dy; - if (vAR > pAR) { dw = w; dh = w / vAR; dx = x; dy = y + (h - dh) / 2; } - else { dh = h; dw = h * vAR; dy = y; dx = x + (w - dw) / 2; } + if (vAR > pAR) { + dw = w; + dh = w / vAR; + dx = x; + dy = y + (h - dh) / 2; + } else { + dh = h; + dw = h * vAR; + dy = y; + dx = x + (w - dw) / 2; + } ctx.drawImage(videoEl, dx, dy, dw, dh); ctx.restore(); + } else if (videoError) { + const cx = x + w / 2; + const cy = y + h / 2; + ctx.fillStyle = "#fca5a5"; + ctx.font = "bold 13px 'Courier New', monospace"; + ctx.textAlign = "center"; + ctx.fillText("⚠ Video error", cx, cy - 22); + ctx.fillStyle = "#fecaca"; + ctx.font = "11px 'Courier New', monospace"; + ctx.fillText(videoError, cx, cy - 4); + if (lastStageEvent) { + ctx.fillStyle = "#9ca3af"; + ctx.font = "10px 'Courier New', monospace"; + ctx.fillText(lastStageEvent, cx, cy + 14); + } + } else if (totalFrames > 0 || loadStage === "fetching-info") { + // Video is loading — show detailed status + const pct = Math.round(videoLoadProgress * 100); + const elapsed = loadStartTime > 0 ? (performance.now() - loadStartTime) / 1000 : 0; + + // Headline + let headline; + if (videoStalled) headline = pct > 0 ? `Stalled at ${pct}%` : "Stalled"; + else if (loadStage === "fetching-info") headline = "Fetching video info…"; + else if (loadStage === "loading-metadata") headline = "Loading metadata…"; + else if (loadStage === "buffering") headline = pct > 0 ? `Buffering ${pct}%` : "Buffering…"; + else headline = "Loading video…"; + + const cx = x + w / 2; + const cy = y + h / 2; + + ctx.fillStyle = videoStalled ? "#fbbf24" : "#c7d2fe"; + ctx.font = "bold 13px 'Courier New', monospace"; + ctx.textAlign = "center"; + ctx.fillText(headline, cx, cy - 22); + + // Progress bar + const pbW = Math.min(w - 40, 320); + const pbX = x + (w - pbW) / 2; + const pbY = cy - 8; + ctx.fillStyle = "#1a1a26"; + roundRect(ctx, pbX, pbY, pbW, 8, 4); + ctx.fill(); + if (videoLoadProgress > 0) { + ctx.fillStyle = videoStalled ? "#f59e0b" : "#6366f1"; + roundRect(ctx, pbX, pbY, Math.max(8, pbW * videoLoadProgress), 8, 4); + ctx.fill(); + } else if (loadStage === "fetching-info" || loadStage === "loading-metadata") { + // Indeterminate shimmer — a sliding block + const t = (performance.now() / 1000) % 1.5; + const bw = pbW * 0.25; + const bx = pbX + (pbW - bw) * (t / 1.5); + ctx.fillStyle = "#4f46e5"; + roundRect(ctx, bx, pbY, bw, 8, 4); + ctx.fill(); + } + ctx.strokeStyle = "#2a2a3e"; + ctx.lineWidth = 0.5; + roundRect(ctx, pbX, pbY, pbW, 8, 4); + ctx.stroke(); + + // Detail line + ctx.fillStyle = "#8b8ba0"; + ctx.font = "10px 'Courier New', monospace"; + const detail = lastStageEvent || ""; + ctx.fillText(detail, cx, cy + 14); + + // Stats line: elapsed + buffered seconds + readyState + const parts = []; + if (elapsed > 0) parts.push(`${elapsed.toFixed(1)}s elapsed`); + if (videoEl && videoEl.buffered?.length > 0 && videoEl.duration > 0) { + const bufEnd = videoEl.buffered.end(videoEl.buffered.length - 1); + parts.push(`${bufEnd.toFixed(1)}s / ${videoEl.duration.toFixed(1)}s buffered`); + } + if (videoEl) { + const rsLabels = ["nothing", "metadata", "current", "future", "enough"]; + parts.push(`readyState=${rsLabels[videoEl.readyState] ?? videoEl.readyState}`); + } + if (parts.length) { + ctx.fillStyle = "#55556a"; + ctx.font = "9px 'Courier New', monospace"; + ctx.fillText(parts.join(" • "), cx, cy + 28); + } } else { ctx.fillStyle = C.textDim; - ctx.font = "12px 'Courier New', monospace"; + ctx.font = "12px 'Courier New', monospace"; ctx.textAlign = "center"; - ctx.fillText(totalFrames ? "Loading preview…" : "⬆ Upload a video above", x + w / 2, y + h / 2 + 4); + ctx.fillText("⬆ Upload a video above", x + w / 2, y + h / 2 + 4); } // Timecode overlay if (totalFrames > 0) { - const f = currentFrame; + const f = currentFrame; const secs = f / fps; - const mm = String(Math.floor(secs / 60)).padStart(2, "0"); - const ss = String(Math.floor(secs % 60)).padStart(2, "0"); - const ff = String(f % Math.max(1, Math.round(fps))).padStart(2, "0"); + const mm = String(Math.floor(secs / 60)).padStart(2, "0"); + const ss = String(Math.floor(secs % 60)).padStart(2, "0"); + const ff = String(f % Math.max(1, Math.round(fps))).padStart(2, "0"); ctx.fillStyle = "rgba(0,0,0,0.72)"; ctx.fillRect(x + 4, y + h - 22, 160, 18); ctx.fillStyle = "#a5b4fc"; - ctx.font = "12px 'Courier New', monospace"; + ctx.font = "12px 'Courier New', monospace"; ctx.textAlign = "left"; ctx.fillText(`${mm}:${ss}:${ff} fr ${f}`, x + 8, y + h - 8); if (vidW && vidH) { ctx.fillStyle = "rgba(0,0,0,0.55)"; ctx.fillRect(x + w - 80, y + 4, 76, 16); ctx.fillStyle = C.textDim; - ctx.font = "10px 'Courier New', monospace"; + ctx.font = "10px 'Courier New', monospace"; ctx.textAlign = "right"; ctx.fillText(`${vidW}×${vidH}`, x + w - 6, y + 14); } @@ -392,109 +673,171 @@ function createVideoTimelineWidget(node, videoWidgetName) { // ── Transport bar ──────────────────────────────────────────────────────── // Layout (left → right): - // [◀◀ PREV] [▶ PLAY / ⏸ PAUSE toggle] [■ STOP] [NEXT ▶▶] + // [⏮ JUMP BACK] [◀ PREV] [▶ PLAY / ⏸ PAUSE] [NEXT ▶] [JUMP FWD ⏭] // - // Returns a map of id → {x,y,w,h} in node-local coords for hit-testing. - // All rects store the ABSOLUTE node-local y so mouse() needs no math. + // JUMP buttons shift the loop region by its own span + 1, so each press + // moves to the "next set" of frames. Disabled when no room remains. function drawTransport(ctx, x, y, w, h) { ctx.fillStyle = "#0e0e16"; ctx.fillRect(x, y, w, h); ctx.strokeStyle = C.border; - ctx.lineWidth = 0.5; + ctx.lineWidth = 0.5; ctx.strokeRect(x, y, w, h); if (!totalFrames) return {}; - const BTN_H = 32; - const STEP_W = 44; // frame-step buttons - const TOGGLE_W = 100; // play/pause toggle — most prominent button - const GAP = 10; - const totalW = STEP_W + GAP + TOGGLE_W + GAP + STEP_W; - const bx0 = x + Math.round((w - totalW) / 2); - const by = y + Math.round((h - BTN_H) / 2); + const BTN_H = 32; + const STEP_W = 44; + const JUMP_W = 52; + const TOGGLE_W = 100; + const GAP = 8; + const totalW = JUMP_W + GAP + STEP_W + GAP + TOGGLE_W + GAP + STEP_W + GAP + JUMP_W; + const bx0 = x + Math.round((w - totalW) / 2); + const by = y + Math.round((h - BTN_H) / 2); + + const span = loopOut - loopIn; + const jumpDelta = span + 1; + const canJumpBack = loopIn - jumpDelta >= 0; + const canJumpFwd = loopOut + jumpDelta <= totalFrames - 1; - // Button descriptors — positions computed left→right const isPlaying = playState === "playing"; + let bx = bx0; const buttons = [ { - id: "btnPrev", label: "◀", sublabel: "PREV", - col: C.stepIcon, isStep: true, - bx: bx0, bw: STEP_W, - }, - { - id: "btnPlayPause", - label: isPlaying ? "⏸ PAUSE" : "▶ PLAY", - col: isPlaying ? C.pauseIcon : C.playIcon, - bx: bx0 + STEP_W + GAP, bw: TOGGLE_W, - isActive: isPlaying, - }, - { - id: "btnNext", label: "▶", sublabel: "NEXT", - col: C.stepIcon, isStep: true, - bx: bx0 + STEP_W + GAP + TOGGLE_W + GAP, bw: STEP_W, + id: "btnJumpBack", + label: "⏮", + sublabel: "JUMP", + col: C.stepIcon, + isStep: true, + bx, + bw: JUMP_W, + disabled: !canJumpBack, }, ]; + bx += JUMP_W + GAP; + buttons.push({ + id: "btnPrev", + label: "◀", + sublabel: "PREV", + col: C.stepIcon, + isStep: true, + bx, + bw: STEP_W, + }); + bx += STEP_W + GAP; + buttons.push({ + id: "btnPlayPause", + label: isPlaying ? "⏸ PAUSE" : "▶ PLAY", + col: isPlaying ? C.pauseIcon : C.playIcon, + bx, + bw: TOGGLE_W, + isActive: isPlaying, + }); + bx += TOGGLE_W + GAP; + buttons.push({ + id: "btnNext", + label: "▶", + sublabel: "NEXT", + col: C.stepIcon, + isStep: true, + bx, + bw: STEP_W, + }); + bx += STEP_W + GAP; + buttons.push({ + id: "btnJumpFwd", + label: "⏭", + sublabel: "JUMP", + col: C.stepIcon, + isStep: true, + bx, + bw: JUMP_W, + disabled: !canJumpFwd, + }); const hitRects = {}; for (const btn of buttons) { - const isActive = !!btn.isActive; - const isHovered = widget._hover === btn.id; + const isActive = !!btn.isActive; + const disabled = !!btn.disabled; + const isHovered = !disabled && widget._hover === btn.id; + + ctx.save(); + if (disabled) ctx.globalAlpha = 0.35; const bgCol = btn.isStep - ? (isHovered ? C.stepBgHov : C.stepBg) - : (isActive ? C.btnBgActive : isHovered ? C.btnBgHov : C.btnBg); + ? isHovered + ? C.stepBgHov + : C.stepBg + : isActive + ? C.btnBgActive + : isHovered + ? C.btnBgHov + : C.btnBg; const bdCol = btn.isStep - ? (isHovered ? C.stepBorderHov : C.stepBorder) - : (isActive ? "#818cf8" : isHovered ? C.btnBorderHov : C.btnBorder); + ? isHovered + ? C.stepBorderHov + : C.stepBorder + : isActive + ? "#818cf8" + : isHovered + ? C.btnBorderHov + : C.btnBorder; - // Background ctx.fillStyle = bgCol; roundRect(ctx, btn.bx, by, btn.bw, BTN_H, 6); ctx.fill(); ctx.strokeStyle = bdCol; - ctx.lineWidth = isActive ? 1.5 : 1; + ctx.lineWidth = isActive ? 1.5 : 1; ctx.stroke(); - // Glow ring on active play/pause only if (isActive && !btn.isStep) { ctx.strokeStyle = "rgba(99,102,241,0.35)"; - ctx.lineWidth = 3; + ctx.lineWidth = 3; roundRect(ctx, btn.bx - 1, by - 1, btn.bw + 2, BTN_H + 2, 7); ctx.stroke(); } - // Label — step buttons: large arrow + small text label; others: single line ctx.textAlign = "center"; if (btn.isStep) { ctx.fillStyle = isHovered ? "#bfdbfe" : btn.col; - ctx.font = "14px 'Courier New', monospace"; + ctx.font = "14px 'Courier New', monospace"; ctx.fillText(btn.label, btn.bx + btn.bw / 2, by + BTN_H / 2 - 1); ctx.fillStyle = isHovered ? "#93c5fd" : "#3b6a96"; - ctx.font = "8px 'Courier New', monospace"; + ctx.font = "8px 'Courier New', monospace"; ctx.fillText(btn.sublabel, btn.bx + btn.bw / 2, by + BTN_H / 2 + 10); } else { - ctx.fillStyle = isActive ? "#ffffff" : (isHovered ? "#e2e2ea" : btn.col); - ctx.font = "11px 'Courier New', monospace"; + ctx.fillStyle = isActive ? "#ffffff" : isHovered ? "#e2e2ea" : btn.col; + ctx.font = "11px 'Courier New', monospace"; ctx.fillText(btn.label, btn.bx + btn.bw / 2, by + BTN_H / 2 + 4); } - // Store absolute rect for hit-testing - hitRects[btn.id] = { x: btn.bx, y: by, w: btn.bw, h: BTN_H }; + ctx.restore(); + + // Store hit rect only if enabled (disabled buttons don't respond) + hitRects[btn.id] = { + x: btn.bx, + y: by, + w: btn.bw, + h: BTN_H, + disabled, + }; } // Progress bar — slim strip at bottom of transport row if (totalFrames > 0) { - const pbY = y + h - 5; - const pbW = w - 20; - const pbX = x + 10; + const pbY = y + h - 5; + const pbW = w - 20; + const pbX = x + 10; const span = loopOut - loopIn; - const pct = span > 0 ? clamp(currentFrame - loopIn, 0, span) / span : 0; + const pct = span > 0 ? clamp(currentFrame - loopIn, 0, span) / span : 0; ctx.fillStyle = "#1a1a26"; - roundRect(ctx, pbX, pbY, pbW, 3, 2); ctx.fill(); + roundRect(ctx, pbX, pbY, pbW, 3, 2); + ctx.fill(); if (pct > 0) { ctx.fillStyle = "#6366f1"; - roundRect(ctx, pbX, pbY, Math.round(pbW * pct), 3, 2); ctx.fill(); + roundRect(ctx, pbX, pbY, Math.round(pbW * pct), 3, 2); + ctx.fill(); } } @@ -509,28 +852,47 @@ function createVideoTimelineWidget(node, videoWidgetName) { ctx.strokeRect(x, y, w, h); if (!totalFrames) return; - const vS = viewStart, vE = effectiveViewEnd(); + const vS = viewStart, + vE = effectiveViewEnd(); const vis = vE - vS; const raw = 40 * (vis / w); - const nice = [1,2,5,10,15,30,60,120,300,600, - Math.round(fps), Math.round(fps*5), Math.round(fps*10), Math.round(fps*30)]; - const tickInterval = nice.find(n => n >= raw) || nice[nice.length - 1]; - const firstTick = Math.ceil(vS / tickInterval) * tickInterval; + const nice = [ + 1, + 2, + 5, + 10, + 15, + 30, + 60, + 120, + 300, + 600, + Math.round(fps), + Math.round(fps * 5), + Math.round(fps * 10), + Math.round(fps * 30), + ]; + const tickInterval = nice.find((n) => n >= raw) || nice[nice.length - 1]; + const firstTick = Math.ceil(vS / tickInterval) * tickInterval; - ctx.font = "9px 'Courier New', monospace"; + ctx.font = "9px 'Courier New', monospace"; ctx.textAlign = "center"; for (let f = firstTick; f <= vE; f += tickInterval) { - const tx = frameToX(f, x, w); - const isSec = fps > 0 && (f % Math.round(fps)) === 0; + const tx = frameToX(f, x, w); + const isSec = fps > 0 && f % Math.round(fps) === 0; ctx.strokeStyle = isSec ? C.rulerMajor : "#2a2a38"; - ctx.lineWidth = isSec ? 1 : 0.5; - ctx.beginPath(); ctx.moveTo(tx, y); ctx.lineTo(tx, y + (isSec ? h : h * 0.5)); ctx.stroke(); + ctx.lineWidth = isSec ? 1 : 0.5; + ctx.beginPath(); + ctx.moveTo(tx, y); + ctx.lineTo(tx, y + (isSec ? h : h * 0.5)); + ctx.stroke(); if (isSec || tickInterval >= 30) { const secs = f / fps; ctx.fillStyle = C.rulerText; ctx.fillText( - `${String(Math.floor(secs/60)).padStart(2,"0")}:${String(Math.floor(secs%60)).padStart(2,"0")}`, - tx, y + h - 2 + `${String(Math.floor(secs / 60)).padStart(2, "0")}:${String(Math.floor(secs % 60)).padStart(2, "0")}`, + tx, + y + h - 2, ); } } @@ -547,44 +909,58 @@ function createVideoTimelineWidget(node, videoWidgetName) { if (!totalFrames || thumbCache.size === 0) { ctx.fillStyle = C.textDim; - ctx.font = "10px monospace"; + ctx.font = "10px monospace"; ctx.textAlign = "center"; ctx.fillText("Loading filmstrip…", x + w / 2, y + h / 2 + 3); return; } - const vS = viewStart, vE = effectiveViewEnd(); - const thumbW = h * (16 / 9); - const step = Math.max(1, Math.round((vE - vS) / Math.max(1, Math.ceil(w / thumbW)))); - const first = Math.floor(vS / step) * step; + const vS = viewStart, + vE = effectiveViewEnd(); + const aspect = vidW && vidH ? vidW / vidH : 16 / 9; + const thumbW = h * aspect; + const step = Math.max( + 1, + Math.round((vE - vS) / Math.max(1, Math.ceil(w / thumbW))), + ); + const first = Math.floor(vS / step) * step; ctx.save(); - ctx.beginPath(); ctx.rect(x, y, w, h); ctx.clip(); + ctx.beginPath(); + ctx.rect(x, y, w, h); + ctx.clip(); for (let f = first; f <= vE + step; f += step) { const cellX = frameToX(f, x, w); const cellW = Math.max(1, frameToX(f + step, x, w) - cellX); - const img = findNearestThumb(f); - if (img) { try { ctx.drawImage(img, cellX, y, cellW, h); } catch (_) {} } - else { ctx.fillStyle = "#090910"; ctx.fillRect(cellX, y, cellW, h); } - ctx.strokeStyle = C.stripBorder; ctx.lineWidth = 0.5; + const img = findNearestThumb(f); + if (img) { + try { + ctx.drawImage(img, cellX, y, cellW, h); + } catch (_) {} + } else { + ctx.fillStyle = "#090910"; + ctx.fillRect(cellX, y, cellW, h); + } + ctx.strokeStyle = C.stripBorder; + ctx.lineWidth = 0.5; ctx.strokeRect(cellX, y, cellW, h); } // Darken outside loop region - const lx = frameToX(loopIn, x, w); + const lx = frameToX(loopIn, x, w); const rx = frameToX(loopOut, x, w); ctx.fillStyle = "rgba(0,0,0,0.6)"; - ctx.fillRect(x, y, lx - x, h); - ctx.fillRect(rx, y, (x + w) - rx, h); + ctx.fillRect(x, y, lx - x, h); + ctx.fillRect(rx, y, x + w - rx, h); ctx.restore(); // Frame labels const lblEvery = Math.max(1, Math.round((vE - vS) / 8)); - ctx.fillStyle = "rgba(255,255,255,0.5)"; - ctx.font = "9px 'Courier New', monospace"; - ctx.textAlign = "center"; + ctx.fillStyle = "rgba(255,255,255,0.5)"; + ctx.font = "9px 'Courier New', monospace"; + ctx.textAlign = "center"; for (let f = Math.ceil(vS / lblEvery) * lblEvery; f <= vE; f += lblEvery) ctx.fillText(String(f), frameToX(f, x, w), y + h - 3); @@ -597,10 +973,14 @@ function createVideoTimelineWidget(node, videoWidgetName) { function findNearestThumb(target) { if (thumbCache.has(target)) return thumbCache.get(target); - let best = null, bestDist = Infinity; + let best = null, + bestDist = Infinity; for (const [f, img] of thumbCache) { const d = Math.abs(f - target); - if (d < bestDist) { bestDist = d; best = img; } + if (d < bestDist) { + bestDist = d; + best = img; + } } return best; } @@ -610,19 +990,30 @@ function createVideoTimelineWidget(node, videoWidgetName) { ctx.fillRect(x, y, w, h); if (activityData.length < 2) return; ctx.save(); - ctx.beginPath(); ctx.rect(x, y, w, h); ctx.clip(); + ctx.beginPath(); + ctx.rect(x, y, w, h); + ctx.clip(); ctx.beginPath(); for (let i = 0; i < activityData.length; i++) { - const frame = Math.round(i * (totalFrames - 1) / (activityData.length - 1)); + const frame = Math.round( + (i * (totalFrames - 1)) / (activityData.length - 1), + ); const ax = frameToX(frame, x, w); const ay = y + h - activityData[i] * h * 0.88; i === 0 ? ctx.moveTo(ax, ay) : ctx.lineTo(ax, ay); } - ctx.lineTo(x + w, y + h); ctx.lineTo(x, y + h); ctx.closePath(); - ctx.fillStyle = C.activityFill; ctx.fill(); - ctx.strokeStyle = C.activity; ctx.lineWidth = 1; ctx.stroke(); + ctx.lineTo(x + w, y + h); + ctx.lineTo(x, y + h); + ctx.closePath(); + ctx.fillStyle = C.activityFill; + ctx.fill(); + ctx.strokeStyle = C.activity; + ctx.lineWidth = 1; + ctx.stroke(); ctx.restore(); - ctx.fillStyle = C.textDim; ctx.font = "9px monospace"; ctx.textAlign = "left"; + ctx.fillStyle = C.textDim; + ctx.font = "9px monospace"; + ctx.textAlign = "left"; ctx.fillText("BRIGHTNESS", x + 3, y + h - 3); } @@ -633,14 +1024,14 @@ function createVideoTimelineWidget(node, videoWidgetName) { if (!totalFrames) return; - const lx = frameToX(loopIn, x, w); + const lx = frameToX(loopIn, x, w); const rx = frameToX(loopOut, x, w); const regionW = rx - lx; - const hovIn = widget._hover === "loopIn" || drag?.type === "loopIn"; - const hovOut = widget._hover === "loopOut" || drag?.type === "loopOut"; + const hovIn = widget._hover === "loopIn" || drag?.type === "loopIn"; + const hovOut = widget._hover === "loopOut" || drag?.type === "loopOut"; const hovBody = widget._hover === "loopBody" || drag?.type === "loopBody"; - const anyHov = hovIn || hovOut || hovBody; + const anyHov = hovIn || hovOut || hovBody; // ── Body fill ─────────────────────────────────────────────────────── ctx.fillStyle = anyHov ? C.loopFillHov : C.loopFill; @@ -650,14 +1041,14 @@ function createVideoTimelineWidget(node, videoWidgetName) { if (anyHov) { // Bright solid border when hovered ctx.strokeStyle = C.loopBorderHov; - ctx.lineWidth = 1.5; + ctx.lineWidth = 1.5; ctx.strokeRect(lx, y, regionW, h); // Dashed highlight on top edge to signal "this whole area is draggable" if (hovBody) { ctx.setLineDash([4, 3]); ctx.strokeStyle = "rgba(165,180,252,0.7)"; - ctx.lineWidth = 1; + ctx.lineWidth = 1; ctx.beginPath(); ctx.moveTo(lx + HANDLE_W + 2, y + 1); ctx.lineTo(rx - HANDLE_W - 2, y + 1); @@ -666,24 +1057,26 @@ function createVideoTimelineWidget(node, videoWidgetName) { } } else { ctx.strokeStyle = C.loopBorder; - ctx.lineWidth = 1; + ctx.lineWidth = 1; ctx.strokeRect(lx, y, regionW, h); } // ── "Drag me" hint label inside body ──────────────────────────────── if (regionW > 80 && !anyHov) { ctx.fillStyle = "rgba(99,102,241,0.4)"; - ctx.font = "9px 'Courier New', monospace"; + ctx.font = "9px 'Courier New', monospace"; ctx.textAlign = "center"; ctx.fillText("↔ drag", lx + regionW / 2, y + h * 0.68); } // ── Frame count label ──────────────────────────────────────────────── if (regionW > 50) { - ctx.fillStyle = anyHov ? "rgba(224,231,255,0.95)" : "rgba(139,139,255,0.75)"; - ctx.font = "9px 'Courier New', monospace"; + ctx.fillStyle = anyHov + ? "rgba(224,231,255,0.95)" + : "rgba(139,139,255,0.75)"; + ctx.font = "9px 'Courier New', monospace"; ctx.textAlign = "center"; - ctx.fillText(`${loopOut - loopIn + 1} fr`, lx + regionW / 2, y + h * 0.40); + ctx.fillText(`${loopOut - loopIn + 1} fr`, lx + regionW / 2, y + h * 0.4); } // ── Playback position line inside loop ─────────────────────────────── @@ -691,13 +1084,16 @@ function createVideoTimelineWidget(node, videoWidgetName) { const posX = frameToX(currentFrame, x, w); if (posX >= lx && posX <= rx) { ctx.strokeStyle = C.posLine; - ctx.lineWidth = 2; - ctx.beginPath(); ctx.moveTo(posX, y); ctx.lineTo(posX, y + h); ctx.stroke(); + ctx.lineWidth = 2; + ctx.beginPath(); + ctx.moveTo(posX, y); + ctx.lineTo(posX, y + h); + ctx.stroke(); } } // ── Handles ───────────────────────────────────────────────────────── - drawLoopHandle(ctx, lx, y, h, "in", hovIn); + drawLoopHandle(ctx, lx, y, h, "in", hovIn); drawLoopHandle(ctx, rx, y, h, "out", hovOut); } @@ -708,16 +1104,14 @@ function createVideoTimelineWidget(node, videoWidgetName) { if (hovered) { ctx.fillStyle = C.loopGlow; const gw = HANDLE_W + 8; - if (side === "in") - ctx.fillRect(cx - gw, y - 2, gw + 2, h + 4); - else - ctx.fillRect(cx - 2, y - 2, gw + 2, h + 4); + if (side === "in") ctx.fillRect(cx - gw, y - 2, gw + 2, h + 4); + else ctx.fillRect(cx - 2, y - 2, gw + 2, h + 4); } // Handle body - ctx.fillStyle = col; + ctx.fillStyle = col; ctx.strokeStyle = hovered ? "#e0e7ff" : "#4f46e5"; - ctx.lineWidth = hovered ? 1.5 : 1; + ctx.lineWidth = hovered ? 1.5 : 1; if (side === "in") { // Left bracket tab @@ -727,14 +1121,16 @@ function createVideoTimelineWidget(node, videoWidgetName) { ctx.lineTo(cx, y + h); ctx.lineTo(cx - HANDLE_W, y + h); ctx.closePath(); - ctx.fill(); ctx.stroke(); + ctx.fill(); + ctx.stroke(); // Arrow chevron ctx.fillStyle = hovered ? "#1e1b4b" : "#312e81"; ctx.beginPath(); ctx.moveTo(cx - HANDLE_W + 3, y + h * 0.28); - ctx.lineTo(cx - 3, y + h * 0.50); + ctx.lineTo(cx - 3, y + h * 0.5); ctx.lineTo(cx - HANDLE_W + 3, y + h * 0.72); - ctx.closePath(); ctx.fill(); + ctx.closePath(); + ctx.fill(); } else { // Right bracket tab ctx.beginPath(); @@ -743,13 +1139,15 @@ function createVideoTimelineWidget(node, videoWidgetName) { ctx.lineTo(cx + HANDLE_W, y + h); ctx.lineTo(cx, y + h); ctx.closePath(); - ctx.fill(); ctx.stroke(); + ctx.fill(); + ctx.stroke(); ctx.fillStyle = hovered ? "#1e1b4b" : "#312e81"; ctx.beginPath(); ctx.moveTo(cx + HANDLE_W - 3, y + h * 0.28); - ctx.lineTo(cx + 3, y + h * 0.50); + ctx.lineTo(cx + 3, y + h * 0.5); ctx.lineTo(cx + HANDLE_W - 3, y + h * 0.72); - ctx.closePath(); ctx.fill(); + ctx.closePath(); + ctx.fill(); } } @@ -757,29 +1155,42 @@ function createVideoTimelineWidget(node, videoWidgetName) { const isHov = widget._hover === "collapse"; ctx.fillStyle = isHov ? "#1e1e2e" : "#16161e"; ctx.fillRect(x, y, w, h); - ctx.strokeStyle = C.border; ctx.lineWidth = 0.5; + ctx.strokeStyle = C.border; + ctx.lineWidth = 0.5; ctx.strokeRect(x, y, w, h); // Chevron - const cx = x + 12, cy = y + h / 2; + const cx = x + 12, + cy = y + h / 2; ctx.strokeStyle = isHov ? "#a5b4fc" : "#44445a"; ctx.lineWidth = 1.5; ctx.beginPath(); if (collapsed) { - ctx.moveTo(cx - 3, cy - 3); ctx.lineTo(cx + 3, cy); ctx.lineTo(cx - 3, cy + 3); + ctx.moveTo(cx - 3, cy - 3); + ctx.lineTo(cx + 3, cy); + ctx.lineTo(cx - 3, cy + 3); } else { - ctx.moveTo(cx - 4, cy - 2); ctx.lineTo(cx, cy + 3); ctx.lineTo(cx + 4, cy - 2); + ctx.moveTo(cx - 4, cy - 2); + ctx.lineTo(cx, cy + 3); + ctx.lineTo(cx + 4, cy - 2); } ctx.stroke(); ctx.fillStyle = isHov ? "#a5b4fc" : C.textDim; - ctx.font = "10px 'Courier New', monospace"; + ctx.font = "10px 'Courier New', monospace"; ctx.textAlign = "left"; ctx.fillText( - collapsed ? "VIDEO PREVIEW (click to expand)" : "VIDEO PREVIEW (click to collapse)", - x + 24, y + h / 2 + 4 + collapsed + ? "VIDEO PREVIEW (click to expand)" + : "VIDEO PREVIEW (click to collapse)", + x + 24, + y + h / 2 + 4, ); if (collapsed && totalFrames > 0) { ctx.textAlign = "right"; - ctx.fillText(`${totalFrames} fr • ${fps.toFixed(0)} fps`, x + w - 4, y + h / 2 + 4); + ctx.fillText( + `${totalFrames} fr • ${fps.toFixed(0)} fps`, + x + w - 4, + y + h / 2 + 4, + ); } } @@ -794,53 +1205,131 @@ function createVideoTimelineWidget(node, videoWidgetName) { ctx.strokeStyle = isHov ? "#818cf8" : "#2a2a3e"; ctx.lineWidth = 1; ctx.beginPath(); - ctx.moveTo(x, y); ctx.lineTo(x + w, y); - ctx.moveTo(x, y + h); ctx.lineTo(x + w, y + h); + ctx.moveTo(x, y); + ctx.lineTo(x + w, y); + ctx.moveTo(x, y + h); + ctx.lineTo(x + w, y + h); ctx.stroke(); // Label — centred vertically ctx.fillStyle = isHov ? "#c7d2fe" : "#55556a"; - ctx.font = "9px 'Courier New', monospace"; + ctx.font = "9px 'Courier New', monospace"; ctx.textAlign = "center"; ctx.fillText("↕ DRAG TO RESIZE", x + w / 2, y + h / 2 + 4); } + function formatBytes(n) { + const units = ["B", "KB", "MB", "GB", "TB"]; + let i = 0; + while (Math.abs(n) >= 1024 && i < units.length - 1) { + n /= 1024; + i++; + } + return `${n.toFixed(1)} ${units[i]}`; + } + + function estimatePeakMemory(nFrames, customNFrames) { + if (!bytesPerFrameF32) return 0; + const bppU8 = bytesPerFrameF32 / 4; + const firstCall = nFrames * bppU8 + nFrames * bytesPerFrameF32; + const residentAfter = nFrames * bytesPerFrameF32 * 2; + const secondCall = + residentAfter + customNFrames * bppU8 + customNFrames * bytesPerFrameF32; + return Math.max(firstCall, secondCall); + } + + function getTargetFps() { + const w = getWidget("target_fps"); + return w?.value || 24.0; + } + function drawInfoBar(ctx, x, y, w) { if (!totalFrames) return; - const dur = ((loopOut - loopIn) / fps).toFixed(2); + const dur = ((loopOut - loopIn) / fps).toFixed(2); const nFrames = loopOut - loopIn + 1; const stateLabel = playState === "playing" ? "▶ PLAYING" : "⏸ PAUSED"; - ctx.fillStyle = C.textDim; - ctx.font = "10px 'Courier New', monospace"; + + // Row 1 — loop summary + ctx.font = "bold 13px 'Courier New', monospace"; ctx.textAlign = "left"; + ctx.fillStyle = "#ffffff"; ctx.fillText( `LOOP ${loopIn} → ${loopOut} (${nFrames} fr / ${dur}s) • ${stateLabel}`, - x + 2, y + 12 + x + 2, + y + 14, ); - const vS = viewStart, vE = effectiveViewEnd(); + const vS = viewStart, + vE = effectiveViewEnd(); const zoomPct = Math.round((totalFrames / Math.max(1, vE - vS)) * 100); ctx.textAlign = "right"; - ctx.fillText(`zoom ${zoomPct}% ⇧+scroll=pan`, x + w - 2, y + 12); + ctx.fillStyle = "#e2e2ea"; + ctx.font = "12px 'Courier New', monospace"; + ctx.fillText(`zoom ${zoomPct}% ⇧+scroll=pan`, x + w - 2, y + 14); + + // Row 2 — memory estimate (color-coded) + if (bytesPerFrameF32 > 0) { + const tFps = getTargetFps(); + const clipDur = nFrames / Math.max(fps, 0.001); + const customN = Math.max(1, Math.round(clipDur * tFps)); + const peak = estimatePeakMemory(nFrames, customN); + const ratio = peak / memoryLimitBytes; + + let color; + if (ratio >= 1) color = "#ef4444"; // red — will be blocked + else if (ratio >= 0.75) color = "#f59e0b"; // amber — risky + else color = "#86efac"; // green — safe + + const icon = ratio >= 1 ? "⛔" : ratio >= 0.75 ? "⚠" : "✓"; + const statusText = + ratio >= 1 + ? "WILL BE BLOCKED" + : ratio >= 0.75 + ? "high memory use" + : "ok"; + + ctx.textAlign = "left"; + ctx.fillStyle = color; + ctx.font = "bold 12px 'Courier New', monospace"; + ctx.fillText( + `${icon} MEM est. ${formatBytes(peak)} / ${formatBytes(memoryLimitBytes)} limit • ${statusText}`, + x + 2, + y + 30, + ); + + ctx.textAlign = "right"; + ctx.fillStyle = "#9ca3af"; + ctx.font = "11px 'Courier New', monospace"; + ctx.fillText( + `${nFrames}+${customN} fr @ ${vidW}×${vidH}`, + x + w - 2, + y + 30, + ); + } } // ─── Widget object ──────────────────────────────────────────────────────── const widget = { - type: "VIDEOTIMELINE_V3", - name: "video_timeline_widget", + type: "VIDEOTIMELINE_V3", + name: "video_timeline_widget", options: {}, - value: null, - _hover: null, + value: null, + _hover: null, _btnRects: {}, // Called externally (e.g. from spinner watches) to update the loop // region without going through drag logic. setLoop(newIn, newOut) { if (!totalFrames) return; - loopIn = Math.max(0, Math.min(newIn, totalFrames - 1)); + loopIn = Math.max(0, Math.min(newIn, totalFrames - 1)); loopOut = Math.max(loopIn, Math.min(newOut, totalFrames - 1)); currentFrame = loopIn; - if (videoEl) { videoEl.pause(); videoEl.currentTime = loopIn / fps; } - cancelAnimationFrame(rafId); rafId = null; playState = "paused"; + if (videoEl) { + videoEl.pause(); + videoEl.currentTime = loopIn / fps; + } + cancelAnimationFrame(rafId); + rafId = null; + playState = "paused"; syncWidgets(); node.setDirtyCanvas(true, false); }, @@ -851,52 +1340,78 @@ function createVideoTimelineWidget(node, videoWidgetName) { const outerW = widgetWidth - PAD * 2; const totalH = widget.computeSize()[1] - 4; - ctx.fillStyle = C.surface; ctx.strokeStyle = C.border; ctx.lineWidth = 1; + ctx.fillStyle = C.surface; + ctx.strokeStyle = C.border; + ctx.lineWidth = 1; roundRect(ctx, outerX, y, outerW, totalH); - ctx.fill(); ctx.stroke(); + ctx.fill(); + ctx.stroke(); const iX = outerX + 6; const iW = outerW - 12; // Collapse toggle row — always visible - const collapseY = y + 4; + const collapseY = y + 4; drawCollapseToggle(ctx, iX, collapseY, iW, COLLAPSE_H); // Sections below — heights vary when preview is collapsed - const previewY = collapseY + COLLAPSE_H + 2; - const resizeBarY = previewY + (collapsed ? 0 : previewH); + const previewY = collapseY + COLLAPSE_H + 2; + const resizeBarY = previewY + (collapsed ? 0 : previewH); const transportY = resizeBarY + (collapsed ? 0 : RESIZE_H + 2); - const rulerY = transportY + (collapsed ? 0 : TRANSPORT_H + 2); - const stripY = rulerY + RULER_H; - const actY = stripY + STRIP_H; - const loopBarY = actY + ACTIVITY_H; - const infoY = loopBarY + LOOP_H + 4; + const rulerY = transportY + (collapsed ? 0 : TRANSPORT_H + 2); + const stripY = rulerY + RULER_H; + const sH = getStripH(); + const actY = stripY + sH; + const loopBarY = actY + ACTIVITY_H; + const infoY = loopBarY + LOOP_H + 4; - layout = { iX, iW, collapseY, previewY, resizeBarY, - transportY, rulerY, stripY, actY, loopBarY, infoY }; + layout = { + iX, + iW, + collapseY, + previewY, + resizeBarY, + transportY, + rulerY, + stripY, + actY, + loopBarY, + infoY, + }; if (!collapsed) { - drawPreview(ctx, iX, previewY, iW, previewH); - drawResizeHandle(ctx, iX, resizeBarY, iW, RESIZE_H); + drawPreview(ctx, iX, previewY, iW, previewH); + drawResizeHandle(ctx, iX, resizeBarY, iW, RESIZE_H); widget._btnRects = drawTransport(ctx, iX, transportY, iW, TRANSPORT_H); } else { widget._btnRects = {}; } - drawRuler(ctx, iX, rulerY, iW, RULER_H); - drawFilmstrip(ctx, iX, stripY, iW, STRIP_H); - drawActivity(ctx, iX, actY, iW, ACTIVITY_H); - drawLoopBar(ctx, iX, loopBarY, iW, LOOP_H); - drawInfoBar(ctx, iX, infoY, iW); + drawRuler(ctx, iX, rulerY, iW, RULER_H); + drawFilmstrip(ctx, iX, stripY, iW, sH); + drawActivity(ctx, iX, actY, iW, ACTIVITY_H); + drawLoopBar(ctx, iX, loopBarY, iW, LOOP_H); + drawInfoBar(ctx, iX, infoY, iW); ctx.restore(); - if (playState === "playing" || (videoEl && videoEl.readyState >= 2)) + if (playState === "playing" || videoReady || (videoEl && !videoReady && totalFrames > 0)) node.setDirtyCanvas(true, false); }, computeSize() { const ph = collapsed ? 0 : previewH + RESIZE_H + 2 + TRANSPORT_H + 2; - return [0, COLLAPSE_H + 10 + ph + RULER_H + STRIP_H + ACTIVITY_H + LOOP_H + INFO_H + 28]; + return [ + 0, + COLLAPSE_H + + 10 + + ph + + RULER_H + + getStripH() + + ACTIVITY_H + + LOOP_H + + INFO_H + + 28, + ]; }, mouse(event, pos) { @@ -912,25 +1427,47 @@ function createVideoTimelineWidget(node, videoWidgetName) { return true; } - const { iX, iW, collapseY, resizeBarY, transportY, rulerY, - stripY, actY, loopBarY } = layout; + const { + iX, + iW, + collapseY, + resizeBarY, + transportY, + rulerY, + stripY, + actY, + loopBarY, + } = layout; // ── Pre-compute zone membership ───────────────────────────────── - const inCollapse = collapseY != null && my >= collapseY && my <= collapseY + COLLAPSE_H; - const inResize = !collapsed && resizeBarY != null - && my >= resizeBarY && my <= resizeBarY + RESIZE_H; - const inTransport = !collapsed && transportY != null - && my >= transportY && my <= transportY + TRANSPORT_H; - const inRuler = rulerY != null && my >= rulerY && my < (stripY ?? rulerY + RULER_H); - const inStrip = stripY != null && my >= stripY && my < (actY ?? stripY + STRIP_H); - const inAct = actY != null && my >= actY && my < actY + ACTIVITY_H; - const inLoopRow = loopBarY != null && my >= loopBarY && my <= loopBarY + LOOP_H; - const inTimeline = (inRuler || inStrip || inAct || inLoopRow); + const inCollapse = + collapseY != null && my >= collapseY && my <= collapseY + COLLAPSE_H; + const inResize = + !collapsed && + resizeBarY != null && + my >= resizeBarY && + my <= resizeBarY + RESIZE_H; + const inTransport = + !collapsed && + transportY != null && + my >= transportY && + my <= transportY + TRANSPORT_H; + const inRuler = + rulerY != null && my >= rulerY && my < (stripY ?? rulerY + RULER_H); + const inStrip = + stripY != null && my >= stripY && my < (actY ?? stripY + getStripH()); + const inAct = actY != null && my >= actY && my < actY + ACTIVITY_H; + const inLoopRow = + loopBarY != null && my >= loopBarY && my <= loopBarY + LOOP_H; + const inTimeline = inRuler || inStrip || inAct || inLoopRow; - // Single hit-test helper for transport buttons + // Single hit-test helper for transport buttons (skips disabled ones) function hitBtn() { - for (const [id, r] of Object.entries(widget._btnRects)) - if (mx >= r.x && mx <= r.x + r.w && my >= r.y && my <= r.y + r.h) return id; + for (const [id, r] of Object.entries(widget._btnRects)) { + if (r.disabled) continue; + if (mx >= r.x && mx <= r.x + r.w && my >= r.y && my <= r.y + r.h) + return id; + } return null; } @@ -951,14 +1488,34 @@ function createVideoTimelineWidget(node, videoWidgetName) { } if (inTransport) { const id = hitBtn(); - if (id === "btnPlayPause") { isPlaying() ? pause() : play(); return true; } - if (id === "btnPrev") { stepFrame(-1); return true; } - if (id === "btnNext") { stepFrame(+1); return true; } + if (id === "btnPlayPause") { + isPlaying() ? pause() : play(); + return true; + } + if (id === "btnPrev") { + stepFrame(-1); + return true; + } + if (id === "btnNext") { + stepFrame(+1); + return true; + } + if (id === "btnJumpBack") { + stepFrame(-((loopOut - loopIn) + 1)); + return true; + } + if (id === "btnJumpFwd") { + stepFrame(+((loopOut - loopIn) + 1)); + return true; + } return false; } if (inTimeline) { drag = detectTarget(mx, my, iX, iW, loopBarY); - if (drag) { setCursor(cursorFor(drag.type, true)); return true; } + if (drag) { + setCursor(cursorFor(drag.type, true)); + return true; + } } return false; } @@ -968,7 +1525,11 @@ function createVideoTimelineWidget(node, videoWidgetName) { // Active drag takes priority if (resizeDrag) { const delta = my - resizeDragStartY; - previewH = clamp(resizeDragStartH + delta, PREVIEW_H_MIN, PREVIEW_H_MAX); + previewH = clamp( + resizeDragStartH + delta, + PREVIEW_H_MIN, + PREVIEW_H_MAX, + ); node.setSize([node.size[0], node.computeSize()[1]]); node.setDirtyCanvas(true, true); widget._hover = "resize"; @@ -987,19 +1548,24 @@ function createVideoTimelineWidget(node, videoWidgetName) { let hov = null; let cur = "default"; if (inCollapse) { - hov = "collapse"; cur = "pointer"; + hov = "collapse"; + cur = "pointer"; } else if (inResize) { - hov = "resize"; cur = "ns-resize"; + hov = "resize"; + cur = "ns-resize"; } else if (inTransport) { - hov = hitBtn(); cur = hov ? "pointer" : "default"; + hov = hitBtn(); + cur = hov ? "pointer" : "default"; } else if (inLoopRow) { const t = detectTarget(mx, my, iX, iW, loopBarY); hov = t?.type ?? null; cur = cursorFor(hov, false); } else if (inRuler) { - hov = "ruler"; cur = "crosshair"; + hov = "ruler"; + cur = "crosshair"; } else if (inStrip || inAct) { - hov = "filmstrip"; cur = "default"; + hov = "filmstrip"; + cur = "default"; } widget._hover = hov; @@ -1011,14 +1577,15 @@ function createVideoTimelineWidget(node, videoWidgetName) { // ── Pointer up ────────────────────────────────────────────────── if (event.type === "pointerup") { resizeDrag = false; - drag = null; + drag = null; // Re-evaluate hover at rest position let hov = null; - if (inCollapse) hov = "collapse"; - else if (inResize) hov = "resize"; + if (inCollapse) hov = "collapse"; + else if (inResize) hov = "resize"; else if (inTransport) hov = hitBtn(); - else if (inLoopRow) hov = detectTarget(mx, my, iX, iW, loopBarY)?.type ?? null; - else if (inRuler) hov = "ruler"; + else if (inLoopRow) + hov = detectTarget(mx, my, iX, iW, loopBarY)?.type ?? null; + else if (inRuler) hov = "ruler"; else if (inStrip || inAct) hov = "filmstrip"; widget._hover = hov; setCursor(hov ? cursorFor(hov, false) : "default"); @@ -1027,24 +1594,38 @@ function createVideoTimelineWidget(node, videoWidgetName) { return false; }, - }; // ─── Drag targeting ────────────────────────────────────────────────────── function detectTarget(mx, my, tX, tW, loopBarY) { if (!totalFrames) return null; - const lx = frameToX(loopIn, tX, tW); + const lx = frameToX(loopIn, tX, tW); const rx = frameToX(loopOut, tX, tW); // Loop bar row only const inLoopRow = my >= loopBarY && my <= loopBarY + LOOP_H; if (inLoopRow) { if (Math.abs(mx - lx) <= HANDLE_W + 4) - return { type: "loopIn", startX: mx, startIn: loopIn, startOut: loopOut }; + return { + type: "loopIn", + startX: mx, + startIn: loopIn, + startOut: loopOut, + }; if (Math.abs(mx - rx) <= HANDLE_W + 4) - return { type: "loopOut", startX: mx, startIn: loopIn, startOut: loopOut }; + return { + type: "loopOut", + startX: mx, + startIn: loopIn, + startOut: loopOut, + }; if (mx > lx + HANDLE_W && mx < rx - HANDLE_W) - return { type: "loopBody", startX: mx, startIn: loopIn, startOut: loopOut }; + return { + type: "loopBody", + startX: mx, + startIn: loopIn, + startOut: loopOut, + }; } return null; } @@ -1052,7 +1633,9 @@ function createVideoTimelineWidget(node, videoWidgetName) { function applyDrag(mx, tX, tW) { if (!drag) return; const frame = xToFrame(mx, tX, tW); - const dxFr = Math.round((mx - drag.startX) / tW * (effectiveViewEnd() - viewStart)); + const dxFr = Math.round( + ((mx - drag.startX) / tW) * (effectiveViewEnd() - viewStart), + ); switch (drag.type) { case "loopIn": loopIn = clamp(frame, 0, loopOut - 1); @@ -1062,7 +1645,7 @@ function createVideoTimelineWidget(node, videoWidgetName) { break; case "loopBody": { const span = drag.startOut - drag.startIn; - loopIn = clamp(drag.startIn + dxFr, 0, totalFrames - 1 - span); + loopIn = clamp(drag.startIn + dxFr, 0, totalFrames - 1 - span); loopOut = loopIn + span; break; } @@ -1070,8 +1653,8 @@ function createVideoTimelineWidget(node, videoWidgetName) { // Always pause and seek preview to loop start when the region changes cancelAnimationFrame(rafId); rafId = null; - playState = "paused"; - currentFrame = loopIn; + playState = "paused"; + currentFrame = loopIn; if (videoEl) { videoEl.pause(); videoEl.currentTime = loopIn / fps; @@ -1082,23 +1665,32 @@ function createVideoTimelineWidget(node, videoWidgetName) { function handleWheel(event, mx) { if (!totalFrames) return; const { iX, iW } = layout; - const vS = viewStart, vE = effectiveViewEnd(), vis = vE - vS; + const vS = viewStart, + vE = effectiveViewEnd(), + vis = vE - vS; if (event.shiftKey) { const pan = Math.round(vis * 0.08 * Math.sign(event.deltaY)); viewStart = clamp(vS + pan, 0, totalFrames - MIN_VIEW); - viewEnd = clamp(vE + pan, viewStart + MIN_VIEW, totalFrames); + viewEnd = clamp(vE + pan, viewStart + MIN_VIEW, totalFrames); } else { const cursorFrame = xToFrame(mx, iX, iW); const factor = event.deltaY > 0 ? 1.15 : 0.87; - let ns = 0, ne = 0; + let ns = 0, + ne = 0; let newSpan = clamp(Math.round(vis * factor), MIN_VIEW, totalFrames); const ratio = clamp((cursorFrame - vS) / vis, 0, 1); ns = Math.round(cursorFrame - ratio * newSpan); ne = ns + newSpan; - if (ns < 0) { ns = 0; ne = newSpan; } - if (ne > totalFrames) { ne = totalFrames; ns = totalFrames - newSpan; } + if (ns < 0) { + ns = 0; + ne = newSpan; + } + if (ne > totalFrames) { + ne = totalFrames; + ns = totalFrames - newSpan; + } viewStart = clamp(ns, 0, totalFrames); - viewEnd = clamp(ne, viewStart + MIN_VIEW, totalFrames); + viewEnd = clamp(ne, viewStart + MIN_VIEW, totalFrames); } event.preventDefault?.(); event.stopPropagation?.(); @@ -1108,7 +1700,10 @@ function createVideoTimelineWidget(node, videoWidgetName) { let lastVideo = null; setInterval(() => { const vw = getWidget(videoWidgetName); - if (vw?.value && vw.value !== lastVideo) { lastVideo = vw.value; fetchVideoInfo(vw.value); } + if (vw?.value && vw.value !== lastVideo) { + lastVideo = vw.value; + fetchVideoInfo(vw.value); + } }, 600); const vw0 = getWidget(videoWidgetName); if (vw0?.value) fetchVideoInfo(vw0.value); @@ -1128,35 +1723,49 @@ app.registerExtension({ onCreated?.apply(this, arguments); const tl = createVideoTimelineWidget(this, "video"); if (this.widgets?.length) this.widgets.splice(1, 0, tl); - else { this.widgets = this.widgets || []; this.widgets.push(tl); } - - // Fix widget label and upload button text - const vw = this.widgets?.find(w => w.name === "video"); - if (vw) { - vw.label = "Choose Video to Upload"; + else { + this.widgets = this.widgets || []; + this.widgets.push(tl); } - // num_frames: read-only, auto-computed - const nf = this.widgets?.find(w => w.name === "num_frames"); + // Fix widget label and upload button text + const vw = this.widgets?.find((w) => w.name === "video"); + if (vw) { + vw.label = "Video Filename"; + } + + // Rename the upload button (added by ComfyUI core when video_upload: True). + // It has no stable name across versions, so match by label/text. + for (const w of this.widgets || []) { + const labelMatch = + typeof w.label === "string" && /choose\s+file\s+to\s+upload/i.test(w.label); + const nameMatch = w.name === "upload" || w.name === "choose file to upload"; + if (labelMatch || nameMatch) { + w.label = "Choose Video to Upload"; + if (typeof w.name === "string" && /choose\s+file/i.test(w.name)) { + w.name = "Choose Video to Upload"; + } + } + } + + // num_frames: auto-computed from loop span — hide it completely so the + // user isn't confused by a read-only "AUTO" field. We still update its + // `value` via syncWidgets() so the backend receives the span. + const nf = this.widgets?.find((w) => w.name === "num_frames"); if (nf) { - nf.disabled = true; - nf.read_only = true; - const _orig = nf.draw?.bind(nf); - nf.draw = function(ctx, node, widget_width, y, H) { - ctx.save(); ctx.globalAlpha = 0.45; - _orig?.(ctx, node, widget_width, y, H); - ctx.restore(); - ctx.fillStyle = "#6366f1"; ctx.font = "9px 'Courier New', monospace"; - ctx.textAlign = "right"; - ctx.fillText("AUTO", widget_width - 8, y + H / 2 + 4); - }; + nf.type = "hidden"; + nf.hidden = true; + nf.computeSize = () => [0, -4]; // collapse layout row + nf.draw = () => {}; nf.mouse = () => false; } // Watch start_frame and end_frame spinners — sync loop region when changed - const tlWidget = this.widgets?.find(w => w.name === "video_timeline_widget"); - const sfWidget = this.widgets?.find(w => w.name === "start_frame"); - const efWidget = this.widgets?.find(w => w.name === "end_frame"); + const tlWidget = this.widgets?.find( + (w) => w.name === "video_timeline_widget", + ); + const sfWidget = this.widgets?.find((w) => w.name === "start_frame"); + const efWidget = this.widgets?.find((w) => w.name === "end_frame"); if (tlWidget && sfWidget && efWidget) { let lastSf = sfWidget.value; let lastEf = efWidget.value; @@ -1164,9 +1773,10 @@ app.registerExtension({ const sf = sfWidget.value; const ef = efWidget.value; if (sf !== lastSf || ef !== lastEf) { - lastSf = sf; lastEf = ef; + lastSf = sf; + lastEf = ef; // Clamp: start can't exceed end and vice-versa - const newIn = Math.min(sf, ef); + const newIn = Math.min(sf, ef); const newOut = Math.max(sf, ef); tlWidget.setLoop(newIn, newOut); }