Compare commits
15
Commits
tara-keyframes
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9dcb9e4d08 | ||
|
|
b29bb93dd4 | ||
|
|
194690985a | ||
|
|
0fc2f69257 | ||
|
|
93bcf3806d | ||
|
|
a6cae2b9c1 | ||
|
|
e6a9d384b7 | ||
|
|
6d434b246e | ||
|
|
9b41672085 | ||
|
|
33fd830581 | ||
|
|
22bb235c1d | ||
|
|
04d8012020 | ||
|
|
1ebfd0c9b6 | ||
|
|
4c8711374f | ||
|
|
d83b9ed0af |
+111
@@ -0,0 +1,111 @@
|
|||||||
|
# Changelog
|
||||||
|
|
||||||
|
All notable changes to **ComfyUI-BFSNodes** are documented in this file.
|
||||||
|
|
||||||
|
## [1.120.0] - 2026-11-10
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **BFS H3 Image Character Swap Conditioning**: native H3 Image conditioning and automatic still-image prompts. The guide is Picture 1; red, blue and green references are Pictures 2, 3 and 4. Supports normal maps, DWPose, overlays and outlines. Requires ComfyUI 0.45.0 or later.
|
||||||
|
- **Add Guide for MiniMax H3 Image (BFS)**: experimental frozen guide reference aligned to the target image positions through a cloned ModelPatcher. Uses native H3 Image latents without an audio stream.
|
||||||
|
- **Video Segment Picker**: optional `point_model` input and MolmoPoint-8B tracking method. Connect the loader from ComfyUI-Molmo2-H3.
|
||||||
|
- Protected-object selections, a normal-map-inside-mask mode with feathered edges, translucent overlays and colored outlines.
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- Bound picker mask caches by memory size, keep cached masks as boolean tensors and preserve existing masks when filling empty stretches.
|
||||||
|
- Track empty stretches and nearest-anchor windows instead of repeatedly recomputing the full clip.
|
||||||
|
- Correct SAM inference tensor handling for point edits, rendering invalidation, normal-map lazy input requests and Apply fix messages.
|
||||||
|
- Stabilize compare preview sizing; expose memory cleanup after comparison.
|
||||||
|
|
||||||
|
## [1.119.5] - 2026-11-09
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **BFS Video Segment Picker (`BFSVideoSegmentPicker`)**: Restored the classic Molmo2 full-video text tracking as the default, with the current SAM 3 tracking available in a dropdown.
|
||||||
|
- Added repair scopes for one frame, the active frame onward, recomputing all frames, and filling only empty masks. Filling empty masks is the default and preserves existing non-empty results.
|
||||||
|
- Preserve the last propagated preview while adding selections, disable auto-propagation by default, and hide the manual eraser in the video picker while retaining SAM-guided remove points.
|
||||||
|
- Shortened the package description to fit Comfy Registry metadata validation.
|
||||||
|
|
||||||
|
## [1.119.4] - 2026-11-09
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **BFS Video Segment Picker (`BFSVideoSegmentPicker`)**:
|
||||||
|
- **Automatic Unmarking / Subject Limiting (`max_subjects`)**: Added `"max_subjects"` parameter (`["auto", "1", "2", "3"]`, default `"auto"`). In `"auto"` mode, the node dynamically inspects the execution workflow graph (`prompt`) to count connected reference pictures (`picture_N`) on downstream conditioning nodes (e.g. `BFSH3KeyframeConditioning`). If only 2 pictures are connected, Subject 3 (`3 · green`) is automatically excluded from tracking, masking, and hybrid guide rendering, preventing unmatched green skeletons or normal maps from reaching MiniMax H3.
|
||||||
|
- **One-Click Subject Clear (`✕`)**: Added a clear button `✕` on each subject badge (`1 · red`, `2 · blue`, `3 · green`) in the UI header. Clicking it instantly removes all picks for that subject across all frames and triggers auto-propagation.
|
||||||
|
- **Explicit Text Tracking (`molmo_descriptions`)**: Added optional `"molmo_descriptions"` multiline input (line $k$ = Subject $k$). When provided, Molmo2 tracks using the exact text descriptions across camera cuts and shots instead of guessing from low-resolution crops, avoiding tracking errors.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [1.119.3] - 2026-11-09
|
||||||
|
|
||||||
|
### Changed
|
||||||
|
- **BFS Video Segment Picker (`BFSVideoSegmentPicker`) & BFS H3 Character Swap Conditioning (`BFSH3KeyframeConditioning`)**:
|
||||||
|
- Limited selectable subjects and reference pictures to 3 (`1 · red`, `2 · blue`, `3 · green` / `picture_1`..`picture_3`) to match the 3-subject training distribution of the Normal Map and DWPose Character Swap LoRAs.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [1.119.2] - 2026-11-09
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **BFS H3 Character Swap Conditioning (`BFSH3KeyframeConditioning`)**:
|
||||||
|
- Added runtime VRAM estimation and peak activation patching (`_patch_vram_estimate`) onto the `MiniMaxH3` model instance so ComfyUI accounts for `minimax_keyframes` (`guide_video`), `minimax_refs` (`Picture 1`–`Picture 4`), and `cross_attn` tokens when computing `minimum_memory_required`, and releases intermediate `q, k, v` and `h` buffers early in `DiTBlock` / `Attention` without modifying ComfyUI core.
|
||||||
|
- Cleared cached DWPose detector/pose sessions (`global_cached_dwpose`) safely in `_free_gpu()` without triggering lazy-import errors on unrelated modules in `sys.modules`.
|
||||||
|
- **BFS Video Segment Picker (`BFSVideoSegmentPicker`)**:
|
||||||
|
- Separated multi-subject DWPose guide rendering into two passes (erase all subject regions to neutral gray first, then draw all subject skeletons on top) so overlapping gray regions of adjacent/hugging subjects never erase each other's skeletons.
|
||||||
|
- Added per-frame DWPose matching exclusion (`exclude`), relaxed overlap fallback (`min_inside=0.25`), and neighbor-frame pose fallback (`±2` frames) so closely interacting subjects do not lose their skeleton on intermediate frames.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [1.119.1] - 2026-11-09
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **BFS Video Segment Picker (`BFSVideoSegmentPicker`)**:
|
||||||
|
- Prevented subjects picked on later frames from being falsely propagated or re-anchored onto other people in earlier frames before `first_pick`.
|
||||||
|
- Restricted Molmo2 tracking (`molmo_relocate`) to search strictly forward from `first_pick` (`image[first_pick:]`), highlighting the selected person on the initial frame (`t=0.0s`) and dims background distractions.
|
||||||
|
- Added visual candidate verification (`molmo_verify_candidate`): compares the reference person crop (`Picture 1`) against each candidate person crop (`Picture 2`) with Molmo2, checks RGB/chroma histogram similarity (`person_color_sim`), and rejects candidates that overlap masks already claimed by other subjects (`claimed_at`).
|
||||||
|
- Bounded backward SAM 3 tracking before `first_pick` to the immediately preceding strip frame (`prev_strip`) with frame-to-frame spatial (`IoU >= 0.15`), non-overlap, and color continuity checks.
|
||||||
|
- Enforced `gc.collect()` and `mm.soft_empty_cache(True)` when freeing GPU models after segmentation/tracking.
|
||||||
|
- **BFS H3 Character Swap Conditioning (`BFSH3KeyframeConditioning`)**:
|
||||||
|
- Added `_free_gpu()` to explicitly unload `MiniMaxH3TEModel_` (15 GB) and `MiniMaxH3VideoVAE` (5 GB) and flush the CUDA cache after `MiniMaxH3ReferenceToVideo` and `MiniMaxH3AddGuide`, preventing partial-unload VRAM retention and CUDA OOM when `MiniMaxH3` (20 GB) starts sampling.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [1.119.0] - 2026-11-09
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **BFS Video Segment Picker (`BFSVideoSegmentPicker`)**:
|
||||||
|
- Interactive multi-subject (1–4) SAM 3.1 video segmentation and shot-by-shot propagation with automatic camera cut detection.
|
||||||
|
- Optional Molmo2 re-anchoring across camera cuts and lost stretches, with per-shot and per-subject caching.
|
||||||
|
- Interactive timeline scrubber displaying per-frame subject coverage and lost stretches.
|
||||||
|
- Hybrid guide video output supporting `normal map`, `openpose + face` (DWPose skeleton + face landmarks over neutral gray), and texture-free styles (`flat color`, `shading`, `blur`, `pixelate`, `gray detail`, `edges`).
|
||||||
|
- Configurable `pose_region` (`convex hull` or `person outline`) and `pose_gray_opacity`.
|
||||||
|
- **BFS H3 Character Swap Conditioning (`BFSH3KeyframeConditioning`)**:
|
||||||
|
- Unified conditioning node supporting `keyframe pins`, `normal map guide`, and `DWPose guide` modes with up to 4 reference pictures (`Picture 1`–`Picture 4`) mapped to red, blue, green, and yellow subject marks.
|
||||||
|
- **Keyframe & Loop Helpers**:
|
||||||
|
- Added `BFSKeyframePlanner`, `BFSKeyframeMarks`, `BFSTrackFromFrame`, `BFSWorkflowLoop`, `BFSXYSweep`, and `BFSFrameLoop`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [1.118.1] - 2026-11-07
|
||||||
|
|
||||||
|
### Fixed
|
||||||
|
- **Panel Server Routes**: Wrapped interactive panel button model executions under `torch.inference_mode()` to match standard ComfyUI node execution.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [1.118.0] - 2026-11-04
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **BFS Segment Picker (`BFSSegmentPicker`)**: Added interactive brush strokes, negative/remove points, and eraser mode on the segmentation canvas.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [1.117.0] - 2026-11-03
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **BFS Video Compare (`BFSVideoCompare`)**: Added current-frame PNG download button in the comparison UI and automatic PNG output when comparing single images.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## [1.116.0] - 2026-11-03
|
||||||
|
|
||||||
|
### Added
|
||||||
|
- **BFS Segment Picker (`BFSSegmentPicker`)**: Interactive SAM 3.1 object picker canvas with automatic concept detection, text search, and point prompts.
|
||||||
@@ -144,7 +144,7 @@ workflow once per shot, and join the results back with the original timing and s
|
|||||||
guide at 2 fps without native guide video-reference latents; use Add Guide separately for motion.
|
guide at 2 fps without native guide video-reference latents; use Add Guide separately for motion.
|
||||||
See [H3 guide modes](MINIMAX_H3_GUIDES.md#video-context-in-clip-with-aligned-guide-latents).
|
See [H3 guide modes](MINIMAX_H3_GUIDES.md#video-context-in-clip-with-aligned-guide-latents).
|
||||||
- **BFS Shot Join** — concatenates, trims to the true lengths, cross-fades soft joins, returns audio. With `comparison` on, its `guide` output is the original video frame for frame.
|
- **BFS Shot Join** — concatenates, trims to the true lengths, cross-fades soft joins, returns audio. With `comparison` on, its `guide` output is the original video frame for frame.
|
||||||
- **BFS Video Compare** — one player for two videos (Join's `guide` and `images`): wipe ↔ / ↕, overlay or side by side, any of guide / target / difference on top, plus a per-pixel difference map and the difference per frame (click to jump).
|
- **BFS Video Compare** — one player for two videos (Join's `guide` and `images`): wipe ↔ / ↕, overlay or side by side, any of guide / target / difference on top, plus a per-pixel difference map and the difference per frame (click to jump). 🖼 PNG downloads the frame on screen as shown; with two images (one frame) ⬇ save PNG downloads it from lossless PNGs at the compared size.
|
||||||
|
|
||||||
**Cast (1.49.0)**: *Find people* tracks faces through the video (InsightFace) and groups them
|
**Cast (1.49.0)**: *Find people* tracks faces through the video (InsightFace) and groups them
|
||||||
into people. Link each person to a reference and every shot uses the reference of the person
|
into people. Link each person to a reference and every shot uses the reference of the person
|
||||||
@@ -252,3 +252,80 @@ The Token / Word Weight Editor accepts `(phrase:1.3)` directives in
|
|||||||
[One workflow](examples/h3_tara_optional_sam_loop.comfy.json) now has independent
|
[One workflow](examples/h3_tara_optional_sam_loop.comfy.json) now has independent
|
||||||
SAM 3.1 and shot-loop switches. Both default OFF. See
|
SAM 3.1 and shot-loop switches. Both default OFF. See
|
||||||
[setup, disabled branches and status](H3_CAST_AND_TOKEN_CONTROLS.md#one-workflow-with-optional-sam-and-loop-111531).
|
[setup, disabled branches and status](H3_CAST_AND_TOKEN_CONTROLS.md#one-workflow-with-optional-sam-and-loop-111531).
|
||||||
|
|
||||||
|
### BFS Segment Picker (1.116.0)
|
||||||
|
|
||||||
|
Segment an image into objects with SAM 3.1 and pick them with the mouse. Connect a
|
||||||
|
Load Checkpoint node with `sam3.1_multiplex_fp16.safetensors` (its MODEL and CLIP),
|
||||||
|
press **▶ Segment** (runs only this node) and the objects appear on a canvas in the
|
||||||
|
node: hover to see one, click to select it. The automatic pass looks for the
|
||||||
|
`concepts` (default `person, animal`); add what it missed with **Find** (a SAM 3 text
|
||||||
|
prompt) or with keep / exclude **points**. Outputs: the selection's mask, one mask per
|
||||||
|
selected object, the image with the selection tinted (`mark_color` at `mark_opacity`,
|
||||||
|
magenta at 0.3 by default, the Colored Mark convention), a numbered overlay and the
|
||||||
|
objects as JSON. The selection and the added objects are saved with the workflow and
|
||||||
|
segmented again on load.
|
||||||
|
|
||||||
|
Hand edits (1.118.0): **🖌 Brush** paints extra area into the mark (e.g. hair SAM left
|
||||||
|
out), **➖ Remove** cuts out what SAM segments at the points you place, and **🧽 Eraser**
|
||||||
|
paints what to cut out. Each becomes a chip; unselect it to switch it off. Brush and
|
||||||
|
eraser strokes belong to the image they were painted on and are dropped when the image
|
||||||
|
changes. Fixed: clicking inside the node no longer shrinks its canvas while the side
|
||||||
|
panel is open.
|
||||||
|
|
||||||
|
### Character swap with marked guides (1.119.0)
|
||||||
|
|
||||||
|
Mark who is replaced in the guide video, then condition MiniMax H3 Character Swap on it.
|
||||||
|
|
||||||
|
**BFS Video Segment Picker (SAM 3)** — the Segment Picker for a video. A strip shows
|
||||||
|
frames of the batch; click a frame, pick the subject tab (1 red, 2 blue, 3 green,
|
||||||
|
4 yellow = Picture 1–4) and click the person. **↔ Propagate** tracks every subject from
|
||||||
|
its picked frames to the whole video with SAM 3, shot by shot (camera cuts are detected
|
||||||
|
and never tracked across). Optional `molmo_clip` (Molmo2 H3 encoder): Molmo2 describes
|
||||||
|
the picked person, follows them through the video and re-anchors shots that have no
|
||||||
|
pick, so a person is found again after a cut. Picking runs only segment the strip;
|
||||||
|
tracking, Molmo2, MoGe and DWPose run on Propagate. Repeated runs reuse what did not
|
||||||
|
change (per subject and per shot). After the first Propagate, picking on a frame
|
||||||
|
propagates by itself (**auto**). The **Whole video** scrubber shows every frame with
|
||||||
|
its masks, the lost stretches in red; **+ add this frame** puts any frame in the strip
|
||||||
|
to fix it. A status line with a progress bar and a log shows what runs.
|
||||||
|
|
||||||
|
Outputs: `masks`, `marked` (tinted), `overlay`, `report`, and `hybrid`: the guide
|
||||||
|
with each subject drawn in its color, by `mark_style`:
|
||||||
|
- `normal map` — its MoGe normal map in gray detail, tinted (connect `normals`:
|
||||||
|
Run MoGe Inference → Render MoGe Geometry, a normal mode);
|
||||||
|
- `openpose + face` — erased to gray (`pose_region`: convex hull or person outline,
|
||||||
|
`pose_gray_opacity`) and drawn as its DWPose skeleton with the eyes and the mouth
|
||||||
|
opening (connect `pose_keypoint`: DWPose Estimator with body, hands and face);
|
||||||
|
- texture-free alternatives: flat color, shading, blur, pixelate, gray detail, edges.
|
||||||
|
|
||||||
|
`hybrid_grow` sets the margin around each subject; `hybrid_base` stacks pickers.
|
||||||
|
`free_after` unloads the models from the GPU when the node is done (default: all).
|
||||||
|
|
||||||
|
**BFS H3 Character Swap Conditioning** — `mode`:
|
||||||
|
- `keyframe pins`: the source video plus one Add Guide per edited keyframe at its own
|
||||||
|
frame (`keyframes` + `keyframe_spec` from BFS Keyframe Planner, or `keyframe_N` /
|
||||||
|
`time_N`). `keyframe_until` drops the keyframes (or every guide) below that sigma on
|
||||||
|
the returned `model`.
|
||||||
|
- `normal map guide` / `DWPose guide`: the Video Segment Picker's `hybrid` as the guide.
|
||||||
|
|
||||||
|
Pictures 1–4 are the references in color order; the prompt is written from the mode
|
||||||
|
and the number of Pictures (or `prompt_override`). Outputs: model, positive, latent,
|
||||||
|
prompt, keyframe_video and `pictures` (the Pictures as a list, e.g. for BFS Video
|
||||||
|
Compare's `reference`).
|
||||||
|
|
||||||
|
Also: **BFS Keyframe Planner** (choose the keyframe frames on the video and export
|
||||||
|
them with their spec), **BFS Keyframe Marks** / **BFS Track From Frame (SAM 3)** (a
|
||||||
|
tracked person's mask at each keyframe), **BFS Workflow Loop** (run a part of the
|
||||||
|
graph once per item, injecting the item and changing widgets per iteration),
|
||||||
|
**BFS XY Sweep** (a grid of any two widgets' values) and **BFS Frame Loop Start/End**.
|
||||||
|
|
||||||
|
### MiniMax H3 Image character swap
|
||||||
|
|
||||||
|
Requires **ComfyUI 0.45.0+** and the H3 Image model, image VAE and matching text encoder.
|
||||||
|
|
||||||
|
Use **BFS H3 Image Character Swap Conditioning** with the Picker's single-image `hybrid` as `guide_image`. Connect character references in red, blue and green order. The generated prompt treats the guide as **Picture 1** and the characters as **Pictures 2–4**. Connect `positive` and `latent` to the sampler; the node also outputs the prompt for inspection. Normal map, DWPose, translucent overlay and outline guides are supported.
|
||||||
|
|
||||||
|
**Add Guide for MiniMax H3 Image (BFS)** is an optional experimental alignment step. Connect the model, positive conditioning, native image latent, image VAE and marked guide; use its returned MODEL and positive with the sampler. It aligns the guide's frozen reference positions to the target canvas. It does not use the video/audio Add Guide latent format. Structural checks passed; visual equivalence to a video-trained guide LoRA is not guaranteed.
|
||||||
|
|
||||||
|
For **MolmoPoint-8B** in Video Segment Picker, connect `point_model` from the loader in [ComfyUI-Molmo2-H3](https://github.com/alisson-anjos/ComfyUI-Molmo2-H3), select `MolmoPoint-8B` in `propagation_method`, and use **Apply fix** with the desired repair scope. SAM 3 still produces the masks.
|
||||||
|
|||||||
+27
@@ -132,6 +132,12 @@ try:
|
|||||||
except Exception as _e: # noqa
|
except Exception as _e: # noqa
|
||||||
print(f"[BFSNodes] Video Compare node not loaded: {_e!r}")
|
print(f"[BFSNodes] Video Compare node not loaded: {_e!r}")
|
||||||
VCMP_NODE_CLASS_MAPPINGS, VCMP_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
|
VCMP_NODE_CLASS_MAPPINGS, VCMP_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
|
||||||
|
try:
|
||||||
|
from .bfs_segment_picker import NODE_CLASS_MAPPINGS as SEGPICK_NODE_CLASS_MAPPINGS
|
||||||
|
from .bfs_segment_picker import NODE_DISPLAY_NAME_MAPPINGS as SEGPICK_NODE_DISPLAY_NAME_MAPPINGS
|
||||||
|
except Exception as _e: # noqa
|
||||||
|
print(f"[BFSNodes] Segment Picker node not loaded: {_e!r}")
|
||||||
|
SEGPICK_NODE_CLASS_MAPPINGS, SEGPICK_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
|
||||||
|
|
||||||
try:
|
try:
|
||||||
from .minimax_h3_clip_video import NODE_CLASS_MAPPINGS as H3CLIP_NODE_CLASS_MAPPINGS
|
from .minimax_h3_clip_video import NODE_CLASS_MAPPINGS as H3CLIP_NODE_CLASS_MAPPINGS
|
||||||
@@ -181,6 +187,7 @@ NODE_CLASS_MAPPINGS = {
|
|||||||
**HSMS_NODE_CLASS_MAPPINGS,
|
**HSMS_NODE_CLASS_MAPPINGS,
|
||||||
**LSURG_NODE_CLASS_MAPPINGS,
|
**LSURG_NODE_CLASS_MAPPINGS,
|
||||||
**VCMP_NODE_CLASS_MAPPINGS,
|
**VCMP_NODE_CLASS_MAPPINGS,
|
||||||
|
**SEGPICK_NODE_CLASS_MAPPINGS,
|
||||||
}
|
}
|
||||||
|
|
||||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||||
@@ -217,8 +224,28 @@ NODE_DISPLAY_NAME_MAPPINGS = {
|
|||||||
**HSMS_NODE_DISPLAY_NAME_MAPPINGS,
|
**HSMS_NODE_DISPLAY_NAME_MAPPINGS,
|
||||||
**LSURG_NODE_DISPLAY_NAME_MAPPINGS,
|
**LSURG_NODE_DISPLAY_NAME_MAPPINGS,
|
||||||
**VCMP_NODE_DISPLAY_NAME_MAPPINGS,
|
**VCMP_NODE_DISPLAY_NAME_MAPPINGS,
|
||||||
|
**SEGPICK_NODE_DISPLAY_NAME_MAPPINGS,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Character swap with marked guides (Video Segment Picker -> H3 Character Swap Conditioning), keyframe planning, and
|
||||||
|
# the loop / sweep helpers. Each module loads on its own: a missing optional dependency only skips that module.
|
||||||
|
for _mod in ("bfs_keyframe_marks", "bfs_video_segment_picker", "bfs_h3_swap_conditioning", "bfs_keyframe_planner",
|
||||||
|
"bfs_frame_loop", "bfs_xy_sweep", "bfs_workflow_loop"):
|
||||||
|
try:
|
||||||
|
import importlib as _il
|
||||||
|
_m = _il.import_module(f".{_mod}", __package__)
|
||||||
|
NODE_CLASS_MAPPINGS.update(_m.NODE_CLASS_MAPPINGS)
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS.update(_m.NODE_DISPLAY_NAME_MAPPINGS)
|
||||||
|
except Exception as _e: # noqa: BLE001
|
||||||
|
print(f"[BFSNodes] {_mod} not loaded: {_e!r}")
|
||||||
|
|
||||||
WEB_DIRECTORY = "./web/js"
|
WEB_DIRECTORY = "./web/js"
|
||||||
|
|
||||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
|
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS", "WEB_DIRECTORY"]
|
||||||
|
|
||||||
|
try:
|
||||||
|
from .bfs_h3_image_swap_conditioning import BFSH3ImageCharacterSwapConditioning
|
||||||
|
NODE_CLASS_MAPPINGS["BFSH3ImageCharacterSwapConditioning"] = BFSH3ImageCharacterSwapConditioning
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS["BFSH3ImageCharacterSwapConditioning"] = "BFS H3 Image Character Swap Conditioning"
|
||||||
|
except ImportError as _e:
|
||||||
|
print(f"[BFSNodes] H3 Image conditioning requires updated ComfyUI: {_e}")
|
||||||
|
|||||||
@@ -0,0 +1,136 @@
|
|||||||
|
"""BFS Frame Loop: run any part of a workflow once per frame, one after another, and collect the results.
|
||||||
|
|
||||||
|
Wire a batch of frames (e.g. the Keyframe Planner's source_frames, or BFS Keyframe Marks' marked_frames) into
|
||||||
|
BFS Frame Loop Start; it gives one frame (plus its mask, index and count). Build the per-frame work after it (an image
|
||||||
|
edit: Qwen 2.1 body swap, Klein, ...), and wire its result into BFS Frame Loop End. The End node expands the body
|
||||||
|
between Start and End again for the next frame until all are done, then returns them as one batch (same order).
|
||||||
|
Models stay outside the loop body. Results are kept on disk between iterations; no global state.
|
||||||
|
|
||||||
|
The expansion follows the BFS H3 Native Shot Loop pattern (start/end graph expansion).
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import torch
|
||||||
|
import torch.nn.functional as F
|
||||||
|
|
||||||
|
try:
|
||||||
|
import folder_paths
|
||||||
|
except Exception: # tests run without ComfyUI
|
||||||
|
folder_paths = None
|
||||||
|
|
||||||
|
CATEGORY = "BFS/Loop"
|
||||||
|
|
||||||
|
|
||||||
|
class BFSFrameLoopStart:
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {"images": ("IMAGE", {"tooltip": "The frames to process one by one."})},
|
||||||
|
"optional": {"masks": ("MASK", {"tooltip": "One mask per frame (optional), handed out with its frame."}),
|
||||||
|
"loop_state": ("BFS_FRAME_LOOP", {"tooltip": "Set by Frame Loop End; leave unconnected."})}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "MASK", "INT", "INT", "BFS_FRAME_LOOP")
|
||||||
|
RETURN_NAMES = ("image", "mask", "index", "count", "loop_state")
|
||||||
|
OUTPUT_TOOLTIPS = ("The current frame.", "Its mask (zeros when no masks are given).", "Index of the current frame (0-based).",
|
||||||
|
"How many frames the loop runs.", "Connect to BFS Frame Loop End.")
|
||||||
|
FUNCTION = "start"
|
||||||
|
CATEGORY = CATEGORY
|
||||||
|
DESCRIPTION = "Hands out one frame per loop iteration; pair it with BFS Frame Loop End."
|
||||||
|
|
||||||
|
def start(self, images, masks=None, loop_state=None):
|
||||||
|
n = int(images.shape[0])
|
||||||
|
if loop_state is None:
|
||||||
|
temp = folder_paths.get_temp_directory() if folder_paths else "."
|
||||||
|
folder = os.path.join(temp, "bfs_frame_loop", uuid.uuid4().hex[:12])
|
||||||
|
os.makedirs(folder, exist_ok=True)
|
||||||
|
loop_state = {"folder": folder, "index": 0, "count": n, "done": []}
|
||||||
|
state = dict(loop_state)
|
||||||
|
if state["count"] != n:
|
||||||
|
raise ValueError(f"The loop started with {state['count']} frames but now gets {n}.")
|
||||||
|
i = int(state["index"])
|
||||||
|
h, w = images.shape[1:3]
|
||||||
|
if masks is not None:
|
||||||
|
m = masks if masks.ndim == 3 else masks[None]
|
||||||
|
if m.shape[0] != n:
|
||||||
|
raise ValueError(f"{m.shape[0]} masks for {n} frames.")
|
||||||
|
mask = m[i:i + 1]
|
||||||
|
if mask.shape[1:] != (h, w):
|
||||||
|
mask = F.interpolate(mask[:, None].float(), size=(h, w), mode="bilinear", align_corners=False)[:, 0]
|
||||||
|
else:
|
||||||
|
mask = torch.zeros(1, h, w)
|
||||||
|
print(f"[BFS Frame Loop] frame {i + 1}/{n}", flush=True)
|
||||||
|
return images[i:i + 1], mask, i, n, state
|
||||||
|
|
||||||
|
|
||||||
|
class BFSFrameLoopEnd:
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {"image": ("IMAGE", {"tooltip": "The processed frame of this iteration."}),
|
||||||
|
"loop_state": ("BFS_FRAME_LOOP", {"tooltip": "From BFS Frame Loop Start."})},
|
||||||
|
"optional": {"resize_to_first": ("BOOLEAN", {"default": True,
|
||||||
|
"tooltip": "Resize every result to the first result's size so they form one batch."})},
|
||||||
|
"hidden": {"dynprompt": "DYNPROMPT", "unique_id": "UNIQUE_ID"}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "INT")
|
||||||
|
RETURN_NAMES = ("images", "count")
|
||||||
|
OUTPUT_TOOLTIPS = ("All processed frames, in order, as one batch.", "How many frames.")
|
||||||
|
FUNCTION = "finish"
|
||||||
|
CATEGORY = CATEGORY
|
||||||
|
DESCRIPTION = "Collects one processed frame per iteration and expands the loop body until all frames are done."
|
||||||
|
|
||||||
|
def finish(self, image, loop_state, resize_to_first=True, dynprompt=None, unique_id=None):
|
||||||
|
state = dict(loop_state)
|
||||||
|
i = int(state["index"])
|
||||||
|
path = Path(state["folder"]) / f"{i:05d}.pt"
|
||||||
|
tmp = path.with_suffix(".tmp"); torch.save(image[:1].detach().cpu().contiguous(), tmp); tmp.replace(path)
|
||||||
|
state["done"] = list(state["done"]) + [str(path)]
|
||||||
|
state["index"] = i + 1
|
||||||
|
if state["index"] < state["count"]:
|
||||||
|
if dynprompt is None:
|
||||||
|
raise ValueError("BFS Frame Loop needs ComfyUI's dynamic graph execution")
|
||||||
|
return self.expand(dynprompt, str(unique_id), state)
|
||||||
|
parts = [torch.load(p, map_location="cpu", weights_only=True) for p in state["done"]]
|
||||||
|
if resize_to_first:
|
||||||
|
h, w = parts[0].shape[1:3]
|
||||||
|
parts = [p if p.shape[1:3] == (h, w) else
|
||||||
|
F.interpolate(p.movedim(-1, 1), size=(h, w), mode="bilinear", align_corners=False).movedim(1, -1) for p in parts]
|
||||||
|
if len({tuple(p.shape[1:]) for p in parts}) != 1:
|
||||||
|
raise ValueError("The processed frames have different sizes; turn on resize_to_first.")
|
||||||
|
(Path(state["folder"]) / "manifest.json").write_text(json.dumps(state, indent=1))
|
||||||
|
return torch.cat(parts), len(parts)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def expand(dynprompt, end, state):
|
||||||
|
from comfy_execution.graph_utils import GraphBuilder, is_link
|
||||||
|
feeds, seen = {}, set()
|
||||||
|
|
||||||
|
def visit(nid):
|
||||||
|
if nid in seen:
|
||||||
|
return
|
||||||
|
seen.add(nid)
|
||||||
|
for value in dynprompt.get_node(nid).get("inputs", {}).values():
|
||||||
|
if is_link(value):
|
||||||
|
parent = str(value[0]); feeds.setdefault(parent, []).append(nid); visit(parent)
|
||||||
|
visit(end)
|
||||||
|
starts = [n for n in seen if dynprompt.get_node(n)["class_type"] == "BFSFrameLoopStart"]
|
||||||
|
if len(starts) != 1:
|
||||||
|
raise ValueError("Connect exactly one BFS Frame Loop Start upstream of this Frame Loop End")
|
||||||
|
start = starts[0]; body = {start}; pending = [start]
|
||||||
|
while pending:
|
||||||
|
for child in feeds.get(pending.pop(), []):
|
||||||
|
if child not in body:
|
||||||
|
body.add(child); pending.append(child)
|
||||||
|
graph = GraphBuilder()
|
||||||
|
clones = {n: graph.node(dynprompt.get_node(n)["class_type"], "Next" if n == end else n) for n in body}
|
||||||
|
for nid, node in clones.items():
|
||||||
|
node.set_override_display_id(dynprompt.get_display_node_id(nid))
|
||||||
|
for key, value in dynprompt.get_node(nid).get("inputs", {}).items():
|
||||||
|
node.set_input(key, clones[str(value[0])].out(value[1]) if is_link(value) and str(value[0]) in body else value)
|
||||||
|
clones[start].set_input("loop_state", state)
|
||||||
|
return {"result": tuple(clones[end].out(i) for i in range(2)), "expand": graph.finalize()}
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {"BFSFrameLoopStart": BFSFrameLoopStart, "BFSFrameLoopEnd": BFSFrameLoopEnd}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {"BFSFrameLoopStart": "BFS Frame Loop Start", "BFSFrameLoopEnd": "BFS Frame Loop End"}
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
"""Aligned image guide for the native MiniMax H3 Image reference stream."""
|
||||||
|
import copy
|
||||||
|
|
||||||
|
import comfy.utils
|
||||||
|
from comfy_api.latest import io
|
||||||
|
|
||||||
|
MARKER = 'bfs_h3_image_guide'
|
||||||
|
|
||||||
|
|
||||||
|
def align_guide_layout(layout, refs):
|
||||||
|
result = copy.copy(layout)
|
||||||
|
result.position_ids = layout.position_ids.clone()
|
||||||
|
target = next((a, b) for a, b, kind in layout.segments if kind == 'video')
|
||||||
|
image_refs = iter(ref for ref in refs if ref['kind'] == 'image')
|
||||||
|
for a, b, kind in layout.segments:
|
||||||
|
if kind != 'ref_img':
|
||||||
|
continue
|
||||||
|
ref = next(image_refs)
|
||||||
|
if ref.get(MARKER):
|
||||||
|
ta, tb = target
|
||||||
|
if b - a != tb - ta:
|
||||||
|
raise ValueError('H3 Image guide must match the target canvas')
|
||||||
|
result.position_ids[a:b] = layout.position_ids[ta:tb]
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def patch_image_guide(model):
|
||||||
|
patched = model.clone()
|
||||||
|
if patched.model_options.get(MARKER):
|
||||||
|
return patched
|
||||||
|
previous = patched.model_options.get('model_function_wrapper')
|
||||||
|
|
||||||
|
def wrapper(model_function, args):
|
||||||
|
conds = args['c']
|
||||||
|
payload = conds.get('minimax_payload', {})
|
||||||
|
refs = payload.get('refs', [])
|
||||||
|
if any(ref.get(MARKER) for ref in refs):
|
||||||
|
layout = align_guide_layout(payload['layout'], refs)
|
||||||
|
args = dict(args, c=dict(conds, minimax_payload=dict(payload, layout=layout)))
|
||||||
|
if previous is not None:
|
||||||
|
return previous(model_function, args)
|
||||||
|
return model_function(args['input'], args['timestep'], **args['c'])
|
||||||
|
|
||||||
|
patched.set_model_unet_function_wrapper(wrapper)
|
||||||
|
patched.model_options[MARKER] = True
|
||||||
|
return patched
|
||||||
|
|
||||||
|
|
||||||
|
class BFSH3ImageAddGuide(io.ComfyNode):
|
||||||
|
@classmethod
|
||||||
|
def define_schema(cls):
|
||||||
|
return io.Schema(node_id='BFSH3ImageAddGuide', display_name='Add Guide for MiniMax H3 Image (BFS)',
|
||||||
|
category='BFS/MiniMax H3',
|
||||||
|
description='Experimental aligned guide for H3 Image. Uses a frozen image reference with target-aligned positions. Connect the returned MODEL and positive to the sampler; use the native H3 Image latent.',
|
||||||
|
inputs=[io.Model.Input('model'), io.Conditioning.Input('positive'), io.Vae.Input('vae'),
|
||||||
|
io.Image.Input('guide'), io.Latent.Input('latent')],
|
||||||
|
outputs=[io.Model.Output(), io.Conditioning.Output(display_name='positive')])
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def execute(cls, model, positive, vae, guide, latent):
|
||||||
|
samples = latent['samples']
|
||||||
|
if samples.is_nested or samples.ndim != 5 or samples.shape[1:3] != (24, 1):
|
||||||
|
raise ValueError('Use the LATENT from MiniMax H3 Image Conditioning or Empty MiniMax H3 Image Latent')
|
||||||
|
if model.model.diffusion_model.__class__.__name__ != 'MiniMaxH3ImageModel':
|
||||||
|
raise ValueError('This guide requires the native MiniMax H3 Image model')
|
||||||
|
h, w = samples.shape[-2:]
|
||||||
|
pixels = comfy.utils.common_upscale(guide[:1, ..., :3].movedim(-1, 1), w * 16, h * 16, 'bicubic', 'center').movedim(1, -1)
|
||||||
|
z = vae.encode(pixels)
|
||||||
|
if z.ndim != 5 or tuple(z.shape[1:]) != (24, 1, h, w):
|
||||||
|
raise ValueError('Guide VAE must produce native H3 Image latents matching the target')
|
||||||
|
guide_ref = {'kind': 'image', 'latent_h': h, 'latent_w': w, 'latent': z, MARKER: True}
|
||||||
|
out = []
|
||||||
|
for cond, options in positive:
|
||||||
|
refs = [r for r in options.get('minimax_refs', []) if not r.get(MARKER)]
|
||||||
|
updated = dict(options, minimax_refs=refs + [guide_ref])
|
||||||
|
out.append([cond, updated])
|
||||||
|
return io.NodeOutput(patch_image_guide(model), out)
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {'BFSH3ImageAddGuide': BFSH3ImageAddGuide}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {'BFSH3ImageAddGuide': 'Add Guide for MiniMax H3 Image (BFS)'}
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
"""Native MiniMax H3 Image conditioning with color-marked character references."""
|
||||||
|
from comfy_api.latest import io
|
||||||
|
from comfy_extras.nodes_minimax_h3 import MiniMaxH3ImageConditioning
|
||||||
|
|
||||||
|
COLORS = ('red', 'blue', 'green')
|
||||||
|
MODES = ('normal map guide', 'DWPose guide', 'translucent overlay guide', 'outline guide')
|
||||||
|
|
||||||
|
|
||||||
|
def image_swap_prompt(count, mode):
|
||||||
|
if mode not in MODES:
|
||||||
|
raise ValueError(f'Unknown guide mode: {mode}')
|
||||||
|
if not 1 <= count <= len(COLORS):
|
||||||
|
raise ValueError('Connect one to three character reference images')
|
||||||
|
marks = {
|
||||||
|
'normal map guide': ('normal map', 'each subject to replace is marked with its surface-normal map colorized in one solid color (its 3D surface, without its real colors)'),
|
||||||
|
'DWPose guide': ('pose skeleton', 'each subject to replace is shown as its pose skeleton (body, hands and face) drawn in one color'),
|
||||||
|
'translucent overlay guide': ('translucent overlay', 'each subject to replace is identified by a translucent color overlay; the appearance beneath the tint is the source appearance to replace'),
|
||||||
|
'outline guide': ('outline', 'each subject to replace is identified by a thin colored outline around that subject'),
|
||||||
|
}
|
||||||
|
mark, explanation = marks[mode]
|
||||||
|
definitions = '\n'.join(
|
||||||
|
f'<Subject {j}> is the subject marked by the {COLORS[j-1]} {mark} in <Picture 1>; its appearance comes from <Picture {j+1}>, its position, pose, head orientation, gaze and facial expression from the {COLORS[j-1]} mark.'
|
||||||
|
for j in range(1, count+1))
|
||||||
|
replacements = ', '.join(f'the {COLORS[j-1]} subject replaced by <Picture {j+1}>' for j in range(1, count+1))
|
||||||
|
retention = '\n'.join(
|
||||||
|
f'<Subject {j}>: attribute_transfer - keeps the position, pose, head orientation, gaze and facial expression of the {COLORS[j-1]} mark and takes the face, hair, body and whole outfit of <Picture {j+1}>.'
|
||||||
|
for j in range(1, count+1))
|
||||||
|
return (
|
||||||
|
'character_swap:\nsubject_definitions:\n'
|
||||||
|
f'<Picture 1> is the source of the edit. In it, {explanation}; the mark says who changes, not where the new look must end, and everything else in the guide is the original image.\n'
|
||||||
|
f'{definitions}\n'
|
||||||
|
f'summary: [image editing + reference generation] The target image is <Picture 1> with {replacements}, following each mark\'s pose and expression. '
|
||||||
|
'Every other person, the layout, the background, lighting, camera perspective and framing stay exactly as in <Picture 1>. '
|
||||||
|
'Keep held objects and foreground occluders in their original positions, shapes and colors; they are separate from the subjects being replaced. '
|
||||||
|
'Remove the normal-map colors, skeletons, overlays and outlines from the final image.\n'
|
||||||
|
f'retention_analysis:\n{retention}\n'
|
||||||
|
'Everyone else, held objects, foreground occluders, the layout and the background: fully_preserved.')
|
||||||
|
|
||||||
|
|
||||||
|
class BFSH3ImageCharacterSwapConditioning(io.ComfyNode):
|
||||||
|
@classmethod
|
||||||
|
def define_schema(cls):
|
||||||
|
return io.Schema(
|
||||||
|
node_id='BFSH3ImageCharacterSwapConditioning',
|
||||||
|
display_name='BFS H3 Image Character Swap Conditioning',
|
||||||
|
category='BFS/MiniMax H3',
|
||||||
|
description='Native H3 Image character swap: the marked guide is Picture 1; red, blue and green subjects take their appearance from Pictures 2, 3 and 4. Produces native image reference latents and a still-image prompt.',
|
||||||
|
inputs=[
|
||||||
|
io.Clip.Input('clip'), io.Vae.Input('vae'),
|
||||||
|
io.Image.Input('guide_image', tooltip='Marked hybrid image from BFS Video Segment Picker; becomes Picture 1.'),
|
||||||
|
io.Autogrow.Input('references', template=io.Autogrow.TemplatePrefix(
|
||||||
|
input=io.Image.Input('reference', tooltip='Character appearance in color order: red, blue, green; becomes Picture 2, 3, 4.'),
|
||||||
|
prefix='reference_', min=1, max=3)),
|
||||||
|
io.Combo.Input('mode', options=MODES, default=MODES[0]),
|
||||||
|
io.Int.Input('width', default=0, min=0, max=16384, step=32, tooltip='0 uses the native H3 Image sizing based on the guide.'),
|
||||||
|
io.Int.Input('height', default=0, min=0, max=16384, step=32),
|
||||||
|
io.String.Input('prompt_override', default='', multiline=True, optional=True,
|
||||||
|
tooltip='Optional complete replacement prompt. Guide remains Picture 1; character references start at Picture 2.'),
|
||||||
|
],
|
||||||
|
outputs=[io.Conditioning.Output(display_name='positive'), io.Latent.Output(display_name='latent'),
|
||||||
|
io.String.Output(display_name='prompt')])
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def execute(cls, clip, vae, guide_image, references, mode=MODES[0], width=0, height=0, prompt_override=''):
|
||||||
|
names = sorted(references, key=lambda name: int(name.rsplit('_', 1)[-1]))
|
||||||
|
pictures = [references[name] for name in names if references[name] is not None]
|
||||||
|
prompt = prompt_override.strip() or image_swap_prompt(len(pictures), mode)
|
||||||
|
images = {'image_1': guide_image}
|
||||||
|
images.update({f'image_{j}': picture for j, picture in enumerate(pictures, 2)})
|
||||||
|
positive, latent = MiniMaxH3ImageConditioning.execute(
|
||||||
|
clip=clip, vae=vae, prompt=prompt, width=width, height=height, images=images).args[:2]
|
||||||
|
return io.NodeOutput(positive, latent, prompt)
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {'BFSH3ImageCharacterSwapConditioning': BFSH3ImageCharacterSwapConditioning}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {'BFSH3ImageCharacterSwapConditioning': 'BFS H3 Image Character Swap Conditioning'}
|
||||||
@@ -0,0 +1,419 @@
|
|||||||
|
"""BFS H3 Character Swap Conditioning: MiniMax H3 Character Swap where the guide says who is replaced.
|
||||||
|
|
||||||
|
Five ways to mark the subjects (`mode`):
|
||||||
|
- "keyframe pins": the source video as an aligned guide (Add Guide at frame 0) plus one Add Guide per edited keyframe
|
||||||
|
at its own frame index (frames of the result with the new people in place, e.g. from BFS Keyframe Planner).
|
||||||
|
- "normal map guide": the guide video with each subject to replace drawn as its normal map in gray detail, tinted with
|
||||||
|
the subject's color (1 red, 2 blue, 3 green = Picture 1..3).
|
||||||
|
- "DWPose guide": the guide video with each subject erased to gray and drawn as its DWPose skeleton (body, hands and
|
||||||
|
face, with the eyes and the mouth opening) in the subject's color.
|
||||||
|
- "translucent overlay guide": the source video with each subject's mask tinted in its color at the Picker's
|
||||||
|
mark_opacity, so the original pose and motion remain visible under the tint.
|
||||||
|
- "outline guide": the source video with a thin colored contour inside each subject's mask; the scene and held
|
||||||
|
objects remain visible.
|
||||||
|
The marked guides come from BFS Video Segment Picker (`hybrid` output). The Pictures are native references and the
|
||||||
|
prompt is written from the mode and the number of Pictures (or prompt_override).
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
|
||||||
|
import torch
|
||||||
|
|
||||||
|
GRAY = 128.0 / 255.0 # background of the keyframe_video preview
|
||||||
|
MAX_KEYFRAMES = 8
|
||||||
|
MAX_PICTURES = 3
|
||||||
|
PINS, NORMALS, POSE, OVERLAY, OUTLINE = ("keyframe pins", "normal map guide", "DWPose guide",
|
||||||
|
"translucent overlay guide", "outline guide")
|
||||||
|
MODES = [PINS, NORMALS, POSE, OVERLAY, OUTLINE]
|
||||||
|
SUBJECT_COLORS = ["red", "blue", "green"]
|
||||||
|
|
||||||
|
|
||||||
|
def pose_caption(k):
|
||||||
|
"""The DWPose guide prompt (k Pictures)."""
|
||||||
|
C = SUBJECT_COLORS
|
||||||
|
defs = '\n'.join(f'<Subject {j}> is the subject marked by the {C[j - 1]} pose skeleton in the guide video; its appearance comes '
|
||||||
|
f'from <Picture {j}>, its pose, motion, facial expression and mouth movement from the {C[j - 1]} skeleton.'
|
||||||
|
for j in range(1, k + 1))
|
||||||
|
ret = '\n'.join(f'<Subject {j}>: attribute_transfer - keeps the pose, motion, expression and lip movement of the {C[j - 1]} skeleton '
|
||||||
|
f'and takes the face, hair, body shape and whole outfit of <Picture {j}> in every frame.' for j in range(1, k + 1))
|
||||||
|
who = ', '.join(f'the {C[j - 1]} subject replaced by <Picture {j}>' for j in range(1, k + 1))
|
||||||
|
return ('character_swap:\nsubject_definitions:\nThe guide video is the source of the edit. In it, each subject to replace is erased to '
|
||||||
|
'plain gray and shown only as its pose skeleton (body, hands and face, with the eyes and the mouth opening) drawn in one '
|
||||||
|
'color; the mark gives the pose and the expression, not the body shape, and everything else in the guide is the real video.\n'
|
||||||
|
+ defs + f'\nsummary: [video editing + reference generation] The target video is the guide video with {who}, from the first '
|
||||||
|
'frame to the last, following each skeleton\'s pose, expression and mouth. Every other person, the layout, the backgrounds, '
|
||||||
|
'the camera and the motion stay exactly as in the guide video.\nretention_analysis:\n' + ret +
|
||||||
|
'\nEveryone else, the layout and the backgrounds: fully_preserved.')
|
||||||
|
|
||||||
|
|
||||||
|
def normals_caption(k):
|
||||||
|
"""The normal map guide prompt (k Pictures)."""
|
||||||
|
C = SUBJECT_COLORS
|
||||||
|
defs = '\n'.join(f'<Subject {j}> is the subject marked by the {C[j - 1]} normal map in the guide video; its appearance comes '
|
||||||
|
f'from <Picture {j}>, its pose and motion from the {C[j - 1]} mark.' for j in range(1, k + 1))
|
||||||
|
ret = '\n'.join(f'<Subject {j}>: attribute_transfer - keeps the pose and motion of the {C[j - 1]} mark and takes the face, '
|
||||||
|
f'hair, body and whole outfit of <Picture {j}> in every frame.' for j in range(1, k + 1))
|
||||||
|
who = ', '.join(f'the {C[j - 1]} subject replaced by <Picture {j}>' for j in range(1, k + 1))
|
||||||
|
return ('character_swap:\nsubject_definitions:\nThe guide video is the source of the edit. In it, each subject to replace is marked '
|
||||||
|
'with its surface-normal map colorized in one solid color (its 3D surface, without its real colors); the mark says who '
|
||||||
|
'changes, not where the new look must end, and everything else in the guide is the real video.\n' + defs +
|
||||||
|
f'\nsummary: [video editing + reference generation] The target video is the guide video with {who}, from the first frame '
|
||||||
|
'to the last, following each mark\'s pose and motion. Every other person, the layout, the backgrounds, the camera and the '
|
||||||
|
'motion stay exactly as in the guide video. Keep held objects and foreground occluders in their original '
|
||||||
|
'positions, shapes and colors; they are separate from the subject being replaced.\nretention_analysis:\n' + ret +
|
||||||
|
'\nEveryone else, held objects, foreground occluders, the layout and the backgrounds: fully_preserved.')
|
||||||
|
|
||||||
|
|
||||||
|
def overlay_caption(k):
|
||||||
|
"""Prompt for the Picker's translucent color overlay, where source appearance is still visible underneath."""
|
||||||
|
C = SUBJECT_COLORS
|
||||||
|
defs = '\n'.join(f'<Subject {j}> is the person under the translucent {C[j - 1]} overlay in the guide video. '
|
||||||
|
f'Replace that person with <Picture {j}> while keeping their position, pose, motion and timing.'
|
||||||
|
for j in range(1, k + 1))
|
||||||
|
ret = '\n'.join(f'<Subject {j}>: attribute_transfer - use the face, hair, body and full outfit of <Picture {j}>; '
|
||||||
|
f'use the {C[j - 1]} marked person only for position, pose, motion and timing.'
|
||||||
|
for j in range(1, k + 1))
|
||||||
|
return ('character_swap:\nsubject_definitions:\nThe guide is the original video with a translucent color overlay on each '
|
||||||
|
'person to replace. The overlay identifies the person; the original appearance visible beneath it is not the target '
|
||||||
|
'appearance. The tint must disappear in the result.\n' + defs +
|
||||||
|
'\nsummary: [video editing + reference generation] Replace every marked person throughout the video with their '
|
||||||
|
'matching Picture. Keep their actions, camera motion, all unmarked people and the background as in the guide. '
|
||||||
|
'Do not render the colored overlays.\nretention_analysis:\n' + ret +
|
||||||
|
'\nUnmarked people and background: fully_preserved.')
|
||||||
|
|
||||||
|
|
||||||
|
def outline_caption(k):
|
||||||
|
"""Prompt for a person contour that leaves the original scene and interactions unobscured."""
|
||||||
|
C = SUBJECT_COLORS
|
||||||
|
defs = '\n'.join(f'<Subject {j}> is the person inside the {C[j - 1]} outline in the guide video. '
|
||||||
|
f'Take their new appearance from <Picture {j}> while retaining their position, pose, motion and timing.'
|
||||||
|
for j in range(1, k + 1))
|
||||||
|
ret = '\n'.join(f'<Subject {j}>: attribute_transfer - use the face, hair, body and full outfit of <Picture {j}>; '
|
||||||
|
f'keep the movement and interactions of the {C[j - 1]} outlined person.'
|
||||||
|
for j in range(1, k + 1))
|
||||||
|
return ('character_swap:\nsubject_definitions:\nThe guide is the original video with a thin colored outline on each '
|
||||||
|
'person to replace. The outline is only an identity marker and must disappear in the result. The source person '
|
||||||
|
'inside the outline provides pose and motion, while their new appearance comes from the matching Picture.\n' + defs +
|
||||||
|
'\nsummary: [video editing + reference generation] Replace only the outlined people throughout the video. '
|
||||||
|
'Preserve the background, other people, and all held or interacting objects with their original colors, positions, '
|
||||||
|
'shapes and contact with the subjects. Preserve foreground occlusions and the camera motion. Do not render the '
|
||||||
|
'colored outlines.\nretention_analysis:\n' + ret +
|
||||||
|
'\nBackground, other people, held objects and occlusions: fully_preserved.')
|
||||||
|
|
||||||
|
|
||||||
|
def _tail(k):
|
||||||
|
subs = '\n'.join(f'<Subject {j}> is the person of the guide video who appears with the look of <Picture {j}> in the '
|
||||||
|
f'keyframes; the appearance comes from <Picture {j}>, the motion from the guide video.' for j in range(1, k + 1))
|
||||||
|
ret = '\n'.join(f'<Subject {j}>: attribute_transfer - keeps the motion of the guide video and takes the face, hair, body and '
|
||||||
|
f'whole outfit of <Picture {j}> in every frame.' for j in range(1, k + 1))
|
||||||
|
summary = (f'\nsummary: [video editing + keyframe completion + reference generation] The target video is the guide video with '
|
||||||
|
f'{k} {"people" if k > 1 else "person"} replaced: exactly the people that differ between the keyframes and the guide video, '
|
||||||
|
f'replaced by {", ".join(f"<Picture {j}>" for j in range(1, k + 1))} from the first frame to the last. Every other '
|
||||||
|
'person, the layout, the backgrounds, the camera and the motion stay exactly as in the guide video.\n'
|
||||||
|
'retention_analysis:\n' + ret + '\nEveryone else, the layout and the backgrounds: fully_preserved.')
|
||||||
|
return subs, summary
|
||||||
|
|
||||||
|
|
||||||
|
def pins_caption(k, times):
|
||||||
|
"""The keyframe pins prompt (k Pictures, keyframes at `times` seconds)."""
|
||||||
|
when = ', '.join(f'{t:.1f}s' for t in times); n = len(times)
|
||||||
|
subs, summary = _tail(k)
|
||||||
|
return ('character_swap:\nsubject_definitions:\nThe guide video is the source of the edit. '
|
||||||
|
f'{n} keyframe{"s" if n > 1 else ""} of the target video {"are" if n > 1 else "is"} pinned at {when}; at those times '
|
||||||
|
'the target video matches the keyframes exactly.\n' + subs + summary)
|
||||||
|
|
||||||
|
|
||||||
|
def _fit(image, h, w):
|
||||||
|
"""[1,H,W,C] -> [h,w,3], center crop to the guide's aspect, then resize."""
|
||||||
|
import comfy.utils
|
||||||
|
x = image[:1, ..., :3].movedim(-1, 1)
|
||||||
|
x = comfy.utils.common_upscale(x, w, h, "lanczos", "center")
|
||||||
|
return x.movedim(1, -1)[0].clamp(0, 1)
|
||||||
|
|
||||||
|
|
||||||
|
class BFSH3KeyframeConditioning:
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
opt = {f"picture_{j}": ("IMAGE", {"tooltip": f"Subject {j} ({SUBJECT_COLORS[j - 1]} mark in the guide modes)."})
|
||||||
|
for j in range(2, MAX_PICTURES + 1)}
|
||||||
|
opt.update({
|
||||||
|
"model": ("MODEL", {"tooltip": "keyframe pins: connect to get it back with keyframe_until applied. Guide modes: passed through."}),
|
||||||
|
"keyframes": ("IMAGE", {"tooltip": "keyframe pins: edited frames, one per keyframe (BFS Keyframe Planner keyframes)."}),
|
||||||
|
"keyframe_spec": ("STRING", {"forceInput": True, "tooltip": "keyframe pins: BFS Keyframe Planner keyframe_spec (the frame of every keyframe)."}),
|
||||||
|
})
|
||||||
|
for i in range(1, MAX_KEYFRAMES + 1):
|
||||||
|
opt[f"keyframe_{i}"] = ("IMAGE", {"tooltip": f"keyframe pins: edited frame {i}, a full frame of the RESULT (swapped person in place)."})
|
||||||
|
opt[f"time_{i}"] = ("FLOAT", {"default": float(i - 1), "min": 0.0, "max": 600.0, "step": 0.05,
|
||||||
|
"tooltip": f"keyframe pins: seconds into the guide where keyframe {i} sits."})
|
||||||
|
return {"required": {
|
||||||
|
"clip": ("CLIP", {"tooltip": "MiniMax H3's text encoder (Qwen3-VL)."}),
|
||||||
|
"vae": ("VAE",),
|
||||||
|
"guide_video": ("IMAGE", {"tooltip": "keyframe pins: the prepared 24-fps source video. Guide modes: the marked guide "
|
||||||
|
"(BFS Video Segment Picker `hybrid`)."}),
|
||||||
|
"width": ("INT", {"forceInput": True}), "height": ("INT", {"forceInput": True}), "length": ("INT", {"forceInput": True}),
|
||||||
|
"picture_1": ("IMAGE", {"tooltip": "Subject 1 (red mark in the guide modes)."}),
|
||||||
|
"mode": (MODES, {"default": PINS, "tooltip": "How the subjects to replace are marked: edited keyframes pinned at their "
|
||||||
|
"frames, or a guide with each subject drawn as a colored normal map, "
|
||||||
|
"DWPose skeleton, translucent color overlay, or colored outline "
|
||||||
|
"(BFS Video Segment Picker)."}),
|
||||||
|
"keyframe_until": ("FLOAT", {"default": 0.7, "min": 0.0, "max": 1.0, "step": 0.01,
|
||||||
|
"tooltip": "keyframe pins only, on the returned model: the guides condition every step whose "
|
||||||
|
"sigma is at least this; below it only the Pictures remain. 0.5 = more faithful to "
|
||||||
|
"the source, 0 = guides in every step. The guide modes keep the guide on every step."}),
|
||||||
|
"keyframe_until_drops": (["keyframes only", "all guides"], {"default": "keyframes only",
|
||||||
|
"tooltip": "keyframe pins only: below keyframe_until drop only the keyframes (the source video "
|
||||||
|
"keeps guiding every step) or every guide."}),
|
||||||
|
"prompt_override": ("STRING", {"multiline": True, "default": "", "tooltip": "Non-empty: used instead of the generated prompt."}),
|
||||||
|
}, "optional": opt}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("MODEL", "CONDITIONING", "LATENT", "STRING", "IMAGE", "IMAGE")
|
||||||
|
RETURN_NAMES = ("model", "positive", "latent", "prompt", "keyframe_video", "pictures")
|
||||||
|
OUTPUT_IS_LIST = (False, False, False, False, False, True)
|
||||||
|
OUTPUT_TOOLTIPS = ("", "", "", "The prompt used.", "keyframe pins: the keyframes at their frames on gray. Guide modes: the guide.",
|
||||||
|
"The Pictures in order (Picture 1..4, any sizes), e.g. for BFS Video Compare's reference.")
|
||||||
|
FUNCTION = "encode"
|
||||||
|
CATEGORY = "BFS/MiniMax H3"
|
||||||
|
DESCRIPTION = ("MiniMax H3 Character Swap conditioning. keyframe pins: the source video plus one Add Guide per edited "
|
||||||
|
"keyframe. Guide modes use the BFS Video Segment Picker hybrid output: normal map, DWPose, translucent "
|
||||||
|
"overlay or colored outline. Red, blue and green marks correspond to Pictures 1, 2 and 3. "
|
||||||
|
"The node writes the matching prompt automatically.")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _collect(kw, fps=24.0):
|
||||||
|
"""[(frame, image[1,H,W,C])] from the Planner batch + spec and/or the keyframe_N / time_N inputs."""
|
||||||
|
out = []
|
||||||
|
batch, spec = kw.get("keyframes"), kw.get("keyframe_spec")
|
||||||
|
if batch is not None:
|
||||||
|
frames = json.loads(spec).get("frames", []) if spec else []
|
||||||
|
if len(frames) != len(batch):
|
||||||
|
raise ValueError(f"{len(batch)} keyframes but keyframe_spec lists {len(frames)} frames: connect the Planner's keyframe_spec.")
|
||||||
|
out += [(int(f), batch[i:i + 1]) for i, f in enumerate(frames)]
|
||||||
|
for i in range(1, MAX_KEYFRAMES + 1):
|
||||||
|
if kw.get(f"keyframe_{i}") is not None:
|
||||||
|
out.append((round(float(kw.get(f"time_{i}", 0.0)) * fps), kw[f"keyframe_{i}"]))
|
||||||
|
return sorted(out, key=lambda x: x[0])
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _free_gpu():
|
||||||
|
"""Unload TE, VAE, and cached preprocessor models (e.g. DWPose) from GPU so MiniMaxH3 has full VRAM."""
|
||||||
|
try:
|
||||||
|
import gc
|
||||||
|
import sys
|
||||||
|
import comfy.model_management as mm
|
||||||
|
mm.unload_all_models()
|
||||||
|
for k, mod in list(sys.modules.items()):
|
||||||
|
if "dwpose" not in k or mod is None:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
dw = getattr(mod, "global_cached_dwpose", None)
|
||||||
|
if dw is not None:
|
||||||
|
dw.det = None
|
||||||
|
dw.pose = None
|
||||||
|
dw.det_filename = None
|
||||||
|
dw.pose_filename = None
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
pass
|
||||||
|
gc.collect()
|
||||||
|
mm.soft_empty_cache(True)
|
||||||
|
except Exception as e: # noqa: BLE001
|
||||||
|
print(f"[BFS H3 keyframe] could not free GPU models: {e!r}", flush=True)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _patch_vram_estimate(model):
|
||||||
|
"""Ensure MiniMaxH3 accounts for guide_video/keyframes/refs in ComfyUI's VRAM estimate so it doesn't OOM."""
|
||||||
|
if model is None:
|
||||||
|
return None
|
||||||
|
inner = getattr(model, "model", None)
|
||||||
|
if inner is None:
|
||||||
|
return model
|
||||||
|
import math
|
||||||
|
import types
|
||||||
|
|
||||||
|
def _extra_conds_shapes(_self, **kwargs):
|
||||||
|
extra = 0
|
||||||
|
payload = kwargs.get("minimax_payload", None)
|
||||||
|
if hasattr(payload, "cond") and isinstance(payload.cond, dict):
|
||||||
|
payload = payload.cond
|
||||||
|
keyframes = kwargs.get("minimax_keyframes", None)
|
||||||
|
if keyframes is None and isinstance(payload, dict):
|
||||||
|
keyframes = payload.get("keyframes", None)
|
||||||
|
refs = kwargs.get("minimax_refs", None)
|
||||||
|
if refs is None and isinstance(payload, dict):
|
||||||
|
refs = payload.get("refs", None)
|
||||||
|
for item in (keyframes or []) + (refs or []):
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
lat = item.get("latent", None)
|
||||||
|
if lat is not None and hasattr(lat, "shape"):
|
||||||
|
extra += math.prod(lat.shape[1:])
|
||||||
|
aud = item.get("audio_latent", None)
|
||||||
|
if aud is not None and hasattr(aud, "shape"):
|
||||||
|
extra += math.prod(aud.shape[1:])
|
||||||
|
cross_attn = kwargs.get("cross_attn", kwargs.get("c_crossattn", None))
|
||||||
|
if hasattr(cross_attn, "cond"):
|
||||||
|
cross_attn = cross_attn.cond
|
||||||
|
if cross_attn is not None and hasattr(cross_attn, "shape") and len(cross_attn.shape) >= 2:
|
||||||
|
extra += int(cross_attn.shape[1]) * 96
|
||||||
|
return {"minimax_cond": [1, 1, extra]} if extra > 0 else {}
|
||||||
|
|
||||||
|
cond_keys = tuple(getattr(inner, "memory_usage_factor_conds", ()) or ())
|
||||||
|
if "minimax_cond" not in cond_keys:
|
||||||
|
inner.memory_usage_factor_conds = cond_keys + ("minimax_cond",)
|
||||||
|
inner.extra_conds_shapes = types.MethodType(_extra_conds_shapes, inner)
|
||||||
|
if getattr(inner, "memory_usage_factor", 0.0) < 0.135:
|
||||||
|
inner.memory_usage_factor = 0.135
|
||||||
|
|
||||||
|
dm = getattr(inner, "diffusion_model", None)
|
||||||
|
if dm is not None and not getattr(dm, "_bfs_low_peak_patched", False):
|
||||||
|
try:
|
||||||
|
import comfy.ldm.minimax.model as mm_mod
|
||||||
|
import comfy.model_management as cmm
|
||||||
|
import comfy.quant_ops
|
||||||
|
from comfy.ldm.modules.attention import AttentionTensorContainer, optimized_attention
|
||||||
|
|
||||||
|
def _attn_forward(self, x, rope_freqs=None, transformer_options={}):
|
||||||
|
s = x.shape[0]
|
||||||
|
dev = x.device
|
||||||
|
q, k, v = self.qkv_proj(x).split(self.heads * self.head_dim, dim=-1)
|
||||||
|
del x
|
||||||
|
v = v.view(s, self.heads, self.head_dim)
|
||||||
|
if rope_freqs is not None:
|
||||||
|
q = q.view(1, s, self.heads, self.head_dim)
|
||||||
|
k = k.view(1, s, self.heads, self.head_dim)
|
||||||
|
qw = cmm.cast_to(self.q_norm.weight, device=dev)
|
||||||
|
kw = cmm.cast_to(self.k_norm.weight, device=dev)
|
||||||
|
rot = rope_freqs.shape[-3] * 2
|
||||||
|
if cmm.in_training:
|
||||||
|
q, k = comfy.quant_ops.ck.rms_rope_split_half(
|
||||||
|
q, k, rope_freqs, qw, kw, epsilon=self.q_norm.eps, rot_dim=rot)
|
||||||
|
else:
|
||||||
|
comfy.quant_ops.ck.rms_rope_split_half_(
|
||||||
|
q, k, rope_freqs, qw, kw, epsilon=self.q_norm.eps, rot_dim=rot)
|
||||||
|
del qw, kw
|
||||||
|
q, k = q[0], k[0]
|
||||||
|
else:
|
||||||
|
q = self.q_norm(q.view(s, self.heads, self.head_dim))
|
||||||
|
k = self.k_norm(k.view(s, self.heads, self.head_dim))
|
||||||
|
q = AttentionTensorContainer(q.transpose(0, 1).unsqueeze(0))
|
||||||
|
k = AttentionTensorContainer(k.transpose(0, 1).unsqueeze(0))
|
||||||
|
v = AttentionTensorContainer(v.transpose(0, 1).unsqueeze(0))
|
||||||
|
out = optimized_attention(
|
||||||
|
q, k, v, self.heads, preferred_attention=self.comfy_attention,
|
||||||
|
mask=None, skip_reshape=True, skip_output_reshape=True,
|
||||||
|
transformer_options=transformer_options,
|
||||||
|
)
|
||||||
|
del q, k, v
|
||||||
|
out = out.transpose(1, 2).reshape(s, self.heads * self.head_dim)
|
||||||
|
return self.out_proj(out)
|
||||||
|
|
||||||
|
def _block_forward(self, x, t_emb, mod_segments, rope_freqs, transformer_options={}, attention=None):
|
||||||
|
attention = self.attn if attention is None else attention
|
||||||
|
shift_msa, scale_msa, gate_msa, shift_mlp, scale_mlp, gate_mlp = self.adaln_proj(t_emb)
|
||||||
|
attn_out = attention(
|
||||||
|
mm_mod._mod_scale_shift(self.norm1(x), shift_msa, scale_msa, mod_segments),
|
||||||
|
rope_freqs=rope_freqs, transformer_options=transformer_options,
|
||||||
|
)
|
||||||
|
del shift_msa, scale_msa
|
||||||
|
x = mm_mod._mod_gate(x, gate_msa, attn_out, mod_segments)
|
||||||
|
del gate_msa, attn_out
|
||||||
|
mlp_out = self.mlp(mm_mod._mod_scale_shift(self.norm2(x), shift_mlp, scale_mlp, mod_segments))
|
||||||
|
del shift_mlp, scale_mlp
|
||||||
|
return mm_mod._mod_gate(x, gate_mlp, mlp_out, mod_segments)
|
||||||
|
|
||||||
|
for blk in getattr(dm, "blocks", ()):
|
||||||
|
blk.forward = types.MethodType(_block_forward, blk)
|
||||||
|
if hasattr(blk, "attn"):
|
||||||
|
blk.attn.forward = types.MethodType(_attn_forward, blk.attn)
|
||||||
|
dm._bfs_low_peak_patched = True
|
||||||
|
except Exception as e: # noqa: BLE001
|
||||||
|
print(f"[BFS H3 keyframe] could not patch block peak VRAM: {e!r}", flush=True)
|
||||||
|
return model
|
||||||
|
|
||||||
|
def encode(self, clip, vae, guide_video, width, height, length, picture_1, mode=PINS, keyframe_until=0.7,
|
||||||
|
keyframe_until_drops="keyframes only", prompt_override="", model=None, **kw):
|
||||||
|
if mode not in MODES:
|
||||||
|
raise ValueError(f"Unknown mode {mode!r}: use one of {MODES}.")
|
||||||
|
from comfy_extras.nodes_minimax_h3 import MiniMaxH3AddGuide, MiniMaxH3ReferenceToVideo
|
||||||
|
pictures = [picture_1] + [kw[f"picture_{j}"] for j in range(2, MAX_PICTURES + 1) if kw.get(f"picture_{j}") is not None]
|
||||||
|
kfs = self._collect(kw)
|
||||||
|
if mode != PINS:
|
||||||
|
return self._guide(clip, vae, guide_video, width, height, length, pictures, prompt_override, model, kw, mode=mode)
|
||||||
|
if not kfs:
|
||||||
|
raise ValueError("No keyframes: connect the Planner's keyframes + keyframe_spec, or keyframe_1 / time_1.")
|
||||||
|
if len(guide_video) != length:
|
||||||
|
raise ValueError(f"guide has {len(guide_video)} frames, length is {length}: connect Prepare's length.")
|
||||||
|
h, w = guide_video.shape[1:3]
|
||||||
|
video = torch.full((length, h, w, 3), GRAY, dtype=guide_video.dtype)
|
||||||
|
placed, frames = [], []
|
||||||
|
for f, img in kfs:
|
||||||
|
f = min(length - 1, max(0, int(f)))
|
||||||
|
if f in placed:
|
||||||
|
print(f"[BFS H3 keyframe] two keyframes at frame {f}", flush=True)
|
||||||
|
frame = _fit(img, h, w).to(video.dtype)
|
||||||
|
video[f] = frame
|
||||||
|
placed.append(f); frames.append(frame)
|
||||||
|
times = [f / 24.0 for f in placed]
|
||||||
|
prompt = prompt_override.strip() or pins_caption(len(pictures), times)
|
||||||
|
print("[BFS H3 keyframe] prompt:\n" + prompt, flush=True)
|
||||||
|
try:
|
||||||
|
positive, latent = MiniMaxH3ReferenceToVideo.execute(
|
||||||
|
clip=clip, vae=vae, prompt=prompt, width=width, height=height, length=length, ref_image_size="match",
|
||||||
|
ref_images={f"ref_image_{i}": p for i, p in enumerate(pictures, 1)}).args[:2]
|
||||||
|
self._free_gpu()
|
||||||
|
positive = MiniMaxH3AddGuide.execute(positive=positive, latent=latent, frame_idx=0, vae=vae, image=guide_video).args[0]
|
||||||
|
for f, frame in zip(placed, frames):
|
||||||
|
positive = MiniMaxH3AddGuide.execute(positive=positive, latent=latent, frame_idx=f, vae=vae, image=frame[None]).args[0]
|
||||||
|
finally:
|
||||||
|
self._free_gpu()
|
||||||
|
print(f"[BFS H3 keyframe] {mode}: {len(placed)} keyframes at frames {placed}; {len(pictures)} Pictures", flush=True)
|
||||||
|
return (self._until(model, keyframe_until, keyframe_until_drops.startswith("keyframes")), positive, latent, prompt, video,
|
||||||
|
pictures)
|
||||||
|
|
||||||
|
def _guide(self, clip, vae, guide_video, width, height, length, pictures, prompt_override, model, kw, mode=NORMALS):
|
||||||
|
"""Marked guide (subjects drawn as color-coded marks) as the only aligned guide, every
|
||||||
|
step; the Pictures in color order; no keyframes."""
|
||||||
|
from comfy_extras.nodes_minimax_h3 import MiniMaxH3AddGuide, MiniMaxH3ReferenceToVideo
|
||||||
|
if len(guide_video) != length:
|
||||||
|
raise ValueError(f"guide has {len(guide_video)} frames, length is {length}: connect Prepare's length.")
|
||||||
|
captions = {NORMALS: normals_caption, POSE: pose_caption, OVERLAY: overlay_caption, OUTLINE: outline_caption}
|
||||||
|
prompt = prompt_override.strip() or captions[mode](len(pictures))
|
||||||
|
print("[BFS H3 keyframe] guide prompt:\n" + prompt, flush=True)
|
||||||
|
try:
|
||||||
|
positive, latent = MiniMaxH3ReferenceToVideo.execute(
|
||||||
|
clip=clip, vae=vae, prompt=prompt, width=width, height=height, length=length, ref_image_size="match",
|
||||||
|
ref_images={f"ref_image_{i}": p for i, p in enumerate(pictures, 1)}).args[:2]
|
||||||
|
self._free_gpu()
|
||||||
|
positive = MiniMaxH3AddGuide.execute(positive=positive, latent=latent, frame_idx=0, vae=vae, image=guide_video).args[0]
|
||||||
|
finally:
|
||||||
|
self._free_gpu()
|
||||||
|
# the guide carries the background: it stays in every step (keyframe_until does not apply)
|
||||||
|
return self._patch_vram_estimate(model), positive, latent, prompt, guide_video, pictures
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _until(model, keyframe_until, keep_source=False):
|
||||||
|
"""The model with the guides dropped below sigma keyframe_until (as Viggle Remake's RemakeKeyframeUntil).
|
||||||
|
keep_source: drop only the keyframes; the source video (the first Add Guide, see encode) stays."""
|
||||||
|
if model is None:
|
||||||
|
return None
|
||||||
|
BFSH3KeyframeConditioning._patch_vram_estimate(model)
|
||||||
|
import comfy.patcher_extension
|
||||||
|
until = float(keyframe_until)
|
||||||
|
|
||||||
|
def release(executor, x, timestep, context, transformer_options, minimax_payload=None, **kwargs):
|
||||||
|
p = minimax_payload or {}
|
||||||
|
if p.get("keyframes") and float(timestep.flatten()[0]) / 1000 < until:
|
||||||
|
refs = p.get("refs") or []
|
||||||
|
kept = list(p["keyframes"][:1]) if keep_source else []
|
||||||
|
minimax_payload = dict(p, keyframes=kept or None, layout=None,
|
||||||
|
cond_video_latents=[k["latent"] for k in kept if k.get("latent") is not None] +
|
||||||
|
[r["latent"] for r in refs if "latent" in r],
|
||||||
|
cond_audio_latents=[k["audio_latent"] for k in kept if k.get("audio_latent") is not None] +
|
||||||
|
[r["audio_latent"] for r in refs if r.get("audio_latent") is not None])
|
||||||
|
return executor(x, timestep, context, transformer_options, minimax_payload=minimax_payload, **kwargs)
|
||||||
|
|
||||||
|
m = model.clone()
|
||||||
|
m.add_wrapper_with_key(comfy.patcher_extension.WrappersMP.DIFFUSION_MODEL, "bfs_h3_keyframe_until", release)
|
||||||
|
return m
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {"BFSH3KeyframeConditioning": BFSH3KeyframeConditioning}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {"BFSH3KeyframeConditioning": "BFS H3 Character Swap Conditioning"}
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
"""BFS Keyframe Marks: the tracked person's mask at every keyframe, and the keyframe frames marked for an image edit.
|
||||||
|
|
||||||
|
Pick the person once (BFS Segment Picker on a guide frame), track it over the whole guide (Run SAM3 Video Track ->
|
||||||
|
SAM3 Track to Mask), then this node takes the mask at each keyframe of a BFS Keyframe Planner keyframe_spec and tints
|
||||||
|
that frame (the Colored Mark convention: magenta at 30 % marks the person an image edit replaces). A keyframe whose mask is empty (the track lost the person, e.g. after a camera cut) is reported, so
|
||||||
|
it can be marked again by hand.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
|
||||||
|
import torch
|
||||||
|
import torch.nn.functional as F
|
||||||
|
|
||||||
|
|
||||||
|
def parse_color(text):
|
||||||
|
s = str(text or "").strip().lstrip("#")
|
||||||
|
if len(s) == 3:
|
||||||
|
s = "".join(ch * 2 for ch in s)
|
||||||
|
try:
|
||||||
|
return int(s[0:2], 16), int(s[2:4], 16), int(s[4:6], 16)
|
||||||
|
except (ValueError, IndexError):
|
||||||
|
return 255, 0, 255
|
||||||
|
|
||||||
|
|
||||||
|
def grow(mask, pixels):
|
||||||
|
"""[N,H,W] mask dilated (pixels > 0) or eroded (pixels < 0) by a square of that radius."""
|
||||||
|
if not pixels:
|
||||||
|
return mask
|
||||||
|
r = abs(int(pixels)); k = 2 * r + 1
|
||||||
|
m = mask[:, None].float()
|
||||||
|
if pixels > 0:
|
||||||
|
return F.max_pool2d(m, k, 1, r)[:, 0]
|
||||||
|
return -F.max_pool2d(-m, k, 1, r)[:, 0]
|
||||||
|
|
||||||
|
|
||||||
|
class BFSKeyframeMarks:
|
||||||
|
DESCRIPTION = __doc__.split("\n\n")[0]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {
|
||||||
|
"guide_video": ("IMAGE", {"tooltip": "The guide frames the track was made on (same frames as the Planner's guide)."}),
|
||||||
|
"masks": ("MASK", {"tooltip": "The person to replace: one mask per guide frame, or one per keyframe (Segment Picker batch_masks on the Planner's source_frames)."}),
|
||||||
|
"keyframe_spec": ("STRING", {"forceInput": True, "tooltip": "BFS Keyframe Planner keyframe_spec: which frames are keyframes."}),
|
||||||
|
"mark_color": ("STRING", {"default": "#FF00FF", "tooltip": "Tint of the person in marked_frames (hex)."}),
|
||||||
|
"mark_opacity": ("FLOAT", {"default": 0.3, "min": 0.0, "max": 1.0, "step": 0.05,
|
||||||
|
"tooltip": "How strongly the person is tinted (0.3 = the Colored Mark default)."}),
|
||||||
|
"grow_pixels": ("INT", {"default": 0, "min": -64, "max": 64,
|
||||||
|
"tooltip": "Grow (positive) or shrink (negative) each mask before tinting, e.g. to cover hair."}),
|
||||||
|
"threshold": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.05, "tooltip": "Mask values above this count as the person."}),
|
||||||
|
}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "MASK", "IMAGE", "STRING")
|
||||||
|
RETURN_NAMES = ("marked_frames", "masks", "source_frames", "report")
|
||||||
|
OUTPUT_TOOLTIPS = ("The keyframe frames with the person tinted, one per keyframe (in keyframe_spec order).",
|
||||||
|
"The person's mask at each keyframe.",
|
||||||
|
"The untouched keyframe frames.",
|
||||||
|
"JSON: per keyframe its frame, time, mask area and whether the mask is empty.")
|
||||||
|
FUNCTION = "mark"
|
||||||
|
CATEGORY = "BFS/MiniMax H3"
|
||||||
|
|
||||||
|
def mark(self, guide_video, masks, keyframe_spec, mark_color, mark_opacity, grow_pixels, threshold):
|
||||||
|
spec = json.loads(keyframe_spec or "{}")
|
||||||
|
frames = [int(f) for f in spec.get("frames", [])]
|
||||||
|
if not frames:
|
||||||
|
raise ValueError("keyframe_spec lists no keyframes: add them in BFS Keyframe Planner.")
|
||||||
|
n, h, w = guide_video.shape[:3]
|
||||||
|
if masks.ndim == 2:
|
||||||
|
masks = masks[None]
|
||||||
|
per_keyframe = masks.shape[0] == len(frames) and masks.shape[0] != n # masks made on the Planner's source_frames
|
||||||
|
if not per_keyframe and masks.shape[0] != n:
|
||||||
|
raise ValueError(f"{masks.shape[0]} masks: give one per guide frame ({n}) or one per keyframe ({len(frames)}).")
|
||||||
|
if masks.shape[1:] != (h, w):
|
||||||
|
masks = F.interpolate(masks[:, None].float(), size=(h, w), mode="bilinear", align_corners=False)[:, 0]
|
||||||
|
color = torch.tensor(parse_color(mark_color), dtype=guide_video.dtype) / 255.0
|
||||||
|
fps = float(spec.get("fps", 24.0))
|
||||||
|
src, out, kept, report = [], [], [], []
|
||||||
|
for i, f in enumerate(frames):
|
||||||
|
f = max(0, min(n - 1, f))
|
||||||
|
j = i if per_keyframe else f
|
||||||
|
m = grow((masks[j:j + 1] > threshold).float(), grow_pixels).clamp(0, 1)
|
||||||
|
img = guide_video[f:f + 1, ..., :3]
|
||||||
|
a = (m[0].to(img.dtype) * float(mark_opacity))[..., None]
|
||||||
|
out.append(img * (1 - a) + color * a)
|
||||||
|
src.append(img); kept.append(m)
|
||||||
|
area = float(m.mean())
|
||||||
|
report.append({"frame": f, "time": round(f / fps, 3), "area": round(area, 4), "empty": area < 1e-4})
|
||||||
|
lost = [r["frame"] for r in report if r["empty"]]
|
||||||
|
if len(lost) == len(report):
|
||||||
|
raise ValueError("No person selected: click the person to replace in the Video Segment Picker, then run again.")
|
||||||
|
if lost:
|
||||||
|
print(f"[BFS Keyframe Marks] no mask at keyframe frame(s) {lost}: mark the person there by hand", flush=True)
|
||||||
|
return torch.cat(out), torch.cat(kept), torch.cat(src), json.dumps({"keyframes": report, "lost": lost})
|
||||||
|
|
||||||
|
|
||||||
|
class BFSTrackFromFrame:
|
||||||
|
"""Track a person over the whole guide from the frame they were picked on (not only frame 0): SAM 3 tracks
|
||||||
|
forward from that frame and backward (the frames before it, reversed), and the two halves are joined. With
|
||||||
|
anchor_frame -1 the anchor is the first keyframe of keyframe_spec, so feed the Planner's source_frames to the
|
||||||
|
Segment Picker and its mask lines up."""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {
|
||||||
|
"guide_video": ("IMAGE", {"tooltip": "The guide frames (the same the Planner gets)."}),
|
||||||
|
"sam3_model": ("MODEL", {"tooltip": "SAM 3.1 MODEL (Load Checkpoint with sam3.1_multiplex_fp16.safetensors)."}),
|
||||||
|
"mask": ("MASK", {"tooltip": "The person on the anchor frame (BFS Segment Picker mask)."}),
|
||||||
|
"anchor_frame": ("INT", {"default": -1, "min": -1, "max": 100000,
|
||||||
|
"tooltip": "Guide frame the mask was made on; -1 = the first keyframe of keyframe_spec."}),
|
||||||
|
}, "optional": {
|
||||||
|
"keyframe_spec": ("STRING", {"forceInput": True, "tooltip": "BFS Keyframe Planner keyframe_spec (for anchor_frame -1)."}),
|
||||||
|
}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("MASK", "INT")
|
||||||
|
RETURN_NAMES = ("masks", "anchor_frame")
|
||||||
|
OUTPUT_TOOLTIPS = ("One mask per guide frame.", "The anchor frame used.")
|
||||||
|
FUNCTION = "track"
|
||||||
|
CATEGORY = "BFS/MiniMax H3"
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _run(frames, model, mask):
|
||||||
|
from comfy_extras.nodes_sam3 import SAM3_VideoTrack, SAM3_TrackToMask
|
||||||
|
data = SAM3_VideoTrack.execute(images=frames, model=model, initial_mask=mask, max_objects=1).args[0]
|
||||||
|
return SAM3_TrackToMask.execute(track_data=data, object_indices="").args[0]
|
||||||
|
|
||||||
|
def track(self, guide_video, sam3_model, mask, anchor_frame, keyframe_spec=None):
|
||||||
|
n, h, w = guide_video.shape[:3]
|
||||||
|
k = int(anchor_frame)
|
||||||
|
if k < 0:
|
||||||
|
frames = json.loads(keyframe_spec or "{}").get("frames", [])
|
||||||
|
if not frames:
|
||||||
|
raise ValueError("anchor_frame -1 needs a keyframe_spec with at least one keyframe.")
|
||||||
|
k = int(frames[0])
|
||||||
|
k = max(0, min(n - 1, k))
|
||||||
|
m = mask[:1] if mask.ndim == 3 else mask[None]
|
||||||
|
if m.shape[1:] != (h, w):
|
||||||
|
m = F.interpolate(m[:, None].float(), size=(h, w), mode="bilinear", align_corners=False)[:, 0]
|
||||||
|
m = (m > 0.5).float()
|
||||||
|
fwd = self._run(guide_video[k:], sam3_model, m) # frames k .. n-1
|
||||||
|
parts = [fwd]
|
||||||
|
if k > 0:
|
||||||
|
bwd = self._run(guide_video[:k + 1].flip(0), sam3_model, m) # frames k .. 0
|
||||||
|
parts = [bwd.flip(0)[:k], fwd]
|
||||||
|
out = torch.cat([p.to(guide_video.device, torch.float32) for p in parts])
|
||||||
|
if out.shape[1:] != (h, w):
|
||||||
|
out = F.interpolate(out[:, None], size=(h, w), mode="bilinear", align_corners=False)[:, 0]
|
||||||
|
print(f"[BFS Track From Frame] anchor frame {k}: tracked {k} frames back, {n - k} forward", flush=True)
|
||||||
|
return out, k
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {"BFSKeyframeMarks": BFSKeyframeMarks, "BFSTrackFromFrame": BFSTrackFromFrame}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {"BFSKeyframeMarks": "BFS Keyframe Marks", "BFSTrackFromFrame": "BFS Track From Frame (SAM 3)"}
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
"""BFS Keyframe Planner: pick keyframe times on the guide video, drop in edited frames and reference pictures.
|
||||||
|
|
||||||
|
The node shows the guide on a small player with a timeline (frames come from its last run: queue it once to load
|
||||||
|
them). "+ Keyframe" marks the current frame; each keyframe card holds the source frame and, optionally, the edited
|
||||||
|
frame for that time (drop an image on the card or upload it). Reference pictures (up to 4) are dropped or uploaded in
|
||||||
|
the node too, or connected as inputs.
|
||||||
|
|
||||||
|
Two ways to use it:
|
||||||
|
- with edited frames: wire keyframes + keyframe_spec + pictures into BFS H3 Keyframe Swap Conditioning;
|
||||||
|
- without: use it to choose the times; source_frames are the original frames there, ready for an image edit
|
||||||
|
(e.g. Segment Picker + the Qwen 2.1 magenta body swap), and the edits come back into the conditioning node.
|
||||||
|
|
||||||
|
All state lives in the hidden `plan` STRING widget (JSON), saved with the workflow:
|
||||||
|
{"keyframes": [{"frame": 53, "edit": "bfs_keyframes/abc.png"}, {"frame": 96, "edit": null}],
|
||||||
|
"pictures": ["bfs_keyframes/ref1.png"]}
|
||||||
|
Image paths are relative to ComfyUI's input folder (the panel uploads there).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import torch
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
try:
|
||||||
|
import folder_paths
|
||||||
|
except Exception: # tests run without ComfyUI
|
||||||
|
folder_paths = None
|
||||||
|
|
||||||
|
MAX_PICTURES = 4
|
||||||
|
STRIP_MAX_FRAMES = 480 # the panel's filmstrip: at most this many frames (evenly spread over longer guides)
|
||||||
|
STRIP_WIDTH = 384
|
||||||
|
|
||||||
|
|
||||||
|
def parse_plan(text):
|
||||||
|
try:
|
||||||
|
p = json.loads(text or "{}") or {}
|
||||||
|
except Exception:
|
||||||
|
p = {}
|
||||||
|
kfs = []
|
||||||
|
for k in p.get("keyframes", []) if isinstance(p.get("keyframes"), list) else []:
|
||||||
|
try:
|
||||||
|
kfs.append({"frame": int(k["frame"]), "edit": k.get("edit") or None})
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
pics = [x for x in p.get("pictures", []) if isinstance(x, str) and x] if isinstance(p.get("pictures"), list) else []
|
||||||
|
return {"keyframes": kfs, "pictures": pics[:MAX_PICTURES]}
|
||||||
|
|
||||||
|
|
||||||
|
def _input_path(rel):
|
||||||
|
base = folder_paths.get_input_directory() if folder_paths else "."
|
||||||
|
path = os.path.normpath(os.path.join(base, rel))
|
||||||
|
if not path.startswith(os.path.normpath(base) + os.sep):
|
||||||
|
raise ValueError(f"image path outside the input folder: {rel}")
|
||||||
|
if not os.path.isfile(path):
|
||||||
|
raise FileNotFoundError(f"keyframe planner image not found in input/: {rel}")
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
def load_image(rel):
|
||||||
|
im = Image.open(_input_path(rel)).convert("RGB")
|
||||||
|
return torch.from_numpy(np.asarray(im, np.float32) / 255.0)[None]
|
||||||
|
|
||||||
|
|
||||||
|
def fit(image, h, w):
|
||||||
|
"""[1,H,W,C] -> [1,h,w,3]: center crop to the guide's aspect, then resize."""
|
||||||
|
import comfy.utils
|
||||||
|
x = image[:1, ..., :3].movedim(-1, 1)
|
||||||
|
return comfy.utils.common_upscale(x, w, h, "lanczos", "center").movedim(1, -1).clamp(0, 1)
|
||||||
|
|
||||||
|
|
||||||
|
def filmstrip(video, fps):
|
||||||
|
"""Small JPEGs of the guide in the temp folder, for the panel. Same video -> same files (hashed)."""
|
||||||
|
n = video.shape[0]
|
||||||
|
idx = np.unique(np.linspace(0, n - 1, min(n, STRIP_MAX_FRAMES)).round().astype(int))
|
||||||
|
h, w = video.shape[1:3]
|
||||||
|
sw = min(STRIP_WIDTH, w); sh = max(1, round(h * sw / w))
|
||||||
|
sample = video[:: max(1, n // 16), ::16, ::16, :3].float().cpu().numpy()
|
||||||
|
key = hashlib.sha1(np.ascontiguousarray(sample).tobytes() + f"{n}x{h}x{w}".encode()).hexdigest()[:16]
|
||||||
|
temp = folder_paths.get_temp_directory() if folder_paths else "."
|
||||||
|
sub = os.path.join("bfs_kfplan", key); out = os.path.join(temp, sub); os.makedirs(out, exist_ok=True)
|
||||||
|
files = []
|
||||||
|
for i in idx:
|
||||||
|
name = f"f{i:05d}.jpg"; path = os.path.join(out, name)
|
||||||
|
if not os.path.exists(path):
|
||||||
|
arr = (video[int(i), ..., :3].float().cpu().numpy() * 255).clip(0, 255).astype(np.uint8)
|
||||||
|
Image.fromarray(arr).resize((sw, sh), Image.BILINEAR).save(path, quality=82)
|
||||||
|
files.append(name)
|
||||||
|
return {"subfolder": sub, "type": "temp", "files": files, "frames": [int(i) for i in idx],
|
||||||
|
"count": int(n), "fps": float(fps), "width": int(w), "height": int(h)}
|
||||||
|
|
||||||
|
|
||||||
|
class BFSKeyframePlanner:
|
||||||
|
DESCRIPTION = __doc__.split("\n\n")[0]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
opt = {f"picture_{j}": ("IMAGE", {"tooltip": f"Reference picture {j}; overrides the one dropped in the panel."})
|
||||||
|
for j in range(1, MAX_PICTURES + 1)}
|
||||||
|
return {
|
||||||
|
"required": {
|
||||||
|
"guide_video": ("IMAGE", {"tooltip": "The guide frames (e.g. BFS H3 Prepare Native Video)."}),
|
||||||
|
"fps": ("FLOAT", {"default": 24.0, "min": 1.0, "max": 120.0, "step": 0.5}),
|
||||||
|
"plan": ("STRING", {"default": "{}", "multiline": True, "tooltip": "Set by the panel: keyframe frames, edited frames and pictures."}),
|
||||||
|
},
|
||||||
|
"optional": opt,
|
||||||
|
"hidden": {"unique_id": "UNIQUE_ID"},
|
||||||
|
}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "IMAGE", "IMAGE", "STRING") + ("IMAGE",) * MAX_PICTURES
|
||||||
|
RETURN_NAMES = ("guide_video", "source_frames", "keyframes", "keyframe_spec") + tuple(f"picture_{j}" for j in range(1, MAX_PICTURES + 1))
|
||||||
|
OUTPUT_TOOLTIPS = ("The guide, unchanged.",
|
||||||
|
"The original guide frames at the keyframe times (one per keyframe), e.g. to edit them.",
|
||||||
|
"The edited frame of every keyframe, at the guide size; a keyframe without one gives its source frame.",
|
||||||
|
"JSON: frames, times (s), fps and which keyframes have an edited frame.") + \
|
||||||
|
tuple(f"Reference picture {j} (None when not set)." for j in range(1, MAX_PICTURES + 1))
|
||||||
|
FUNCTION = "plan_keyframes"
|
||||||
|
CATEGORY = "BFS/MiniMax H3"
|
||||||
|
OUTPUT_NODE = True
|
||||||
|
|
||||||
|
def plan_keyframes(self, guide_video, fps, plan, unique_id=None, **kw):
|
||||||
|
n, h, w = guide_video.shape[0], guide_video.shape[1], guide_video.shape[2]
|
||||||
|
state = parse_plan(plan)
|
||||||
|
ui = {"bfs_kfplan": [filmstrip(guide_video, fps)]}
|
||||||
|
kfs = sorted({max(0, min(n - 1, k["frame"])): k for k in state["keyframes"]}.items())
|
||||||
|
src, edit, spec = [], [], {"frames": [], "times": [], "edited": [], "fps": float(fps), "length": int(n)}
|
||||||
|
for f, k in kfs:
|
||||||
|
frame = guide_video[f:f + 1, ..., :3]
|
||||||
|
src.append(frame)
|
||||||
|
edit.append(fit(load_image(k["edit"]), h, w).to(frame) if k["edit"] else frame)
|
||||||
|
spec["frames"].append(int(f)); spec["times"].append(round(f / float(fps), 4)); spec["edited"].append(bool(k["edit"]))
|
||||||
|
if not kfs:
|
||||||
|
print("[BFS Keyframe Planner] no keyframes yet: add them in the panel (outputs hold the first frame)", flush=True)
|
||||||
|
src = edit = [guide_video[:1, ..., :3]]
|
||||||
|
pictures = []
|
||||||
|
for j in range(1, MAX_PICTURES + 1):
|
||||||
|
p = kw.get(f"picture_{j}")
|
||||||
|
if p is None and j <= len(state["pictures"]):
|
||||||
|
p = load_image(state["pictures"][j - 1])
|
||||||
|
pictures.append(p)
|
||||||
|
return {"ui": ui, "result": (guide_video, torch.cat(src), torch.cat(edit), json.dumps(spec), *pictures)}
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {"BFSKeyframePlanner": BFSKeyframePlanner}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {"BFSKeyframePlanner": "BFS Keyframe Planner"}
|
||||||
@@ -0,0 +1,438 @@
|
|||||||
|
"""BFS Segment Picker: segment an image into objects with SAM 3.1, pick them with the mouse, get their masks.
|
||||||
|
|
||||||
|
Connect SAM 3.1 from a Load Checkpoint node (sam3.1_multiplex_fp16.safetensors gives its MODEL and CLIP). The node
|
||||||
|
segments the first image of its input automatically (one SAM 3 text prompt per concept: "person, animal" by
|
||||||
|
default) and shows every object on a canvas in the node. Click objects to select them; the outputs are the mask of the
|
||||||
|
selection, one mask per selected object, the image with the selection tinted (the Colored Mark convention: magenta at
|
||||||
|
30 % marks the subject a body swap replaces) and a numbered overlay of everything found.
|
||||||
|
|
||||||
|
When the automatic pass misses something, add it in the panel: type what to find (a SAM 3 text prompt, e.g. "red
|
||||||
|
umbrella"), place keep / exclude points on it, or paint it with the brush (to grow the mark past what SAM found, e.g.
|
||||||
|
over hair). Added objects appear at once (the panel asks the server through
|
||||||
|
/bfs/segpick/*), and the workflow stores how each one was made, so a reload or another machine segments them again.
|
||||||
|
|
||||||
|
All state lives in the hidden `selection` STRING widget (JSON):
|
||||||
|
{"selected": ["a3", "t1.0"], "added": [{"id": "t1", "text": "red umbrella"},
|
||||||
|
{"id": "p2", "points": [{"x": .41, "y": .62, "label": 1}]},
|
||||||
|
{"id": "b3", "image": "9f2c...", "strokes": [{"r": .02, "pts": [[.4, .3], [.45, .32]]}]}], "next": 4}
|
||||||
|
An entry with "remove": true (r<n> points, e<n> eraser strokes) is cut out of the selection instead of added to it.
|
||||||
|
Points are relative to the image (0..1), a stroke radius to its width. A brush entry belongs to the image it was painted
|
||||||
|
on (`image`, see image_key): on another image it is dropped, while text and point entries are segmented again. Ids: a<k> automatic objects (left to right),
|
||||||
|
t<n>.<k> the k-th match of a text entry, p<n> a point entry, b<n> a brush entry.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import uuid
|
||||||
|
from collections import OrderedDict
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import torch
|
||||||
|
from PIL import Image, ImageDraw, ImageFont
|
||||||
|
|
||||||
|
try:
|
||||||
|
import folder_paths
|
||||||
|
except Exception: # tests run without ComfyUI
|
||||||
|
folder_paths = None
|
||||||
|
|
||||||
|
PALETTE = [(230, 70, 60), (45, 125, 225), (50, 180, 90), (235, 205, 35), (205, 55, 180), (30, 200, 205),
|
||||||
|
(240, 130, 35), (135, 75, 210), (120, 200, 60), (225, 95, 150), (80, 160, 230), (190, 150, 90)]
|
||||||
|
DEFAULT_CONCEPTS = "person, animal"
|
||||||
|
PREVIEW_SIDE = 1024 # long side of the images the panel shows (outputs keep full resolution)
|
||||||
|
MAX_SESSIONS = 6
|
||||||
|
_SESSIONS: "OrderedDict[str, dict]" = OrderedDict()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------------- state
|
||||||
|
|
||||||
|
def parse_state(text: str | None) -> dict:
|
||||||
|
"""The selection widget's JSON, tolerant of empty or hand-edited values."""
|
||||||
|
try:
|
||||||
|
s = json.loads(text) if text and text.strip() else {}
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
s = {}
|
||||||
|
if not isinstance(s, dict):
|
||||||
|
s = {}
|
||||||
|
added = []
|
||||||
|
for e in s.get("added") or []:
|
||||||
|
if not isinstance(e, dict) or not e.get("id"):
|
||||||
|
continue
|
||||||
|
n = len(added)
|
||||||
|
if str(e.get("text") or "").strip():
|
||||||
|
added.append({"id": str(e["id"]), "text": str(e["text"]).strip()})
|
||||||
|
elif e.get("points"):
|
||||||
|
pts = [{"x": float(p["x"]), "y": float(p["y"]), "label": 1 if p.get("label", 1) else 0}
|
||||||
|
for p in e["points"] if isinstance(p, dict) and "x" in p and "y" in p]
|
||||||
|
if any(p["label"] for p in pts):
|
||||||
|
added.append({"id": str(e["id"]), "points": pts})
|
||||||
|
elif e.get("strokes"):
|
||||||
|
strokes = [{"r": float(st["r"]), "pts": [[float(x), float(y)] for x, y in st["pts"]]}
|
||||||
|
for st in e["strokes"] if isinstance(st, dict) and st.get("pts")]
|
||||||
|
if strokes:
|
||||||
|
added.append({"id": str(e["id"]), "image": str(e.get("image") or ""), "strokes": strokes})
|
||||||
|
if e.get("remove") and len(added) > n:
|
||||||
|
added[-1]["remove"] = True
|
||||||
|
selected = [str(i) for i in s.get("selected") or [] if isinstance(i, (str, int))]
|
||||||
|
return {"selected": list(dict.fromkeys(selected)), "added": added, "next": int(s.get("next") or len(added) + 1)}
|
||||||
|
|
||||||
|
|
||||||
|
def split_concepts(text: str) -> list[str]:
|
||||||
|
return list(dict.fromkeys(c.strip() for c in str(text or "").replace("\n", ",").split(",") if c.strip()))
|
||||||
|
|
||||||
|
|
||||||
|
def entry_key(entry: dict) -> str:
|
||||||
|
"""What an added entry asks SAM for: an entry whose text or points change is segmented again."""
|
||||||
|
return json.dumps({k: entry[k] for k in ("text", "points", "strokes", "remove") if k in entry}, sort_keys=True)
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------------------------------------ geometry
|
||||||
|
|
||||||
|
def box_of(mask: torch.Tensor) -> list[int]:
|
||||||
|
ys, xs = torch.nonzero(mask, as_tuple=True)
|
||||||
|
if len(xs) == 0:
|
||||||
|
return [0, 0, 0, 0]
|
||||||
|
x0, y0, x1, y1 = int(xs.min()), int(ys.min()), int(xs.max()) + 1, int(ys.max()) + 1
|
||||||
|
return [x0, y0, x1 - x0, y1 - y0]
|
||||||
|
|
||||||
|
|
||||||
|
def dedupe(found: list[tuple[torch.Tensor, dict]], overlap: float = .8) -> list[tuple[torch.Tensor, dict]]:
|
||||||
|
"""Drop a mask that mostly repeats an earlier one (the animal prompt can find a person again, and the reverse)."""
|
||||||
|
kept: list[tuple[torch.Tensor, dict]] = []
|
||||||
|
for m, info in found:
|
||||||
|
a = int(m.sum())
|
||||||
|
if any(int((m & o).sum()) > overlap * min(a, int(o.sum())) for o, _ in kept):
|
||||||
|
continue
|
||||||
|
kept.append((m, info))
|
||||||
|
return kept
|
||||||
|
|
||||||
|
|
||||||
|
def compose(masks: list[torch.Tensor], h: int, w: int) -> tuple[torch.Tensor, torch.Tensor]:
|
||||||
|
"""(union [1, h, w], individual [n, h, w]) as float masks; an empty selection gives zero masks."""
|
||||||
|
if not masks:
|
||||||
|
z = torch.zeros(1, h, w)
|
||||||
|
return z, z.clone()
|
||||||
|
stack = torch.stack([m.float() for m in masks])
|
||||||
|
return (stack.amax(0, keepdim=True)), stack
|
||||||
|
|
||||||
|
|
||||||
|
def paint(strokes: list[dict], h: int, w: int) -> torch.Tensor:
|
||||||
|
"""Brush strokes (relative points, radius relative to the width) as a bool mask [h, w]."""
|
||||||
|
img = Image.new("L", (w, h), 0)
|
||||||
|
draw = ImageDraw.Draw(img)
|
||||||
|
for st in strokes:
|
||||||
|
r = max(1., st["r"] * w)
|
||||||
|
pts = [(x * w, y * h) for x, y in st["pts"]]
|
||||||
|
if len(pts) > 1:
|
||||||
|
draw.line(pts, fill=255, width=round(2 * r), joint="curve")
|
||||||
|
for x, y in pts:
|
||||||
|
draw.ellipse([x - r, y - r, x + r, y + r], fill=255)
|
||||||
|
return torch.from_numpy(np.array(img) > 127)
|
||||||
|
|
||||||
|
|
||||||
|
def tint(image: torch.Tensor, mask: torch.Tensor, color: tuple[int, int, int], opacity: float) -> torch.Tensor:
|
||||||
|
"""image [1, h, w, 3] with the mask's pixels blended toward the color."""
|
||||||
|
c = torch.tensor(color, dtype=image.dtype) / 255.
|
||||||
|
a = (mask[0].to(image.dtype) * float(opacity))[..., None]
|
||||||
|
out = image.clone()
|
||||||
|
out[0, ..., :3] = image[0, ..., :3] * (1 - a) + c * a
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def parse_color(text: str) -> tuple[int, int, int]:
|
||||||
|
s = str(text or "").strip().lstrip("#")
|
||||||
|
if len(s) == 3:
|
||||||
|
s = "".join(ch * 2 for ch in s)
|
||||||
|
try:
|
||||||
|
return int(s[0:2], 16), int(s[2:4], 16), int(s[4:6], 16)
|
||||||
|
except (ValueError, IndexError):
|
||||||
|
return 255, 0, 255
|
||||||
|
|
||||||
|
|
||||||
|
# ----------------------------------------------------------------------------------------------------------------- SAM
|
||||||
|
|
||||||
|
def _text_cond(clip, text: str, max_detections: int):
|
||||||
|
cond = clip.encode_from_tokens_scheduled(clip.tokenize(text))
|
||||||
|
# SAM3_Detect keeps one detection per text prompt unless the prompt asks for more (the multi-prompt form)
|
||||||
|
return [[cond[0][0], {**cond[0][1], "sam3_multi_cond": [{"cond": cond[0][0], "attention_mask": cond[0][1].get("attention_mask"),
|
||||||
|
"max_detections": int(max_detections)}]}]]
|
||||||
|
|
||||||
|
|
||||||
|
def detect_text(sam: tuple, frame: torch.Tensor, text: str, threshold: float, max_detections: int) -> list[tuple[torch.Tensor, dict]]:
|
||||||
|
from comfy_extras.nodes_sam3 import SAM3_Detect
|
||||||
|
model, clip = sam
|
||||||
|
h, w = frame.shape[1:3]
|
||||||
|
# ComfyUI can materialize SAM weights as inference tensors. Its detector must then
|
||||||
|
# run under inference_mode too; otherwise autograd asks those weights for a version counter.
|
||||||
|
with torch.inference_mode():
|
||||||
|
masks, boxes = SAM3_Detect.execute(model=model, image=frame, conditioning=_text_cond(clip, text, max_detections),
|
||||||
|
threshold=float(threshold), refine_iterations=1, individual_masks=True).args
|
||||||
|
masks = masks.cpu() > .5 if masks.numel() else torch.zeros(0, h, w, dtype=torch.bool)
|
||||||
|
boxes = boxes[0] if boxes else []
|
||||||
|
return [(m, {"score": round(float(b.get("score", 0)), 3)}) for m, b in zip(masks, boxes)]
|
||||||
|
|
||||||
|
|
||||||
|
def detect_points(sam: tuple, frame: torch.Tensor, points: list[dict], threshold: float) -> torch.Tensor:
|
||||||
|
from comfy_extras.nodes_sam3 import SAM3_Detect
|
||||||
|
model, _ = sam
|
||||||
|
h, w = frame.shape[1:3]
|
||||||
|
pos = [{"x": p["x"] * w, "y": p["y"] * h} for p in points if p["label"]]
|
||||||
|
neg = [{"x": p["x"] * w, "y": p["y"] * h} for p in points if not p["label"]]
|
||||||
|
with torch.inference_mode():
|
||||||
|
masks = SAM3_Detect.execute(model=model, image=frame, positive_coords=json.dumps(pos), negative_coords=json.dumps(neg),
|
||||||
|
threshold=float(threshold), refine_iterations=2).args[0]
|
||||||
|
return (masks[0].cpu() > .5) if masks.numel() else torch.zeros(h, w, dtype=torch.bool)
|
||||||
|
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------------------------------------------------- session
|
||||||
|
|
||||||
|
def image_key(frame: torch.Tensor) -> str:
|
||||||
|
"""Which image this is, from a 64x64 thumbnail and the size: what a brush entry is tied to."""
|
||||||
|
small = torch.nn.functional.interpolate(frame[..., :3].movedim(-1, 1).float(), size=(64, 64), mode="area")
|
||||||
|
digest = hashlib.sha1((small * 255).round().byte().numpy().tobytes())
|
||||||
|
digest.update(str(tuple(frame.shape)).encode())
|
||||||
|
return digest.hexdigest()[:12]
|
||||||
|
|
||||||
|
|
||||||
|
def _session_key(frame: torch.Tensor, spec: dict, sam: tuple) -> str:
|
||||||
|
digest = hashlib.sha1(image_key(frame).encode())
|
||||||
|
digest.update(json.dumps(spec, sort_keys=True).encode())
|
||||||
|
digest.update(str(id(getattr(sam[0], "model", sam[0]))).encode()) # another SAM checkpoint segments again
|
||||||
|
return digest.hexdigest()[:16]
|
||||||
|
|
||||||
|
|
||||||
|
def _remember(sid: str, sess: dict) -> dict:
|
||||||
|
_SESSIONS[sid] = sess
|
||||||
|
_SESSIONS.move_to_end(sid)
|
||||||
|
while len(_SESSIONS) > MAX_SESSIONS:
|
||||||
|
_SESSIONS.popitem(last=False)
|
||||||
|
return sess
|
||||||
|
|
||||||
|
|
||||||
|
def auto_objects(sam: tuple, frame: torch.Tensor, concepts: list[str], threshold: float, max_per_concept: int, min_area: float) -> list[dict]:
|
||||||
|
h, w = frame.shape[1:3]
|
||||||
|
found = []
|
||||||
|
for concept in concepts:
|
||||||
|
for m, info in detect_text(sam, frame, concept, threshold, max_per_concept):
|
||||||
|
area = int(m.sum())
|
||||||
|
if area < max(64, min_area * h * w):
|
||||||
|
continue
|
||||||
|
# Generic "objects" may segment the entire scene. Such a mask is
|
||||||
|
# unsafe as an auto subject: it paints the background into the guide.
|
||||||
|
if concept.strip().casefold() in {"object", "objects"} and area >= .25 * h * w:
|
||||||
|
_, _, bw, bh = box_of(m)
|
||||||
|
if bw >= .98 * w and bh >= .98 * h:
|
||||||
|
continue
|
||||||
|
found.append((m, {**info, "label": concept}))
|
||||||
|
found = dedupe(found)
|
||||||
|
found.sort(key=lambda f: box_of(f[0])[0]) # left to right, so "a1" reads naturally
|
||||||
|
return [{"id": f"a{k + 1}", "label": info["label"], "source": "auto", "score": info["score"], "mask": m}
|
||||||
|
for k, (m, info) in enumerate(found)]
|
||||||
|
|
||||||
|
|
||||||
|
def entry_objects(sam: tuple, frame: torch.Tensor, entry: dict, threshold: float, max_per_concept: int) -> list[dict]:
|
||||||
|
objs = _entry_objects(sam, frame, entry, threshold, max_per_concept)
|
||||||
|
if entry.get("remove"):
|
||||||
|
for o in objs:
|
||||||
|
o["remove"], o["label"] = True, "eraser" if entry.get("strokes") else "remove"
|
||||||
|
return objs
|
||||||
|
|
||||||
|
|
||||||
|
def _entry_objects(sam: tuple, frame: torch.Tensor, entry: dict, threshold: float, max_per_concept: int) -> list[dict]:
|
||||||
|
if entry.get("text"):
|
||||||
|
hits = dedupe(detect_text(sam, frame, entry["text"], threshold, max_per_concept))
|
||||||
|
hits.sort(key=lambda f: box_of(f[0])[0])
|
||||||
|
return [{"id": f"{entry['id']}.{k}", "label": entry["text"], "source": "text", "entry": entry["id"],
|
||||||
|
"score": info["score"], "mask": m} for k, (m, info) in enumerate(hits)]
|
||||||
|
if entry.get("strokes"):
|
||||||
|
m = paint(entry["strokes"], *frame.shape[1:3])
|
||||||
|
return [{"id": entry["id"], "label": "brush", "source": "brush", "entry": entry["id"], "score": None, "mask": m}] if m.any() else []
|
||||||
|
m = detect_points(sam, frame, entry["points"], threshold)
|
||||||
|
if int(m.sum()) < 16:
|
||||||
|
return []
|
||||||
|
return [{"id": entry["id"], "label": "points", "source": "points", "entry": entry["id"], "score": None, "mask": m}]
|
||||||
|
|
||||||
|
|
||||||
|
def session_for(sam: tuple, frame: torch.Tensor, spec: dict) -> tuple[str, dict]:
|
||||||
|
sid = _session_key(frame, spec, sam)
|
||||||
|
if sid in _SESSIONS:
|
||||||
|
_SESSIONS.move_to_end(sid)
|
||||||
|
_SESSIONS[sid]["sam"] = sam
|
||||||
|
return sid, _SESSIONS[sid]
|
||||||
|
objs = auto_objects(sam, frame, split_concepts(spec["concepts"]), spec["threshold"], spec["max_per_concept"], spec["min_area"])
|
||||||
|
# the panel's /bfs/segpick/add segments with the SAM the node last ran with
|
||||||
|
return sid, _remember(sid, {"frame": frame.cpu(), "image_key": image_key(frame), "spec": spec, "auto": objs, "entries": {}, "sam": sam})
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_entry(sess: dict, entry: dict) -> list[dict]:
|
||||||
|
"""The objects of an added entry, segmented once per (entry id, what it asks for)."""
|
||||||
|
key = entry_key(entry)
|
||||||
|
have = sess["entries"].get(entry["id"])
|
||||||
|
if have is None or have["key"] != key:
|
||||||
|
spec = sess["spec"]
|
||||||
|
have = {"key": key, "objects": entry_objects(sess["sam"], sess["frame"], entry, spec["threshold"], spec["max_per_concept"])}
|
||||||
|
sess["entries"][entry["id"]] = have
|
||||||
|
return have["objects"]
|
||||||
|
|
||||||
|
|
||||||
|
def all_objects(sess: dict, state: dict) -> list[dict]:
|
||||||
|
objs = list(sess["auto"])
|
||||||
|
for e in state["added"]:
|
||||||
|
if e.get("strokes") and e["image"] and e["image"] != sess["image_key"]:
|
||||||
|
continue # painted on another image
|
||||||
|
objs += ensure_entry(sess, e)
|
||||||
|
return objs
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------------------------------------- panel files
|
||||||
|
|
||||||
|
def _temp_dir() -> str:
|
||||||
|
base = folder_paths.get_temp_directory() if folder_paths else os.path.join(os.path.dirname(__file__), ".temp")
|
||||||
|
d = os.path.join(base, "bfs_segpick")
|
||||||
|
os.makedirs(d, exist_ok=True)
|
||||||
|
return d
|
||||||
|
|
||||||
|
|
||||||
|
def _preview_size(h: int, w: int) -> tuple[int, int]:
|
||||||
|
s = min(1.0, PREVIEW_SIDE / max(h, w))
|
||||||
|
return max(1, round(w * s)), max(1, round(h * s))
|
||||||
|
|
||||||
|
|
||||||
|
def _save(img: Image.Image, prefix: str) -> dict:
|
||||||
|
name = f"{prefix}_{uuid.uuid4().hex[:10]}.png"
|
||||||
|
img.save(os.path.join(_temp_dir(), name), compress_level=1)
|
||||||
|
return {"filename": name, "subfolder": "bfs_segpick", "type": "temp"}
|
||||||
|
|
||||||
|
|
||||||
|
def panel_object(o: dict, size: tuple[int, int], h: int, w: int) -> dict:
|
||||||
|
"""What the panel needs about one object: its mask as a small PNG (white on transparent) and where it is."""
|
||||||
|
m = Image.fromarray(o["mask"].numpy().astype(np.uint8) * 255).resize(size, Image.NEAREST)
|
||||||
|
rgba = Image.new("RGBA", size, (255, 255, 255, 0))
|
||||||
|
rgba.putalpha(m)
|
||||||
|
x, y, bw, bh = box_of(o["mask"])
|
||||||
|
if "_file" not in o:
|
||||||
|
o["_file"] = _save(rgba, f"m_{o['id']}")
|
||||||
|
return {"id": o["id"], "label": o["label"], "source": o["source"], "entry": o.get("entry"), "score": o["score"],
|
||||||
|
"remove": bool(o.get("remove")), "area": round(float(o["mask"].float().mean()), 5), "box": [round(x / w, 4), round(y / h, 4), round(bw / w, 4), round(bh / h, 4)],
|
||||||
|
"mask": o["_file"]}
|
||||||
|
|
||||||
|
|
||||||
|
def panel_data(sid: str, sess: dict, objects: list[dict]) -> dict:
|
||||||
|
frame = sess["frame"]
|
||||||
|
h, w = frame.shape[1:3]
|
||||||
|
size = _preview_size(h, w)
|
||||||
|
if "_image" not in sess:
|
||||||
|
rgb = (frame[0, ..., :3].clamp(0, 1) * 255).byte().numpy()
|
||||||
|
sess["_image"] = _save(Image.fromarray(rgb).resize(size, Image.LANCZOS), "img")
|
||||||
|
return {"session": sid, "image_key": sess["image_key"], "width": int(w), "height": int(h), "image": sess["_image"],
|
||||||
|
"objects": [panel_object(o, size, h, w) for o in objects]}
|
||||||
|
|
||||||
|
|
||||||
|
def overlay(frame: torch.Tensor, objects: list[dict], selected: set[str]) -> torch.Tensor:
|
||||||
|
rgb = (frame[0, ..., :3].clamp(0, 1) * 255).byte().numpy().astype(np.float32)
|
||||||
|
h, w = rgb.shape[:2]
|
||||||
|
for n, o in enumerate(objects):
|
||||||
|
m = o["mask"].numpy()
|
||||||
|
a = .6 if o["id"] in selected else .3
|
||||||
|
rgb[m] = rgb[m] * (1 - a) + np.array(PALETTE[n % len(PALETTE)], np.float32) * a
|
||||||
|
canvas = Image.fromarray(rgb.clip(0, 255).astype(np.uint8))
|
||||||
|
draw = ImageDraw.Draw(canvas)
|
||||||
|
size = max(14, h // 28)
|
||||||
|
try:
|
||||||
|
font = ImageFont.load_default(size=size)
|
||||||
|
except TypeError:
|
||||||
|
font = ImageFont.load_default()
|
||||||
|
for n, o in enumerate(objects):
|
||||||
|
x, y, bw, bh = box_of(o["mask"])
|
||||||
|
color = PALETTE[n % len(PALETTE)]
|
||||||
|
if o["id"] in selected:
|
||||||
|
draw.rectangle([x, y, x + bw, y + bh], outline=color, width=max(2, h // 300))
|
||||||
|
draw.text((x + 4, y + 4), o["id"], fill=(255, 255, 255), font=font, stroke_width=2, stroke_fill=(0, 0, 0))
|
||||||
|
return torch.from_numpy(np.asarray(canvas, np.float32) / 255.)[None]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------------------------------------------- node
|
||||||
|
|
||||||
|
class BFSSegmentPicker:
|
||||||
|
DESCRIPTION = __doc__.split("\n\n")[0] + " Click objects on the canvas to select them; add missed ones by text, points or the brush."
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {
|
||||||
|
"required": {
|
||||||
|
"image": ("IMAGE",),
|
||||||
|
"sam3_model": ("MODEL", {"tooltip": "SAM 3.1: MODEL of a Load Checkpoint node with sam3.1_multiplex_fp16.safetensors."}),
|
||||||
|
"sam3_clip": ("CLIP", {"tooltip": "SAM 3.1: CLIP of the same Load Checkpoint node; turns the concepts and Find texts into SAM prompts."}),
|
||||||
|
"concepts": ("STRING", {"default": DEFAULT_CONCEPTS, "tooltip": "What the automatic pass looks for, comma separated (one SAM 3 text prompt each), e.g. person, animal, car, chair. Empty = no automatic pass, only what you add."}),
|
||||||
|
"threshold": ("FLOAT", {"default": .4, "min": .05, "max": .95, "step": .05, "tooltip": "SAM 3 detection score threshold: lower finds more (and more false hits)."}),
|
||||||
|
"max_per_concept": ("INT", {"default": 8, "min": 1, "max": 32, "tooltip": "Most objects one concept or text prompt can return."}),
|
||||||
|
"min_area": ("FLOAT", {"default": .002, "min": 0., "max": .5, "step": .001, "tooltip": "Automatic objects smaller than this share of the image are dropped."}),
|
||||||
|
"mark_color": ("STRING", {"default": "#FF00FF", "tooltip": "Tint of the selection in the marked output (hex)."}),
|
||||||
|
"mark_opacity": ("FLOAT", {"default": .3, "min": 0., "max": 1., "step": .05, "tooltip": "How strongly the marked output tints the selection (0.3 = the Colored Mark default)."}),
|
||||||
|
"selection": ("STRING", {"default": "{}", "multiline": True, "tooltip": "Set by the canvas: the selected objects and the ones added by text or points."}),
|
||||||
|
},
|
||||||
|
"hidden": {"unique_id": "UNIQUE_ID"},
|
||||||
|
}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("MASK", "MASK", "IMAGE", "IMAGE", "STRING")
|
||||||
|
RETURN_NAMES = ("mask", "masks", "marked", "overlay", "objects")
|
||||||
|
OUTPUT_TOOLTIPS = ("Union of the selected objects, minus the selected remove / eraser ones (one mask).", "One mask per selected object (remove ones already cut out), in selection order.",
|
||||||
|
"The image with the selection tinted (mark_color, mark_opacity).", "Every object found, numbered; the selection stronger.",
|
||||||
|
"JSON: size, selected ids and every object's id, label, score, box (relative x, y, w, h) and area.")
|
||||||
|
FUNCTION = "pick"
|
||||||
|
CATEGORY = "BFS/Mask"
|
||||||
|
OUTPUT_NODE = True
|
||||||
|
|
||||||
|
def pick(self, image, sam3_model, sam3_clip, concepts, threshold, max_per_concept, min_area, mark_color, mark_opacity, selection, unique_id=None):
|
||||||
|
frame = image[:1, ..., :3].float().cpu()
|
||||||
|
h, w = frame.shape[1:3]
|
||||||
|
spec = {"concepts": ", ".join(split_concepts(concepts)), "threshold": round(float(threshold), 4),
|
||||||
|
"max_per_concept": int(max_per_concept), "min_area": round(float(min_area), 5)}
|
||||||
|
state = parse_state(selection)
|
||||||
|
sid, sess = session_for((sam3_model, sam3_clip), frame, spec)
|
||||||
|
objects = all_objects(sess, state)
|
||||||
|
by_id = {o["id"]: o for o in objects}
|
||||||
|
chosen = [by_id[i] for i in state["selected"] if i in by_id]
|
||||||
|
cut = [o["mask"] for o in chosen if o.get("remove")]
|
||||||
|
cut = torch.stack(cut).any(0) if cut else torch.zeros(h, w, dtype=torch.bool)
|
||||||
|
mask, masks = compose([o["mask"] & ~cut for o in chosen if not o.get("remove")], h, w)
|
||||||
|
marked = tint(frame, mask, parse_color(mark_color), mark_opacity)
|
||||||
|
info = {"width": int(w), "height": int(h), "selected": [o["id"] for o in chosen],
|
||||||
|
"objects": [{k: v for k, v in panel_object(o, _preview_size(h, w), h, w).items() if k != "mask"} for o in objects]}
|
||||||
|
return {"ui": {"bfs_segpick": [panel_data(sid, sess, objects)]},
|
||||||
|
"result": (mask, masks, marked, overlay(frame, objects, set(info["selected"])), json.dumps(info))}
|
||||||
|
|
||||||
|
|
||||||
|
# -------------------------------------------------------------------------------------------------------------- routes
|
||||||
|
|
||||||
|
def _routes():
|
||||||
|
try:
|
||||||
|
from aiohttp import web
|
||||||
|
from server import PromptServer
|
||||||
|
except Exception:
|
||||||
|
return
|
||||||
|
|
||||||
|
@PromptServer.instance.routes.post("/bfs/segpick/add")
|
||||||
|
async def add(request):
|
||||||
|
"""Segment one added entry ({session, entry: {id, text} | {id, points}}) on the session's image now."""
|
||||||
|
body = await request.json()
|
||||||
|
sess = _SESSIONS.get(str(body.get("session") or ""))
|
||||||
|
if sess is None:
|
||||||
|
return web.json_response({"error": "This image is no longer in memory: run the node once, then add again."}, status=404)
|
||||||
|
state = parse_state(json.dumps({"added": [body.get("entry") or {}]}))
|
||||||
|
if not state["added"]:
|
||||||
|
return web.json_response({"error": "Give a text prompt, at least one keep point or a brush stroke."}, status=400)
|
||||||
|
try:
|
||||||
|
objs = ensure_entry(sess, state["added"][0])
|
||||||
|
except Exception as e: # noqa: BLE001 - shown in the panel
|
||||||
|
return web.json_response({"error": f"SAM 3 failed: {e}"}, status=500)
|
||||||
|
h, w = sess["frame"].shape[1:3]
|
||||||
|
size = _preview_size(h, w)
|
||||||
|
return web.json_response({"objects": [panel_object(o, size, h, w) for o in objs]})
|
||||||
|
|
||||||
|
|
||||||
|
_routes()
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {"BFSSegmentPicker": BFSSegmentPicker}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {"BFSSegmentPicker": "BFS Segment Picker (SAM 3)"}
|
||||||
+14
-5
@@ -1685,7 +1685,16 @@ async def _off_loop(fn):
|
|||||||
"""Run slow work in a thread so the server keeps answering (and the panel gets progress events)."""
|
"""Run slow work in a thread so the server keeps answering (and the panel gets progress events)."""
|
||||||
import asyncio
|
import asyncio
|
||||||
_outside_prompt()
|
_outside_prompt()
|
||||||
return await asyncio.get_running_loop().run_in_executor(None, fn)
|
return await asyncio.get_running_loop().run_in_executor(None, lambda: _like_a_node(fn))
|
||||||
|
|
||||||
|
|
||||||
|
def _like_a_node(fn):
|
||||||
|
"""Run a panel button's model work under torch.inference_mode(), as ComfyUI runs every node (execution.py). After
|
||||||
|
the workflow ran once, the VLM and SAM keep tensors made in inference mode; using them outside it raises
|
||||||
|
"Inference tensors do not track version counter" (issue #14)."""
|
||||||
|
import torch
|
||||||
|
with torch.inference_mode():
|
||||||
|
return fn()
|
||||||
|
|
||||||
|
|
||||||
def _outside_prompt() -> None:
|
def _outside_prompt() -> None:
|
||||||
@@ -5850,10 +5859,10 @@ try:
|
|||||||
preview_color = (mask_override or {}).get("preview_color") or (
|
preview_color = (mask_override or {}).get("preview_color") or (
|
||||||
seg.get("color_guide_color") if seg.get("color_guide") else "#ff283c")
|
seg.get("color_guide_color") if seg.get("color_guide") else "#ff283c")
|
||||||
out = await asyncio.get_running_loop().run_in_executor(
|
out = await asyncio.get_running_loop().run_in_executor(
|
||||||
None, lambda: mask_preview(path, a, seg["start"], seg["end"] - seg["start"], spec,
|
None, lambda: _like_a_node(lambda: mask_preview(path, a, seg["start"], seg["end"] - seg["start"], spec,
|
||||||
int(body.get("count", 6)),
|
int(body.get("count", 6)),
|
||||||
preview_color,
|
preview_color,
|
||||||
seg.get("color_guide_opacity") if seg.get("color_guide") else 0.55))
|
seg.get("color_guide_opacity") if seg.get("color_guide") else 0.55)))
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=500)
|
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=500)
|
||||||
return web.json_response(out)
|
return web.json_response(out)
|
||||||
@@ -5878,8 +5887,8 @@ try:
|
|||||||
import asyncio
|
import asyncio
|
||||||
_outside_prompt()
|
_outside_prompt()
|
||||||
out = await asyncio.get_running_loop().run_in_executor(
|
out = await asyncio.get_running_loop().run_in_executor(
|
||||||
None, lambda: [dict(vlm_shot(clip, path, a, s["start"], s["end"], cfg), start=s["start"], end=s["end"])
|
None, lambda: _like_a_node(lambda: [dict(vlm_shot(clip, path, a, s["start"], s["end"], cfg),
|
||||||
for s in pick])
|
start=s["start"], end=s["end"]) for s in pick]))
|
||||||
except Exception as exc: # noqa: BLE001
|
except Exception as exc: # noqa: BLE001
|
||||||
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=500)
|
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=500)
|
||||||
return web.json_response({"segs": out})
|
return web.json_response({"segs": out})
|
||||||
|
|||||||
+28
-3
@@ -67,6 +67,17 @@ def ref_pictures(reference) -> list:
|
|||||||
return out[:MAX_REFS]
|
return out[:MAX_REFS]
|
||||||
|
|
||||||
|
|
||||||
|
def save_stills(frames: dict, out_dir: str) -> dict:
|
||||||
|
"""{source: [1, H, W, 3] in 0..1} as lossless PNGs -> {source: file name}."""
|
||||||
|
from PIL import Image
|
||||||
|
names = {}
|
||||||
|
for k, v in frames.items():
|
||||||
|
names[k] = f"{k}.png"
|
||||||
|
Image.fromarray((v[0, ..., :3].float().clamp(0, 1).cpu().numpy() * 255).round().astype(np.uint8)).save(
|
||||||
|
os.path.join(out_dir, names[k]))
|
||||||
|
return names
|
||||||
|
|
||||||
|
|
||||||
def save_refs(reference, out_dir: str, height: int) -> list:
|
def save_refs(reference, out_dir: str, height: int) -> list:
|
||||||
"""The reference pictures as PNGs no taller than the player (no upscale) -> file names."""
|
"""The reference pictures as PNGs no taller than the player (no upscale) -> file names."""
|
||||||
from PIL import Image
|
from PIL import Image
|
||||||
@@ -129,6 +140,8 @@ class BFSVideoCompare:
|
|||||||
"size) or a batch; up to 12."}),
|
"size) or a batch; up to 12."}),
|
||||||
"label_guide": ("STRING", {"default": "guide"}),
|
"label_guide": ("STRING", {"default": "guide"}),
|
||||||
"label_target": ("STRING", {"default": "target"}),
|
"label_target": ("STRING", {"default": "target"}),
|
||||||
|
"free_memory_after": ("BOOLEAN", {"default": True, "tooltip":
|
||||||
|
"After the workflow finishes, release cached node outputs and unloaded model weights from RAM. The next run reloads models."}),
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -146,10 +159,10 @@ class BFSVideoCompare:
|
|||||||
"where they differ, pixel by pixel.")
|
"where they differ, pixel by pixel.")
|
||||||
|
|
||||||
def compare(self, guide, target, fps=24.0, audio=None, gain=2.0, size="guide", preview_height=720,
|
def compare(self, guide, target, fps=24.0, audio=None, gain=2.0, size="guide", preview_height=720,
|
||||||
label_guide="guide", label_target="target", reference=None):
|
label_guide="guide", label_target="target", reference=None, free_memory_after=True):
|
||||||
one = lambda v: v[0] if isinstance(v, list) and v else v
|
one = lambda v: v[0] if isinstance(v, list) and v else v
|
||||||
fps, audio, gain, size, preview_height, label_guide, label_target = map(
|
fps, audio, gain, size, preview_height, label_guide, label_target, free_memory_after = map(
|
||||||
one, (fps, audio, gain, size, preview_height, label_guide, label_target))
|
one, (fps, audio, gain, size, preview_height, label_guide, label_target, free_memory_after))
|
||||||
guide, target = video_batches(guide, "guide"), video_batches(target, "target")
|
guide, target = video_batches(guide, "guide"), video_batches(target, "target")
|
||||||
ng, nt = sum(x.shape[0] for x in guide), sum(x.shape[0] for x in target)
|
ng, nt = sum(x.shape[0] for x in guide), sum(x.shape[0] for x in target)
|
||||||
if len(guide) > 1 or len(target) > 1:
|
if len(guide) > 1 or len(target) > 1:
|
||||||
@@ -195,6 +208,9 @@ class BFSVideoCompare:
|
|||||||
w.close()
|
w.close()
|
||||||
diff_map = torch.cat(maps, 0)
|
diff_map = torch.cat(maps, 0)
|
||||||
mae, mse = torch.cat(mae), torch.cat(mse)
|
mae, mse = torch.cat(mae), torch.cat(mse)
|
||||||
|
# a single frame is a picture: lossless PNGs at the compared size, which the player's ⬇ save PNG downloads
|
||||||
|
stills = save_stills({"guide": video_chunk(guide, 0, 1, H, W), "target": video_chunk(target, 0, 1, H, W),
|
||||||
|
"difference": diff_map[:1]}, out_dir) if n == 1 else {}
|
||||||
mean = float(mae.mean())
|
mean = float(mae.mean())
|
||||||
worst, best = int(mae.argmax()), int(mae.argmin())
|
worst, best = int(mae.argmax()), int(mae.argmin())
|
||||||
report = "\n".join([
|
report = "\n".join([
|
||||||
@@ -210,7 +226,16 @@ class BFSVideoCompare:
|
|||||||
ui = {"bfs_compare": [{**files, "fps": float(fps), "frames": n, "width": PW, "height": PH,
|
ui = {"bfs_compare": [{**files, "fps": float(fps), "frames": n, "width": PW, "height": PH,
|
||||||
"labels": [label_guide, label_target], "mae": [round(float(x), 2) for x in mae],
|
"labels": [label_guide, label_target], "mae": [round(float(x), 2) for x in mae],
|
||||||
"refs": refs,
|
"refs": refs,
|
||||||
|
"stills": {k: {"filename": f, "subfolder": sub, "type": "temp"} for k, f in stills.items()},
|
||||||
"report": report, "notes": notes}]}
|
"report": report, "notes": notes}]}
|
||||||
|
if free_memory_after:
|
||||||
|
try:
|
||||||
|
from server import PromptServer
|
||||||
|
except ImportError: # the comparison can also run outside ComfyUI
|
||||||
|
PromptServer = None
|
||||||
|
queue = getattr(getattr(PromptServer, "instance", None), "prompt_queue", None)
|
||||||
|
if queue is not None:
|
||||||
|
queue.set_flag("free_memory", True)
|
||||||
return {"ui": ui, "result": (diff_map, report, mean)}
|
return {"ui": ui, "result": (diff_map, report, mean)}
|
||||||
|
|
||||||
|
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,227 @@
|
|||||||
|
"""BFS Workflow Loop: run any part of any workflow once per item, injecting the item (and per-iteration parameters)
|
||||||
|
into nodes you pick, and collect the results in order.
|
||||||
|
|
||||||
|
Connect the IMAGE the loop should produce to `result`, and the batch to iterate over to `items` (frames, keyframes,
|
||||||
|
marked frames...) and optionally `item_masks`. In the panel pick, among the nodes upstream of `result`:
|
||||||
|
- inject: a node whose output is replaced, on iteration i, by item i (e.g. a Load Image that stands for "the frame",
|
||||||
|
or a mask loader for "its mask"),
|
||||||
|
- params: widgets that take one value per iteration (a seed, a prompt line, a strength...), cycled if shorter.
|
||||||
|
On run the node does not compute `result` itself (lazy input): it expands one copy per iteration of the part of the graph
|
||||||
|
that depends on the injected nodes or the changed widgets; models and everything else are shared and run once.
|
||||||
|
|
||||||
|
State lives in the hidden `loop` STRING widget (JSON), saved with the workflow:
|
||||||
|
{"inject": [{"node": "6", "slot": 0, "source": "items"}],
|
||||||
|
"params": [{"node": "34", "widget": "noise_seed", "values": [1, 2, 3]}]}
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
|
||||||
|
import torch
|
||||||
|
import torch.nn.functional as F
|
||||||
|
|
||||||
|
from .bfs_xy_sweep import _cast
|
||||||
|
|
||||||
|
CATEGORY = "BFS/Loop"
|
||||||
|
MAX_ITEMS = 256
|
||||||
|
|
||||||
|
|
||||||
|
def parse_loop(text):
|
||||||
|
try:
|
||||||
|
s = json.loads(text or "{}") or {}
|
||||||
|
except Exception:
|
||||||
|
s = {}
|
||||||
|
inject = [{"node": str(i["node"]), "slot": int(i.get("slot", 0)), "source": i.get("source", "items")}
|
||||||
|
for i in s.get("inject", []) if isinstance(i, dict) and i.get("node") not in (None, "")]
|
||||||
|
params = [{"node": str(p["node"]), "widget": str(p["widget"]), "values": list(p["values"])}
|
||||||
|
for p in s.get("params", []) if isinstance(p, dict) and p.get("node") not in (None, "") and p.get("widget")
|
||||||
|
and isinstance(p.get("values"), list) and p["values"]]
|
||||||
|
return inject, params
|
||||||
|
|
||||||
|
|
||||||
|
class BFSWorkflowLoop:
|
||||||
|
DESCRIPTION = __doc__.split("\n\n")[0]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {
|
||||||
|
"result": ("IMAGE", {"lazy": True, "tooltip": "What one iteration produces; everything upstream of it can be looped."}),
|
||||||
|
"loop": ("STRING", {"default": "{}", "multiline": True, "tooltip": "Set by the panel: injected nodes and per-iteration widgets."}),
|
||||||
|
}, "optional": {
|
||||||
|
"items": ("IMAGE", {"tooltip": "The batch to iterate over: item i is injected on iteration i."}),
|
||||||
|
"items_b": ("IMAGE", {"tooltip": "A second batch, same length (e.g. the marked frames next to the plain ones)."}),
|
||||||
|
"item_masks": ("MASK", {"tooltip": "One mask per item, injected where the panel says (source 'item_masks')."}),
|
||||||
|
"count": ("INT", {"default": 0, "min": 0, "max": MAX_ITEMS,
|
||||||
|
"tooltip": "Iterations when there are no items (params only); 0 = as many as the longest value list."}),
|
||||||
|
}, "hidden": {"dynprompt": "DYNPROMPT", "unique_id": "UNIQUE_ID"}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "INT")
|
||||||
|
RETURN_NAMES = ("images", "count")
|
||||||
|
OUTPUT_TOOLTIPS = ("The result of every iteration, in order, as one batch (resized to the first one's size).", "Iterations run.")
|
||||||
|
FUNCTION = "run_loop"
|
||||||
|
CATEGORY = CATEGORY
|
||||||
|
|
||||||
|
def check_lazy_status(self, result=None, **kw):
|
||||||
|
return []
|
||||||
|
|
||||||
|
def run_loop(self, loop="{}", result=None, items=None, items_b=None, item_masks=None, count=0, dynprompt=None, unique_id=None):
|
||||||
|
inject, params = parse_loop(loop)
|
||||||
|
if not inject and not params:
|
||||||
|
raise ValueError("BFS Workflow Loop: pick a node to inject the items into, or a widget to change, in the panel.")
|
||||||
|
if dynprompt is None:
|
||||||
|
raise ValueError("BFS Workflow Loop needs ComfyUI's dynamic graph execution")
|
||||||
|
from comfy_execution.graph_utils import GraphBuilder, is_link
|
||||||
|
me = str(unique_id); mine = dynprompt.get_node(me)["inputs"]
|
||||||
|
link = mine.get("result")
|
||||||
|
if not is_link(link):
|
||||||
|
raise ValueError("Connect what one iteration produces to `result`.")
|
||||||
|
n = int(items.shape[0]) if items is not None else (int(count) or max(len(p["values"]) for p in params) if params else 0)
|
||||||
|
if any(i["source"] == "items" for i in inject) and items is None:
|
||||||
|
raise ValueError("The panel injects `items`, but nothing is connected to items.")
|
||||||
|
if any(i["source"] == "items_b" for i in inject):
|
||||||
|
if items_b is None:
|
||||||
|
raise ValueError("The panel injects `items_b`, but nothing is connected to items_b.")
|
||||||
|
if items is not None and items_b.shape[0] != n:
|
||||||
|
raise ValueError(f"{items_b.shape[0]} items_b for {n} items.")
|
||||||
|
n = n or int(items_b.shape[0])
|
||||||
|
if any(i["source"] == "item_masks" for i in inject):
|
||||||
|
if item_masks is None:
|
||||||
|
raise ValueError("The panel injects `item_masks`, but nothing is connected to item_masks.")
|
||||||
|
if items is not None and item_masks.shape[0] != n:
|
||||||
|
raise ValueError(f"{item_masks.shape[0]} masks for {n} items.")
|
||||||
|
if n < 1:
|
||||||
|
raise ValueError("Nothing to iterate over: connect items or give parameter values.")
|
||||||
|
if n > MAX_ITEMS:
|
||||||
|
raise ValueError(f"{n} iterations; at most {MAX_ITEMS}.")
|
||||||
|
root = str(link[0])
|
||||||
|
up, feeds, stack = set(), {}, [root]
|
||||||
|
while stack:
|
||||||
|
nid = stack.pop()
|
||||||
|
if nid in up:
|
||||||
|
continue
|
||||||
|
up.add(nid)
|
||||||
|
for v in dynprompt.get_node(nid).get("inputs", {}).values():
|
||||||
|
if is_link(v):
|
||||||
|
p = str(v[0]); feeds.setdefault(p, set()).add(nid); stack.append(p)
|
||||||
|
for i in inject:
|
||||||
|
if i["node"] not in up:
|
||||||
|
raise ValueError(f"Injected node {i['node']} is not upstream of `result` (pick it again in the panel).")
|
||||||
|
for p in params:
|
||||||
|
if p["node"] not in up:
|
||||||
|
raise ValueError(f"Node {p['node']} is not upstream of `result` (pick it again in the panel).")
|
||||||
|
if p["widget"] not in dynprompt.get_node(p["node"]).get("inputs", {}):
|
||||||
|
raise ValueError(f"Node {p['node']} has no widget '{p['widget']}' in this prompt.")
|
||||||
|
injected = {(i["node"], i["slot"]): i["source"] for i in inject}
|
||||||
|
# the body: everything that depends on an injected output or a changed widget and leads to `result`
|
||||||
|
seeds = [p["node"] for p in params] + [c for i in inject for c in feeds.get(i["node"], ()) if c in up]
|
||||||
|
body, stack = set(), list(seeds)
|
||||||
|
while stack:
|
||||||
|
nid = stack.pop()
|
||||||
|
if nid in body:
|
||||||
|
continue
|
||||||
|
body.add(nid)
|
||||||
|
stack.extend(c for c in feeds.get(nid, ()) if c in up)
|
||||||
|
body -= {i["node"] for i in inject if i["node"] not in {p["node"] for p in params}}
|
||||||
|
graph = GraphBuilder()
|
||||||
|
outs = []
|
||||||
|
for it in range(n):
|
||||||
|
picks = {}
|
||||||
|
for src in set(injected.values()):
|
||||||
|
pk = graph.node("BFSLoopPick", f"pick{it}_{src}")
|
||||||
|
if src == "item_masks":
|
||||||
|
pk.set_input("masks", mine["item_masks"])
|
||||||
|
else:
|
||||||
|
pk.set_input("images", mine[src])
|
||||||
|
pk.set_input("index", it)
|
||||||
|
picks[src] = pk
|
||||||
|
over = {(p["node"], p["widget"]): p["values"][it % len(p["values"])] for p in params}
|
||||||
|
clones = {nid: graph.node(dynprompt.get_node(nid)["class_type"], f"i{it}_{nid}") for nid in body}
|
||||||
|
for nid, node in clones.items():
|
||||||
|
node.set_override_display_id(dynprompt.get_display_node_id(nid))
|
||||||
|
for key, value in dynprompt.get_node(nid).get("inputs", {}).items():
|
||||||
|
if (nid, key) in over:
|
||||||
|
node.set_input(key, _cast(over[(nid, key)], value))
|
||||||
|
elif is_link(value) and (str(value[0]), int(value[1])) in injected:
|
||||||
|
src = injected[(str(value[0]), int(value[1]))]
|
||||||
|
node.set_input(key, picks[src].out(1 if src == "item_masks" else 0))
|
||||||
|
elif is_link(value) and str(value[0]) in body:
|
||||||
|
node.set_input(key, clones[str(value[0])].out(value[1]))
|
||||||
|
else:
|
||||||
|
node.set_input(key, value)
|
||||||
|
if root in clones:
|
||||||
|
outs.append(clones[root].out(link[1]))
|
||||||
|
elif (root, int(link[1])) in injected:
|
||||||
|
src = injected[(root, int(link[1]))]; outs.append(picks[src].out(1 if src == "item_masks" else 0))
|
||||||
|
else:
|
||||||
|
outs.append(link)
|
||||||
|
coll = graph.node("BFSLoopCollect", "collect")
|
||||||
|
for i, o in enumerate(outs):
|
||||||
|
coll.set_input(f"image_{i}", o)
|
||||||
|
coll.set_input("n", n)
|
||||||
|
return {"result": (coll.out(0), coll.out(1)), "expand": graph.finalize()}
|
||||||
|
|
||||||
|
|
||||||
|
class BFSLoopPick:
|
||||||
|
"""Internal: item `index` of a batch (image and/or mask)."""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {"index": ("INT", {"default": 0, "min": 0, "max": 100000})},
|
||||||
|
"optional": {"images": ("IMAGE",), "masks": ("MASK",)}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "MASK")
|
||||||
|
FUNCTION = "pick"
|
||||||
|
CATEGORY = CATEGORY
|
||||||
|
|
||||||
|
def pick(self, index, images=None, masks=None):
|
||||||
|
img = images[index:index + 1] if images is not None else torch.zeros(1, 64, 64, 3)
|
||||||
|
if masks is not None:
|
||||||
|
m = masks if masks.ndim == 3 else masks[None]
|
||||||
|
msk = m[index:index + 1]
|
||||||
|
else:
|
||||||
|
msk = torch.zeros(1, img.shape[1], img.shape[2])
|
||||||
|
return img, msk
|
||||||
|
|
||||||
|
|
||||||
|
class BFSLoopCollect:
|
||||||
|
"""Internal: every iteration's result, in order, as one batch."""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {"n": ("INT", {"default": 1, "min": 1, "max": MAX_ITEMS})},
|
||||||
|
"optional": {f"image_{i}": ("IMAGE",) for i in range(MAX_ITEMS)}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "INT")
|
||||||
|
FUNCTION = "collect"
|
||||||
|
CATEGORY = CATEGORY
|
||||||
|
|
||||||
|
def collect(self, n, **kw):
|
||||||
|
parts = [kw[f"image_{i}"] for i in range(n)]
|
||||||
|
h, w = parts[0].shape[1:3]
|
||||||
|
parts = [p if p.shape[1:3] == (h, w) else
|
||||||
|
F.interpolate(p.movedim(-1, 1), size=(h, w), mode="bilinear", align_corners=False).movedim(1, -1) for p in parts]
|
||||||
|
return torch.cat(parts), n
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {"BFSWorkflowLoop": BFSWorkflowLoop, "BFSLoopPick": BFSLoopPick, "BFSLoopCollect": BFSLoopCollect}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {"BFSWorkflowLoop": "BFS Workflow Loop", "BFSLoopPick": "BFS Loop Pick (internal)",
|
||||||
|
"BFSLoopCollect": "BFS Loop Collect (internal)"}
|
||||||
|
|
||||||
|
|
||||||
|
class BFSLoopSlot:
|
||||||
|
"""A stand-in for "the current item" inside a BFS Workflow Loop body: pick it in the loop's Inject list and it gets
|
||||||
|
item i on iteration i. Outside a loop it gives a small gray image (no file to load, so validation never fails)."""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {"label": ("STRING", {"default": "item", "tooltip": "Only a name, to tell slots apart in the loop's list."})}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "MASK")
|
||||||
|
RETURN_NAMES = ("image", "mask")
|
||||||
|
FUNCTION = "slot"
|
||||||
|
CATEGORY = CATEGORY
|
||||||
|
|
||||||
|
def slot(self, label="item"):
|
||||||
|
return torch.full((1, 64, 64, 3), 0.5), torch.zeros(1, 64, 64)
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS["BFSLoopSlot"] = BFSLoopSlot
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS["BFSLoopSlot"] = "BFS Loop Slot"
|
||||||
+241
@@ -0,0 +1,241 @@
|
|||||||
|
"""BFS XY Sweep: run any workflow over a grid of widget values and get one labeled grid back.
|
||||||
|
|
||||||
|
Connect the IMAGE you want to compare (a decoded video or an image) to `result`. In the panel, pick for X (and
|
||||||
|
optionally Y) any widget of any node upstream of that input (the panel lists them, like TinyTerra's advanced xyPlot)
|
||||||
|
and type the values. On run the node does not compute `result` itself (lazy input): it expands one copy of the part of
|
||||||
|
the graph that depends on the swept widgets per combination (everything else, models included, is shared and runs
|
||||||
|
once), then a collector builds the grid.
|
||||||
|
|
||||||
|
All state lives in the hidden `sweep` STRING widget (JSON), saved with the workflow:
|
||||||
|
{"x": {"node": "44", "widget": "lora_name", "values": ["a.safetensors", "b.safetensors"]},
|
||||||
|
"y": {"node": "101", "widget": "keyframe_until", "values": [0.5, 0.7, 0.9]}}
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
import torch
|
||||||
|
import torch.nn.functional as F
|
||||||
|
from PIL import Image, ImageDraw, ImageFont
|
||||||
|
|
||||||
|
try:
|
||||||
|
import folder_paths
|
||||||
|
except Exception: # tests run without ComfyUI
|
||||||
|
folder_paths = None
|
||||||
|
|
||||||
|
CATEGORY = "BFS/Loop"
|
||||||
|
MAX_CELLS = 64
|
||||||
|
|
||||||
|
|
||||||
|
def parse_sweep(text):
|
||||||
|
try:
|
||||||
|
s = json.loads(text or "{}") or {}
|
||||||
|
except Exception:
|
||||||
|
s = {}
|
||||||
|
axes = []
|
||||||
|
for key in ("x", "y"):
|
||||||
|
a = s.get(key) or {}
|
||||||
|
if a.get("node") not in (None, "") and a.get("widget") and isinstance(a.get("values"), list) and a["values"]:
|
||||||
|
axes.append({"node": str(a["node"]), "widget": str(a["widget"]), "values": list(a["values"]), "title": a.get("title", "")})
|
||||||
|
return axes
|
||||||
|
|
||||||
|
|
||||||
|
def _cast(value, like):
|
||||||
|
"""A typed value from the panel, cast to the type the widget had in the prompt."""
|
||||||
|
if isinstance(like, bool):
|
||||||
|
return value if isinstance(value, bool) else str(value).strip().lower() in ("1", "true", "yes", "on")
|
||||||
|
if isinstance(like, int) and not isinstance(like, bool):
|
||||||
|
return int(round(float(value)))
|
||||||
|
if isinstance(like, float):
|
||||||
|
return float(value)
|
||||||
|
return value if isinstance(value, str) else str(value)
|
||||||
|
|
||||||
|
|
||||||
|
def _short(v):
|
||||||
|
s = str(v)
|
||||||
|
if "/" in s or "\\" in s:
|
||||||
|
s = s.replace("\\", "/").rsplit("/", 1)[-1]
|
||||||
|
for ext in (".safetensors", ".ckpt", ".pt", ".png", ".jpg", ".mp4"):
|
||||||
|
if s.endswith(ext):
|
||||||
|
s = s[: -len(ext)]
|
||||||
|
return s if len(s) <= 40 else "…" + s[-39:]
|
||||||
|
|
||||||
|
|
||||||
|
def _font(size):
|
||||||
|
for name in ("DejaVuSans.ttf", "arial.ttf", "LiberationSans-Regular.ttf"):
|
||||||
|
try:
|
||||||
|
return ImageFont.truetype(name, size)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
return ImageFont.load_default()
|
||||||
|
|
||||||
|
|
||||||
|
class BFSXYSweep:
|
||||||
|
DESCRIPTION = __doc__.split("\n\n")[0]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {
|
||||||
|
"result": ("IMAGE", {"lazy": True, "tooltip": "The image or decoded video to compare; the sweep runs everything upstream of it."}),
|
||||||
|
"frames_per_cell": ("INT", {"default": 4, "min": 1, "max": 16, "tooltip": "For videos: how many frames (evenly spread) each cell shows."}),
|
||||||
|
"cell_height": ("INT", {"default": 256, "min": 64, "max": 1024, "step": 16}),
|
||||||
|
"save_videos": ("BOOLEAN", {"default": True, "tooltip": "Save every multi-frame result as an mp4 in output/bfs_sweep/, named by its values."}),
|
||||||
|
"fps": ("FLOAT", {"default": 24.0, "min": 1.0, "max": 120.0}),
|
||||||
|
"sweep": ("STRING", {"default": "{}", "multiline": True, "tooltip": "Set by the panel: the swept widgets and their values."}),
|
||||||
|
}, "hidden": {"dynprompt": "DYNPROMPT", "unique_id": "UNIQUE_ID"}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "IMAGE", "STRING")
|
||||||
|
RETURN_NAMES = ("grid", "results", "labels")
|
||||||
|
OUTPUT_TOOLTIPS = ("The labeled grid.", "Every result (the cells' frames), resized to one size.", "JSON: the values of every cell, in order.")
|
||||||
|
FUNCTION = "sweep_run"
|
||||||
|
CATEGORY = CATEGORY
|
||||||
|
OUTPUT_NODE = True
|
||||||
|
|
||||||
|
def check_lazy_status(self, result=None, **kw):
|
||||||
|
return [] # never evaluate `result` here: the expansion evaluates one copy per combination
|
||||||
|
|
||||||
|
def sweep_run(self, result=None, frames_per_cell=4, cell_height=256, save_videos=True, fps=24.0, sweep="{}",
|
||||||
|
dynprompt=None, unique_id=None):
|
||||||
|
axes = parse_sweep(sweep)
|
||||||
|
if not axes:
|
||||||
|
raise ValueError("BFS XY Sweep: choose at least the X widget and its values in the panel.")
|
||||||
|
if dynprompt is None:
|
||||||
|
raise ValueError("BFS XY Sweep needs ComfyUI's dynamic graph execution")
|
||||||
|
from comfy_execution.graph_utils import GraphBuilder, is_link
|
||||||
|
me = str(unique_id)
|
||||||
|
link = dynprompt.get_node(me)["inputs"].get("result")
|
||||||
|
if not is_link(link):
|
||||||
|
raise ValueError("Connect the image to compare to `result`.")
|
||||||
|
root = str(link[0])
|
||||||
|
# nodes upstream of `result` and who feeds whom inside that set
|
||||||
|
up, feeds, stack = set(), {}, [root]
|
||||||
|
while stack:
|
||||||
|
nid = stack.pop()
|
||||||
|
if nid in up:
|
||||||
|
continue
|
||||||
|
up.add(nid)
|
||||||
|
for v in dynprompt.get_node(nid).get("inputs", {}).values():
|
||||||
|
if is_link(v):
|
||||||
|
p = str(v[0]); feeds.setdefault(p, set()).add(nid); stack.append(p)
|
||||||
|
for a in axes:
|
||||||
|
if a["node"] not in up:
|
||||||
|
raise ValueError(f"Swept node {a['node']} is not upstream of `result` (pick it again in the panel).")
|
||||||
|
if a["widget"] not in dynprompt.get_node(a["node"]).get("inputs", {}):
|
||||||
|
raise ValueError(f"Node {a['node']} has no widget '{a['widget']}' in this prompt.")
|
||||||
|
# the body: what depends on a swept node and leads to `result` (cloned per cell); the rest is shared
|
||||||
|
body, stack = set(), [a["node"] for a in axes]
|
||||||
|
while stack:
|
||||||
|
nid = stack.pop()
|
||||||
|
if nid in body:
|
||||||
|
continue
|
||||||
|
body.add(nid)
|
||||||
|
stack.extend(c for c in feeds.get(nid, ()) if c in up)
|
||||||
|
xs = axes[0]["values"]; ys = axes[1]["values"] if len(axes) > 1 else [None]
|
||||||
|
cells = [(x, y) for y in ys for x in xs]
|
||||||
|
if len(cells) > MAX_CELLS:
|
||||||
|
raise ValueError(f"{len(cells)} combinations; at most {MAX_CELLS}.")
|
||||||
|
graph = GraphBuilder()
|
||||||
|
outs, labels = [], []
|
||||||
|
for ci, (x, y) in enumerate(cells):
|
||||||
|
over = {(axes[0]["node"], axes[0]["widget"]): x}
|
||||||
|
if y is not None:
|
||||||
|
over[(axes[1]["node"], axes[1]["widget"])] = y
|
||||||
|
clones = {n: graph.node(dynprompt.get_node(n)["class_type"], f"c{ci}_{n}") for n in body}
|
||||||
|
for nid, node in clones.items():
|
||||||
|
node.set_override_display_id(dynprompt.get_display_node_id(nid))
|
||||||
|
for key, value in dynprompt.get_node(nid).get("inputs", {}).items():
|
||||||
|
if (nid, key) in over:
|
||||||
|
node.set_input(key, _cast(over[(nid, key)], value))
|
||||||
|
elif is_link(value) and str(value[0]) in body:
|
||||||
|
node.set_input(key, clones[str(value[0])].out(value[1]))
|
||||||
|
else:
|
||||||
|
node.set_input(key, value)
|
||||||
|
outs.append(clones[root].out(link[1]) if root in clones else link)
|
||||||
|
labels.append({"x": x, "y": y})
|
||||||
|
meta = {"axes": [{k: a[k] for k in ("node", "widget", "title")} for a in axes], "cells": labels,
|
||||||
|
"xs": xs, "ys": ys if len(axes) > 1 else [], "frames_per_cell": frames_per_cell, "cell_height": cell_height,
|
||||||
|
"save_videos": save_videos, "fps": fps}
|
||||||
|
coll = graph.node("BFSXYSweepCollect", "collect")
|
||||||
|
coll.set_input("meta", json.dumps(meta))
|
||||||
|
for i, o in enumerate(outs):
|
||||||
|
coll.set_input(f"image_{i}", o)
|
||||||
|
return {"result": (coll.out(0), coll.out(1), coll.out(2)), "expand": graph.finalize()}
|
||||||
|
|
||||||
|
|
||||||
|
class BFSXYSweepCollect:
|
||||||
|
"""Internal: receives every cell's result from BFS XY Sweep and draws the grid."""
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def INPUT_TYPES(cls):
|
||||||
|
return {"required": {"meta": ("STRING", {"default": "{}"})},
|
||||||
|
"optional": {f"image_{i}": ("IMAGE",) for i in range(MAX_CELLS)}}
|
||||||
|
|
||||||
|
RETURN_TYPES = ("IMAGE", "IMAGE", "STRING")
|
||||||
|
RETURN_NAMES = ("grid", "results", "labels")
|
||||||
|
FUNCTION = "collect"
|
||||||
|
CATEGORY = CATEGORY
|
||||||
|
OUTPUT_NODE = True
|
||||||
|
|
||||||
|
def collect(self, meta, **kw):
|
||||||
|
m = json.loads(meta)
|
||||||
|
cells = m["cells"]; k = int(m["frames_per_cell"]); ch = int(m["cell_height"])
|
||||||
|
imgs = [kw.get(f"image_{i}") for i in range(len(cells))]
|
||||||
|
strips, saved = [], []
|
||||||
|
for i, (img, lab) in enumerate(zip(imgs, cells)):
|
||||||
|
idx = np.unique(np.linspace(0, img.shape[0] - 1, min(k, img.shape[0])).round().astype(int))
|
||||||
|
fr = img[idx, ..., :3].float()
|
||||||
|
w = max(1, round(fr.shape[2] * ch / fr.shape[1]))
|
||||||
|
fr = F.interpolate(fr.movedim(-1, 1), size=(ch, w), mode="bilinear", align_corners=False).movedim(1, -1)
|
||||||
|
strips.append(torch.cat(list(fr), dim=1))
|
||||||
|
if m.get("save_videos") and img.shape[0] > 1 and folder_paths:
|
||||||
|
saved.append(self._save(img, lab, m, i))
|
||||||
|
sw = max(s.shape[1] for s in strips)
|
||||||
|
strips = [F.pad(s, (0, 0, 0, sw - s.shape[1])) for s in strips]
|
||||||
|
xs, ys = m["xs"], m["ys"] or [None]
|
||||||
|
ax = m["axes"]
|
||||||
|
xl = f"{ax[0]['title'] or ax[0]['node']}.{ax[0]['widget']}"
|
||||||
|
yl = f"{ax[1]['title'] or ax[1]['node']}.{ax[1]['widget']}" if len(ax) > 1 else ""
|
||||||
|
pad, top, left = 6, 34, (170 if len(ax) > 1 else 0)
|
||||||
|
H = top + len(ys) * (ch + pad); W = left + len(xs) * (sw + pad)
|
||||||
|
canvas = Image.new("RGB", (W, H), (24, 24, 24)); d = ImageDraw.Draw(canvas); f = _font(16)
|
||||||
|
for c, x in enumerate(xs):
|
||||||
|
d.text((left + c * (sw + pad) + 4, 8), f"{xl} = {_short(x)}", fill=(230, 230, 230), font=f)
|
||||||
|
for r, y in enumerate(ys):
|
||||||
|
if y is not None:
|
||||||
|
d.text((6, top + r * (ch + pad) + ch // 2 - 10), f"{yl} =\n{_short(y)}", fill=(230, 230, 230), font=f)
|
||||||
|
for c in range(len(xs)):
|
||||||
|
s = strips[r * len(xs) + c]
|
||||||
|
arr = (s.clamp(0, 1).cpu().numpy() * 255).astype(np.uint8)
|
||||||
|
canvas.paste(Image.fromarray(arr), (left + c * (sw + pad), top + r * (ch + pad)))
|
||||||
|
grid = torch.from_numpy(np.asarray(canvas, np.float32) / 255.0)[None]
|
||||||
|
results = torch.stack(strips)
|
||||||
|
info = {"cells": cells, "videos": saved}
|
||||||
|
ui = {}
|
||||||
|
if folder_paths:
|
||||||
|
out = os.path.join(folder_paths.get_output_directory(), "bfs_sweep"); os.makedirs(out, exist_ok=True)
|
||||||
|
name = f"grid_{len(os.listdir(out)):05d}.png"; canvas.save(os.path.join(out, name))
|
||||||
|
ui = {"images": [{"filename": name, "subfolder": "bfs_sweep", "type": "output"}]}
|
||||||
|
return {"ui": ui, "result": (grid, results, json.dumps(info))}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _save(img, lab, m, i):
|
||||||
|
import av
|
||||||
|
out = os.path.join(folder_paths.get_output_directory(), "bfs_sweep"); os.makedirs(out, exist_ok=True)
|
||||||
|
tag = "_".join(_short(v).replace(" ", "") for v in (lab["x"], lab["y"]) if v is not None)
|
||||||
|
tag = "".join(ch if ch.isalnum() or ch in "._-" else "-" for ch in tag)[:80]
|
||||||
|
path = os.path.join(out, f"{i:02d}_{tag}.mp4")
|
||||||
|
frames = (img[..., :3].clamp(0, 1).cpu().numpy() * 255).astype(np.uint8)
|
||||||
|
h, w = frames.shape[1] // 2 * 2, frames.shape[2] // 2 * 2
|
||||||
|
with av.open(path, "w") as c:
|
||||||
|
st = c.add_stream("libx264", rate=int(round(float(m.get("fps", 24)))))
|
||||||
|
st.width, st.height, st.pix_fmt = w, h, "yuv420p"; st.options = {"crf": "16"}
|
||||||
|
for fr in frames:
|
||||||
|
for p in st.encode(av.VideoFrame.from_ndarray(fr[:h, :w], format="rgb24")):
|
||||||
|
c.mux(p)
|
||||||
|
for p in st.encode():
|
||||||
|
c.mux(p)
|
||||||
|
return path
|
||||||
|
|
||||||
|
|
||||||
|
NODE_CLASS_MAPPINGS = {"BFSXYSweep": BFSXYSweep, "BFSXYSweepCollect": BFSXYSweepCollect}
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS = {"BFSXYSweep": "BFS XY Sweep", "BFSXYSweepCollect": "BFS XY Sweep Collect (internal)"}
|
||||||
@@ -420,3 +420,7 @@ NODE_DISPLAY_NAME_MAPPINGS = {
|
|||||||
'BFSMiniMaxH3GuideTarget': 'MiniMax-H3 Guide Target — Image / Video (BFS)',
|
'BFSMiniMaxH3GuideTarget': 'MiniMax-H3 Guide Target — Image / Video (BFS)',
|
||||||
'BFSMiniMaxH3IdentityReference': 'MiniMax-H3 Identity Reference + RoPE (BFS)',
|
'BFSMiniMaxH3IdentityReference': 'MiniMax-H3 Identity Reference + RoPE (BFS)',
|
||||||
}
|
}
|
||||||
|
|
||||||
|
from .bfs_h3_image_guide import BFSH3ImageAddGuide
|
||||||
|
NODE_CLASS_MAPPINGS['BFSH3ImageAddGuide'] = BFSH3ImageAddGuide
|
||||||
|
NODE_DISPLAY_NAME_MAPPINGS['BFSH3ImageAddGuide'] = 'Add Guide for MiniMax H3 Image (BFS)'
|
||||||
|
|||||||
+2
-2
@@ -1,7 +1,7 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "bfsnodes"
|
name = "bfsnodes"
|
||||||
description = "Custom ComfyUI nodes for reference-guided video: BFS Sampler (guide-driven IC-LoRAs, auto crop, mask inpainting, temporal chunking), LTX Multiple Controls / Identity Transfer / Edit Anything, Head-Swap and Anime2Real conditioning, MiniMax-H3 latent guides, RoPE layouts and loaders, LoRA Surgery, and Shot Planner / Shot Loop (tabbed panel): split long videos into shots at camera cuts, face-tracked cast, SAM 3 mask per shot with full frame, paste, mask, crop and crop + mask generation modes, optional translucent Colored Mark overlay for manually selected regions, optional Qwen3-VL to suggest settings, describe references ({details}) and the person to replace ({target}), continuity between shots, H3 conditioning with an optional duet and prompt writer, join with the original timing, audio and a side-by-side comparison. H3 Duet: pin a source clip beside the video, training-free."
|
description = "ComfyUI nodes for video workflows: SAM 3 segmentation and Molmo2 tracking; MiniMax H3 character swapping and sampling; shot planning and looping; LTX video controls and identity transfer; model loaders, LoRA tools, and image/video utilities."
|
||||||
version = "1.115.33"
|
version = "1.120.0"
|
||||||
license = {file = "LICENSE"}
|
license = {file = "LICENSE"}
|
||||||
dependencies = [
|
dependencies = [
|
||||||
"torch",
|
"torch",
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
import importlib.util
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
import torch
|
||||||
|
if os.environ.get('COMFYUI_PATH'):
|
||||||
|
sys.path.insert(0, os.environ['COMFYUI_PATH'])
|
||||||
|
from comfy.ldm.minimax.model import PackedLayout
|
||||||
|
spec = importlib.util.spec_from_file_location('image_guide', Path(__file__).resolve().parents[1] / 'bfs_h3_image_guide.py')
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
|
||||||
|
class ImageGuideTest(unittest.TestCase):
|
||||||
|
def test_only_guide_positions_are_aligned_and_inputs_stay_unchanged(self):
|
||||||
|
refs = [{'kind': 'image', 'latent_h': 8, 'latent_w': 8},
|
||||||
|
{'kind': 'image', 'latent_h': 8, 'latent_w': 8, module.MARKER: True}]
|
||||||
|
original = PackedLayout(4, 1, 8, 8, 0, refs=refs)
|
||||||
|
positions = original.position_ids.clone()
|
||||||
|
result = module.align_guide_layout(original, refs)
|
||||||
|
ref_segments = [(a,b) for a,b,k in result.segments if k == 'ref_img']
|
||||||
|
a,b = ref_segments[1]
|
||||||
|
ta,tb = next((a,b) for a,b,k in result.segments if k == 'video')
|
||||||
|
self.assertTrue(torch.equal(result.position_ids[a:b], result.position_ids[ta:tb]))
|
||||||
|
a,b = ref_segments[0]
|
||||||
|
self.assertTrue(torch.equal(result.position_ids[a:b], positions[a:b]))
|
||||||
|
self.assertTrue(torch.equal(original.position_ids, positions))
|
||||||
|
self.assertTrue(torch.equal(result.img_update, original.img_update))
|
||||||
|
|
||||||
|
def test_incompatible_guide_grid_is_rejected(self):
|
||||||
|
refs = [{'kind': 'image', 'latent_h': 4, 'latent_w': 4, module.MARKER: True}]
|
||||||
|
layout = PackedLayout(4, 1, 8, 8, 0, refs=refs)
|
||||||
|
with self.assertRaisesRegex(ValueError, 'match the target'):
|
||||||
|
module.align_guide_layout(layout, refs)
|
||||||
@@ -0,0 +1,60 @@
|
|||||||
|
import importlib.util
|
||||||
|
import sys
|
||||||
|
import os
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
import torch
|
||||||
|
|
||||||
|
if os.environ.get('COMFYUI_PATH'):
|
||||||
|
sys.path.insert(0, os.environ['COMFYUI_PATH'])
|
||||||
|
spec = importlib.util.spec_from_file_location('image_swap', Path(__file__).resolve().parents[1] / 'bfs_h3_image_swap_conditioning.py')
|
||||||
|
module = importlib.util.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(module)
|
||||||
|
|
||||||
|
|
||||||
|
class Clip:
|
||||||
|
def tokenize(self, prompt, **kwargs):
|
||||||
|
self.prompt, self.images = prompt, kwargs['images']
|
||||||
|
return {'text': prompt}
|
||||||
|
|
||||||
|
def encode_from_tokens_scheduled(self, tokens):
|
||||||
|
return [[torch.zeros(1, 4, 8), {}]]
|
||||||
|
|
||||||
|
|
||||||
|
class Vae:
|
||||||
|
def encode(self, pixels):
|
||||||
|
return torch.zeros(1, 24, 1, pixels.shape[1] // 16, pixels.shape[2] // 16)
|
||||||
|
|
||||||
|
|
||||||
|
class ImageSwapTest(unittest.TestCase):
|
||||||
|
def test_modes_use_still_image_language_and_shifted_picture_numbers(self):
|
||||||
|
for mode in module.MODES:
|
||||||
|
text = module.image_swap_prompt(3, mode)
|
||||||
|
for word in ('video', 'motion', 'frame', 'timing'):
|
||||||
|
self.assertNotIn(word, text.lower())
|
||||||
|
for j, color in enumerate(module.COLORS, 1):
|
||||||
|
self.assertIn(f'<Subject {j}> is the subject marked by the {color}', text)
|
||||||
|
self.assertIn(f'appearance comes from <Picture {j+1}>', text)
|
||||||
|
self.assertIn('foreground occluders', text)
|
||||||
|
|
||||||
|
def test_native_conditioning_order_and_image_latent(self):
|
||||||
|
clip = Clip()
|
||||||
|
guide = torch.full((1, 32, 32, 3), .1)
|
||||||
|
red = torch.full_like(guide, .2)
|
||||||
|
blue = torch.full_like(guide, .3)
|
||||||
|
result = module.BFSH3ImageCharacterSwapConditioning.execute(
|
||||||
|
clip, Vae(), guide, {'reference_2': blue, 'reference_0': red}, width=128, height=96)
|
||||||
|
positive, latent, prompt = result.args
|
||||||
|
self.assertEqual(tuple(latent['samples'].shape), (1, 24, 1, 6, 8))
|
||||||
|
self.assertEqual(len(positive[0][1]['minimax_refs']), 3)
|
||||||
|
self.assertEqual([round(float(i.mean()), 1) for i in clip.images], [.1, .2, .3])
|
||||||
|
self.assertIn('red subject replaced by <Picture 2>', prompt)
|
||||||
|
self.assertIn('blue subject replaced by <Picture 3>', prompt)
|
||||||
|
|
||||||
|
def test_override_preserved(self):
|
||||||
|
clip = Clip()
|
||||||
|
image = torch.zeros(1, 32, 32, 3)
|
||||||
|
result = module.BFSH3ImageCharacterSwapConditioning.execute(
|
||||||
|
clip, Vae(), image, {'reference_0': image}, width=64, height=64, prompt_override='Custom edit')
|
||||||
|
self.assertEqual(result.args[2], 'Custom edit')
|
||||||
|
self.assertEqual(clip.prompt, 'Custom edit')
|
||||||
@@ -0,0 +1,72 @@
|
|||||||
|
import importlib.util
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
SPEC = importlib.util.spec_from_file_location("bfs_h3_swap_conditioning", ROOT / "bfs_h3_swap_conditioning.py")
|
||||||
|
SC = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(SC)
|
||||||
|
NODE = SC.BFSH3KeyframeConditioning
|
||||||
|
|
||||||
|
|
||||||
|
class InputsTest(unittest.TestCase):
|
||||||
|
def test_modes(self):
|
||||||
|
self.assertEqual(SC.MODES, ["keyframe pins", "normal map guide", "DWPose guide", "translucent overlay guide", "outline guide"])
|
||||||
|
|
||||||
|
def test_pictures_grouped_first(self):
|
||||||
|
t = NODE.INPUT_TYPES()
|
||||||
|
self.assertEqual(list(t["required"])[:7], ["clip", "vae", "guide_video", "width", "height", "length", "picture_1"])
|
||||||
|
self.assertEqual(list(t["optional"])[:5], ["picture_2", "picture_3", "model", "keyframes", "keyframe_spec"])
|
||||||
|
|
||||||
|
def test_outputs(self):
|
||||||
|
self.assertEqual(len(NODE.RETURN_TYPES), len(NODE.RETURN_NAMES))
|
||||||
|
self.assertEqual(len(NODE.OUTPUT_IS_LIST), len(NODE.RETURN_TYPES))
|
||||||
|
self.assertEqual(NODE.RETURN_NAMES[-1], "pictures")
|
||||||
|
self.assertTrue(NODE.OUTPUT_IS_LIST[-1])
|
||||||
|
|
||||||
|
def test_unknown_mode(self):
|
||||||
|
import torch
|
||||||
|
with self.assertRaises(ValueError):
|
||||||
|
NODE().encode(None, None, torch.zeros(1, 8, 8, 3), 8, 8, 1, torch.zeros(1, 8, 8, 3), mode="gray video")
|
||||||
|
|
||||||
|
|
||||||
|
class PromptTest(unittest.TestCase):
|
||||||
|
def test_guide_prompts_name_each_subject_once(self):
|
||||||
|
for fn, word in ((SC.normals_caption, "normal map"), (SC.pose_caption, "pose skeleton")):
|
||||||
|
for k in range(1, SC.MAX_PICTURES + 1):
|
||||||
|
p = fn(k)
|
||||||
|
self.assertTrue(p.startswith("character_swap:\nsubject_definitions:\n"))
|
||||||
|
for j in range(1, k + 1):
|
||||||
|
self.assertIn(f"<Subject {j}> is the subject marked by the {SC.SUBJECT_COLORS[j - 1]} {word}", p)
|
||||||
|
self.assertNotIn(f"<Picture {k + 1}>", p)
|
||||||
|
|
||||||
|
def test_pins_prompt(self):
|
||||||
|
p = SC.pins_caption(2, [0.0, 1.5])
|
||||||
|
self.assertIn("2 keyframes of the target video are pinned at 0.0s, 1.5s", p)
|
||||||
|
self.assertIn("<Picture 2>", p)
|
||||||
|
self.assertIn("1 keyframe of the target video is pinned at 2.0s", SC.pins_caption(1, [2.0]))
|
||||||
|
|
||||||
|
def test_overlay_prompt(self):
|
||||||
|
prompt = SC.overlay_caption(2)
|
||||||
|
self.assertIn("translucent red overlay", prompt)
|
||||||
|
self.assertIn("translucent blue overlay", prompt)
|
||||||
|
self.assertIn("original appearance visible beneath it is not the target appearance", prompt)
|
||||||
|
self.assertIn("The tint must disappear in the result", prompt)
|
||||||
|
self.assertNotIn("<Picture 3>", prompt)
|
||||||
|
|
||||||
|
def test_outline_prompt_preserves_props(self):
|
||||||
|
prompt = SC.outline_caption(2)
|
||||||
|
self.assertIn("red outline", prompt)
|
||||||
|
self.assertIn("blue outline", prompt)
|
||||||
|
self.assertIn("held or interacting objects", prompt)
|
||||||
|
self.assertIn("foreground occlusions", prompt)
|
||||||
|
self.assertNotIn("<Picture 3>", prompt)
|
||||||
|
|
||||||
|
def test_normal_map_prompt_preserves_occluders(self):
|
||||||
|
prompt = SC.normals_caption(1)
|
||||||
|
self.assertIn("held objects and foreground occluders", prompt)
|
||||||
|
self.assertIn("original positions, shapes and colors", prompt)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,265 @@
|
|||||||
|
import importlib.util
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
import types
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import torch
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
SPEC = importlib.util.spec_from_file_location("bfs_segment_picker", ROOT / "bfs_segment_picker.py")
|
||||||
|
SP = importlib.util.module_from_spec(SPEC)
|
||||||
|
SPEC.loader.exec_module(SP)
|
||||||
|
ORIGINAL_DETECT_TEXT = SP.detect_text
|
||||||
|
ORIGINAL_DETECT_POINTS = SP.detect_points
|
||||||
|
|
||||||
|
|
||||||
|
def rect(h, w, x0, y0, x1, y1):
|
||||||
|
m = torch.zeros(h, w, dtype=torch.bool)
|
||||||
|
m[y0:y1, x0:x1] = True
|
||||||
|
return m
|
||||||
|
|
||||||
|
|
||||||
|
class StateTest(unittest.TestCase):
|
||||||
|
def test_empty_and_broken_values(self):
|
||||||
|
for v in (None, "", "{}", "not json", "[1, 2]"):
|
||||||
|
self.assertEqual(SP.parse_state(v), {"selected": [], "added": [], "next": 1})
|
||||||
|
|
||||||
|
def test_keeps_valid_entries_only(self):
|
||||||
|
s = SP.parse_state(json.dumps({
|
||||||
|
"selected": ["a1", "a1", "t1.0", 3],
|
||||||
|
"added": [{"id": "t1", "text": " red car "}, {"id": "p2", "points": [{"x": .5, "y": .5, "label": 1}]},
|
||||||
|
{"id": "p3", "points": [{"x": .1, "y": .1, "label": 0}]}, # only exclude points: nothing to keep
|
||||||
|
{"text": "no id"}, {"id": "t4", "text": " "}],
|
||||||
|
"next": 5}))
|
||||||
|
self.assertEqual(s["selected"], ["a1", "t1.0", "3"])
|
||||||
|
self.assertEqual(s["added"], [{"id": "t1", "text": "red car"}, {"id": "p2", "points": [{"x": .5, "y": .5, "label": 1}]}])
|
||||||
|
self.assertEqual(s["next"], 5)
|
||||||
|
|
||||||
|
def test_concepts(self):
|
||||||
|
self.assertEqual(SP.split_concepts("person, animal,\ncar, person ,,"), ["person", "animal", "car"])
|
||||||
|
self.assertEqual(SP.split_concepts(""), [])
|
||||||
|
|
||||||
|
def test_entry_key_changes_with_what_it_asks(self):
|
||||||
|
a = SP.entry_key({"id": "t1", "text": "car"})
|
||||||
|
self.assertEqual(a, SP.entry_key({"id": "t9", "text": "car"}))
|
||||||
|
self.assertNotEqual(a, SP.entry_key({"id": "t1", "text": "bus"}))
|
||||||
|
|
||||||
|
|
||||||
|
class GeometryTest(unittest.TestCase):
|
||||||
|
def test_box(self):
|
||||||
|
self.assertEqual(SP.box_of(rect(10, 20, 3, 2, 8, 6)), [3, 2, 5, 4])
|
||||||
|
self.assertEqual(SP.box_of(torch.zeros(4, 4, dtype=torch.bool)), [0, 0, 0, 0])
|
||||||
|
|
||||||
|
def test_dedupe_drops_repeats_keeps_neighbours(self):
|
||||||
|
a = rect(20, 20, 0, 0, 10, 10)
|
||||||
|
repeat = rect(20, 20, 0, 0, 10, 9) # 90 % inside a
|
||||||
|
other = rect(20, 20, 8, 8, 18, 18) # touches a a little
|
||||||
|
kept = SP.dedupe([(a, {"n": 1}), (repeat, {"n": 2}), (other, {"n": 3})])
|
||||||
|
self.assertEqual([i["n"] for _, i in kept], [1, 3])
|
||||||
|
|
||||||
|
def test_compose(self):
|
||||||
|
u, each = SP.compose([], 4, 5)
|
||||||
|
self.assertEqual(tuple(u.shape), (1, 4, 5)); self.assertEqual(float(u.sum()), 0.)
|
||||||
|
a, b = rect(4, 5, 0, 0, 2, 2), rect(4, 5, 1, 1, 3, 3)
|
||||||
|
u, each = SP.compose([a, b], 4, 5)
|
||||||
|
self.assertEqual(tuple(each.shape), (2, 4, 5))
|
||||||
|
self.assertEqual(float(u.sum()), float((a | b).sum()))
|
||||||
|
|
||||||
|
def test_tint_only_inside(self):
|
||||||
|
img = torch.zeros(1, 4, 4, 3)
|
||||||
|
m = rect(4, 4, 0, 0, 2, 2).float()[None]
|
||||||
|
out = SP.tint(img, m, (255, 0, 255), .3)
|
||||||
|
self.assertAlmostEqual(float(out[0, 0, 0, 0]), .3, places=5)
|
||||||
|
self.assertAlmostEqual(float(out[0, 0, 0, 1]), 0., places=5)
|
||||||
|
self.assertEqual(float(out[0, 3, 3].sum()), 0.)
|
||||||
|
|
||||||
|
def test_color(self):
|
||||||
|
self.assertEqual(SP.parse_color("#FF00FF"), (255, 0, 255))
|
||||||
|
self.assertEqual(SP.parse_color("0f0"), (0, 255, 0))
|
||||||
|
self.assertEqual(SP.parse_color("nope"), (255, 0, 255))
|
||||||
|
|
||||||
|
|
||||||
|
class InferenceModeTest(unittest.TestCase):
|
||||||
|
def test_text_and_point_sam_calls_run_in_inference_mode(self):
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
class FakeDetect:
|
||||||
|
@staticmethod
|
||||||
|
def execute(**kwargs):
|
||||||
|
calls.append(torch.is_inference_mode_enabled())
|
||||||
|
return types.SimpleNamespace(args=(torch.ones(1, 4, 4), [[{"score": 0.9}]]))
|
||||||
|
|
||||||
|
parent = types.ModuleType("comfy_extras")
|
||||||
|
parent.__path__ = []
|
||||||
|
child = types.ModuleType("comfy_extras.nodes_sam3")
|
||||||
|
child.SAM3_Detect = FakeDetect
|
||||||
|
frame = torch.zeros(1, 4, 4, 3)
|
||||||
|
with patch.dict(sys.modules, {"comfy_extras": parent, "comfy_extras.nodes_sam3": child}), \
|
||||||
|
patch.object(SP, "_text_cond", return_value=[]):
|
||||||
|
self.assertEqual(len(ORIGINAL_DETECT_TEXT((None, None), frame, "person", 0.4, 1)), 1)
|
||||||
|
self.assertTrue(ORIGINAL_DETECT_POINTS((None, None), frame, [{"x": 0.5, "y": 0.5, "label": 1}], 0.4).any())
|
||||||
|
self.assertEqual(calls, [True, True])
|
||||||
|
|
||||||
|
|
||||||
|
class AutoObjectsTest(unittest.TestCase):
|
||||||
|
def test_rejects_scene_sized_generic_object_but_keeps_person(self):
|
||||||
|
h, w = 20, 40
|
||||||
|
scene = torch.zeros(h, w, dtype=torch.bool)
|
||||||
|
scene[::2, :] = True
|
||||||
|
scene[-1, :] = True
|
||||||
|
person = rect(h, w, 10, 2, 28, 18)
|
||||||
|
|
||||||
|
def detect_text(sam, frame, concept, threshold, max_detections):
|
||||||
|
mask = person if concept == "person" else scene
|
||||||
|
return [(mask, {"score": .9})]
|
||||||
|
|
||||||
|
frame = torch.zeros(1, h, w, 3)
|
||||||
|
with patch.object(SP, "detect_text", side_effect=detect_text):
|
||||||
|
objects = SP.auto_objects((None, None), frame, ["objects", "person"], .4, 8, 0.)
|
||||||
|
with patch.object(SP, "detect_text", return_value=[(scene, {"score": .9})]):
|
||||||
|
large_person = SP.auto_objects((None, None), frame, ["person"], .4, 8, 0.)
|
||||||
|
self.assertEqual([(o["id"], o["label"]) for o in objects], [("a1", "person")])
|
||||||
|
self.assertEqual(len(large_person), 1)
|
||||||
|
|
||||||
|
|
||||||
|
class SessionTest(unittest.TestCase):
|
||||||
|
"""The node with SAM replaced: what is segmented once, what is segmented again, what the outputs hold."""
|
||||||
|
|
||||||
|
def setUp(self):
|
||||||
|
self.calls = []
|
||||||
|
h, w = 20, 40
|
||||||
|
people = [rect(h, w, 25, 2, 35, 18), rect(h, w, 2, 2, 12, 18)]
|
||||||
|
|
||||||
|
def detect_text(sam, frame, text, threshold, max_detections):
|
||||||
|
self.calls.append(("text", text))
|
||||||
|
if text == "person":
|
||||||
|
return [(m, {"score": .9}) for m in people]
|
||||||
|
if text == "dog":
|
||||||
|
return [(rect(h, w, 15, 10, 20, 15), {"score": .7})]
|
||||||
|
return []
|
||||||
|
|
||||||
|
def detect_points(sam, frame, points, threshold):
|
||||||
|
self.calls.append(("points", len(points)))
|
||||||
|
return rect(h, w, 14, 0, 19, 6)
|
||||||
|
|
||||||
|
SP.detect_text, SP.detect_points = detect_text, detect_points
|
||||||
|
SP._SESSIONS.clear()
|
||||||
|
self.frame = torch.rand(1, h, w, 3)
|
||||||
|
self.sam = (object(), object())
|
||||||
|
self.tmp = Path(__file__).resolve().parent / ".segpick_tmp"
|
||||||
|
SP._temp_dir = lambda: str(self.tmp.mkdir(exist_ok=True) or self.tmp)
|
||||||
|
|
||||||
|
def tearDown(self):
|
||||||
|
for f in self.tmp.glob("*.png"):
|
||||||
|
f.unlink()
|
||||||
|
if self.tmp.exists():
|
||||||
|
self.tmp.rmdir()
|
||||||
|
|
||||||
|
def run_node(self, selection, concepts="person"):
|
||||||
|
return SP.BFSSegmentPicker().pick(self.frame, self.sam[0], self.sam[1], concepts, .4, 8, 0., "#FF00FF", .3, json.dumps(selection))
|
||||||
|
|
||||||
|
def test_auto_ids_left_to_right_and_selection(self):
|
||||||
|
out = self.run_node({"selected": ["a2"]})
|
||||||
|
mask, masks, marked, overlay, info = out["result"]
|
||||||
|
info = json.loads(info)
|
||||||
|
self.assertEqual([o["id"] for o in info["objects"]], ["a1", "a2"])
|
||||||
|
self.assertLess(info["objects"][0]["box"][0], info["objects"][1]["box"][0])
|
||||||
|
self.assertEqual(info["selected"], ["a2"])
|
||||||
|
self.assertEqual(float(mask[0, 10, 30]), 1.) # a2 is the person on the right
|
||||||
|
self.assertEqual(float(mask[0, 10, 5]), 0.)
|
||||||
|
self.assertEqual(tuple(masks.shape), (1, 20, 40))
|
||||||
|
self.assertGreater(float(marked[0, 10, 30, 0]), float(self.frame[0, 10, 30, 0]) * .7 - 1e-6)
|
||||||
|
panel = out["ui"]["bfs_segpick"][0]
|
||||||
|
self.assertEqual(len(panel["objects"]), 2)
|
||||||
|
self.assertTrue((self.tmp / panel["objects"][0]["mask"]["filename"]).exists())
|
||||||
|
|
||||||
|
def test_segments_once_then_only_new_entries(self):
|
||||||
|
self.run_node({"selected": ["a1"]})
|
||||||
|
self.run_node({"selected": ["a2"]}) # only the selection changed
|
||||||
|
self.assertEqual(self.calls, [("text", "person")])
|
||||||
|
sel = {"selected": ["t1.0", "p2"], "added": [{"id": "t1", "text": "dog"},
|
||||||
|
{"id": "p2", "points": [{"x": .37, "y": .05, "label": 1}]}]}
|
||||||
|
_, masks, *_ , info = self.run_node(sel)["result"]
|
||||||
|
self.assertEqual(json.loads(info)["selected"], ["t1.0", "p2"])
|
||||||
|
self.assertEqual(tuple(masks.shape), (2, 20, 40))
|
||||||
|
self.run_node(sel)
|
||||||
|
self.assertEqual(self.calls, [("text", "person"), ("text", "dog"), ("points", 1)])
|
||||||
|
sel["added"][0]["text"] = "cat" # an edited entry is segmented again
|
||||||
|
_, _, _, _, info = self.run_node(sel)["result"]
|
||||||
|
self.assertEqual(self.calls[-1], ("text", "cat"))
|
||||||
|
self.assertEqual(json.loads(info)["selected"], ["p2"]) # "cat" found nothing, so t1.0 is gone
|
||||||
|
|
||||||
|
def test_brush_is_dropped_on_another_image(self):
|
||||||
|
key = SP.image_key(self.frame)
|
||||||
|
sel = {"selected": ["a1", "b1"], "added": [{"id": "b1", "image": key, "strokes": [{"r": .05, "pts": [[.5, .5]]}]}]}
|
||||||
|
out = self.run_node(sel)
|
||||||
|
self.assertEqual(out["ui"]["bfs_segpick"][0]["image_key"], key)
|
||||||
|
self.assertEqual(json.loads(out["result"][4])["selected"], ["a1", "b1"])
|
||||||
|
self.frame = torch.rand(1, 20, 40, 3) # another image: the strokes no longer apply
|
||||||
|
mask, _, _, _, info = self.run_node(sel)["result"]
|
||||||
|
self.assertEqual(json.loads(info)["selected"], ["a1"])
|
||||||
|
self.assertEqual(float(mask[0, 10, 20]), 0.)
|
||||||
|
|
||||||
|
def test_remove_entries_cut_the_selection(self):
|
||||||
|
key = SP.image_key(self.frame)
|
||||||
|
sel = {"selected": ["a1", "e1", "r2"], "added": [
|
||||||
|
{"id": "e1", "image": key, "remove": True, "strokes": [{"r": .05, "pts": [[.175, .5]]}]},
|
||||||
|
{"id": "r2", "remove": True, "points": [{"x": .4, "y": .1, "label": 1}]}]}
|
||||||
|
out = self.run_node(sel)
|
||||||
|
mask, masks, _, _, info = out["result"]
|
||||||
|
self.assertEqual(float(mask[0, 10, 7]), 0.) # erased out of a1
|
||||||
|
self.assertEqual(float(mask[0, 10, 3]), 1.) # the rest of a1 stays
|
||||||
|
self.assertEqual(tuple(masks.shape), (1, 20, 40)) # remove entries are not masks of their own
|
||||||
|
self.assertEqual(float(masks[0, 10, 7]), 0.)
|
||||||
|
self.assertEqual(json.loads(info)["selected"], ["a1", "e1", "r2"])
|
||||||
|
panel = {o["id"]: o for o in out["ui"]["bfs_segpick"][0]["objects"]}
|
||||||
|
self.assertEqual((panel["e1"]["label"], panel["e1"]["remove"]), ("eraser", True))
|
||||||
|
self.assertEqual((panel["r2"]["label"], panel["r2"]["remove"]), ("remove", True))
|
||||||
|
self.assertFalse(panel["a1"]["remove"])
|
||||||
|
sel["selected"] = ["a1", "r2"] # unselecting the eraser gives a1 back whole
|
||||||
|
self.assertEqual(float(self.run_node(sel)["result"][0][0, 10, 7]), 1.)
|
||||||
|
|
||||||
|
def test_unknown_ids_are_ignored(self):
|
||||||
|
mask, masks, *_ = self.run_node({"selected": ["a9", "t5.0"]})["result"]
|
||||||
|
self.assertEqual(float(mask.sum()), 0.)
|
||||||
|
self.assertEqual(tuple(masks.shape), (1, 20, 40))
|
||||||
|
|
||||||
|
def test_another_sam_is_another_session(self):
|
||||||
|
self.run_node({})
|
||||||
|
self.sam = (object(), object())
|
||||||
|
self.run_node({})
|
||||||
|
self.assertEqual(self.calls, [("text", "person"), ("text", "person")])
|
||||||
|
|
||||||
|
def test_other_concepts_are_another_session(self):
|
||||||
|
self.run_node({}, concepts="person")
|
||||||
|
self.run_node({}, concepts="person, dog")
|
||||||
|
self.assertEqual(self.calls, [("text", "person"), ("text", "person"), ("text", "dog")])
|
||||||
|
|
||||||
|
|
||||||
|
class BrushTest(unittest.TestCase):
|
||||||
|
def test_state_keeps_strokes(self):
|
||||||
|
s = SP.parse_state(json.dumps({"added": [{"id": "b1", "strokes": [{"r": .1, "pts": [[.2, .5], [.8, .5]]}]},
|
||||||
|
{"id": "b2", "strokes": [{"r": .1, "pts": []}]}]}))
|
||||||
|
self.assertEqual(s["added"], [{"id": "b1", "image": "", "strokes": [{"r": .1, "pts": [[.2, .5], [.8, .5]]}]}])
|
||||||
|
self.assertNotEqual(SP.entry_key(s["added"][0]), SP.entry_key({"id": "b1", "strokes": [{"r": .2, "pts": [[.2, .5]]}]}))
|
||||||
|
r = SP.parse_state(json.dumps({"added": [{"id": "r1", "remove": True, "points": [{"x": .5, "y": .5, "label": 1}]},
|
||||||
|
{"id": "p2", "remove": False, "points": [{"x": .5, "y": .5, "label": 1}]}]}))
|
||||||
|
self.assertEqual([e.get("remove") for e in r["added"]], [True, None])
|
||||||
|
|
||||||
|
def test_paint_covers_the_stroke_only(self):
|
||||||
|
m = SP.paint([{"r": .1, "pts": [[.2, .5], [.8, .5]]}], 50, 100)
|
||||||
|
self.assertTrue(m[25, 20] and m[25, 50] and m[25, 80])
|
||||||
|
self.assertFalse(m[5, 50] or m[25, 5] or m[25, 95])
|
||||||
|
|
||||||
|
def test_brush_entry_needs_no_sam(self):
|
||||||
|
frame = torch.zeros(1, 50, 100, 3)
|
||||||
|
objs = SP.entry_objects(None, frame, {"id": "b3", "strokes": [{"r": .05, "pts": [[.5, .5]]}]}, .4, 8)
|
||||||
|
self.assertEqual([(o["id"], o["source"]) for o in objs], [("b3", "brush")])
|
||||||
|
self.assertTrue(objs[0]["mask"][25, 50])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -1759,3 +1759,19 @@ class PromptReferenceContractTest(unittest.TestCase):
|
|||||||
["fox wearing blue pants", "city street"], "Use Picture 2 for the setting", {},
|
["fox wearing blue pants", "city street"], "Use Picture 2 for the setting", {},
|
||||||
style="captions", source_facts="A dancer is moving in a street.")
|
style="captions", source_facts="A dancer is moving in a street.")
|
||||||
self.assertEqual(generator.call_count, 2)
|
self.assertEqual(generator.call_count, 2)
|
||||||
|
|
||||||
|
|
||||||
|
class PanelInferenceModeTest(unittest.TestCase):
|
||||||
|
"""Issue #14: panel buttons reuse models a workflow run left with inference tensors; they must run like a node."""
|
||||||
|
|
||||||
|
def test_panel_work_runs_in_inference_mode(self):
|
||||||
|
import asyncio
|
||||||
|
with torch.inference_mode():
|
||||||
|
cache = torch.zeros(3) # e.g. a VLM buffer created while the workflow ran
|
||||||
|
|
||||||
|
def work():
|
||||||
|
self.assertTrue(torch.is_inference_mode_enabled())
|
||||||
|
cache.add_(1)
|
||||||
|
return cache.sum().item()
|
||||||
|
|
||||||
|
self.assertEqual(asyncio.run(SL._off_loop(work)), 3.0)
|
||||||
|
|||||||
@@ -107,6 +107,19 @@ class CompareNodeTest(unittest.TestCase):
|
|||||||
with self.assertRaisesRegex(ValueError, "nonempty"):
|
with self.assertRaisesRegex(ValueError, "nonempty"):
|
||||||
VC.BFSVideoCompare().compare([], t)
|
VC.BFSVideoCompare().compare([], t)
|
||||||
|
|
||||||
|
def test_a_single_frame_also_sends_lossless_pngs_at_the_compared_size(self):
|
||||||
|
from PIL import Image
|
||||||
|
guide, target = torch.rand(1, 40, 60, 3), torch.rand(1, 20, 30, 3)
|
||||||
|
ui = VC.BFSVideoCompare().compare(guide, target, preview_height=16)["ui"]["bfs_compare"][0]
|
||||||
|
self.assertEqual(sorted(ui["stills"]), ["difference", "guide", "target"])
|
||||||
|
for k, f in ui["stills"].items():
|
||||||
|
im = Image.open(os.path.join(_TMP, f["subfolder"], f["filename"]))
|
||||||
|
self.assertEqual(im.size, (60, 40)) # the guide's size, not the 16 px preview
|
||||||
|
g = Image.open(os.path.join(_TMP, ui["stills"]["guide"]["subfolder"], ui["stills"]["guide"]["filename"]))
|
||||||
|
self.assertEqual(g.getpixel((7, 5)), tuple(int(round(float(c) * 255)) for c in guide[0, 5, 7])) # lossless
|
||||||
|
video = VC.BFSVideoCompare().compare(torch.rand(3, 8, 8, 3), torch.rand(3, 8, 8, 3), preview_height=8)
|
||||||
|
self.assertEqual(video["ui"]["bfs_compare"][0]["stills"], {})
|
||||||
|
|
||||||
def test_reference_pictures_go_to_the_player_no_taller_than_it(self):
|
def test_reference_pictures_go_to_the_player_no_taller_than_it(self):
|
||||||
from PIL import Image
|
from PIL import Image
|
||||||
guide, target = torch.rand(4, 32, 48, 3), torch.rand(4, 32, 48, 3)
|
guide, target = torch.rand(4, 32, 48, 3), torch.rand(4, 32, 48, 3)
|
||||||
|
|||||||
@@ -0,0 +1,132 @@
|
|||||||
|
import importlib
|
||||||
|
import sys
|
||||||
|
import types
|
||||||
|
import unittest
|
||||||
|
from pathlib import Path
|
||||||
|
from unittest.mock import patch
|
||||||
|
|
||||||
|
import torch
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
pkg = types.ModuleType("bfsnodes")
|
||||||
|
pkg.__path__ = [str(ROOT)]
|
||||||
|
sys.modules["bfsnodes"] = pkg
|
||||||
|
|
||||||
|
VSP = importlib.import_module("bfsnodes.bfs_video_segment_picker")
|
||||||
|
|
||||||
|
|
||||||
|
class MaskQualityTest(unittest.TestCase):
|
||||||
|
def test_rendered_edge_is_soft_and_stays_inside_selection(self):
|
||||||
|
mask = torch.zeros(20, 20, dtype=torch.bool)
|
||||||
|
mask[3:17, 3:17] = True
|
||||||
|
alpha = VSP.feather_inside_mask(mask, 4)
|
||||||
|
self.assertEqual(float(alpha[2, 10]), 0.0)
|
||||||
|
self.assertGreater(float(alpha[10, 10]), 0.95)
|
||||||
|
self.assertGreater(float(alpha[3, 10]), 0.0)
|
||||||
|
self.assertLess(float(alpha[3, 10]), 0.5)
|
||||||
|
|
||||||
|
def test_anchor_windows_cover_each_frame_once_and_keep_earlier_ties(self):
|
||||||
|
windows = list(VSP.anchor_windows([0, 53, 69, 125], 0, 209))
|
||||||
|
assigned = {i: a for a, lo, hi in windows for i in range(lo, hi)}
|
||||||
|
self.assertEqual(len(assigned), 209)
|
||||||
|
self.assertEqual(assigned[26], 0)
|
||||||
|
self.assertEqual(assigned[27], 53)
|
||||||
|
self.assertEqual(assigned[61], 53)
|
||||||
|
self.assertEqual(assigned[62], 69)
|
||||||
|
|
||||||
|
def test_fill_empty_tracks_only_gaps_and_preserves_existing_masks(self):
|
||||||
|
frames = torch.zeros(8, 4, 4, 3)
|
||||||
|
masks = torch.zeros(8, 4, 4, dtype=torch.bool)
|
||||||
|
masks[[0, 3, 7], 1, 1] = True
|
||||||
|
report = [{"frame": i, "source": "empty"} for i in range(8)]
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
class FakeTracker:
|
||||||
|
def track(self, video, model, mask, anchor_frame):
|
||||||
|
calls.append((len(video), anchor_frame))
|
||||||
|
out = torch.zeros(len(video), 4, 4)
|
||||||
|
out[:, 0, 0] = 1
|
||||||
|
return out, anchor_frame
|
||||||
|
|
||||||
|
with patch.object(VSP, "BFSTrackFromFrame", FakeTracker):
|
||||||
|
filled = VSP.fill_empty_mask_runs(frames, None, masks, report, [(0, 8)], active_frame=3)
|
||||||
|
self.assertEqual(filled, 5)
|
||||||
|
self.assertEqual(calls, [(3, 2), (4, 0)])
|
||||||
|
self.assertTrue(bool(masks[3, 1, 1]))
|
||||||
|
self.assertFalse(bool(masks[3, 0, 0]))
|
||||||
|
self.assertTrue(bool(masks[5, 0, 0]))
|
||||||
|
|
||||||
|
def test_fill_scope_does_not_request_molmo_when_masks_are_cached(self):
|
||||||
|
selection = '{"frames":{"1":{"1":{"selected":["a1"]}}}}'
|
||||||
|
prompt = {"7": {"inputs": {"molmo_clip": ["6", 0]}}}
|
||||||
|
with patch.object(VSP, "video_key", return_value="video"), \
|
||||||
|
patch.object(VSP, "_LAST_TRACK_CACHE", {("video", 1): (None, None, None)}):
|
||||||
|
need = VSP.BFSVideoSegmentPicker().check_lazy_status(
|
||||||
|
selection=selection, mark_style="flat color", track=True, image=torch.zeros(2, 4, 4, 3),
|
||||||
|
molmo_clip=None, propagation_method="Molmo2 (classic)",
|
||||||
|
propagate_scope="fill only empty masks", unique_id="7", prompt=prompt)
|
||||||
|
self.assertEqual(need, [])
|
||||||
|
|
||||||
|
|
||||||
|
class AutoUnmarkingTest(unittest.TestCase):
|
||||||
|
def test_find_connected_picture_count(self):
|
||||||
|
# 1. 2 pictures connected downstream
|
||||||
|
mock_prompt = {
|
||||||
|
"100": {
|
||||||
|
"class_type": "BFSH3KeyframeConditioning",
|
||||||
|
"inputs": {
|
||||||
|
"guide_video": ["10", 4],
|
||||||
|
"picture_1": ["20", 0],
|
||||||
|
"picture_2": ["21", 0],
|
||||||
|
"picture_3": None,
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"10": {
|
||||||
|
"class_type": "BFSVideoSegmentPicker",
|
||||||
|
"inputs": {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
self.assertEqual(VSP.find_connected_picture_count(mock_prompt, "10"), 2)
|
||||||
|
|
||||||
|
# 2. 3 pictures connected downstream
|
||||||
|
mock_prompt["100"]["inputs"]["picture_3"] = ["22", 0]
|
||||||
|
self.assertEqual(VSP.find_connected_picture_count(mock_prompt, "10"), 3)
|
||||||
|
|
||||||
|
# 3. Only 1 picture connected
|
||||||
|
mock_prompt["100"]["inputs"]["picture_2"] = None
|
||||||
|
mock_prompt["100"]["inputs"]["picture_3"] = None
|
||||||
|
self.assertEqual(VSP.find_connected_picture_count(mock_prompt, "10"), 1)
|
||||||
|
|
||||||
|
# 4. No downstream node connected
|
||||||
|
mock_prompt["100"]["inputs"]["guide_video"] = ["99", 0]
|
||||||
|
self.assertIsNone(VSP.find_connected_picture_count(mock_prompt, "10"))
|
||||||
|
|
||||||
|
# 5. Empty prompt or None
|
||||||
|
self.assertIsNone(VSP.find_connected_picture_count(None, "10"))
|
||||||
|
self.assertIsNone(VSP.find_connected_picture_count({}, "10"))
|
||||||
|
|
||||||
|
def test_input_types_controls(self):
|
||||||
|
t = VSP.BFSVideoSegmentPicker.INPUT_TYPES()
|
||||||
|
opt = t["optional"]
|
||||||
|
self.assertIn("max_subjects", opt)
|
||||||
|
self.assertEqual(opt["max_subjects"][0], ["auto", "1", "2", "3"])
|
||||||
|
self.assertEqual(opt["max_subjects"][1]["default"], "auto")
|
||||||
|
|
||||||
|
self.assertIn("molmo_descriptions", opt)
|
||||||
|
self.assertEqual(opt["molmo_descriptions"][0], "STRING")
|
||||||
|
self.assertFalse(opt["molmo_descriptions"][1].get("multiline"))
|
||||||
|
self.assertEqual(opt["protect_mask"][0], "MASK")
|
||||||
|
self.assertIn(VSP.PROTECTED_NORMAL_STYLE, opt["mark_style"][0])
|
||||||
|
|
||||||
|
def test_locked_objects_survive_selection_round_trip(self):
|
||||||
|
import json
|
||||||
|
state = {"frames": {"7": {"1": {"selected": ["a1"]},
|
||||||
|
"lock": {"selected": ["lock_p1"], "added": [
|
||||||
|
{"id": "lock_p1", "points": [{"x": 0.4, "y": 0.5, "label": 1}]}]}}}}
|
||||||
|
frames = VSP.parse_frames(json.dumps(state))
|
||||||
|
self.assertEqual(frames["7"]["1"]["selected"], ["a1"])
|
||||||
|
self.assertEqual(frames["7"]["lock"]["selected"], ["lock_p1"])
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
unittest.main()
|
||||||
@@ -0,0 +1,302 @@
|
|||||||
|
/**
|
||||||
|
* BFS Keyframe Planner: the panel of the node. A small player of the guide (frames from the node's last run), a timeline
|
||||||
|
* with the keyframes on it, one card per keyframe (source frame + an optional edited frame, dropped or uploaded) and up
|
||||||
|
* to 4 reference pictures.
|
||||||
|
*
|
||||||
|
* All state lives in the hidden `plan` STRING widget (JSON: keyframe frames, edited frame and picture paths in input/),
|
||||||
|
* so a workflow saves, reloads and shares the plan. Uploaded images go to input/bfs_keyframes/.
|
||||||
|
*/
|
||||||
|
import { app } from "../../scripts/app.js";
|
||||||
|
import { api } from "../../scripts/api.js";
|
||||||
|
import { wheelToCanvas } from "./bfs_wheel.js";
|
||||||
|
import { createApp, h, ref, reactive, computed, watch, onBeforeUnmount } from "./vendor/vue.esm-browser.prod.mjs";
|
||||||
|
|
||||||
|
const DEFAULT_W = 560, DEFAULT_H = 640, MAX_PICTURES = 4, SUBFOLDER = "bfs_keyframes";
|
||||||
|
|
||||||
|
const tempUrl = (d, f) => api.apiURL(`/view?filename=${encodeURIComponent(f)}&subfolder=${encodeURIComponent(d.subfolder)}&type=${d.type || "temp"}`);
|
||||||
|
const inputUrl = p => {
|
||||||
|
const i = p.lastIndexOf("/");
|
||||||
|
return api.apiURL(`/view?filename=${encodeURIComponent(p.slice(i + 1))}&subfolder=${encodeURIComponent(i < 0 ? "" : p.slice(0, i))}&type=input`);
|
||||||
|
};
|
||||||
|
|
||||||
|
function parsePlan(text) {
|
||||||
|
let s = {};
|
||||||
|
try { s = JSON.parse(text || "{}") || {}; } catch { s = {}; }
|
||||||
|
const keyframes = (Array.isArray(s.keyframes) ? s.keyframes : []).filter(k => Number.isFinite(+k.frame))
|
||||||
|
.map(k => ({ frame: Math.round(+k.frame), edit: k.edit || null }));
|
||||||
|
const pictures = (Array.isArray(s.pictures) ? s.pictures : []).slice(0, MAX_PICTURES).map(p => p || null);
|
||||||
|
while (pictures.length < MAX_PICTURES) pictures.push(null);
|
||||||
|
return { keyframes, pictures };
|
||||||
|
}
|
||||||
|
|
||||||
|
async function upload(file) {
|
||||||
|
const body = new FormData();
|
||||||
|
body.append("image", file);
|
||||||
|
body.append("subfolder", SUBFOLDER);
|
||||||
|
body.append("type", "input");
|
||||||
|
const r = await api.fetchApi("/upload/image", { method: "POST", body });
|
||||||
|
if (r.status !== 200) throw new Error(`upload failed (${r.status})`);
|
||||||
|
const j = await r.json();
|
||||||
|
return j.subfolder ? `${j.subfolder}/${j.name}` : j.name;
|
||||||
|
}
|
||||||
|
|
||||||
|
function pickFile() {
|
||||||
|
return new Promise(resolve => {
|
||||||
|
const inp = document.createElement("input");
|
||||||
|
inp.type = "file"; inp.accept = "image/*";
|
||||||
|
inp.onchange = () => resolve(inp.files?.[0] || null);
|
||||||
|
inp.click();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
let styled = false;
|
||||||
|
function styles() {
|
||||||
|
if (styled) return;
|
||||||
|
styled = true;
|
||||||
|
const css = document.createElement("style");
|
||||||
|
css.textContent = `
|
||||||
|
.bkp{position:absolute;inset:0;display:flex;flex-direction:column;gap:6px;padding:6px;box-sizing:border-box;font:12px system-ui,sans-serif;color:#ddd}
|
||||||
|
.bkp button{background:#333;color:#ddd;border:1px solid #555;border-radius:4px;padding:3px 8px;cursor:pointer;font:inherit;line-height:1.2}
|
||||||
|
.bkp button:hover:not(:disabled){background:#444}
|
||||||
|
.bkp button:disabled{opacity:.4;cursor:default}
|
||||||
|
.bkp button.pri{background:#2d5a8a;border-color:#3d6fa5;color:#fff}
|
||||||
|
.bkp button.danger:hover:not(:disabled){background:#6a2a2a}
|
||||||
|
.bkp .stage{flex:1;min-height:150px;position:relative;display:flex;align-items:center;justify-content:center;background:#121212;border-radius:6px;overflow:hidden}
|
||||||
|
.bkp .stage img{max-width:100%;max-height:100%;object-fit:contain;user-select:none;-webkit-user-drag:none}
|
||||||
|
.bkp .stage.onkf{box-shadow:inset 0 0 0 2px #e0a030}
|
||||||
|
.bkp .badge{position:absolute;left:6px;top:6px;background:rgba(0,0,0,.7);padding:2px 7px;border-radius:10px;font-variant-numeric:tabular-nums}
|
||||||
|
.bkp .badge.kf{left:auto;right:6px;background:#e0a030;color:#111;font-weight:600}
|
||||||
|
.bkp .empty{color:#888;text-align:center;padding:16px;line-height:1.7}
|
||||||
|
.bkp .ctl{display:flex;gap:4px;align-items:center}
|
||||||
|
.bkp .ctl .sp{flex:1}
|
||||||
|
.bkp .time{font-variant-numeric:tabular-nums;color:#bbb;min-width:110px;text-align:center}
|
||||||
|
.bkp .tl{position:relative;height:26px;background:#1d1d1d;border:1px solid #333;border-radius:5px;cursor:pointer;touch-action:none}
|
||||||
|
.bkp .tl .fill{position:absolute;left:0;top:0;bottom:0;background:#2a3a4d;border-radius:5px 0 0 5px;pointer-events:none}
|
||||||
|
.bkp .tl .head{position:absolute;top:-2px;bottom:-2px;width:2px;margin-left:-1px;background:#fff;pointer-events:none}
|
||||||
|
.bkp .tl .mk{position:absolute;top:3px;width:10px;height:18px;margin-left:-5px;border-radius:3px;background:#e0a030;border:1px solid #111;cursor:pointer}
|
||||||
|
.bkp .tl .mk.done{background:#3fbf6a}
|
||||||
|
.bkp .tl .mk.cur{outline:2px solid #fff}
|
||||||
|
.bkp .sec{display:flex;align-items:center;gap:6px;color:#999;font-size:11px;text-transform:uppercase;letter-spacing:.05em}
|
||||||
|
.bkp .sec .n{color:#ccc}
|
||||||
|
.bkp .row{display:flex;gap:6px;overflow-x:auto;padding-bottom:3px;min-height:96px}
|
||||||
|
.bkp .card{flex:0 0 auto;display:flex;flex-direction:column;gap:3px;background:#1d1d1d;border:1px solid #383838;border-radius:6px;padding:4px}
|
||||||
|
.bkp .card.cur{border-color:#e0a030}
|
||||||
|
.bkp .card .pair{display:flex;gap:3px}
|
||||||
|
.bkp .thumb{width:72px;height:54px;border-radius:4px;background:#111 center/cover no-repeat;position:relative;cursor:pointer;flex:0 0 auto}
|
||||||
|
.bkp .thumb .lbl{position:absolute;left:2px;bottom:2px;font-size:9px;background:rgba(0,0,0,.7);padding:0 3px;border-radius:2px}
|
||||||
|
.bkp .drop{white-space:pre-line;width:72px;height:54px;border:1px dashed #555;border-radius:4px;display:flex;align-items:center;justify-content:center;text-align:center;color:#888;font-size:10px;cursor:pointer;line-height:1.2;flex:0 0 auto}
|
||||||
|
.bkp .drop:hover,.bkp .drop.over{border-color:#3fbf6a;color:#cfc}
|
||||||
|
.bkp .card .foot{display:flex;align-items:center;gap:4px;font-size:11px;font-variant-numeric:tabular-nums}
|
||||||
|
.bkp .card .foot .sp{flex:1}
|
||||||
|
.bkp .x{color:#999;cursor:pointer;padding:0 3px;border-radius:3px}.bkp .x:hover{color:#f66;background:#2a1a1a}
|
||||||
|
.bkp .pics{display:flex;gap:6px}
|
||||||
|
.bkp .pic{position:relative}
|
||||||
|
.bkp .pic .thumb{width:60px;height:60px}
|
||||||
|
.bkp .pic .drop{width:60px;height:60px}
|
||||||
|
.bkp .pic .x{position:absolute;right:1px;top:1px;background:rgba(0,0,0,.7)}
|
||||||
|
.bkp .pic .tag{font-size:10px;color:#888;text-align:center}
|
||||||
|
.bkp .hint{color:#777;font-size:11px}.bkp .err{color:#f77;font-size:11px}
|
||||||
|
`;
|
||||||
|
document.head.appendChild(css);
|
||||||
|
}
|
||||||
|
|
||||||
|
function Panel(io) {
|
||||||
|
const plan = reactive(parsePlan(io.get()));
|
||||||
|
const strip = ref(null); // {subfolder, type, files, frames, count, fps, width, height} from the last run
|
||||||
|
const pos = ref(0); // index into strip.frames
|
||||||
|
const playing = ref(false);
|
||||||
|
const err = ref("");
|
||||||
|
const over = ref(""); // drop target being hovered
|
||||||
|
let timer = null;
|
||||||
|
|
||||||
|
const save = () => io.set(JSON.stringify({
|
||||||
|
keyframes: [...plan.keyframes].sort((a, b) => a.frame - b.frame),
|
||||||
|
pictures: plan.pictures.map(p => p || null),
|
||||||
|
}));
|
||||||
|
watch(plan, save, { deep: true });
|
||||||
|
|
||||||
|
const count = computed(() => strip.value?.count || 0);
|
||||||
|
const fps = computed(() => strip.value?.fps || io.fps() || 24);
|
||||||
|
const frame = computed(() => strip.value ? strip.value.frames[pos.value] : 0);
|
||||||
|
const sorted = computed(() => [...plan.keyframes].sort((a, b) => a.frame - b.frame));
|
||||||
|
const curKf = computed(() => plan.keyframes.find(k => k.frame === frame.value) || null);
|
||||||
|
const nearest = f => { // strip index of the frame nearest to f
|
||||||
|
const fr = strip.value?.frames || [];
|
||||||
|
let best = 0;
|
||||||
|
for (let i = 1; i < fr.length; i++) if (Math.abs(fr[i] - f) < Math.abs(fr[best] - f)) best = i;
|
||||||
|
return best;
|
||||||
|
};
|
||||||
|
const frameUrl = f => strip.value ? tempUrl(strip.value, strip.value.files[nearest(f)]) : "";
|
||||||
|
const fmt = f => `${(f / fps.value).toFixed(2)}s`;
|
||||||
|
|
||||||
|
const stop = () => { playing.value = false; if (timer) { clearInterval(timer); timer = null; } };
|
||||||
|
const play = () => {
|
||||||
|
if (!strip.value) return;
|
||||||
|
if (playing.value) return stop();
|
||||||
|
playing.value = true;
|
||||||
|
const fr = strip.value.frames;
|
||||||
|
const step = fr.length > 1 ? (fr[fr.length - 1] - fr[0]) / (fr.length - 1) : 1;
|
||||||
|
timer = setInterval(() => { pos.value = (pos.value + 1) % fr.length; }, 1000 * step / fps.value);
|
||||||
|
};
|
||||||
|
const seekIdx = i => { if (strip.value) pos.value = Math.max(0, Math.min(strip.value.frames.length - 1, i)); };
|
||||||
|
const seekFrame = f => seekIdx(nearest(f));
|
||||||
|
onBeforeUnmount(stop);
|
||||||
|
|
||||||
|
const addHere = () => {
|
||||||
|
if (!strip.value || curKf.value) return;
|
||||||
|
plan.keyframes.push({ frame: frame.value, edit: null });
|
||||||
|
};
|
||||||
|
const removeKf = k => { plan.keyframes.splice(plan.keyframes.indexOf(k), 1); };
|
||||||
|
const jumpKf = d => {
|
||||||
|
const list = sorted.value;
|
||||||
|
if (!list.length) return;
|
||||||
|
const f = frame.value;
|
||||||
|
const t = d > 0 ? list.find(k => k.frame > f) || list[0] : [...list].reverse().find(k => k.frame < f) || list[list.length - 1];
|
||||||
|
seekFrame(t.frame);
|
||||||
|
};
|
||||||
|
|
||||||
|
const withFile = async (file, apply) => {
|
||||||
|
if (!file) return;
|
||||||
|
err.value = "";
|
||||||
|
try { apply(await upload(file)); } catch (e) { err.value = String(e.message || e); }
|
||||||
|
};
|
||||||
|
const dropProps = (key, apply) => ({
|
||||||
|
class: ["drop", over.value === key ? "over" : ""],
|
||||||
|
onDragover: e => { e.preventDefault(); e.stopPropagation(); over.value = key; },
|
||||||
|
onDragleave: () => { if (over.value === key) over.value = ""; },
|
||||||
|
onDrop: e => { e.preventDefault(); e.stopPropagation(); over.value = ""; withFile(e.dataTransfer?.files?.[0], apply); },
|
||||||
|
onClick: async e => { e.stopPropagation(); withFile(await pickFile(), apply); },
|
||||||
|
});
|
||||||
|
|
||||||
|
// timeline: click / drag to seek
|
||||||
|
const tlDown = e => {
|
||||||
|
if (!strip.value) return;
|
||||||
|
const el = e.currentTarget, r = el.getBoundingClientRect();
|
||||||
|
const go = ev => {
|
||||||
|
const x = Math.max(0, Math.min(1, (ev.clientX - r.left) / r.width));
|
||||||
|
seekFrame(Math.round(x * Math.max(0, count.value - 1)));
|
||||||
|
};
|
||||||
|
stop(); go(e);
|
||||||
|
el.setPointerCapture?.(e.pointerId);
|
||||||
|
const move = ev => go(ev);
|
||||||
|
const up = () => { el.removeEventListener("pointermove", move); el.removeEventListener("pointerup", up); };
|
||||||
|
el.addEventListener("pointermove", move); el.addEventListener("pointerup", up);
|
||||||
|
};
|
||||||
|
const pct = f => `${count.value > 1 ? (100 * f / (count.value - 1)) : 0}%`;
|
||||||
|
|
||||||
|
io.expose({
|
||||||
|
executed: d => {
|
||||||
|
if (!d) return;
|
||||||
|
const keep = frame.value;
|
||||||
|
strip.value = d;
|
||||||
|
seekFrame(keep);
|
||||||
|
for (const f of d.files) { const im = new Image(); im.src = tempUrl(d, f); } // preload for smooth playback
|
||||||
|
},
|
||||||
|
reload: () => { Object.assign(plan, parsePlan(io.get())); },
|
||||||
|
});
|
||||||
|
|
||||||
|
return () => {
|
||||||
|
const s = strip.value;
|
||||||
|
const cards = sorted.value.map((k, i) => h("div", { class: ["card", k.frame === frame.value ? "cur" : ""], key: k.frame }, [
|
||||||
|
h("div", { class: "pair" }, [
|
||||||
|
h("div", { class: "thumb", title: "source frame: click to go there", style: s ? { backgroundImage: `url("${frameUrl(k.frame)}")` } : {},
|
||||||
|
onClick: () => { stop(); seekFrame(k.frame); } }, [h("span", { class: "lbl" }, "source")]),
|
||||||
|
k.edit
|
||||||
|
? h("div", { class: "thumb", title: "edited frame: click to replace", style: { backgroundImage: `url("${inputUrl(k.edit)}")` },
|
||||||
|
onClick: async () => withFile(await pickFile(), p => { k.edit = p; }) }, [h("span", { class: "lbl" }, "edited")])
|
||||||
|
: h("div", { ...dropProps(`kf${k.frame}`, p => { k.edit = p; }), title: "drop or click: the edited frame for this time (optional)" },
|
||||||
|
"drop edited\nframe"),
|
||||||
|
]),
|
||||||
|
h("div", { class: "foot" }, [
|
||||||
|
h("span", null, `#${i + 1} · ${fmt(k.frame)}`), h("span", { class: "sp" }),
|
||||||
|
k.edit ? h("span", { class: "x", title: "clear the edited frame", onClick: () => { k.edit = null; } }, "⟲") : null,
|
||||||
|
h("span", { class: "x", title: "remove this keyframe", onClick: () => removeKf(k) }, "✕"),
|
||||||
|
]),
|
||||||
|
]));
|
||||||
|
const linked = j => !!io.node.inputs?.find(x => x.name === `picture_${j}`)?.link;
|
||||||
|
const pics = plan.pictures.map((p, j) => h("div", { class: "pic", key: j }, [
|
||||||
|
linked(j + 1)
|
||||||
|
? h("div", { class: "drop", title: "this picture comes from the connected input" }, "input")
|
||||||
|
: p
|
||||||
|
? h("div", { class: "thumb", style: { backgroundImage: `url("${inputUrl(p)}")` }, title: "click to replace",
|
||||||
|
onClick: async () => withFile(await pickFile(), q => { plan.pictures[j] = q; }) })
|
||||||
|
: h("div", { ...dropProps(`pic${j}`, q => { plan.pictures[j] = q; }), title: "drop or click: a reference picture" }, "+ picture"),
|
||||||
|
p && !linked(j + 1) ? h("span", { class: "x", title: "remove", onClick: () => { plan.pictures[j] = null; } }, "✕") : null,
|
||||||
|
h("div", { class: "tag" }, `Picture ${j + 1}`),
|
||||||
|
]));
|
||||||
|
return h("div", { class: "bkp" }, [
|
||||||
|
h("div", { class: ["stage", curKf.value ? "onkf" : ""] }, s
|
||||||
|
? [h("img", { src: tempUrl(s, s.files[pos.value]), draggable: false }),
|
||||||
|
h("span", { class: "badge" }, `${fmt(frame.value)} · frame ${frame.value}/${count.value - 1}`),
|
||||||
|
curKf.value ? h("span", { class: "badge kf" }, curKf.value.edit ? "keyframe ✓" : "keyframe") : null]
|
||||||
|
: [h("div", { class: "empty" }, ["Queue this node once to load the guide frames.", h("br"),
|
||||||
|
"Then scrub, press + Keyframe, and drop the edited frames on the cards."])]),
|
||||||
|
h("div", { class: "ctl" }, [
|
||||||
|
h("button", { disabled: !s, title: "previous keyframe", onClick: () => { stop(); jumpKf(-1); } }, "⏮"),
|
||||||
|
h("button", { disabled: !s, title: "previous frame", onClick: () => { stop(); seekIdx(pos.value - 1); } }, "◀"),
|
||||||
|
h("button", { disabled: !s, title: playing.value ? "pause" : "play", onClick: play }, playing.value ? "⏸" : "▶"),
|
||||||
|
h("button", { disabled: !s, title: "next frame", onClick: () => { stop(); seekIdx(pos.value + 1); } }, "▶|"),
|
||||||
|
h("button", { disabled: !s, title: "next keyframe", onClick: () => { stop(); jumpKf(1); } }, "⏭"),
|
||||||
|
h("span", { class: "time" }, s ? `${fmt(frame.value)} / ${fmt(count.value - 1)}` : "--"),
|
||||||
|
h("span", { class: "sp" }),
|
||||||
|
curKf.value
|
||||||
|
? h("button", { class: "danger", onClick: () => removeKf(curKf.value) }, "- Keyframe")
|
||||||
|
: h("button", { class: "pri", disabled: !s, onClick: addHere, title: "mark the current frame as a keyframe" }, "+ Keyframe"),
|
||||||
|
]),
|
||||||
|
h("div", { class: "tl", onPointerdown: tlDown }, s ? [
|
||||||
|
h("div", { class: "fill", style: { width: pct(frame.value) } }),
|
||||||
|
...sorted.value.map(k => h("div", { class: ["mk", k.edit ? "done" : "", k.frame === frame.value ? "cur" : ""], key: k.frame,
|
||||||
|
style: { left: pct(k.frame) }, title: `${fmt(k.frame)}${k.edit ? " (edited)" : " (no edited frame yet)"}`,
|
||||||
|
onPointerdown: e => { e.stopPropagation(); stop(); seekFrame(k.frame); } })),
|
||||||
|
h("div", { class: "head", style: { left: pct(frame.value) } }),
|
||||||
|
] : []),
|
||||||
|
h("div", { class: "sec" }, ["Keyframes ", h("span", { class: "n" }, String(plan.keyframes.length)),
|
||||||
|
plan.keyframes.length ? h("span", null, ` · ${plan.keyframes.filter(k => k.edit).length} edited`) : null]),
|
||||||
|
h("div", { class: "row" }, cards.length ? cards : [h("div", { class: "hint" }, "No keyframes yet.")]),
|
||||||
|
h("div", { class: "sec" }, "References"),
|
||||||
|
h("div", { class: "pics" }, pics),
|
||||||
|
err.value ? h("div", { class: "err" }, err.value) : h("div", { class: "hint" }, "Outputs follow the plan on the next run. Without edited frames, source_frames are ready to edit."),
|
||||||
|
]);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
app.registerExtension({
|
||||||
|
name: "BFSNodes.KeyframePlanner",
|
||||||
|
async nodeCreated(node) {
|
||||||
|
if (node.comfyClass !== "BFSKeyframePlanner") return;
|
||||||
|
styles();
|
||||||
|
const widget = node.widgets?.find(w => w.name === "plan");
|
||||||
|
if (widget) { // the JSON stays a widget (saved with the workflow) but is never shown
|
||||||
|
widget.type = "hidden"; widget.hidden = true; widget.computeSize = () => [0, -4];
|
||||||
|
widget.options = { ...(widget.options || {}), hidden: true, serialize: true };
|
||||||
|
widget.serializeValue = () => widget.value;
|
||||||
|
}
|
||||||
|
const fpsW = node.widgets?.find(w => w.name === "fps");
|
||||||
|
const host = document.createElement("div");
|
||||||
|
host.style.cssText = "position:relative;width:100%;height:100%;min-width:440px;min-height:520px";
|
||||||
|
node.addDOMWidget("bfs_keyframe_planner", "div", host, { serialize: false, hideOnZoom: false, hideInPanel: true });
|
||||||
|
wheelToCanvas(host);
|
||||||
|
const handle = {};
|
||||||
|
createApp({
|
||||||
|
setup: () => Panel({
|
||||||
|
node,
|
||||||
|
get: () => widget?.value ?? "{}",
|
||||||
|
set: v => { if (widget) widget.value = v; node.graph?.setDirtyCanvas(true); },
|
||||||
|
fps: () => Number(fpsW?.value) || 24,
|
||||||
|
expose: o => Object.assign(handle, o),
|
||||||
|
}),
|
||||||
|
}).mount(host);
|
||||||
|
const onExecuted = node.onExecuted;
|
||||||
|
node.onExecuted = function (output) {
|
||||||
|
onExecuted?.apply(this, arguments);
|
||||||
|
handle.executed?.(output?.bfs_kfplan?.[0]);
|
||||||
|
};
|
||||||
|
const onConfigure = node.onConfigure;
|
||||||
|
node.onConfigure = function () {
|
||||||
|
const r = onConfigure?.apply(this, arguments);
|
||||||
|
setTimeout(() => handle.reload?.(), 0);
|
||||||
|
return r;
|
||||||
|
};
|
||||||
|
node.size = [Math.max(node.size[0], DEFAULT_W), Math.max(node.size[1], DEFAULT_H)];
|
||||||
|
},
|
||||||
|
});
|
||||||
@@ -0,0 +1,430 @@
|
|||||||
|
/**
|
||||||
|
* BFS Segment Picker: the canvas of the node. It shows the objects SAM 3 found in the image, colored; hover one to
|
||||||
|
* see it, click to select it. Objects the automatic pass missed are added by text (a SAM 3 prompt), by points
|
||||||
|
* (keep / exclude), segmented at once by /bfs/segpick/add, or painted with the brush (to grow the mark by hand).
|
||||||
|
* ➖ Remove (points) and 🧽 Eraser (brush) make the same kinds of objects, cut out of the mark instead of added to it.
|
||||||
|
*
|
||||||
|
* All state lives in the hidden `selection` STRING widget (JSON: selected ids and the added entries), so a workflow
|
||||||
|
* saves, reloads and shares the selection. The objects themselves come from the node's last run (onExecuted).
|
||||||
|
*/
|
||||||
|
import { app } from "../../scripts/app.js";
|
||||||
|
import { api } from "../../scripts/api.js";
|
||||||
|
import { wheelToCanvas } from "./bfs_wheel.js";
|
||||||
|
import { createApp, h, ref, reactive, computed, watch, nextTick, onMounted, onBeforeUnmount } from "./vendor/vue.esm-browser.prod.mjs";
|
||||||
|
|
||||||
|
// same order as PALETTE in bfs_segment_picker.py, so the panel and the overlay output agree
|
||||||
|
const PALETTE = [[230, 70, 60], [45, 125, 225], [50, 180, 90], [235, 205, 35], [205, 55, 180], [30, 200, 205],
|
||||||
|
[240, 130, 35], [135, 75, 210], [120, 200, 60], [225, 95, 150], [80, 160, 230], [190, 150, 90]];
|
||||||
|
const CUT = [25, 25, 25]; // remove / eraser objects: a dark hole, unlike any PALETTE color
|
||||||
|
const rgb = (c, a = 1) => `rgba(${c[0]},${c[1]},${c[2]},${a})`;
|
||||||
|
const DEFAULT_W = 760, DEFAULT_H = 820;
|
||||||
|
|
||||||
|
const fileUrl = f => api.apiURL(`/view?filename=${encodeURIComponent(f.filename)}&subfolder=${encodeURIComponent(f.subfolder || "")}&type=${f.type || "temp"}`);
|
||||||
|
|
||||||
|
function parseState(text) {
|
||||||
|
let s = {};
|
||||||
|
try { s = JSON.parse(text || "{}") || {}; } catch { s = {}; }
|
||||||
|
return { selected: Array.isArray(s.selected) ? s.selected.map(String) : [], added: Array.isArray(s.added) ? s.added : [],
|
||||||
|
next: Number(s.next) || (Array.isArray(s.added) ? s.added.length + 1 : 1) };
|
||||||
|
}
|
||||||
|
|
||||||
|
const STYLE_ID = "bfs-segpick-style";
|
||||||
|
export function styles() {
|
||||||
|
if (document.getElementById(STYLE_ID)) return;
|
||||||
|
const css = document.createElement("style");
|
||||||
|
css.id = STYLE_ID;
|
||||||
|
css.textContent = `
|
||||||
|
.bsp{position:absolute;inset:0;display:flex;flex-direction:column;gap:6px;padding:6px;box-sizing:border-box;font:12px system-ui,sans-serif;color:#ddd}
|
||||||
|
.bsp .bar{display:flex;flex-wrap:wrap;gap:6px;align-items:center}
|
||||||
|
.bsp button{background:#333;color:#ddd;border:1px solid #555;border-radius:4px;padding:3px 8px;cursor:pointer;font:inherit}
|
||||||
|
.bsp button:hover:not(:disabled){background:#444}
|
||||||
|
.bsp button:disabled{opacity:.45;cursor:default}
|
||||||
|
.bsp button.pri{background:#2d5a8a;border-color:#3d6fa5;color:#fff}
|
||||||
|
.bsp button.on{background:#7a3d8a;border-color:#9a55aa;color:#fff}
|
||||||
|
.bsp input[type=text]{background:#222;color:#ddd;border:1px solid #555;border-radius:4px;padding:3px 6px;font:inherit;flex:1;min-width:120px}
|
||||||
|
.bsp .stage{flex:1;min-height:200px;position:relative;display:flex;align-items:center;justify-content:center;background:#151515;border-radius:4px;overflow:hidden}
|
||||||
|
.bsp .frame{position:relative}
|
||||||
|
.bsp .frame img,.bsp .frame canvas{position:absolute;inset:0;width:100%;height:100%;user-select:none}
|
||||||
|
.bsp .frame canvas{cursor:pointer}
|
||||||
|
.bsp .frame.points canvas,.bsp .frame.remove canvas{cursor:crosshair}
|
||||||
|
.bsp .frame.brush canvas,.bsp .frame.eraser canvas{cursor:none}
|
||||||
|
.bsp .chip.cut{border-style:dashed}
|
||||||
|
.bsp .frame.remove .pt.pos{background:#e74c3c}.bsp .frame.remove .pt.neg{background:#2ecc71}
|
||||||
|
.bsp input[type=range]{width:90px}
|
||||||
|
.bsp .pt{position:absolute;width:12px;height:12px;margin:-6px 0 0 -6px;border-radius:50%;border:2px solid #fff;pointer-events:auto;cursor:pointer;box-shadow:0 0 3px #000}
|
||||||
|
.bsp .pt.pos{background:#2ecc71}.bsp .pt.neg{background:#e74c3c}
|
||||||
|
.bsp .empty{color:#888;text-align:center;padding:20px;line-height:1.6}
|
||||||
|
.bsp .tip{position:absolute;left:6px;top:6px;background:rgba(0,0,0,.75);padding:2px 6px;border-radius:3px;pointer-events:none}
|
||||||
|
.bsp .list{display:flex;flex-wrap:wrap;gap:4px;max-height:96px;overflow:auto}
|
||||||
|
.bsp .chip{display:flex;align-items:center;gap:4px;padding:2px 6px;border-radius:10px;border:1px solid #444;background:#222;cursor:pointer}
|
||||||
|
.bsp .chip.sel{border-color:#fff;background:#3a3a3a}
|
||||||
|
.bsp .chip .sw{width:10px;height:10px;border-radius:2px}
|
||||||
|
.bsp .chip .x{color:#999;padding:0 2px}.bsp .chip .x:hover{color:#f66}
|
||||||
|
.bsp .hint{color:#888}.bsp .err{color:#f77}
|
||||||
|
`;
|
||||||
|
document.head.appendChild(css);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** One loaded object: a colored canvas to draw and its mask alpha for hit tests, at the preview resolution. */
|
||||||
|
function loadMask(o, color) {
|
||||||
|
return new Promise((resolve, reject) => {
|
||||||
|
const img = new Image();
|
||||||
|
img.crossOrigin = "anonymous";
|
||||||
|
img.onload = () => {
|
||||||
|
const w = img.naturalWidth, hh = img.naturalHeight;
|
||||||
|
const c = document.createElement("canvas"); c.width = w; c.height = hh;
|
||||||
|
const g = c.getContext("2d");
|
||||||
|
g.drawImage(img, 0, 0);
|
||||||
|
const alpha = g.getImageData(0, 0, w, hh).data;
|
||||||
|
const hit = new Uint8Array(w * hh);
|
||||||
|
for (let i = 0; i < hit.length; i++) hit[i] = alpha[i * 4 + 3] > 127 ? 1 : 0;
|
||||||
|
g.globalCompositeOperation = "source-in";
|
||||||
|
g.fillStyle = rgb(color); g.fillRect(0, 0, w, hh);
|
||||||
|
// the outline: mask pixels with a background pixel within r, so it stays visible once scaled down
|
||||||
|
const r = Math.max(1, Math.round(Math.max(w, hh) / 400));
|
||||||
|
const e = document.createElement("canvas"); e.width = w; e.height = hh;
|
||||||
|
const eg = e.getContext("2d"), px = eg.createImageData(w, hh), [cr, cg, cb] = color;
|
||||||
|
for (let y = 0; y < hh; y++) for (let x = 0; x < w; x++) {
|
||||||
|
if (!hit[y * w + x]) continue;
|
||||||
|
let edge = false;
|
||||||
|
for (let dy = -r; dy <= r && !edge; dy++) for (let dx = -r; dx <= r; dx++) {
|
||||||
|
const xx = x + dx, yy = y + dy;
|
||||||
|
if (xx < 0 || yy < 0 || xx >= w || yy >= hh || !hit[yy * w + xx]) { edge = true; break; }
|
||||||
|
}
|
||||||
|
if (edge) { const i = (y * w + x) * 4; px.data[i] = cr; px.data[i + 1] = cg; px.data[i + 2] = cb; px.data[i + 3] = 255; }
|
||||||
|
}
|
||||||
|
eg.putImageData(px, 0, 0);
|
||||||
|
resolve({ canvas: c, edge: e, hit, w, h: hh });
|
||||||
|
};
|
||||||
|
img.onerror = reject;
|
||||||
|
img.src = fileUrl(o.mask);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
export function Panel(io, options = {}) { // exported: BFS Video Segment Picker shows one per frame
|
||||||
|
const showEraser = options.showEraser !== false;
|
||||||
|
const showRemove = options.showRemove !== false;
|
||||||
|
const state = reactive(parseState(io.get()));
|
||||||
|
const data = ref(null); // {session, width, height, image, objects} from the last run
|
||||||
|
const loaded = new Map(); // object id -> loadMask result (not reactive: canvases and big arrays)
|
||||||
|
const ready = ref(0); // bumps when masks finish loading, to redraw
|
||||||
|
const hover = ref(null);
|
||||||
|
const mode = ref("select"); // "select" | "points" | "brush" | "remove" (points) | "eraser" (brush)
|
||||||
|
const points = ref([]); // the point entry being built
|
||||||
|
const strokes = ref([]); // the brush entry being built: [{r, pts: [[x, y], ...]}], relative to the image
|
||||||
|
const brush = ref(.02); // brush radius, relative to the image width
|
||||||
|
let painting = false, cursor = null;
|
||||||
|
const text = ref("");
|
||||||
|
const busy = ref(""); const error = ref("");
|
||||||
|
const stageEl = ref(null); const canvasEl = ref(null);
|
||||||
|
const fit = reactive({ w: 0, h: 0 });
|
||||||
|
|
||||||
|
const save = () => io.set(JSON.stringify({ selected: state.selected, added: state.added, next: state.next }));
|
||||||
|
const entryId = kind => `${options.entryPrefix || ""}${kind}${state.next}`;
|
||||||
|
const objects = computed(() => data.value?.objects || []);
|
||||||
|
const colorOf = id => {
|
||||||
|
const i = Math.max(0, objects.value.findIndex(o => o.id === id));
|
||||||
|
return objects.value[i]?.remove ? CUT : PALETTE[i % PALETTE.length];
|
||||||
|
};
|
||||||
|
const isSel = id => state.selected.includes(id);
|
||||||
|
const pointing = () => mode.value === "points" || mode.value === "remove";
|
||||||
|
const brushing = () => mode.value === "brush" || mode.value === "eraser";
|
||||||
|
const cutting = () => mode.value === "remove" || mode.value === "eraser";
|
||||||
|
function setMode(m) {
|
||||||
|
if (m !== mode.value) { points.value = []; strokes.value = []; } // unfinished points / strokes mean something else in another mode
|
||||||
|
mode.value = m; hover.value = null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadAll() {
|
||||||
|
const objs = objects.value;
|
||||||
|
await Promise.all(objs.map(async (o, i) => {
|
||||||
|
const key = `${o.id}|${o.mask.filename}`;
|
||||||
|
if (loaded.get(o.id)?.key === key) return;
|
||||||
|
try { loaded.set(o.id, { key, ...(await loadMask(o, o.remove ? CUT : (options.maskColor || PALETTE[i % PALETTE.length]))) }); } catch { /* shown as missing */ }
|
||||||
|
}));
|
||||||
|
ready.value++;
|
||||||
|
}
|
||||||
|
|
||||||
|
function relayout() {
|
||||||
|
const el = stageEl.value, d = data.value;
|
||||||
|
if (!el || !d) return;
|
||||||
|
const W = el.clientWidth - 2, H = el.clientHeight - 2, s = Math.min(W / d.width, H / d.height);
|
||||||
|
fit.w = Math.max(1, Math.floor(d.width * s)); fit.h = Math.max(1, Math.floor(d.height * s));
|
||||||
|
}
|
||||||
|
|
||||||
|
function draw() {
|
||||||
|
const c = canvasEl.value, d = data.value;
|
||||||
|
if (!c || !d) return;
|
||||||
|
const first = loaded.values().next().value;
|
||||||
|
const W = first?.w || fit.w, H = first?.h || fit.h;
|
||||||
|
if (c.width !== W || c.height !== H) { c.width = W; c.height = H; }
|
||||||
|
const g = c.getContext("2d");
|
||||||
|
g.clearRect(0, 0, W, H);
|
||||||
|
// biggest first, so small objects stay visible on top of the ones around them
|
||||||
|
const order = [...objects.value].sort((a, b) => b.area - a.area);
|
||||||
|
// the image itself is never tinted: an object is an outline, filled only while hovered (light) or selected
|
||||||
|
for (const o of order) {
|
||||||
|
const m = loaded.get(o.id);
|
||||||
|
if (!m) continue;
|
||||||
|
const sel = isSel(o.id), hov = hover.value === o.id;
|
||||||
|
if (sel || hov) { g.globalAlpha = sel ? (hov ? .6 : .45) : .3; g.drawImage(m.canvas, 0, 0, W, H); }
|
||||||
|
if (mode.value !== "select" && !sel && !hov) continue;
|
||||||
|
g.globalAlpha = sel || hov ? 1 : .7;
|
||||||
|
g.drawImage(m.edge, 0, 0, W, H);
|
||||||
|
}
|
||||||
|
g.globalAlpha = 1;
|
||||||
|
if (!brushing()) return;
|
||||||
|
g.strokeStyle = g.fillStyle = cutting() ? "rgba(20,20,20,.6)" : "rgba(255,0,255,.55)"; g.lineCap = g.lineJoin = "round";
|
||||||
|
for (const st of strokes.value) {
|
||||||
|
const [x0, y0] = st.pts[0];
|
||||||
|
g.beginPath(); g.arc(x0 * W, y0 * H, st.r * W, 0, 2 * Math.PI); g.fill();
|
||||||
|
if (st.pts.length < 2) continue;
|
||||||
|
g.lineWidth = 2 * st.r * W; g.beginPath(); g.moveTo(x0 * W, y0 * H);
|
||||||
|
for (const [x, y] of st.pts.slice(1)) g.lineTo(x * W, y * H);
|
||||||
|
g.stroke();
|
||||||
|
}
|
||||||
|
if (cursor) {
|
||||||
|
g.lineWidth = Math.max(1, W / 500); g.strokeStyle = "#fff";
|
||||||
|
g.beginPath(); g.arc(cursor.x * W, cursor.y * H, brush.value * W, 0, 2 * Math.PI); g.stroke();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
watch([ready, hover, mode, brush, () => strokes.value.length, () => [...state.selected], () => fit.w, () => fit.h], () => nextTick(draw));
|
||||||
|
|
||||||
|
function at(e) {
|
||||||
|
const r = canvasEl.value.getBoundingClientRect();
|
||||||
|
return { x: Math.min(1, Math.max(0, (e.clientX - r.left) / r.width)), y: Math.min(1, Math.max(0, (e.clientY - r.top) / r.height)) };
|
||||||
|
}
|
||||||
|
function objectAt(p) {
|
||||||
|
let best = null;
|
||||||
|
for (const o of objects.value) {
|
||||||
|
const m = loaded.get(o.id);
|
||||||
|
if (!m) continue;
|
||||||
|
const i = Math.min(m.h - 1, Math.floor(p.y * m.h)) * m.w + Math.min(m.w - 1, Math.floor(p.x * m.w));
|
||||||
|
if (m.hit[i] && (!best || o.area < best.area)) best = o; // the smallest one under the mouse: inner objects stay pickable
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
function toggle(id) {
|
||||||
|
state.selected = isSel(id) ? state.selected.filter(i => i !== id) : [...state.selected, id];
|
||||||
|
save();
|
||||||
|
}
|
||||||
|
function onPointer(e) {
|
||||||
|
if (!data.value) return;
|
||||||
|
e.preventDefault(); e.stopPropagation();
|
||||||
|
const p = at(e);
|
||||||
|
if (pointing()) {
|
||||||
|
points.value = [...points.value, { x: +p.x.toFixed(4), y: +p.y.toFixed(4), label: (e.button === 2 || e.shiftKey) ? 0 : 1 }];
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (brushing()) {
|
||||||
|
canvasEl.value.setPointerCapture(e.pointerId);
|
||||||
|
painting = true;
|
||||||
|
strokes.value = [...strokes.value, { r: +brush.value.toFixed(4), pts: [[+p.x.toFixed(4), +p.y.toFixed(4)]] }];
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
const o = objectAt(p);
|
||||||
|
if (options.autoPointOnClick && (!o || !o.entry)) {
|
||||||
|
add({ id: entryId("p"), points: [{ x: +p.x.toFixed(4), y: +p.y.toFixed(4), label: 1 }] });
|
||||||
|
} else if (o) toggle(o.id);
|
||||||
|
}
|
||||||
|
function onMove(e) {
|
||||||
|
if (data.value && brushing()) {
|
||||||
|
cursor = at(e);
|
||||||
|
const pts = strokes.value.at(-1)?.pts, last = pts?.at(-1);
|
||||||
|
// a point every ~quarter radius keeps the stroke smooth without bloating the saved JSON
|
||||||
|
if (painting && Math.hypot(cursor.x - last[0], cursor.y - last[1]) > brush.value / 4) pts.push([+cursor.x.toFixed(4), +cursor.y.toFixed(4)]);
|
||||||
|
draw();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (!data.value || pointing()) { hover.value = null; return; }
|
||||||
|
hover.value = objectAt(at(e))?.id ?? null;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function add(entry) {
|
||||||
|
if (!data.value) { error.value = "Run the node first (▶ Segment)."; return; }
|
||||||
|
busy.value = entry.text ? `Finding “${entry.text}”…` : entry.strokes ? "Adding the painted area…" : "Segmenting the points…"; error.value = "";
|
||||||
|
try {
|
||||||
|
const r = await api.fetchApi("/bfs/segpick/add", { method: "POST", headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ session: data.value.session, entry }) });
|
||||||
|
const j = await r.json();
|
||||||
|
if (!r.ok || j.error) throw new Error(j.error || r.statusText);
|
||||||
|
if (!j.objects.length) { error.value = entry.text ? `Nothing found for “${entry.text}”: try other words or a lower threshold.` : "The points gave an empty mask."; return; }
|
||||||
|
state.added = [...state.added.filter(a => a.id !== entry.id), entry];
|
||||||
|
state.next = Math.max(state.next, Number(entry.id.match(/(\d+)$/)?.[1] || 0) + 1);
|
||||||
|
data.value = { ...data.value, objects: [...data.value.objects.filter(o => o.entry !== entry.id), ...j.objects] };
|
||||||
|
state.selected = [...state.selected, ...j.objects.map(o => o.id).filter(id => !isSel(id))]; // what you add is what you want
|
||||||
|
save();
|
||||||
|
await loadAll();
|
||||||
|
} catch (err) {
|
||||||
|
error.value = String(err.message || err);
|
||||||
|
} finally { busy.value = ""; }
|
||||||
|
}
|
||||||
|
const addText = () => { const t = text.value.trim(); if (t) { add({ id: entryId("t"), text: t }); text.value = ""; } };
|
||||||
|
const addPoints = async () => {
|
||||||
|
if (!points.value.some(p => p.label)) return;
|
||||||
|
await add(cutting() ? { id: entryId("r"), remove: true, points: points.value } : { id: entryId("p"), points: points.value });
|
||||||
|
if (!error.value) setMode("select");
|
||||||
|
};
|
||||||
|
const addBrush = async () => {
|
||||||
|
if (!strokes.value.length) return;
|
||||||
|
const stroke = { image: data.value.image_key, strokes: strokes.value };
|
||||||
|
await add(cutting() ? { id: entryId("e"), remove: true, ...stroke } : { id: entryId("b"), ...stroke });
|
||||||
|
if (!error.value) setMode("select");
|
||||||
|
};
|
||||||
|
function removeEntry(entryId) {
|
||||||
|
state.added = state.added.filter(a => a.id !== entryId);
|
||||||
|
const gone = objects.value.filter(o => o.entry === entryId).map(o => o.id);
|
||||||
|
state.selected = state.selected.filter(i => !gone.includes(i));
|
||||||
|
if (data.value) data.value = { ...data.value, objects: data.value.objects.filter(o => o.entry !== entryId) };
|
||||||
|
gone.forEach(i => loaded.delete(i));
|
||||||
|
save(); ready.value++;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function runNode() {
|
||||||
|
error.value = "";
|
||||||
|
try {
|
||||||
|
const p = await app.graphToPrompt();
|
||||||
|
await api.queuePrompt(0, p, { partialExecutionTargets: [String(io.node.id)] });
|
||||||
|
busy.value = "Segmenting…";
|
||||||
|
} catch (err) { error.value = String(err.message || err); }
|
||||||
|
}
|
||||||
|
|
||||||
|
io.expose({
|
||||||
|
executed: d => {
|
||||||
|
busy.value = "";
|
||||||
|
if (!d) return;
|
||||||
|
if (data.value?.session !== d.session) loaded.clear();
|
||||||
|
if (data.value && data.value.image_key !== d.image_key) strokes.value = [];
|
||||||
|
// brush entries belong to the image they were painted on: another image drops them (text and points segment again)
|
||||||
|
const stale = state.added.filter(a => a.strokes && a.image && a.image !== d.image_key).map(a => a.id);
|
||||||
|
if (stale.length) { state.added = state.added.filter(a => !stale.includes(a.id)); save(); }
|
||||||
|
data.value = d;
|
||||||
|
// ids of objects that no longer exist (another image, other concepts) leave the selection
|
||||||
|
const ids = new Set(d.objects.map(o => o.id));
|
||||||
|
const keep = state.selected.filter(i => ids.has(i));
|
||||||
|
if (keep.length !== state.selected.length) { state.selected = keep; save(); }
|
||||||
|
nextTick(() => { relayout(); loadAll(); });
|
||||||
|
},
|
||||||
|
reload: () => { Object.assign(state, parseState(io.get())); ready.value++; },
|
||||||
|
});
|
||||||
|
|
||||||
|
let ro;
|
||||||
|
const done = () => { busy.value = ""; }; // a failed or fully cached run sends no output to this node
|
||||||
|
onMounted(() => {
|
||||||
|
ro = new ResizeObserver(() => relayout()); ro.observe(stageEl.value);
|
||||||
|
api.addEventListener("execution_error", done); api.addEventListener("execution_success", done);
|
||||||
|
});
|
||||||
|
onBeforeUnmount(() => {
|
||||||
|
ro?.disconnect();
|
||||||
|
api.removeEventListener("execution_error", done); api.removeEventListener("execution_success", done);
|
||||||
|
});
|
||||||
|
watch(data, () => nextTick(relayout));
|
||||||
|
|
||||||
|
return () => {
|
||||||
|
const d = data.value, objs = objects.value;
|
||||||
|
const hovered = objs.find(o => o.id === hover.value);
|
||||||
|
return h("div", { class: "bsp", onKeydown: e => e.stopPropagation() }, [
|
||||||
|
h("div", { class: "bar" }, [
|
||||||
|
h("button", { class: "pri", onClick: runNode, title: "Run this node only (and what feeds it) to segment the image" }, d ? "↻ Segment again" : "▶ Segment"),
|
||||||
|
h("button", { class: mode.value === "select" ? "on" : "", onClick: () => setMode("select") }, "👆 Select"),
|
||||||
|
h("button", { class: mode.value === "points" ? "on" : "", disabled: !d, onClick: () => setMode("points"),
|
||||||
|
title: "Place points on something the automatic pass missed" }, "📍 Points"),
|
||||||
|
h("button", { class: mode.value === "brush" ? "on" : "", disabled: !d, onClick: () => setMode("brush"),
|
||||||
|
title: "Paint extra area into the mark by hand (e.g. hair SAM left out)" }, "🖌 Brush"),
|
||||||
|
showRemove ? h("button", { class: mode.value === "remove" ? "on" : "", disabled: !d, onClick: () => setMode("remove"),
|
||||||
|
title: "Place points on something to cut out of the mark (SAM segments it)" }, "➖ Remove") : null,
|
||||||
|
showEraser ? h("button", { class: mode.value === "eraser" ? "on" : "", disabled: !d, onClick: () => setMode("eraser"),
|
||||||
|
title: "Paint an area to subtract manually from the selected mask" }, "🧽 Eraser") : null,
|
||||||
|
h("span", { class: "hint" }, d ? `${objs.length} objects · ${state.selected.length} selected` : ""),
|
||||||
|
h("span", { style: "flex:1" }),
|
||||||
|
h("button", { disabled: !objs.length, onClick: () => { state.selected = objs.map(o => o.id); save(); } }, "All"),
|
||||||
|
h("button", { disabled: !state.selected.length, onClick: () => { state.selected = []; save(); } }, "None"),
|
||||||
|
]),
|
||||||
|
h("div", { class: "bar" }, [
|
||||||
|
h("input", { type: "text", value: text.value, placeholder: "Find by text (SAM 3 prompt), e.g. red umbrella, the man on the left",
|
||||||
|
disabled: !d, onInput: e => { text.value = e.target.value; }, onKeydown: e => { e.stopPropagation(); if (e.key === "Enter") addText(); } }),
|
||||||
|
h("button", { disabled: !d || !text.value.trim() || !!busy.value, onClick: addText }, "🔎 Find"),
|
||||||
|
pointing() ? [
|
||||||
|
h("button", { class: "pri", disabled: !points.value.some(p => p.label) || !!busy.value, onClick: addPoints },
|
||||||
|
`${cutting() ? "✓ Remove this" : "✓ Create object"} (${points.value.length})`),
|
||||||
|
h("button", { disabled: !points.value.length, onClick: () => { points.value = []; } }, "Clear points"),
|
||||||
|
] : null,
|
||||||
|
brushing() ? [
|
||||||
|
h("input", { type: "range", min: .003, max: .1, step: .001, value: brush.value, title: "Brush size",
|
||||||
|
onInput: e => { brush.value = +e.target.value; } }),
|
||||||
|
h("button", { class: "pri", disabled: !strokes.value.length || !!busy.value, onClick: addBrush },
|
||||||
|
`${cutting() ? "✓ Erase painted area" : "✓ Add painted area"} (${strokes.value.length})`),
|
||||||
|
h("button", { disabled: !strokes.value.length, onClick: () => { strokes.value = strokes.value.slice(0, -1); } }, "Undo"),
|
||||||
|
h("button", { disabled: !strokes.value.length, onClick: () => { strokes.value = []; } }, "Clear"),
|
||||||
|
] : null,
|
||||||
|
]),
|
||||||
|
h("div", { class: "stage", ref: stageEl, onContextmenu: e => e.preventDefault() }, d ? [
|
||||||
|
h("div", { class: ["frame", mode.value !== "select" && mode.value], style: `width:${fit.w}px;height:${fit.h}px` }, [
|
||||||
|
h("img", { src: fileUrl(d.image), draggable: false }),
|
||||||
|
h("canvas", { ref: canvasEl, onPointerdown: onPointer, onPointermove: onMove, onPointerup: () => { painting = false; },
|
||||||
|
onPointerleave: () => { hover.value = null; cursor = null; if (!painting) draw(); } }),
|
||||||
|
...points.value.map((p, k) => h("div", { class: ["pt", p.label ? "pos" : "neg"], style: `left:${p.x * 100}%;top:${p.y * 100}%`,
|
||||||
|
title: "click to remove", onPointerdown: e => { e.stopPropagation(); points.value = points.value.filter((_, j) => j !== k); } })),
|
||||||
|
]),
|
||||||
|
hovered ? h("div", { class: "tip" }, `${hovered.id} · ${hovered.label}${hovered.score != null ? ` · ${hovered.score}` : ""}`) : null,
|
||||||
|
mode.value === "points" ? h("div", { class: "tip" }, "click = keep · right-click or shift+click = exclude · click a point to remove it") : null,
|
||||||
|
mode.value === "remove" ? h("div", { class: "tip" }, "click on what to cut out of the mark · right-click or shift+click = keep it in · then ✓ Remove this") : null,
|
||||||
|
mode.value === "brush" ? h("div", { class: "tip" }, "paint what the mark should also cover · then ✓ Add painted area") : null,
|
||||||
|
mode.value === "eraser" ? h("div", { class: "tip" }, "paint what to cut out of the mark · then ✓ Erase painted area") : null,
|
||||||
|
] : h("div", { class: "empty" }, ["Connect an image and press ▶ Segment (or run the workflow).", h("br"),
|
||||||
|
"Then click objects to select them; add what was missed by text or 📍 points."])),
|
||||||
|
busy.value ? h("div", { class: "hint" }, busy.value) : null,
|
||||||
|
error.value ? h("div", { class: "err" }, error.value) : null,
|
||||||
|
objs.length ? h("div", { class: "list" }, objs.map(o => h("div", {
|
||||||
|
class: ["chip", isSel(o.id) && "sel", o.remove && "cut"], title: `${o.label}${o.score != null ? ` · score ${o.score}` : ""} · ${(o.area * 100).toFixed(1)}% of the image`,
|
||||||
|
onClick: () => toggle(o.id), onPointerenter: () => { hover.value = o.id; }, onPointerleave: () => { hover.value = null; },
|
||||||
|
}, [
|
||||||
|
h("span", { class: "sw", style: `background:${rgb(colorOf(o.id))}` }),
|
||||||
|
`${o.remove ? "➖ " : ""}${o.id} ${o.label}`,
|
||||||
|
o.entry ? h("span", { class: "x", title: "remove this added object", onClick: e => { e.stopPropagation(); removeEntry(o.entry); } }, "×") : null,
|
||||||
|
]))) : null,
|
||||||
|
d ? h("div", { class: "hint" }, "The outputs follow the selection on the next run.") : null,
|
||||||
|
]);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
app.registerExtension({
|
||||||
|
name: "BFSNodes.SegmentPicker",
|
||||||
|
async nodeCreated(node) {
|
||||||
|
if (node.comfyClass !== "BFSSegmentPicker") return;
|
||||||
|
styles();
|
||||||
|
const widget = node.widgets?.find(w => w.name === "selection");
|
||||||
|
if (widget) { // the JSON stays a widget (saved with the workflow) but is never shown
|
||||||
|
widget.type = "hidden"; widget.hidden = true; widget.computeSize = () => [0, -4];
|
||||||
|
widget.options = { ...(widget.options || {}), hidden: true, serialize: true };
|
||||||
|
widget.serializeValue = () => widget.value;
|
||||||
|
}
|
||||||
|
const host = document.createElement("div");
|
||||||
|
host.style.cssText = "position:relative;width:100%;height:100%;min-width:420px;min-height:460px";
|
||||||
|
// hideInPanel: the side panel's legacy renderer writes its own width into the widget, shrinking the canvas in the node
|
||||||
|
node.addDOMWidget("bfs_segment_picker", "div", host, { serialize: false, hideOnZoom: false, hideInPanel: true });
|
||||||
|
wheelToCanvas(host);
|
||||||
|
const handle = {};
|
||||||
|
createApp({
|
||||||
|
setup: () => Panel({
|
||||||
|
node,
|
||||||
|
get: () => widget?.value ?? "{}",
|
||||||
|
set: v => { if (widget) widget.value = v; node.graph?.setDirtyCanvas(true); },
|
||||||
|
expose: o => Object.assign(handle, o),
|
||||||
|
}),
|
||||||
|
}).mount(host);
|
||||||
|
const onExecuted = node.onExecuted;
|
||||||
|
node.onExecuted = function (output) {
|
||||||
|
onExecuted?.apply(this, arguments);
|
||||||
|
handle.executed?.(output?.bfs_segpick?.[0]);
|
||||||
|
};
|
||||||
|
const onConfigure = node.onConfigure;
|
||||||
|
node.onConfigure = function () {
|
||||||
|
const r = onConfigure?.apply(this, arguments);
|
||||||
|
setTimeout(() => handle.reload?.(), 0);
|
||||||
|
return r;
|
||||||
|
};
|
||||||
|
node.size = [Math.max(node.size[0], DEFAULT_W), Math.max(node.size[1], DEFAULT_H)];
|
||||||
|
},
|
||||||
|
});
|
||||||
@@ -9,6 +9,8 @@
|
|||||||
* whatever you do in the player (dragging the divider, pausing, scrubbing) until ■ stop. Both draw the two videos into
|
* whatever you do in the player (dragging the divider, pausing, scrubbing) until ■ stop. Both draw the two videos into
|
||||||
* a canvas and record it in the browser, in real time. The browser's file (WebM, or a fragmented MP4 many apps won't
|
* a canvas and record it in the browser, in real time. The browser's file (WebM, or a fragmented MP4 many apps won't
|
||||||
* play) is then converted by ComfyUI to the chosen format: MP4 (H.264), a small MP4 (under 10 MB) or a GIF.
|
* play) is then converted by ComfyUI to the chosen format: MP4 (H.264), a small MP4 (under 10 MB) or a GIF.
|
||||||
|
* 🖼 PNG downloads the frame on screen as a picture, drawn the same way. With a single frame (two images) the node
|
||||||
|
* also sends lossless PNGs of guide / target / difference: ⬇ save PNG draws from those, at the compared size.
|
||||||
*/
|
*/
|
||||||
import { app } from "../../scripts/app.js";
|
import { app } from "../../scripts/app.js";
|
||||||
import { api } from "../../scripts/api.js";
|
import { api } from "../../scripts/api.js";
|
||||||
@@ -109,6 +111,7 @@ function buildPlayer(host) {
|
|||||||
selFormat.title = "Format of what save / rec download. MP4: H.264 + AAC, plays and previews anywhere. MP4 small: "
|
selFormat.title = "Format of what save / rec download. MP4: H.264 + AAC, plays and previews anywhere. MP4 small: "
|
||||||
+ "the same, sized to stay under 10 MB. GIF: no sound, under 10 MB. browser file: the recording as it is, no conversion.";
|
+ "the same, sized to stay under 10 MB. GIF: no sound, under 10 MB. browser file: the recording as it is, no conversion.";
|
||||||
const btnRec = el("button", { title: "Record what you do in the player until stop, then download it", onclick: () => (rec.mr ? stopRec() : startRec()) }, "● rec");
|
const btnRec = el("button", { title: "Record what you do in the player until stop, then download it", onclick: () => (rec.mr ? stopRec() : startRec()) }, "● rec");
|
||||||
|
const btnShot = el("button", { title: "Download the frame on screen as a PNG, as shown (layout, divider, labels, refs)", onclick: () => snapshot() }, "🖼 PNG");
|
||||||
const chkLabels = el("input", { type: "checkbox" });
|
const chkLabels = el("input", { type: "checkbox" });
|
||||||
chkLabels.checked = true;
|
chkLabels.checked = true;
|
||||||
chkLabels.addEventListener("change", () => { st.labels = chkLabels.checked; });
|
chkLabels.addEventListener("change", () => { st.labels = chkLabels.checked; });
|
||||||
@@ -131,7 +134,7 @@ function buildPlayer(host) {
|
|||||||
stage,
|
stage,
|
||||||
refBox,
|
refBox,
|
||||||
el("div", { class: "row" }, [btnPrev, btnPlay, btnNext, scrub, btnMute, info]),
|
el("div", { class: "row" }, [btnPrev, btnPlay, btnNext, scrub, btnMute, info]),
|
||||||
el("div", { class: "row" }, [btnSave, btnRec, selFormat, el("label", { title: "Draw the names on the saved / recorded video" }, [chkLabels, "labels"]), lblRefs]),
|
el("div", { class: "row" }, [btnSave, btnShot, btnRec, selFormat, el("label", { title: "Draw the names on the saved / recorded video" }, [chkLabels, "labels"]), lblRefs]),
|
||||||
strip,
|
strip,
|
||||||
notes,
|
notes,
|
||||||
]));
|
]));
|
||||||
@@ -267,6 +270,9 @@ function buildPlayer(host) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const last = new Map(); // video -> its last whole frame, drawn while it seeks (a seeking video draws blank)
|
const last = new Map(); // video -> its last whole frame, drawn while it seeks (a seeking video draws blank)
|
||||||
|
const stills = {}; // source -> loaded lossless PNG (single-frame comparisons only)
|
||||||
|
const single = () => st.data?.frames === 1;
|
||||||
|
const still = s => (stills[s]?.complete && stills[s].naturalWidth ? stills[s] : null);
|
||||||
function pic(v) {
|
function pic(v) {
|
||||||
if (v.readyState < 2 || v.seeking) return last.get(v) || v;
|
if (v.readyState < 2 || v.seeking) return last.get(v) || v;
|
||||||
let c = last.get(v);
|
let c = last.get(v);
|
||||||
@@ -306,8 +312,8 @@ function buildPlayer(host) {
|
|||||||
|
|
||||||
function drawVideos(ctx, area) {
|
function drawVideos(ctx, area) {
|
||||||
const cw = ctx.canvas.width, ch = area ?? ctx.canvas.height, m = st.mode, p = st.pos;
|
const cw = ctx.canvas.width, ch = area ?? ctx.canvas.height, m = st.mode, p = st.pos;
|
||||||
const B = pic(vBot), T = pic(vTop);
|
const B = still(st.bottom) || pic(vBot), T = still(st.top) || pic(vTop);
|
||||||
const aw = vBot.videoWidth || st.data.width, ah = vBot.videoHeight || st.data.height;
|
const aw = B.naturalWidth || vBot.videoWidth || st.data.width, ah = B.naturalHeight || vBot.videoHeight || st.data.height;
|
||||||
const name = s => s === "guide" ? (st.data?.labels?.[0] || "guide") : s === "target" ? (st.data?.labels?.[1] || "target") : "difference";
|
const name = s => s === "guide" ? (st.data?.labels?.[0] || "guide") : s === "target" ? (st.data?.labels?.[1] || "target") : "difference";
|
||||||
ctx.fillStyle = "#000"; ctx.fillRect(0, 0, cw, ch);
|
ctx.fillStyle = "#000"; ctx.fillRect(0, 0, cw, ch);
|
||||||
if (m === "side") { // bottom left, top right, as on the stage
|
if (m === "side") { // bottom left, top right, as on the stage
|
||||||
@@ -378,6 +384,29 @@ function buildPlayer(host) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// the canvas for save / rec / PNG: the videos at their size (side by side twice as wide), the reference strip below
|
||||||
|
function sheet(aw, ah) {
|
||||||
|
const canvas = el("canvas");
|
||||||
|
canvas.width = (st.mode === "side" ? 2 * aw + Math.round(2 * aw * 0.004) : aw) & ~1;
|
||||||
|
canvas.videoHeight = ah & ~1;
|
||||||
|
canvas.height = canvas.videoHeight + stripHeight(canvas.videoHeight);
|
||||||
|
return canvas;
|
||||||
|
}
|
||||||
|
|
||||||
|
function snapshot() { // the frame on screen -> PNG (a single frame draws from the lossless stills)
|
||||||
|
if (!st.data) return;
|
||||||
|
if (st.playing) toggle();
|
||||||
|
const s = still(st.bottom);
|
||||||
|
const canvas = sheet(s?.naturalWidth || vBot.videoWidth || st.data.width, s?.naturalHeight || vBot.videoHeight || st.data.height);
|
||||||
|
compose(canvas.getContext("2d"));
|
||||||
|
canvas.toBlob(blob => {
|
||||||
|
if (!blob) { alert("BFS Video Compare: could not make the PNG."); return; }
|
||||||
|
const href = URL.createObjectURL(blob);
|
||||||
|
download(href, `${stem()}${single() ? "" : `_f${st.frame}`}.png`);
|
||||||
|
setTimeout(() => URL.revokeObjectURL(href), 60000);
|
||||||
|
}, "image/png");
|
||||||
|
}
|
||||||
|
|
||||||
function buttons() {
|
function buttons() {
|
||||||
if (rec.busy) { // uploading / converting the finished recording
|
if (rec.busy) { // uploading / converting the finished recording
|
||||||
btnRec.classList.remove("rec");
|
btnRec.classList.remove("rec");
|
||||||
@@ -391,18 +420,17 @@ function buildPlayer(host) {
|
|||||||
btnRec.classList.toggle("rec", busy);
|
btnRec.classList.toggle("rec", busy);
|
||||||
btnRec.textContent = busy ? (rec.full ? "■ stop" : `■ stop ${((performance.now() - rec.t0) / 1000).toFixed(0)}s`) : "● rec";
|
btnRec.textContent = busy ? (rec.full ? "■ stop" : `■ stop ${((performance.now() - rec.t0) / 1000).toFixed(0)}s`) : "● rec";
|
||||||
btnSave.disabled = busy;
|
btnSave.disabled = busy;
|
||||||
btnSave.textContent = busy && rec.full ? `⬇ ${Math.round(100 * (vBot.currentTime / (vBot.duration || 1)))}%` : "⬇ save";
|
btnSave.textContent = busy && rec.full ? `⬇ ${Math.round(100 * (vBot.currentTime / (vBot.duration || 1)))}%` : single() ? "⬇ save PNG" : "⬇ save";
|
||||||
|
btnShot.disabled = busy;
|
||||||
|
// one frame is a picture: save downloads it as PNG, a video recording has nothing to show
|
||||||
|
for (const b of [btnRec, selFormat, btnShot]) b.style.display = single() ? "none" : "";
|
||||||
}
|
}
|
||||||
|
|
||||||
function startRec(full = false) {
|
function startRec(full = false) {
|
||||||
if (!st.data || rec.mr || rec.busy) return false;
|
if (!st.data || rec.mr || rec.busy) return false;
|
||||||
if (!window.MediaRecorder || !HTMLCanvasElement.prototype.captureStream) { alert("This browser can't record a canvas (MediaRecorder)."); return false; }
|
if (!window.MediaRecorder || !HTMLCanvasElement.prototype.captureStream) { alert("This browser can't record a canvas (MediaRecorder)."); return false; }
|
||||||
const aw = vBot.videoWidth || st.data.width, ah = vBot.videoHeight || st.data.height;
|
|
||||||
const side = st.mode === "side", canvas = el("canvas");
|
|
||||||
// the size of the layout it starts in (side by side is twice as wide); a later mode change is fitted into it
|
// the size of the layout it starts in (side by side is twice as wide); a later mode change is fitted into it
|
||||||
canvas.width = (side ? 2 * aw + Math.round(2 * aw * 0.004) : aw) & ~1;
|
const canvas = sheet(vBot.videoWidth || st.data.width, vBot.videoHeight || st.data.height);
|
||||||
canvas.videoHeight = ah & ~1; // the videos; the reference strip goes below them
|
|
||||||
canvas.height = canvas.videoHeight + stripHeight(canvas.videoHeight);
|
|
||||||
const ctx = canvas.getContext("2d");
|
const ctx = canvas.getContext("2d");
|
||||||
compose(ctx);
|
compose(ctx);
|
||||||
const stream = canvas.captureStream(Math.max(30, Math.round(fps())));
|
const stream = canvas.captureStream(Math.max(30, Math.round(fps())));
|
||||||
@@ -437,6 +465,7 @@ function buildPlayer(host) {
|
|||||||
|
|
||||||
async function saveFull() { // once from the first frame to the last, then stop and download
|
async function saveFull() { // once from the first frame to the last, then stop and download
|
||||||
if (!st.data || rec.mr || rec.busy) return;
|
if (!st.data || rec.mr || rec.busy) return;
|
||||||
|
if (single()) { snapshot(); return; }
|
||||||
if (st.playing) toggle();
|
if (st.playing) toggle();
|
||||||
for (const v of videos()) v.loop = false;
|
for (const v of videos()) v.loop = false;
|
||||||
const seeked = v => new Promise(res => { const t = setTimeout(res, 2000); v.addEventListener("seeked", () => { clearTimeout(t); res(); }, { once: true }); });
|
const seeked = v => new Promise(res => { const t = setTimeout(res, 2000); v.addEventListener("seeked", () => { clearTimeout(t); res(); }, { once: true }); });
|
||||||
@@ -458,6 +487,9 @@ function buildPlayer(host) {
|
|||||||
lblRefs.style.display = d.refs?.length ? "flex" : "none";
|
lblRefs.style.display = d.refs?.length ? "flex" : "none";
|
||||||
showRefs();
|
showRefs();
|
||||||
st.data = d;
|
st.data = d;
|
||||||
|
for (const k of Object.keys(stills)) delete stills[k];
|
||||||
|
for (const [k, f] of Object.entries(d.stills || {})) { const im = new Image(); im.crossOrigin = "anonymous"; im.src = url(f); stills[k] = im; }
|
||||||
|
buttons();
|
||||||
notes.textContent = (d.notes || []).join("\n");
|
notes.textContent = (d.notes || []).join("\n");
|
||||||
empty.style.display = "none";
|
empty.style.display = "none";
|
||||||
scrub.max = String(Math.max(0, d.frames - 1));
|
scrub.max = String(Math.max(0, d.frames - 1));
|
||||||
@@ -478,6 +510,11 @@ app.registerExtension({
|
|||||||
const r = created?.apply(this, arguments);
|
const r = created?.apply(this, arguments);
|
||||||
styles();
|
styles();
|
||||||
const host = el("div", { style: "width:100%;height:100%;min-height:320px" });
|
const host = el("div", { style: "width:100%;height:100%;min-height:320px" });
|
||||||
|
// Keep clicks/drags inside the interactive player from reaching LiteGraph. Otherwise a
|
||||||
|
// pointer event on a video/control can be interpreted as a canvas gesture and the node
|
||||||
|
// may recalculate its DOM-widget height, making the compare panel collapse after use.
|
||||||
|
for (const type of ["pointerdown", "pointerup", "pointermove", "click", "dblclick", "contextmenu"])
|
||||||
|
host.addEventListener(type, e => e.stopPropagation());
|
||||||
this._bfsCompare = buildPlayer(host);
|
this._bfsCompare = buildPlayer(host);
|
||||||
wheelToCanvas(host); // the wheel zooms the canvas over the player too
|
wheelToCanvas(host); // the wheel zooms the canvas over the player too
|
||||||
this.addDOMWidget("bfs_video_compare", "div", host, { serialize: false, hideOnZoom: false, getMinHeight: () => 320 });
|
this.addDOMWidget("bfs_video_compare", "div", host, { serialize: false, hideOnZoom: false, getMinHeight: () => 320 });
|
||||||
|
|||||||
@@ -0,0 +1,385 @@
|
|||||||
|
/**
|
||||||
|
* BFS Video Segment Picker: a strip with every frame of the batch on top; click a frame to show it in the image
|
||||||
|
* Segment Picker's canvas (the same Panel, one selection per frame). The strip marks each frame: ✓ picked by hand,
|
||||||
|
* ↔ tracked from a picked frame, ⚠ lost or nothing. State: the hidden `selection` widget, {"frames": {"i": selection}}.
|
||||||
|
*/
|
||||||
|
import { app } from "../../scripts/app.js";
|
||||||
|
import { api } from "../../scripts/api.js";
|
||||||
|
import { wheelToCanvas } from "./bfs_wheel.js";
|
||||||
|
import { Panel, styles as pickerStyles } from "./bfs_segment_picker.js";
|
||||||
|
import { createApp, h, ref, reactive } from "./vendor/vue.esm-browser.prod.mjs";
|
||||||
|
|
||||||
|
const fileUrl = f => api.apiURL(`/view?filename=${encodeURIComponent(f.filename)}&subfolder=${encodeURIComponent(f.subfolder || "")}&type=${f.type || "temp"}`);
|
||||||
|
|
||||||
|
const SUBJECTS = [["1", "#eb2828", "red"], ["2", "#236eeb", "blue"], ["3", "#3ccd3c", "green"]];
|
||||||
|
function parse(text) { // {frame: {subject: selection}}; a bare selection (older state) is subject 1
|
||||||
|
let s = {};
|
||||||
|
try { s = JSON.parse(text || "{}") || {}; } catch { s = {}; }
|
||||||
|
const f = (s.frames && typeof s.frames === "object") ? s.frames : {};
|
||||||
|
const out = {};
|
||||||
|
for (const [k, v] of Object.entries(f)) {
|
||||||
|
if (v && (v.selected || v.added)) out[k] = { "1": v };
|
||||||
|
else if (v && typeof v === "object") {
|
||||||
|
out[k] = Object.fromEntries(Object.entries(v).filter(([sk]) => sk === "lock" || (+sk >= 1 && +sk <= SUBJECTS.length)));
|
||||||
|
} else out[k] = {};
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
let styled = false;
|
||||||
|
function styles() {
|
||||||
|
if (styled) return;
|
||||||
|
styled = true;
|
||||||
|
const css = document.createElement("style");
|
||||||
|
css.textContent = `
|
||||||
|
.bvsp{--card:#1b1d21;--line:#2e3138;--muted:#8b919c;--text:#e3e6eb;--accent:#3b82f6;--ok:#22c55e;--trk:#60a5fa;--bad:#f43f5e;
|
||||||
|
position:absolute;inset:0;display:flex;flex-direction:column;gap:6px;padding:6px;box-sizing:border-box;font:12px system-ui,sans-serif;color:var(--text);overflow:hidden}
|
||||||
|
.bvsp .card{background:var(--card);border:1px solid var(--line);border-radius:8px;padding:6px 8px;display:flex;flex-direction:column;gap:6px;flex:0 0 auto}
|
||||||
|
.bvsp .row{display:flex;align-items:center;gap:8px;min-width:0}
|
||||||
|
.bvsp .sp{flex:1}
|
||||||
|
.bvsp .muted{color:var(--muted);font-size:11px}
|
||||||
|
.bvsp .btn{background:#2a2d34;border:1px solid #3a3e47;color:var(--text);border-radius:6px;padding:4px 10px;cursor:pointer;font:inherit;white-space:nowrap}
|
||||||
|
.bvsp .btn:hover{background:#343843}
|
||||||
|
.bvsp .btn.pri{background:var(--accent);border-color:#4b8ff7;color:#fff;font-weight:600}
|
||||||
|
.bvsp .btn.pri:hover{background:#4b8ff7}
|
||||||
|
.bvsp .chk{display:flex;align-items:center;gap:4px;color:var(--muted);font-size:11px;cursor:pointer;user-select:none}
|
||||||
|
.bvsp .subj{display:flex;gap:4px}
|
||||||
|
.bvsp .subj .t{display:flex;align-items:center;gap:5px;border:1px solid var(--line);background:#23262c;border-radius:999px;padding:3px 10px 3px 6px;cursor:pointer;color:var(--muted);font-weight:600}
|
||||||
|
.bvsp .subj .t .clr{display:inline-flex;align-items:center;justify-content:center;width:14px;height:14px;border-radius:50%;margin-left:4px;font-size:10px;line-height:1;background:rgba(255,255,255,.15);color:#fff;cursor:pointer}
|
||||||
|
.bvsp .subj .t .clr:hover{background:#f43f5e;color:#fff}
|
||||||
|
.bvsp .subj .t i{width:10px;height:10px;border-radius:50%;display:block}
|
||||||
|
.bvsp .subj .t.on{color:#fff;border-color:currentColor;background:#2b2f37}
|
||||||
|
.bvsp .subj .t.lock.on{color:#fbbf24;background:#44351b}
|
||||||
|
.bvsp .stat{display:flex;align-items:center;gap:8px;font-size:11px;color:#cfd6e0}
|
||||||
|
.bvsp .stat .pb{flex:0 0 110px;height:5px;background:#2a2d34;border-radius:3px;overflow:hidden}
|
||||||
|
.bvsp .stat .pb i{display:block;height:100%}
|
||||||
|
.bvsp .stat .txt{flex:1;white-space:nowrap;overflow:hidden;text-overflow:ellipsis}
|
||||||
|
.bvsp .stat .tog{cursor:pointer;color:var(--trk);user-select:none}
|
||||||
|
.bvsp .log{max-height:90px;overflow:auto;background:#141518;border:1px solid var(--line);border-radius:6px;padding:4px 6px;font:11px ui-monospace,monospace;color:#aab}
|
||||||
|
.bvsp .counts{display:flex;gap:10px;font-size:11px;color:var(--muted)}
|
||||||
|
.bvsp .counts b{font-weight:700}
|
||||||
|
.bvsp .strip{display:flex;gap:5px;overflow-x:auto;padding-bottom:2px}
|
||||||
|
.bvsp .fr{position:relative;flex:0 0 auto;width:96px;height:58px;border-radius:6px;background:#111 center/cover no-repeat;cursor:pointer;border:2px solid transparent;outline:1px solid var(--line)}
|
||||||
|
.bvsp .fr.on{border-color:#f5b041}
|
||||||
|
.bvsp .fr .b{position:absolute;left:3px;bottom:3px;font-size:10px;background:rgba(0,0,0,.7);padding:0 4px;border-radius:3px}
|
||||||
|
.bvsp .fr .s{position:absolute;right:3px;top:3px;font-size:11px;font-weight:700;padding:0 4px;border-radius:3px;background:rgba(0,0,0,.7)}
|
||||||
|
.bvsp .fr .s.picked{color:var(--ok)}.bvsp .fr .s.tracked{color:var(--trk)}.bvsp .fr .s.lost{color:var(--bad)}
|
||||||
|
.bvsp .fr .dots{position:absolute;left:3px;top:3px;display:flex;gap:2px}
|
||||||
|
.bvsp .fr .dots i{width:8px;height:8px;border-radius:50%;border:1px solid #000;display:block}
|
||||||
|
.bvsp .tl{display:flex;gap:10px;align-items:center}
|
||||||
|
.bvsp .tl img{height:72px;border-radius:6px;border:1px solid var(--line);flex:0 0 auto}
|
||||||
|
.bvsp .tl .scrub{flex:1;display:flex;flex-direction:column;gap:6px;min-width:140px}
|
||||||
|
.bvsp .tl .track{position:relative;height:18px}
|
||||||
|
.bvsp .tl .track .bg{position:absolute;left:0;right:0;top:7px;height:4px;border-radius:2px}
|
||||||
|
.bvsp .tl .track input{position:absolute;inset:0;width:100%;margin:0;background:transparent;-webkit-appearance:none;appearance:none}
|
||||||
|
.bvsp .tl .track input::-webkit-slider-runnable-track{background:transparent;height:18px}
|
||||||
|
.bvsp .tl .track input::-moz-range-track{background:transparent}
|
||||||
|
.bvsp .tl .track input::-webkit-slider-thumb{-webkit-appearance:none;width:14px;height:14px;margin-top:2px;border-radius:50%;background:#fff;border:2px solid var(--accent);cursor:pointer}
|
||||||
|
.bvsp .tl .track input::-moz-range-thumb{width:12px;height:12px;border-radius:50%;background:#fff;border:2px solid var(--accent);cursor:pointer}
|
||||||
|
.bvsp .tl .fn{font-variant-numeric:tabular-nums;color:var(--text);font-weight:600}
|
||||||
|
.bvsp .chips{display:flex;flex-wrap:wrap;gap:4px;align-items:center}
|
||||||
|
.bvsp .chips .c{display:flex;align-items:center;gap:4px;border:1px solid #6b2a37;background:#2e1a1f;color:#fecdd3;border-radius:999px;padding:1px 8px;cursor:pointer;font-size:11px}
|
||||||
|
.bvsp .chips .c:hover{background:#45222b}
|
||||||
|
.bvsp .chips .c i{width:8px;height:8px;border-radius:50%;display:block}
|
||||||
|
.bvsp .chips .okk{color:var(--ok);font-size:11px}
|
||||||
|
.bvsp .host{position:absolute;inset:0}
|
||||||
|
.bvsp .empty{color:var(--muted);text-align:center;padding:12px}
|
||||||
|
`;
|
||||||
|
document.head.appendChild(css);
|
||||||
|
}
|
||||||
|
|
||||||
|
// A run the panel starts for picking ("Segment again", "+ frame") only segments the strip: the selection it sends is marked
|
||||||
|
// segment_only, so the node skips tracking and Molmo2. "Propagate" and a normal workflow run track as usual.
|
||||||
|
const propagating = new Set();
|
||||||
|
let wrapped = false;
|
||||||
|
function wrapQueue() {
|
||||||
|
if (wrapped) return;
|
||||||
|
wrapped = true;
|
||||||
|
const orig = api.queuePrompt.bind(api);
|
||||||
|
api.queuePrompt = async function (number, p, opts) {
|
||||||
|
const targets = opts?.partialExecutionTargets || [];
|
||||||
|
for (const id of targets) {
|
||||||
|
const out = p?.output?.[String(id)];
|
||||||
|
if (out?.class_type !== "BFSVideoSegmentPicker" || propagating.has(String(id))) continue;
|
||||||
|
try {
|
||||||
|
const sel = JSON.parse(out.inputs.selection || "{}");
|
||||||
|
sel.segment_only = true;
|
||||||
|
out.inputs.selection = JSON.stringify(sel);
|
||||||
|
} catch { /* leave it */ }
|
||||||
|
}
|
||||||
|
return orig(number, p, opts);
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
app.registerExtension({
|
||||||
|
name: "BFSNodes.VideoSegmentPicker",
|
||||||
|
async nodeCreated(node) {
|
||||||
|
if (node.comfyClass !== "BFSVideoSegmentPicker") return;
|
||||||
|
pickerStyles(); styles();
|
||||||
|
const widget = node.widgets?.find(w => w.name === "selection");
|
||||||
|
if (widget) {
|
||||||
|
widget.type = "hidden"; widget.hidden = true; widget.computeSize = () => [0, -4];
|
||||||
|
widget.options = { ...(widget.options || {}), hidden: true, serialize: true };
|
||||||
|
widget.serializeValue = () => widget.value;
|
||||||
|
}
|
||||||
|
const frames = reactive({ v: parse(widget?.value) });
|
||||||
|
let focusFrame = 0;
|
||||||
|
const scopeWidget = node.widgets?.find(w => w.name === "propagate_scope");
|
||||||
|
const validScopes = ["all frames", "only this frame", "from active frame onward", "from first selected frame onward", "fill only empty masks"];
|
||||||
|
const scope = ref(validScopes.includes(scopeWidget?.value) ? scopeWidget.value : "fill only empty masks");
|
||||||
|
if (scopeWidget) scopeWidget.value = scope.value;
|
||||||
|
const save = () => { if (widget) widget.value = JSON.stringify({ frames: frames.v, active_frame: focusFrame }); node.graph?.setDirtyCanvas(true); };
|
||||||
|
const hasPicks = k => Object.values(frames.v).some(f => f?.[k]?.selected?.length);
|
||||||
|
const clearSubject = k => {
|
||||||
|
let changed = false;
|
||||||
|
for (const f of Object.keys(frames.v)) {
|
||||||
|
if (frames.v[f]?.[k]) {
|
||||||
|
delete frames.v[f][k];
|
||||||
|
changed = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (changed) {
|
||||||
|
scope.value = "all frames";
|
||||||
|
if (scopeWidget) scopeWidget.value = "all frames";
|
||||||
|
save();
|
||||||
|
mountPanel();
|
||||||
|
if (propagatedOnce()) propagate();
|
||||||
|
}
|
||||||
|
};
|
||||||
|
const run = reactive({ data: null }); // {frames: [panel data per frame], report: [...]}
|
||||||
|
const active = ref(0);
|
||||||
|
const subject = ref("1"); // which subject the clicks select (its color = Picture number)
|
||||||
|
|
||||||
|
const root = document.createElement("div");
|
||||||
|
root.style.cssText = "position:relative;width:100%;height:100%;min-width:440px;min-height:420px";
|
||||||
|
node.addDOMWidget("bfs_video_segment_picker", "div", root, { serialize: false, hideOnZoom: false, hideInPanel: true });
|
||||||
|
wheelToCanvas(root);
|
||||||
|
|
||||||
|
// the image picker's Panel for the active frame, remounted when the frame changes
|
||||||
|
let panelApp = null, handle = {};
|
||||||
|
const panelHost = document.createElement("div");
|
||||||
|
panelHost.className = "host";
|
||||||
|
const mountPanel = () => {
|
||||||
|
panelApp?.unmount(); panelHost.innerHTML = ""; handle = {};
|
||||||
|
const i = run.data?.indices?.[active.value] ?? active.value; // the real frame index of the strip entry
|
||||||
|
focusFrame = i;
|
||||||
|
const sj = subject.value;
|
||||||
|
const el = document.createElement("div"); el.style.cssText = "position:absolute;inset:0"; panelHost.appendChild(el);
|
||||||
|
panelApp = createApp({ setup: () => Panel({
|
||||||
|
node,
|
||||||
|
get: () => JSON.stringify(frames.v[String(i)]?.[sj] || {}),
|
||||||
|
set: v => {
|
||||||
|
focusFrame = i;
|
||||||
|
const before = JSON.stringify(frames.v[String(i)]?.[sj]?.selected || []);
|
||||||
|
try { frames.v[String(i)] = { ...(frames.v[String(i)] || {}), [sj]: JSON.parse(v) }; } catch { /* keep */ }
|
||||||
|
save();
|
||||||
|
if (JSON.stringify(frames.v[String(i)]?.[sj]?.selected || []) !== before) autoPropagate();
|
||||||
|
},
|
||||||
|
expose: o => Object.assign(handle, o),
|
||||||
|
}, { showEraser: false, showRemove: false, autoPointOnClick: sj === "lock", entryPrefix: sj === "lock" ? "lock_" : "",
|
||||||
|
maskColor: sj === "lock" ? [251, 191, 36] : null }) });
|
||||||
|
panelApp.mount(el);
|
||||||
|
const d = run.data?.frames?.[active.value];
|
||||||
|
if (d) setTimeout(() => handle.executed?.(d), 0);
|
||||||
|
};
|
||||||
|
|
||||||
|
const pickedSubjects = i => Object.entries(frames.v[String(run.data?.indices?.[i] ?? i)] || {}).filter(([, v]) => (v?.selected || []).length).map(([k]) => k);
|
||||||
|
const status = i => {
|
||||||
|
const own = (frames.v[String(run.data?.indices?.[i] ?? i)]?.[subject.value]?.selected || []).length > 0;
|
||||||
|
const r = subject.value === "lock"
|
||||||
|
? (run.data?.protected_frames?.[i] ? "tracked protected" : "")
|
||||||
|
: (run.data?.report?.[i]?.subjects?.[subject.value] || run.data?.report?.[i]?.source || "");
|
||||||
|
if (own) return ["picked", "✓"];
|
||||||
|
if (r.startsWith("tracked")) return ["tracked", "↔"];
|
||||||
|
if (r.startsWith("lost")) return ["lost", "⚠"];
|
||||||
|
return ["", ""];
|
||||||
|
};
|
||||||
|
const busy = ref("");
|
||||||
|
const live = reactive({ text: "", progress: null, elapsed: 0, log: [], open: false, running: false });
|
||||||
|
api.addEventListener("bfs_vsegpick_status", e => {
|
||||||
|
const d = e.detail || {};
|
||||||
|
if (String(d.node) !== String(node.id)) return;
|
||||||
|
live.text = d.text; live.elapsed = d.elapsed || 0; live.running = (d.progress ?? 0) < 1;
|
||||||
|
if (d.progress !== null && d.progress !== undefined) live.progress = d.progress;
|
||||||
|
if (d.log) { live.log.push(`[${(d.elapsed || 0).toFixed(1)}s] ${d.text}`); if (live.log.length > 60) live.log.shift(); }
|
||||||
|
});
|
||||||
|
wrapQueue();
|
||||||
|
const segmentOnly = async () => { // a picking run: the new frame is segmented, nothing is tracked
|
||||||
|
busy.value = "segmenting…";
|
||||||
|
try { const p = await app.graphToPrompt(); await api.queuePrompt(0, p, { partialExecutionTargets: [String(node.id)] }); }
|
||||||
|
catch (e) { busy.value = String(e.message || e); }
|
||||||
|
};
|
||||||
|
const propagate = async () => {
|
||||||
|
busy.value = "propagating…";
|
||||||
|
try {
|
||||||
|
const p = await app.graphToPrompt();
|
||||||
|
propagating.add(String(node.id));
|
||||||
|
try { await api.queuePrompt(0, p, { partialExecutionTargets: [String(node.id)] }); } finally { propagating.delete(String(node.id)); }
|
||||||
|
} catch (e) { busy.value = String(e.message || e); }
|
||||||
|
};
|
||||||
|
// after the first Propagate, a pick that changes a frame's selection propagates by itself (1.5 s after the last
|
||||||
|
// click): only the shots whose picks changed are tracked again, the rest comes from the cache
|
||||||
|
const auto = ref(localStorage.getItem("bfs_vsegpick_auto") === "1");
|
||||||
|
let autoTimer = null;
|
||||||
|
const propagatedOnce = () => (run.data?.report || []).some(r => /tracked|lost|Molmo/.test(JSON.stringify(r)));
|
||||||
|
const autoPropagate = () => {
|
||||||
|
if (!auto.value || !propagatedOnce()) return;
|
||||||
|
clearTimeout(autoTimer);
|
||||||
|
busy.value = "auto-propagate in 1.5 s…";
|
||||||
|
autoTimer = setTimeout(propagate, 1500);
|
||||||
|
};
|
||||||
|
const addFrame = f => { // put a frame in the strip (an empty pick) and run, so it can be picked on
|
||||||
|
const n = run.data?.count || 0;
|
||||||
|
f = Math.max(0, Math.min(n - 1, Math.round(+f)));
|
||||||
|
if (!Number.isFinite(f)) return;
|
||||||
|
focusFrame = f;
|
||||||
|
if (!frames.v[String(f)]) { frames.v[String(f)] = {}; save(); }
|
||||||
|
else save();
|
||||||
|
segmentOnly();
|
||||||
|
};
|
||||||
|
const SUBJ = { "1": "red", "2": "blue", "3": "green" };
|
||||||
|
const tlPos = ref(0); // index into run.data.timeline (whole-video scrubber)
|
||||||
|
const counts = () => {
|
||||||
|
const r = run.data?.report || []; const n = run.data?.frames?.length || 0;
|
||||||
|
let picked = 0, tracked = 0, lost = 0;
|
||||||
|
for (let i = 0; i < n; i++) { const [c] = status(i); if (c === "picked") picked++; else if (c === "tracked") tracked++; else if (c === "lost") lost++; }
|
||||||
|
return { picked, tracked, lost, n };
|
||||||
|
};
|
||||||
|
const tlAt = () => run.data.timeline[Math.min(tlPos.value, run.data.timeline.length - 1)];
|
||||||
|
const lostTrack = () => { // the scrubber's track: lost stretches in red over the video's length
|
||||||
|
const n = Math.max(1, (run.data?.count || 1) - 1);
|
||||||
|
const runs = [...(run.data?.lost || [])];
|
||||||
|
if (!runs.length) return "#3a3e47";
|
||||||
|
const stops = ["#3a3e47 0%"];
|
||||||
|
for (const r of runs.sort((a, b) => a.start - b.start)) {
|
||||||
|
const a = (100 * r.start / n).toFixed(2), b = (100 * Math.max(r.end, r.start + 0.5) / n).toFixed(2);
|
||||||
|
stops.push(`#3a3e47 ${a}%`, `#f43f5e ${a}%`, `#f43f5e ${b}%`, `#3a3e47 ${b}%`);
|
||||||
|
}
|
||||||
|
return `linear-gradient(90deg, ${stops.join(", ")}, #3a3e47 100%)`;
|
||||||
|
};
|
||||||
|
createApp({
|
||||||
|
setup: () => () => h("div", { class: "bvsp" }, [
|
||||||
|
h("div", { class: "card" }, [
|
||||||
|
h("div", { class: "row" }, [
|
||||||
|
h("div", { class: "subj" }, [...SUBJECTS.map(([k, col, name]) => h("span", {
|
||||||
|
class: ["t", subject.value === k ? "on" : ""], style: subject.value === k ? { color: col } : null,
|
||||||
|
title: `Subject ${k} (${name}) = Picture ${k}: clicks select this subject`,
|
||||||
|
onClick: () => { if (subject.value !== k) { subject.value = k; mountPanel(); } } },
|
||||||
|
[
|
||||||
|
h("i", { style: { background: col } }),
|
||||||
|
h("span", { style: "color:#fff" }, `${k} · ${name}`),
|
||||||
|
hasPicks(k) ? h("span", {
|
||||||
|
class: "clr",
|
||||||
|
title: `Clear all picks for Subject ${k} (${name})`,
|
||||||
|
onClick: (e) => { e.stopPropagation(); clearSubject(k); }
|
||||||
|
}, "✕") : null,
|
||||||
|
])), h("span", {
|
||||||
|
class: ["t", "lock", subject.value === "lock" ? "on" : ""],
|
||||||
|
title: "Everything outside the selected subject is already protected. Use this padlock only for an object inside that subject's mask; click it to segment with SAM 3.",
|
||||||
|
onClick: () => { if (subject.value !== "lock") { subject.value = "lock"; mountPanel(); } },
|
||||||
|
}, [
|
||||||
|
h("span", null, "🔒"),
|
||||||
|
hasPicks("lock") ? h("span", {
|
||||||
|
class: "clr", title: "Clear all protected-object picks",
|
||||||
|
onClick: e => { e.stopPropagation(); clearSubject("lock"); },
|
||||||
|
}, "✕") : null,
|
||||||
|
])]),
|
||||||
|
h("span", { class: "sp" }),
|
||||||
|
h("select", { value: scope.value, title: "Choose how a new frame selection changes the propagated masks",
|
||||||
|
style: "max-width:190px;background:#23262c;color:#e3e6eb;border:1px solid #3a3e47;border-radius:6px;padding:4px;font:inherit",
|
||||||
|
onChange: e => { scope.value = e.target.value; if (scopeWidget) scopeWidget.value = e.target.value; node.graph?.setDirtyCanvas(true); } },
|
||||||
|
["from active frame onward", "only this frame", "fill only empty masks", "all frames", "from first selected frame onward"].map(v => h("option", { value: v },
|
||||||
|
v === "from active frame onward" ? "From this frame onward" : v === "only this frame" ? "Only this frame"
|
||||||
|
: v === "fill only empty masks" ? "Fill only empty masks" : v === "all frames" ? "Recompute all frames" : "From earliest selected frame"))),
|
||||||
|
h("label", { class: "chk", title: "After a selection changes, apply the selected fix mode automatically after 1.5 seconds" }, [
|
||||||
|
h("input", { type: "checkbox", checked: auto.value, onChange: e => { auto.value = e.target.checked;
|
||||||
|
try { localStorage.setItem("bfs_vsegpick_auto", e.target.checked ? "1" : "0"); } catch { /* ignore */ } } }), "auto"]),
|
||||||
|
h("button", { class: "btn pri", onClick: propagate, title: "Apply the selected fix mode to the current selection" }, "↔ Apply fix"),
|
||||||
|
]),
|
||||||
|
live.text || busy.value ? h("div", { class: "stat" }, [
|
||||||
|
h("span", { class: "pb" }, [h("i", { style: { width: `${Math.round(100 * (live.progress ?? 0))}%`,
|
||||||
|
background: live.running ? "#3b82f6" : "#22c55e" } })]),
|
||||||
|
h("span", { class: "txt", title: live.text }, busy.value && !live.running ? busy.value
|
||||||
|
: `${live.running ? "⏳" : "✓"} ${live.text} · ${live.elapsed}s`),
|
||||||
|
h("span", { class: "tog", onClick: () => { live.open = !live.open; } }, live.open ? "▾ log" : "▸ log"),
|
||||||
|
]) : h("div", { class: "muted" }, subject.value === "lock"
|
||||||
|
? "🔒 Outside the subject is already protected. Click an object inside the subject mask to keep it unchanged, then Apply fix."
|
||||||
|
: "Pick a subject, click a frame below, then click its person. Subject N = Picture N."),
|
||||||
|
live.open ? h("div", { class: "log" }, live.log.length ? live.log.map((l, i) => h("div", { key: i }, l)) : "no log yet") : null,
|
||||||
|
]),
|
||||||
|
h("div", { class: "card" }, [
|
||||||
|
h("div", { class: "row" }, [
|
||||||
|
h("span", { style: "font-weight:600" }, "Frames"),
|
||||||
|
run.data ? h("span", { class: "counts" }, (() => { const c = counts(); return [
|
||||||
|
h("span", null, [h("b", { style: "color:#22c55e" }, "✓ "), `${c.picked} picked`]),
|
||||||
|
h("span", null, [h("b", { style: "color:#60a5fa" }, "↔ "), `${c.tracked} tracked`]),
|
||||||
|
h("span", null, [h("b", { style: "color:#f43f5e" }, "⚠ "), `${c.lost} lost`])]; })()) : null,
|
||||||
|
]),
|
||||||
|
run.data?.frames?.length
|
||||||
|
? h("div", { class: "strip" }, run.data.frames.map((d, i) => {
|
||||||
|
const [cls, mark] = status(i);
|
||||||
|
return h("div", { class: ["fr", i === active.value ? "on" : ""], key: i, title: run.data.report?.[i]?.source || "",
|
||||||
|
style: { backgroundImage: `url("${fileUrl(d.image)}")` },
|
||||||
|
onClick: () => { if (active.value !== i) { active.value = i; focusFrame = run.data.indices?.[i] ?? i; save(); mountPanel(); } } },
|
||||||
|
[h("span", { class: "b" }, `f${run.data.indices?.[i] ?? i}`), mark ? h("span", { class: ["s", cls] }, mark) : null,
|
||||||
|
h("span", { class: "dots" }, pickedSubjects(i).map(k => k === "lock"
|
||||||
|
? h("span", { title: "Protected object picked", style: "color:#fbbf24;background:#181818;border-radius:3px;padding:0 2px" }, "🔒")
|
||||||
|
: h("i", { style: { background: SUBJECTS[(+k - 1) % SUBJECTS.length][1] } })))]);
|
||||||
|
}))
|
||||||
|
: h("div", { class: "empty" }, "Run the node once: the frames appear here. Click a frame, then the person."),
|
||||||
|
]),
|
||||||
|
run.data?.timeline?.length ? h("div", { class: "card" }, [
|
||||||
|
h("div", { class: "tl" }, [
|
||||||
|
h("img", { src: fileUrl(tlAt()), title: "every frame with its masks: scrub to find a wrong or missing mark" }),
|
||||||
|
h("div", { class: "scrub" }, [
|
||||||
|
h("div", { class: "row" }, [
|
||||||
|
h("span", { style: "font-weight:600" }, "Whole video"),
|
||||||
|
h("span", { class: "fn" }, `f${tlAt().frame}`),
|
||||||
|
h("span", { class: "muted" }, `/ ${(run.data.count || 1) - 1}`),
|
||||||
|
h("span", { class: "sp" }),
|
||||||
|
h("button", { class: "btn", onClick: () => addFrame(tlAt().frame),
|
||||||
|
title: "put this frame in the strip to pick or fix the person on it" }, "+ add this frame"),
|
||||||
|
]),
|
||||||
|
h("div", { class: "track" }, [
|
||||||
|
h("div", { class: "bg", style: { background: lostTrack() } }),
|
||||||
|
h("input", { type: "range", min: 0, max: run.data.timeline.length - 1, value: tlPos.value,
|
||||||
|
onInput: e => { tlPos.value = +e.target.value; } }),
|
||||||
|
]),
|
||||||
|
h("div", { class: "chips" }, (run.data.lost || []).length
|
||||||
|
? (run.data.lost || []).slice(0, 12).map(r => h("span", { class: "c", title: "add the middle frame of this stretch to the strip",
|
||||||
|
onClick: () => addFrame((r.start + r.end) / 2) },
|
||||||
|
[h("i", { style: { background: SUBJECTS[(+r.subject - 1) % SUBJECTS.length]?.[1] || "#999" } }), `f${r.start}–${r.end}`]))
|
||||||
|
: [h("span", { class: "okk" }, "✓ no lost stretches")]),
|
||||||
|
]),
|
||||||
|
]),
|
||||||
|
]) : null,
|
||||||
|
h("div", { ref: el => { if (el && !el.contains(panelHost)) { el.appendChild(panelHost); if (!panelApp) mountPanel(); } }, style: "flex:1;position:relative;min-height:260px;border:1px solid #2e3138;border-radius:8px;overflow:hidden" }),
|
||||||
|
]),
|
||||||
|
}).mount(root);
|
||||||
|
|
||||||
|
const onExecuted = node.onExecuted;
|
||||||
|
node.onExecuted = function (output) {
|
||||||
|
onExecuted?.apply(this, arguments);
|
||||||
|
const d = output?.bfs_vsegpick?.[0];
|
||||||
|
if (!d) return;
|
||||||
|
run.data = d; busy.value = "";
|
||||||
|
const focusIndex = d.indices?.indexOf?.(focusFrame) ?? -1;
|
||||||
|
if (focusIndex >= 0) active.value = focusIndex;
|
||||||
|
if (active.value >= d.frames.length) active.value = 0;
|
||||||
|
mountPanel();
|
||||||
|
};
|
||||||
|
const onConfigure = node.onConfigure;
|
||||||
|
node.onConfigure = function () {
|
||||||
|
const r = onConfigure?.apply(this, arguments);
|
||||||
|
setTimeout(() => {
|
||||||
|
let saved = {};
|
||||||
|
try { saved = JSON.parse(widget?.value || "{}"); } catch { /* ignore */ }
|
||||||
|
frames.v = parse(widget?.value); focusFrame = Number.isInteger(+saved.active_frame) ? +saved.active_frame : 0;
|
||||||
|
if (scopeWidget?.value) scope.value = scopeWidget.value;
|
||||||
|
mountPanel();
|
||||||
|
}, 0);
|
||||||
|
return r;
|
||||||
|
};
|
||||||
|
node.size = [Math.max(node.size[0], 780), Math.max(node.size[1], 1180)];
|
||||||
|
},
|
||||||
|
});
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
/**
|
||||||
|
* BFS Workflow Loop: the panel. Lists the nodes upstream of `result`. "Inject" picks a node whose output is replaced
|
||||||
|
* by the current item (or its mask) on every iteration; "Change per iteration" picks widgets with one value per
|
||||||
|
* iteration (one per line, or "a, b, c" / "start:stop:step" for numbers). State lives in the hidden `loop` widget.
|
||||||
|
*/
|
||||||
|
import { app } from "../../scripts/app.js";
|
||||||
|
import { wheelToCanvas } from "./bfs_wheel.js";
|
||||||
|
import { createApp, h, reactive, computed, watch } from "./vendor/vue.esm-browser.prod.mjs";
|
||||||
|
|
||||||
|
const SKIP_TYPES = new Set(["hidden", "converted-widget", "button"]);
|
||||||
|
const SKIP_NAMES = new Set(["control_after_generate", "upload", "image_upload", "videopreview", "loop"]);
|
||||||
|
|
||||||
|
function upstreamNodes(node) {
|
||||||
|
const graph = node.graph || app.graph;
|
||||||
|
const getLink = id => graph.links[id] ?? graph.links.get?.(id);
|
||||||
|
const inp = node.inputs?.find(i => i.name === "result");
|
||||||
|
const link = inp?.link != null ? getLink(inp.link) : null;
|
||||||
|
if (!link) return [];
|
||||||
|
const seen = new Set(), order = [], stack = [link.origin_id];
|
||||||
|
while (stack.length) {
|
||||||
|
const id = stack.pop();
|
||||||
|
if (seen.has(id)) continue;
|
||||||
|
seen.add(id);
|
||||||
|
const n = graph.getNodeById(id);
|
||||||
|
if (!n) continue;
|
||||||
|
order.push(n);
|
||||||
|
for (const i of n.inputs || []) if (i.link != null) { const l = getLink(i.link); if (l) stack.push(l.origin_id); }
|
||||||
|
}
|
||||||
|
return order.sort((a, b) => a.id - b.id);
|
||||||
|
}
|
||||||
|
const widgetsOf = n => (n?.widgets || []).filter(w => w.name && !SKIP_TYPES.has(w.type) && !SKIP_NAMES.has(w.name) && !w.hidden);
|
||||||
|
const kindOf = w => (w?.type === "combo" || Array.isArray(w?.options?.values)) ? "combo"
|
||||||
|
: (w?.type === "toggle" || typeof w?.value === "boolean") ? "bool" : (w?.type === "number" || typeof w?.value === "number") ? "number" : "text";
|
||||||
|
|
||||||
|
function parseValues(text, kind) {
|
||||||
|
if (kind === "number") {
|
||||||
|
const out = [];
|
||||||
|
for (const part of String(text).split(/[,\n]/).map(s => s.trim()).filter(Boolean)) {
|
||||||
|
const r = part.split(":").map(Number);
|
||||||
|
if (r.length === 3 && r.every(Number.isFinite) && r[2] !== 0) {
|
||||||
|
for (let v = r[0], i = 0; (r[2] > 0 ? v <= r[1] + 1e-9 : v >= r[1] - 1e-9) && i < 256; v += r[2], i++) out.push(+v.toFixed(6));
|
||||||
|
} else if (Number.isFinite(+part)) out.push(+part);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
if (kind === "bool") return String(text).split(/[,\n]/).map(s => s.trim().toLowerCase()).filter(Boolean).map(s => ["1", "true", "yes", "on"].includes(s));
|
||||||
|
return String(text).split("\n").map(s => s.trim()).filter(Boolean);
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseState(text) {
|
||||||
|
let s = {};
|
||||||
|
try { s = JSON.parse(text || "{}") || {}; } catch { s = {}; }
|
||||||
|
return {
|
||||||
|
inject: (s.inject || []).map(i => ({ node: String(i.node ?? ""), slot: +i.slot || 0, source: i.source || "items" })),
|
||||||
|
params: (s.params || []).map(p => ({ node: String(p.node ?? ""), widget: p.widget || "", text: p.text ?? (p.values || []).join("\n") })),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
let styled = false;
|
||||||
|
function styles() {
|
||||||
|
if (styled) return;
|
||||||
|
styled = true;
|
||||||
|
const css = document.createElement("style");
|
||||||
|
css.textContent = `
|
||||||
|
.bwl{position:absolute;inset:0;display:flex;flex-direction:column;gap:8px;padding:6px;box-sizing:border-box;font:12px system-ui,sans-serif;color:#ddd;overflow:auto}
|
||||||
|
.bwl .sec{display:flex;align-items:center;gap:6px;color:#9ab;font-size:11px;text-transform:uppercase;letter-spacing:.05em}
|
||||||
|
.bwl .row{background:#1d1d1d;border:1px solid #363636;border-radius:6px;padding:5px;display:flex;flex-direction:column;gap:4px}
|
||||||
|
.bwl .line{display:flex;gap:4px;align-items:center}
|
||||||
|
.bwl select,.bwl textarea{background:#262626;color:#ddd;border:1px solid #4a4a4a;border-radius:4px;padding:3px 5px;font:inherit;min-width:0;flex:1}
|
||||||
|
.bwl textarea{resize:vertical;min-height:38px}
|
||||||
|
.bwl button{background:#333;color:#ddd;border:1px solid #555;border-radius:4px;padding:2px 8px;cursor:pointer;font:inherit}
|
||||||
|
.bwl button:hover{background:#444}
|
||||||
|
.bwl .x{color:#999;cursor:pointer;padding:0 4px}.bwl .x:hover{color:#f66}
|
||||||
|
.bwl .hint{color:#888;font-size:11px}.bwl .warn{color:#f7a}
|
||||||
|
.bwl .foot{color:#bbb}.bwl .foot b{color:#fff}
|
||||||
|
`;
|
||||||
|
document.head.appendChild(css);
|
||||||
|
}
|
||||||
|
|
||||||
|
function Panel(io) {
|
||||||
|
const st = reactive(parseState(io.get()));
|
||||||
|
const tick = reactive({ n: 0 });
|
||||||
|
const nodes = computed(() => (tick.n, upstreamNodes(io.node)));
|
||||||
|
const byId = id => nodes.value.find(n => String(n.id) === String(id));
|
||||||
|
const save = () => io.set(JSON.stringify({
|
||||||
|
inject: st.inject.filter(i => i.node).map(i => ({ node: i.node, slot: i.slot, source: i.source })),
|
||||||
|
params: st.params.filter(p => p.node && p.widget).map(p => {
|
||||||
|
const w = widgetsOf(byId(p.node)).find(x => x.name === p.widget);
|
||||||
|
return { node: p.node, widget: p.widget, values: parseValues(p.text, kindOf(w)), text: p.text };
|
||||||
|
}),
|
||||||
|
}));
|
||||||
|
watch(st, save, { deep: true });
|
||||||
|
io.expose({ refresh: () => { tick.n++; }, reload: () => { Object.assign(st, parseState(io.get())); tick.n++; } });
|
||||||
|
|
||||||
|
const nodeSelect = (obj, onPick) => h("select", { value: obj.node, onChange: e => { obj.node = e.target.value; onPick?.(); } },
|
||||||
|
[h("option", { value: "" }, "— pick a node —"), ...nodes.value.map(n => h("option", { value: String(n.id) }, `[${n.id}] ${n.title || n.type}`))]);
|
||||||
|
|
||||||
|
const injectRow = (i, idx) => {
|
||||||
|
const n = byId(i.node);
|
||||||
|
const outs = (n?.outputs || []).map((o, k) => ({ k, label: `${o.name || o.type} (${o.type})` }));
|
||||||
|
return h("div", { class: "row", key: "i" + idx }, [
|
||||||
|
h("div", { class: "line" }, [nodeSelect(i, () => { i.slot = 0; }), h("span", { class: "x", onClick: () => st.inject.splice(idx, 1) }, "✕")]),
|
||||||
|
n ? h("div", { class: "line" }, [
|
||||||
|
h("select", { value: String(i.slot), onChange: e => { i.slot = +e.target.value; } }, outs.map(o => h("option", { value: String(o.k) }, `output: ${o.label}`))),
|
||||||
|
h("select", { value: i.source, onChange: e => { i.source = e.target.value; } },
|
||||||
|
[h("option", { value: "items" }, "← item i (items)"), h("option", { value: "items_b" }, "← item i (items_b)"),
|
||||||
|
h("option", { value: "item_masks" }, "← mask i (item_masks)")]),
|
||||||
|
]) : null,
|
||||||
|
]);
|
||||||
|
};
|
||||||
|
const paramRow = (p, idx) => {
|
||||||
|
const n = byId(p.node);
|
||||||
|
const w = widgetsOf(n).find(x => x.name === p.widget);
|
||||||
|
const kind = kindOf(w);
|
||||||
|
const vals = parseValues(p.text, kind);
|
||||||
|
return h("div", { class: "row", key: "p" + idx }, [
|
||||||
|
h("div", { class: "line" }, [nodeSelect(p, () => { p.widget = ""; p.text = ""; }), h("span", { class: "x", onClick: () => st.params.splice(idx, 1) }, "✕")]),
|
||||||
|
n ? h("select", { value: p.widget, onChange: e => { p.widget = e.target.value; const ww = widgetsOf(n).find(x => x.name === p.widget); p.text = ww ? String(ww.value ?? "") : ""; } },
|
||||||
|
[h("option", { value: "" }, "— pick a widget —"), ...widgetsOf(n).map(x => h("option", { value: x.name }, `${x.name} (${String(x.value).split(/[\\/]/).pop().slice(0, 36)})`))]) : null,
|
||||||
|
w ? h("textarea", { value: p.text, placeholder: kind === "number" ? "1, 2, 3 or 1:10:1" : "one value per line (iteration 1, 2, 3...)",
|
||||||
|
onInput: e => { p.text = e.target.value; } }) : null,
|
||||||
|
w ? h("div", { class: "hint" }, kind === "combo" ? `${vals.length} values; options: ${(w.options?.values || []).slice(0, 6).map(v => String(v).split(/[\\/]/).pop()).join(", ")}…`
|
||||||
|
: `${vals.length} values${vals.length ? ": " + vals.slice(0, 8).join(", ") + (vals.length > 8 ? "…" : "") : ""}`) : null,
|
||||||
|
]);
|
||||||
|
};
|
||||||
|
const itemsLinked = () => !!io.node.inputs?.find(x => x.name === "items")?.link;
|
||||||
|
|
||||||
|
return () => h("div", { class: "bwl" }, [
|
||||||
|
nodes.value.length ? null : h("div", { class: "warn" }, "Connect what one iteration produces to `result`; its upstream nodes appear here."),
|
||||||
|
h("div", { class: "sec" }, ["Inject the current item into", h("span", { style: "flex:1" }),
|
||||||
|
h("button", { onClick: () => st.inject.push({ node: "", slot: 0, source: "items" }) }, "+")]),
|
||||||
|
...st.inject.map(injectRow),
|
||||||
|
st.inject.length ? null : h("div", { class: "hint" }, "e.g. the Load Image that stands for “the frame”: on iteration i it gets item i."),
|
||||||
|
h("div", { class: "sec" }, ["Change per iteration", h("span", { style: "flex:1" }),
|
||||||
|
h("button", { onClick: () => st.params.push({ node: "", widget: "", text: "" }) }, "+")]),
|
||||||
|
...st.params.map(paramRow),
|
||||||
|
st.params.length ? null : h("div", { class: "hint" }, "e.g. a seed or a prompt line per iteration (cycled when shorter than the items)."),
|
||||||
|
h("div", { class: "foot" }, [itemsLinked() ? "Iterations: one per item." : "No items connected: iterations = the longest value list (or count).",
|
||||||
|
h("span", { style: "flex:1" }), h("button", { onClick: () => { tick.n++; }, title: "re-read the graph" }, "↻")]),
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
app.registerExtension({
|
||||||
|
name: "BFSNodes.WorkflowLoop",
|
||||||
|
async nodeCreated(node) {
|
||||||
|
if (node.comfyClass !== "BFSWorkflowLoop") return;
|
||||||
|
styles();
|
||||||
|
const widget = node.widgets?.find(w => w.name === "loop");
|
||||||
|
if (widget) {
|
||||||
|
widget.type = "hidden"; widget.hidden = true; widget.computeSize = () => [0, -4];
|
||||||
|
widget.options = { ...(widget.options || {}), hidden: true, serialize: true };
|
||||||
|
widget.serializeValue = () => widget.value;
|
||||||
|
}
|
||||||
|
const host = document.createElement("div");
|
||||||
|
host.style.cssText = "position:relative;width:100%;height:100%;min-width:340px;min-height:360px";
|
||||||
|
node.addDOMWidget("bfs_workflow_loop", "div", host, { serialize: false, hideOnZoom: false, hideInPanel: true });
|
||||||
|
wheelToCanvas(host);
|
||||||
|
const handle = {};
|
||||||
|
createApp({ setup: () => Panel({ node, get: () => widget?.value ?? "{}",
|
||||||
|
set: v => { if (widget) widget.value = v; node.graph?.setDirtyCanvas(true); }, expose: o => Object.assign(handle, o) }) }).mount(host);
|
||||||
|
const onConn = node.onConnectionsChange;
|
||||||
|
node.onConnectionsChange = function () { const r = onConn?.apply(this, arguments); handle.refresh?.(); return r; };
|
||||||
|
const onConfigure = node.onConfigure;
|
||||||
|
node.onConfigure = function () { const r = onConfigure?.apply(this, arguments); setTimeout(() => handle.reload?.(), 0); return r; };
|
||||||
|
node.size = [Math.max(node.size[0], 380), Math.max(node.size[1], 440)];
|
||||||
|
},
|
||||||
|
});
|
||||||
@@ -0,0 +1,213 @@
|
|||||||
|
/**
|
||||||
|
* BFS XY Sweep: the panel. Lists every node upstream of the `result` input and its widgets (like TinyTerra's
|
||||||
|
* advanced xyPlot); for X and Y pick a node + widget and its values. Combo widgets (LoRA names, samplers...) get a
|
||||||
|
* filterable checklist; numbers take "0.5, 0.7, 0.9" or a range "start:stop:step"; text takes one value per line.
|
||||||
|
* State lives in the hidden `sweep` STRING widget (JSON), saved with the workflow.
|
||||||
|
*/
|
||||||
|
import { app } from "../../scripts/app.js";
|
||||||
|
import { wheelToCanvas } from "./bfs_wheel.js";
|
||||||
|
import { createApp, h, reactive, computed, watch } from "./vendor/vue.esm-browser.prod.mjs";
|
||||||
|
|
||||||
|
const SKIP_TYPES = new Set(["hidden", "converted-widget", "button"]);
|
||||||
|
const SKIP_NAMES = new Set(["control_after_generate", "upload", "image_upload", "videopreview", "sweep"]);
|
||||||
|
|
||||||
|
function upstreamNodes(node) {
|
||||||
|
const graph = node.graph || app.graph;
|
||||||
|
const inp = node.inputs?.find(i => i.name === "result");
|
||||||
|
const link = inp?.link != null ? graph.links[inp.link] ?? graph.links.get?.(inp.link) : null;
|
||||||
|
if (!link) return [];
|
||||||
|
const seen = new Set(), order = [], stack = [link.origin_id];
|
||||||
|
while (stack.length) {
|
||||||
|
const id = stack.pop();
|
||||||
|
if (seen.has(id)) continue;
|
||||||
|
seen.add(id);
|
||||||
|
const n = graph.getNodeById(id);
|
||||||
|
if (!n) continue;
|
||||||
|
order.push(n);
|
||||||
|
for (const i of n.inputs || []) {
|
||||||
|
if (i.link == null) continue;
|
||||||
|
const l = graph.links[i.link] ?? graph.links.get?.(i.link);
|
||||||
|
if (l) stack.push(l.origin_id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return order.sort((a, b) => a.id - b.id);
|
||||||
|
}
|
||||||
|
|
||||||
|
function widgetsOf(n) {
|
||||||
|
return (n.widgets || []).filter(w => w.name && !SKIP_TYPES.has(w.type) && !SKIP_NAMES.has(w.name) && !w.hidden
|
||||||
|
&& !(w.options && w.options.hidden));
|
||||||
|
}
|
||||||
|
|
||||||
|
function kindOf(w) {
|
||||||
|
if (!w) return "text";
|
||||||
|
if (w.type === "combo" || Array.isArray(w.options?.values)) return "combo";
|
||||||
|
if (w.type === "toggle" || typeof w.value === "boolean") return "bool";
|
||||||
|
if (w.type === "number" || typeof w.value === "number") return "number";
|
||||||
|
return "text";
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseNumbers(text) {
|
||||||
|
const out = [];
|
||||||
|
for (const part of String(text).split(/[,\n]/).map(s => s.trim()).filter(Boolean)) {
|
||||||
|
const r = part.split(":").map(Number);
|
||||||
|
if (r.length === 3 && r.every(Number.isFinite) && r[2] !== 0) {
|
||||||
|
for (let v = r[0], i = 0; (r[2] > 0 ? v <= r[1] + 1e-9 : v >= r[1] - 1e-9) && i < 256; v += r[2], i++) out.push(+v.toFixed(6));
|
||||||
|
} else if (Number.isFinite(+part)) out.push(+part);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
|
||||||
|
function parseState(text) {
|
||||||
|
let s = {};
|
||||||
|
try { s = JSON.parse(text || "{}") || {}; } catch { s = {}; }
|
||||||
|
const axis = a => ({ node: a?.node != null ? String(a.node) : "", widget: a?.widget || "", values: Array.isArray(a?.values) ? a.values : [],
|
||||||
|
text: a?.text ?? (Array.isArray(a?.values) ? a.values.join(", ") : ""), title: a?.title || "", filter: "" });
|
||||||
|
return { x: axis(s.x), y: axis(s.y) };
|
||||||
|
}
|
||||||
|
|
||||||
|
let styled = false;
|
||||||
|
function styles() {
|
||||||
|
if (styled) return;
|
||||||
|
styled = true;
|
||||||
|
const css = document.createElement("style");
|
||||||
|
css.textContent = `
|
||||||
|
.bxy{position:absolute;inset:0;display:flex;flex-direction:column;gap:8px;padding:6px;box-sizing:border-box;font:12px system-ui,sans-serif;color:#ddd;overflow:auto}
|
||||||
|
.bxy .ax{background:#1d1d1d;border:1px solid #363636;border-radius:6px;padding:6px;display:flex;flex-direction:column;gap:5px}
|
||||||
|
.bxy .hd{display:flex;align-items:center;gap:6px}
|
||||||
|
.bxy .tag{font-weight:700;background:#2d5a8a;color:#fff;border-radius:4px;padding:1px 7px}
|
||||||
|
.bxy .tag.y{background:#7a3d8a}
|
||||||
|
.bxy select,.bxy input,.bxy textarea{background:#262626;color:#ddd;border:1px solid #4a4a4a;border-radius:4px;padding:3px 5px;font:inherit;min-width:0}
|
||||||
|
.bxy select{flex:1}
|
||||||
|
.bxy textarea{resize:vertical;min-height:40px}
|
||||||
|
.bxy .opts{max-height:120px;overflow:auto;display:flex;flex-direction:column;gap:1px;border:1px solid #333;border-radius:4px;padding:3px}
|
||||||
|
.bxy .opts label{display:flex;gap:5px;align-items:center;white-space:nowrap;overflow:hidden;text-overflow:ellipsis;cursor:pointer}
|
||||||
|
.bxy .opts label:hover{background:#2a2a2a}
|
||||||
|
.bxy .chips{display:flex;flex-wrap:wrap;gap:3px}
|
||||||
|
.bxy .chip{background:#2a2a2a;border:1px solid #444;border-radius:9px;padding:0 6px;font-size:11px}
|
||||||
|
.bxy .hint{color:#888;font-size:11px}
|
||||||
|
.bxy .foot{display:flex;align-items:center;gap:6px;color:#bbb}
|
||||||
|
.bxy .foot .n{font-weight:700;color:#fff}
|
||||||
|
.bxy .warn{color:#f7a}
|
||||||
|
.bxy button{background:#333;color:#ddd;border:1px solid #555;border-radius:4px;padding:2px 8px;cursor:pointer;font:inherit}
|
||||||
|
.bxy button:hover{background:#444}
|
||||||
|
`;
|
||||||
|
document.head.appendChild(css);
|
||||||
|
}
|
||||||
|
|
||||||
|
function Panel(io) {
|
||||||
|
const st = reactive(parseState(io.get()));
|
||||||
|
const tick = reactive({ n: 0 }); // bumps on graph changes so the node lists refresh
|
||||||
|
const nodes = computed(() => (tick.n, upstreamNodes(io.node)));
|
||||||
|
const nodeById = id => nodes.value.find(n => String(n.id) === String(id));
|
||||||
|
const widgetOf = a => widgetsOf(nodeById(a.node) || {}).find(w => w.name === a.widget);
|
||||||
|
|
||||||
|
const save = () => {
|
||||||
|
const out = {};
|
||||||
|
for (const k of ["x", "y"]) {
|
||||||
|
const a = st[k];
|
||||||
|
if (!a.node || !a.widget) continue;
|
||||||
|
const kind = kindOf(widgetOf(a));
|
||||||
|
let values = a.values;
|
||||||
|
if (kind === "number") values = parseNumbers(a.text);
|
||||||
|
else if (kind === "text") values = String(a.text).split("\n").map(s => s.trim()).filter(Boolean);
|
||||||
|
else if (kind === "bool") values = [false, true].filter(v => a.values.includes(v));
|
||||||
|
const n = nodeById(a.node);
|
||||||
|
out[k] = { node: a.node, widget: a.widget, values, text: a.text, title: n ? (n.title || n.type) : a.title };
|
||||||
|
}
|
||||||
|
io.set(JSON.stringify(out));
|
||||||
|
};
|
||||||
|
watch(st, save, { deep: true });
|
||||||
|
|
||||||
|
const count = computed(() => {
|
||||||
|
const len = a => {
|
||||||
|
if (!a.node || !a.widget) return 0;
|
||||||
|
const kind = kindOf(widgetOf(a));
|
||||||
|
if (kind === "number") return parseNumbers(a.text).length;
|
||||||
|
if (kind === "text") return String(a.text).split("\n").filter(s => s.trim()).length;
|
||||||
|
return a.values.length;
|
||||||
|
};
|
||||||
|
const x = len(st.x), y = len(st.y);
|
||||||
|
return x * (y || 1);
|
||||||
|
});
|
||||||
|
|
||||||
|
io.expose({ refresh: () => { tick.n++; }, reload: () => { Object.assign(st, parseState(io.get())); tick.n++; } });
|
||||||
|
|
||||||
|
const axisView = (k, label) => {
|
||||||
|
const a = st[k];
|
||||||
|
const n = nodeById(a.node);
|
||||||
|
const ws = n ? widgetsOf(n) : [];
|
||||||
|
const w = widgetOf(a);
|
||||||
|
const kind = kindOf(w);
|
||||||
|
const pickNode = e => { a.node = e.target.value; a.widget = ""; a.values = []; a.text = ""; };
|
||||||
|
const pickWidget = e => {
|
||||||
|
a.widget = e.target.value; a.values = []; const ww = widgetOf(a);
|
||||||
|
a.text = ww && kindOf(ww) !== "combo" ? String(ww.value ?? "") : "";
|
||||||
|
};
|
||||||
|
let editor = null;
|
||||||
|
if (w && kind === "combo") {
|
||||||
|
const all = (w.options?.values || []).map(String);
|
||||||
|
const shown = a.filter ? all.filter(v => v.toLowerCase().includes(a.filter.toLowerCase())) : all;
|
||||||
|
editor = [
|
||||||
|
h("input", { placeholder: "filter options…", value: a.filter, onInput: e => { a.filter = e.target.value; } }),
|
||||||
|
h("div", { class: "opts" }, shown.slice(0, 400).map(v => h("label", { title: v }, [
|
||||||
|
h("input", { type: "checkbox", checked: a.values.includes(v),
|
||||||
|
onChange: e => { a.values = e.target.checked ? [...a.values, v] : a.values.filter(x => x !== v); } }), v]))),
|
||||||
|
a.values.length ? h("div", { class: "chips" }, a.values.map(v => h("span", { class: "chip", title: v }, v.split(/[\\/]/).pop()))) : null,
|
||||||
|
];
|
||||||
|
} else if (w && kind === "bool") {
|
||||||
|
editor = h("div", { class: "chips" }, [false, true].map(v => h("label", null, [
|
||||||
|
h("input", { type: "checkbox", checked: a.values.includes(v),
|
||||||
|
onChange: e => { a.values = e.target.checked ? [...a.values, v] : a.values.filter(x => x !== v); } }), String(v)])));
|
||||||
|
} else if (w && kind === "number") {
|
||||||
|
editor = [h("input", { value: a.text, placeholder: "0.5, 0.7, 0.9 or 0.5:0.9:0.2", onInput: e => { a.text = e.target.value; } }),
|
||||||
|
h("div", { class: "hint" }, `→ ${parseNumbers(a.text).join(", ") || "no values"} (now ${w.value})`)];
|
||||||
|
} else if (w) {
|
||||||
|
editor = [h("textarea", { value: a.text, placeholder: "one value per line", onInput: e => { a.text = e.target.value; } })];
|
||||||
|
}
|
||||||
|
return h("div", { class: "ax" }, [
|
||||||
|
h("div", { class: "hd" }, [h("span", { class: ["tag", k] }, label),
|
||||||
|
h("select", { value: a.node, onChange: pickNode }, [h("option", { value: "" }, k === "y" ? "— none —" : "— pick a node —"),
|
||||||
|
...nodes.value.map(nd => h("option", { value: String(nd.id) }, `[${nd.id}] ${nd.title || nd.type}`))]),
|
||||||
|
]),
|
||||||
|
n ? h("select", { value: a.widget, onChange: pickWidget }, [h("option", { value: "" }, "— pick a widget —"),
|
||||||
|
...ws.map(ww => h("option", { value: ww.name }, `${ww.name} (${String(ww.value).split(/[\\/]/).pop().slice(0, 40)})`))]) : null,
|
||||||
|
...(Array.isArray(editor) ? editor : [editor]),
|
||||||
|
]);
|
||||||
|
};
|
||||||
|
|
||||||
|
return () => h("div", { class: "bxy" }, [
|
||||||
|
nodes.value.length ? null : h("div", { class: "warn" }, "Connect the image to compare to `result`; its upstream nodes appear here."),
|
||||||
|
axisView("x", "X"),
|
||||||
|
axisView("y", "Y"),
|
||||||
|
h("div", { class: "foot" }, [h("span", null, ["cells: ", h("span", { class: "n" }, String(count.value))]),
|
||||||
|
count.value > 64 ? h("span", { class: "warn" }, "max 64") : null, h("span", { style: "flex:1" }),
|
||||||
|
h("button", { onClick: () => { tick.n++; }, title: "re-read the graph" }, "↻")]),
|
||||||
|
h("div", { class: "hint" }, "Only the nodes that depend on the swept widgets run once per cell; models and the rest run once."),
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
|
app.registerExtension({
|
||||||
|
name: "BFSNodes.XYSweep",
|
||||||
|
async nodeCreated(node) {
|
||||||
|
if (node.comfyClass !== "BFSXYSweep") return;
|
||||||
|
styles();
|
||||||
|
const widget = node.widgets?.find(w => w.name === "sweep");
|
||||||
|
if (widget) {
|
||||||
|
widget.type = "hidden"; widget.hidden = true; widget.computeSize = () => [0, -4];
|
||||||
|
widget.options = { ...(widget.options || {}), hidden: true, serialize: true };
|
||||||
|
widget.serializeValue = () => widget.value;
|
||||||
|
}
|
||||||
|
const host = document.createElement("div");
|
||||||
|
host.style.cssText = "position:relative;width:100%;height:100%;min-width:340px;min-height:380px";
|
||||||
|
node.addDOMWidget("bfs_xy_sweep", "div", host, { serialize: false, hideOnZoom: false, hideInPanel: true });
|
||||||
|
wheelToCanvas(host);
|
||||||
|
const handle = {};
|
||||||
|
createApp({ setup: () => Panel({ node, get: () => widget?.value ?? "{}",
|
||||||
|
set: v => { if (widget) widget.value = v; node.graph?.setDirtyCanvas(true); }, expose: o => Object.assign(handle, o) }) }).mount(host);
|
||||||
|
const onConn = node.onConnectionsChange;
|
||||||
|
node.onConnectionsChange = function () { const r = onConn?.apply(this, arguments); handle.refresh?.(); return r; };
|
||||||
|
const onConfigure = node.onConfigure;
|
||||||
|
node.onConfigure = function () { const r = onConfigure?.apply(this, arguments); setTimeout(() => handle.reload?.(), 0); return r; };
|
||||||
|
node.size = [Math.max(node.size[0], 380), Math.max(node.size[1], 520)];
|
||||||
|
},
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user