Video Segment Picker: keep painted regions, share objects across colors, shift to erase

- A brush region no longer disappears after switching color and back: added objects are kept in the frame data, and
  strokes not applied yet are applied to their color before the color or frame changes or a run starts.
- Objects added on a frame (brush, points, text) are shared by every color; each color selects its own. Removing one
  removes it for all colors.
- Shift while painting does the opposite of the tool: the brush erases from the mark, the eraser paints back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Alisson Anjos
2026-11-13 21:27:18 +00:00
co-authored by Claude Opus 5.5
parent 8862f44812
commit ffde58f24c
7 changed files with 105 additions and 29 deletions
+9
View File
@@ -2,6 +2,15 @@
All notable changes to **ComfyUI-BFSNodes** are documented in this file.
## [1.121.1] - 2026-11-13
### Fixed
- **Video Segment Picker**: a region painted with the brush no longer disappears when you switch to another color and back. Strokes not applied yet are applied to their color before the color or frame changes or a run starts, and the tool and brush size stay selected.
### Changed
- **Video Segment Picker**: objects added on a frame (brush, points, text) are shared by every color there; each color selects its own. Removing an added object removes it for every color.
- **Segment Picker / Video Segment Picker**: hold **shift** while painting to do the opposite of the tool: the brush erases from the mark, the eraser paints back. Erasing and painting over the same spot keeps whatever was painted last.
## [1.121.0] - 2026-11-12
### Added
+9 -1
View File
@@ -268,7 +268,8 @@ segmented again on load.
Hand edits (1.118.0): **🖌 Brush** paints extra area into the mark (e.g. hair SAM left
out), **➖ Remove** cuts out what SAM segments at the points you place, and **🧽 Eraser**
paints what to cut out. Each becomes a chip; unselect it to switch it off. Brush and
paints what to cut out; **shift** while painting does the opposite of the tool (the brush
erases, the eraser paints back). Each becomes a chip; unselect it to switch it off. Brush and
eraser strokes belong to the image they were painted on and are dropped when the image
changes. Fixed: clicking inside the node no longer shrinks its canvas while the side
panel is open.
@@ -290,6 +291,13 @@ propagates by itself (**auto**). The **Whole video** scrubber shows every frame
its masks, the lost stretches in red; **+ add this frame** puts any frame in the strip
to fix it. A status line with a progress bar and a log shows what runs.
Hand fixes on a picked frame (1.121.1): **🖌 Brush** paints into the active color's
mark and **shift** while painting erases from it (e.g. a stray piece of mask over a
hand). Strokes not applied yet are applied when you change color or frame, or press
Apply fix, so they are never lost. What any color adds on a frame (brush, points, text)
is shared by every color there: each color selects from the same objects, and ✕ on an
added object removes it for all of them.
Outputs: `masks`, `marked` (tinted), `overlay`, `report`, and `hybrid`: the guide
with each subject drawn in its color, by `mark_style`:
- `normal map` — its MoGe normal map in gray detail, tinted (connect `normals`:
+6 -4
View File
@@ -68,7 +68,7 @@ def parse_state(text: str | None) -> dict:
if any(p["label"] for p in pts):
added.append({"id": str(e["id"]), "points": pts})
elif e.get("strokes"):
strokes = [{"r": float(st["r"]), "pts": [[float(x), float(y)] for x, y in st["pts"]]}
strokes = [{"r": float(st["r"]), "pts": [[float(x), float(y)] for x, y in st["pts"]], **({"erase": True} if st.get("erase") else {})}
for st in e["strokes"] if isinstance(st, dict) and st.get("pts")]
if strokes:
added.append({"id": str(e["id"]), "image": str(e.get("image") or ""), "strokes": strokes})
@@ -118,16 +118,18 @@ def compose(masks: list[torch.Tensor], h: int, w: int) -> tuple[torch.Tensor, to
def paint(strokes: list[dict], h: int, w: int) -> torch.Tensor:
"""Brush strokes (relative points, radius relative to the width) as a bool mask [h, w]."""
"""Brush strokes (relative points, radius relative to the width) as a bool mask [h, w]; an "erase" stroke clears what
the strokes before it painted (shift while painting in the panel)."""
img = Image.new("L", (w, h), 0)
draw = ImageDraw.Draw(img)
for st in strokes:
r = max(1., st["r"] * w)
pts = [(x * w, y * h) for x, y in st["pts"]]
fill = 0 if st.get("erase") else 255
if len(pts) > 1:
draw.line(pts, fill=255, width=round(2 * r), joint="curve")
draw.line(pts, fill=fill, width=round(2 * r), joint="curve")
for x, y in pts:
draw.ellipse([x - r, y - r, x + r, y + r], fill=255)
draw.ellipse([x - r, y - r, x + r, y + r], fill=fill)
return torch.from_numpy(np.array(img) > 127)
+18 -3
View File
@@ -689,6 +689,21 @@ def parse_frames(text):
return out
def shared_state(frame_state, key):
"""One subject's selection on a frame, with the objects every subject added there: a region painted or found while
another color was active can be selected by this one too. Its own entries win an id clash (older states)."""
own = dict(frame_state.get(key) or {})
added = list(own.get("added") or [])
ids = {str(e.get("id")) for e in added if isinstance(e, dict)}
for k, v in frame_state.items():
for e in (v.get("added") or []) if k != key and isinstance(v, dict) else []:
if isinstance(e, dict) and str(e.get("id")) not in ids:
ids.add(str(e.get("id")))
added.append(e)
own["added"] = added
return own
def find_connected_picture_count(prompt, unique_id):
"""Inspects the prompt workflow graph to find downstream nodes consuming this node's output,
and returns the maximum index of connected picture inputs (e.g. 2 if only picture_1 and picture_2 are linked)."""
@@ -936,7 +951,7 @@ class BFSVideoSegmentPicker:
sid, sess = sp.session_for((sam3_model, sam3_clip), frame, spec)
chosen_all = set()
for sj in subjects:
state = sp.parse_state(json.dumps(states.get(str(i), {}).get(str(sj), {})))
state = sp.parse_state(json.dumps(shared_state(states.get(str(i), {}), str(sj))))
objects = sp.all_objects(sess, state)
by_id = {o["id"]: o for o in objects}
chosen = [by_id[k] for k in state["selected"] if k in by_id]
@@ -945,7 +960,7 @@ class BFSVideoSegmentPicker:
cut = torch.stack(cut).any(0) if cut else torch.zeros(h, w, dtype=torch.bool)
mask, _ = sp.compose([o["mask"] & ~cut for o in chosen if not o.get("remove")], h, w)
own[sj][i] = mask.reshape(h, w).float()
lock_state = sp.parse_state(json.dumps(states.get(str(i), {}).get("lock", {})))
lock_state = sp.parse_state(json.dumps(shared_state(states.get(str(i), {}), "lock")))
lock_objects = sp.all_objects(sess, lock_state)
lock_by_id = {o["id"]: o for o in lock_objects}
lock_chosen = [lock_by_id[k] for k in lock_state["selected"] if k in lock_by_id]
@@ -959,7 +974,7 @@ class BFSVideoSegmentPicker:
st(f"frame {i}: lock overlaps the selected subject; pick the object by text or add negative points", None, log=True)
lock_mask = torch.zeros_like(lock_mask)
own_locked[i] = lock_mask
objects = sp.all_objects(sess, sp.parse_state(json.dumps({"added": [a for st in states.get(str(i), {}).values() for a in st.get("added", [])]})))
objects = sp.all_objects(sess, sp.parse_state(json.dumps({"added": shared_state(states.get(str(i), {}), "1")["added"]})))
panels.append(sp.panel_data(sid, sess, objects))
overlays.append(sp.overlay(frame, objects, chosen_all))
cuts = detect_cuts(image) if n > 1 else []
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "bfsnodes"
description = "ComfyUI nodes for video workflows: SAM 3 segmentation and Molmo2 tracking; MiniMax H3 character swapping and sampling; shot planning and looping; LTX video controls and identity transfer; model loaders, LoRA tools, and image/video utilities."
version = "1.121.0"
version = "1.121.1"
license = {file = "LICENSE"}
dependencies = [
"torch",
+37 -15
View File
@@ -3,6 +3,7 @@
* see it, click to select it. Objects the automatic pass missed are added by text (a SAM 3 prompt), by points
* (keep / exclude), segmented at once by /bfs/segpick/add, or painted with the brush (to grow the mark by hand).
* ➖ Remove (points) and 🧽 Eraser (brush) make the same kinds of objects, cut out of the mark instead of added to it.
* Shift while painting does the opposite of the mode: with the brush it erases the mark, with the eraser it paints back.
*
* All state lives in the hidden `selection` STRING widget (JSON: selected ids and the added entries), so a workflow
* saves, reloads and shares the selection. The objects themselves come from the node's last run (onExecuted).
@@ -109,11 +110,11 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
const loaded = new Map(); // object id -> loadMask result (not reactive: canvases and big arrays)
const ready = ref(0); // bumps when masks finish loading, to redraw
const hover = ref(null);
const mode = ref("select"); // "select" | "points" | "brush" | "remove" (points) | "eraser" (brush)
const mode = ref(options.mode || "select"); // "select" | "points" | "brush" | "remove" (points) | "eraser" (brush)
const points = ref([]); // the point entry being built
const strokes = ref([]); // the brush entry being built: [{r, pts: [[x, y], ...]}], relative to the image
const brush = ref(.02); // brush radius, relative to the image width
let painting = false, cursor = null;
const brush = ref(options.brush || .02); // brush radius, relative to the image width
let painting = false, cursor = null, shiftHeld = false;
const text = ref("");
const busy = ref(""); const error = ref("");
const stageEl = ref(null); const canvasEl = ref(null);
@@ -133,6 +134,7 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
function setMode(m) {
if (m !== mode.value) { points.value = []; strokes.value = []; } // unfinished points / strokes mean something else in another mode
mode.value = m; hover.value = null;
options.onMode?.(m);
}
async function loadAll() {
@@ -174,8 +176,9 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
}
g.globalAlpha = 1;
if (!brushing()) return;
g.strokeStyle = g.fillStyle = cutting() ? "rgba(20,20,20,.6)" : "rgba(255,0,255,.55)"; g.lineCap = g.lineJoin = "round";
g.lineCap = g.lineJoin = "round";
for (const st of strokes.value) {
g.strokeStyle = g.fillStyle = cutting() !== !!st.erase ? "rgba(20,20,20,.6)" : "rgba(255,0,255,.55)";
const [x0, y0] = st.pts[0];
g.beginPath(); g.arc(x0 * W, y0 * H, st.r * W, 0, 2 * Math.PI); g.fill();
if (st.pts.length < 2) continue;
@@ -184,7 +187,7 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
g.stroke();
}
if (cursor) {
g.lineWidth = Math.max(1, W / 500); g.strokeStyle = "#fff";
g.lineWidth = Math.max(1, W / 500); g.strokeStyle = cutting() !== shiftHeld ? "#f43f5e" : "#fff";
g.beginPath(); g.arc(cursor.x * W, cursor.y * H, brush.value * W, 0, 2 * Math.PI); g.stroke();
}
}
@@ -220,7 +223,9 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
if (brushing()) {
canvasEl.value.setPointerCapture(e.pointerId);
painting = true;
strokes.value = [...strokes.value, { r: +brush.value.toFixed(4), pts: [[+p.x.toFixed(4), +p.y.toFixed(4)]] }];
const st = { r: +brush.value.toFixed(4), pts: [[+p.x.toFixed(4), +p.y.toFixed(4)]] };
if (e.shiftKey) st.erase = true; // shift: the opposite of the mode (erase with the brush, paint back with the eraser)
strokes.value = [...strokes.value, st];
return;
}
const o = objectAt(p);
@@ -230,7 +235,7 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
}
function onMove(e) {
if (data.value && brushing()) {
cursor = at(e);
cursor = at(e); shiftHeld = e.shiftKey;
const pts = strokes.value.at(-1)?.pts, last = pts?.at(-1);
// a point every ~quarter radius keeps the stroke smooth without bloating the saved JSON
if (painting && Math.hypot(cursor.x - last[0], cursor.y - last[1]) > brush.value / 4) pts.push([+cursor.x.toFixed(4), +cursor.y.toFixed(4)]);
@@ -249,12 +254,13 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
body: JSON.stringify({ session: data.value.session, entry }) });
const j = await r.json();
if (!r.ok || j.error) throw new Error(j.error || r.statusText);
if (!j.objects.length) { error.value = entry.text ? `Nothing found for “${entry.text}”: try other words or a lower threshold.` : "The points gave an empty mask."; return; }
if (!j.objects.length) { error.value = entry.text ? `Nothing found for “${entry.text}”: try other words or a lower threshold.` : entry.strokes ? "The painted area is empty." : "The points gave an empty mask."; return; }
state.added = [...state.added.filter(a => a.id !== entry.id), entry];
state.next = Math.max(state.next, Number(entry.id.match(/(\d+)$/)?.[1] || 0) + 1);
data.value = { ...data.value, objects: [...data.value.objects.filter(o => o.entry !== entry.id), ...j.objects] };
state.selected = [...state.selected, ...j.objects.map(o => o.id).filter(id => !isSel(id))]; // what you add is what you want
save();
options.onObjects?.(data.value.objects);
await loadAll();
} catch (err) {
error.value = String(err.message || err);
@@ -266,10 +272,23 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
await add(cutting() ? { id: entryId("r"), remove: true, points: points.value } : { id: entryId("p"), points: points.value });
if (!error.value) setMode("select");
};
// The strokes become up to two entries, in painting order: what they add (cut strokes painted as holes in it) and what
// they cut (painting strokes after a cut painted back), so erasing and painting over the same spot ends as painted last.
const commitStrokes = async () => {
const all = strokes.value;
if (!all.length || !data.value) return;
const cutOf = st => cutting() !== !!st.erase;
const part = cutPart => {
const k = all.findIndex(st => cutOf(st) === cutPart);
return k < 0 ? [] : all.slice(k).map(st => (cutOf(st) === cutPart ? { r: st.r, pts: st.pts } : { r: st.r, pts: st.pts, erase: true }));
};
const keep = part(false), cut = part(true), image = data.value.image_key;
strokes.value = [];
if (keep.length) await add({ id: entryId("b"), image, strokes: keep });
if (cut.length) await add({ id: entryId("e"), remove: true, image, strokes: cut });
};
const addBrush = async () => {
if (!strokes.value.length) return;
const stroke = { image: data.value.image_key, strokes: strokes.value };
await add(cutting() ? { id: entryId("e"), remove: true, ...stroke } : { id: entryId("b"), ...stroke });
await commitStrokes();
if (!error.value) setMode("select");
};
function removeEntry(entryId) {
@@ -279,6 +298,8 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
if (data.value) data.value = { ...data.value, objects: data.value.objects.filter(o => o.entry !== entryId) };
gone.forEach(i => loaded.delete(i));
save(); ready.value++;
options.onRemoveEntry?.(entryId);
if (data.value) options.onObjects?.(data.value.objects);
}
async function runNode() {
@@ -307,6 +328,7 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
nextTick(() => { relayout(); loadAll(); });
},
reload: () => { Object.assign(state, parseState(io.get())); ready.value++; },
commit: commitStrokes, // paint strokes not added yet become entries (the video picker calls it before it remounts)
});
let ro;
@@ -352,9 +374,9 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
] : null,
brushing() ? [
h("input", { type: "range", min: .003, max: .1, step: .001, value: brush.value, title: "Brush size",
onInput: e => { brush.value = +e.target.value; } }),
onInput: e => { brush.value = +e.target.value; options.onBrush?.(brush.value); } }),
h("button", { class: "pri", disabled: !strokes.value.length || !!busy.value, onClick: addBrush },
`${cutting() ? "✓ Erase painted area" : "✓ Add painted area"} (${strokes.value.length})`),
`${cutting() ? "✓ Erase painted area" : "✓ Apply painted area"} (${strokes.value.length})`),
h("button", { disabled: !strokes.value.length, onClick: () => { strokes.value = strokes.value.slice(0, -1); } }, "Undo"),
h("button", { disabled: !strokes.value.length, onClick: () => { strokes.value = []; } }, "Clear"),
] : null,
@@ -370,8 +392,8 @@ export function Panel(io, options = {}) { // exported: BFS Video Segment Picke
hovered ? h("div", { class: "tip" }, `${hovered.id} · ${hovered.label}${hovered.score != null ? ` · ${hovered.score}` : ""}`) : null,
mode.value === "points" ? h("div", { class: "tip" }, "click = keep · right-click or shift+click = exclude · click a point to remove it") : null,
mode.value === "remove" ? h("div", { class: "tip" }, "click on what to cut out of the mark · right-click or shift+click = keep it in · then ✓ Remove this") : null,
mode.value === "brush" ? h("div", { class: "tip" }, "paint what the mark should also cover · then ✓ Add painted area") : null,
mode.value === "eraser" ? h("div", { class: "tip" }, "paint what to cut out of the mark · then ✓ Erase painted area") : null,
mode.value === "brush" ? h("div", { class: "tip" }, "paint what the mark should also cover · shift = erase · then ✓ Apply painted area") : null,
mode.value === "eraser" ? h("div", { class: "tip" }, "paint what to cut out of the mark · shift = paint back · then ✓ Erase painted area") : null,
] : h("div", { class: "empty" }, ["Connect an image and press ▶ Segment (or run the workflow).", h("br"),
"Then click objects to select them; add what was missed by text or 📍 points."])),
busy.value ? h("div", { class: "hint" }, busy.value) : null,
+25 -5
View File
@@ -2,6 +2,7 @@
* BFS Video Segment Picker: a strip with every frame of the batch on top; click a frame to show it in the image
* Segment Picker's canvas (the same Panel, one selection per frame). The strip marks each frame: ✓ picked by hand,
* ↔ tracked from a picked frame, ⚠ lost or nothing. State: the hidden `selection` widget, {"frames": {"i": selection}}.
* What a color adds on a frame (brush, points, text) is shared by every color there: each one selects from the same objects.
*/
import { app } from "../../scripts/app.js";
import { api } from "../../scripts/api.js";
@@ -157,8 +158,11 @@ app.registerExtension({
// the image picker's Panel for the active frame, remounted when the frame changes
let panelApp = null, handle = {};
const ui = { mode: "select", brush: .02 }; // the tool and brush size survive a color or frame change
const panelHost = document.createElement("div");
panelHost.className = "host";
// paint strokes not applied yet are applied (to the color they were painted with) before the panel changes
const switchPanel = async change => { try { await handle.commit?.(); } catch { /* shown in the panel */ } change(); mountPanel(); };
const mountPanel = () => {
panelApp?.unmount(); panelHost.innerHTML = ""; handle = {};
const i = run.data?.indices?.[active.value] ?? active.value; // the real frame index of the strip entry
@@ -176,8 +180,22 @@ app.registerExtension({
if (JSON.stringify(frames.v[String(i)]?.[sj]?.selected || []) !== before) autoPropagate();
},
expose: o => Object.assign(handle, o),
}, { showEraser: false, showRemove: false, autoPointOnClick: sj === "lock", entryPrefix: sj === "lock" ? "lock_" : "",
maskColor: sj === "lock" ? [251, 191, 36] : null }) });
}, { showEraser: false, showRemove: false, autoPointOnClick: sj === "lock",
// ids stay unique on the frame, since every color sees the objects the others added
entryPrefix: sj === "lock" ? "lock_" : sj === "1" ? "" : `s${sj}_`,
maskColor: sj === "lock" ? [251, 191, 36] : null,
mode: sj === "lock" ? "select" : ui.mode, brush: ui.brush,
onMode: m => { if (sj !== "lock") ui.mode = m; }, onBrush: b => { ui.brush = b; },
// added objects live in the run's frame data too, so the next mount (another color, back here) still has them
onObjects: objs => { const fd = run.data?.frames?.[active.value]; if (fd) run.data.frames[active.value] = { ...fd, objects: objs }; },
onRemoveEntry: id => { // an added object goes for every color: no color keeps selecting a gone object
for (const v of Object.values(frames.v[String(i)] || {})) {
if (!v || typeof v !== "object") continue;
if (v.added) v.added = v.added.filter(a => a.id !== id);
if (v.selected) v.selected = v.selected.filter(s => s !== id && !s.startsWith(`${id}.`));
}
save();
} }) });
panelApp.mount(el);
const d = run.data?.frames?.[active.value];
if (d) setTimeout(() => handle.executed?.(d), 0);
@@ -206,11 +224,13 @@ app.registerExtension({
wrapQueue();
const segmentOnly = async () => { // a picking run: the new frame is segmented, nothing is tracked
busy.value = "segmenting…";
try { await handle.commit?.(); } catch { /* shown in the panel */ }
try { const p = await app.graphToPrompt(); await api.queuePrompt(0, p, { partialExecutionTargets: [String(node.id)] }); }
catch (e) { busy.value = String(e.message || e); }
};
const propagate = async () => {
busy.value = "propagating…";
try { await handle.commit?.(); } catch { /* shown in the panel */ }
try {
const p = await app.graphToPrompt();
propagating.add(String(node.id));
@@ -264,7 +284,7 @@ app.registerExtension({
h("div", { class: "subj" }, [...SUBJECTS.map(([k, col, name]) => h("span", {
class: ["t", subject.value === k ? "on" : ""], style: subject.value === k ? { color: col } : null,
title: `Subject ${k} (${name}) = Picture ${k}: clicks select this subject`,
onClick: () => { if (subject.value !== k) { subject.value = k; mountPanel(); } } },
onClick: () => { if (subject.value !== k) switchPanel(() => { subject.value = k; }); } },
[
h("i", { style: { background: col } }),
h("span", { style: "color:#fff" }, `${k} · ${name}`),
@@ -276,7 +296,7 @@ app.registerExtension({
])), h("span", {
class: ["t", "lock", subject.value === "lock" ? "on" : ""],
title: "Everything outside the selected subject is already protected. Use this padlock only for an object inside that subject's mask; click it to segment with SAM 3.",
onClick: () => { if (subject.value !== "lock") { subject.value = "lock"; mountPanel(); } },
onClick: () => { if (subject.value !== "lock") switchPanel(() => { subject.value = "lock"; }); },
}, [
h("span", null, "🔒"),
hasPicks("lock") ? h("span", {
@@ -320,7 +340,7 @@ app.registerExtension({
const [cls, mark] = status(i);
return h("div", { class: ["fr", i === active.value ? "on" : ""], key: i, title: run.data.report?.[i]?.source || "",
style: { backgroundImage: `url("${fileUrl(d.image)}")` },
onClick: () => { if (active.value !== i) { active.value = i; focusFrame = run.data.indices?.[i] ?? i; save(); mountPanel(); } } },
onClick: () => { if (active.value !== i) switchPanel(() => { active.value = i; focusFrame = run.data.indices?.[i] ?? i; save(); }); } },
[h("span", { class: "b" }, `f${run.data.indices?.[i] ?? i}`), mark ? h("span", { class: ["s", cls] }, mark) : null,
h("span", { class: "dots" }, pickedSubjects(i).map(k => k === "lock"
? h("span", { title: "Protected object picked", style: "color:#fbbf24;background:#181818;border-radius:3px;padding:0 2px" }, "🔒")