Compare commits

...
Author SHA1 Message Date
ausboss cecffcc592 Camera Move: keep moving through keys, steady the depth
The camera stopped at every key: each stretch between two keys eased in
and out on its own. Each value now follows a monotone cubic through the
keys, so it keeps its speed through a key, never runs past one, and only
eases at the first and last key.

The moved picture shook. The depth model guesses the lens again for every
frame, the depth-scale smoothing kept the frame-to-frame wobble instead of
removing it, and still things got a slightly different depth each frame.
Every frame now uses the clip's median lens, each frame's median depth
follows the smoothed line, and the depth of spots that hold still is
averaged over nearby frames. With the camera held at one angle the guide
now moves 0.02 px a frame between frames, against 0.28 before and 0.04 in
the source clip.
2026-11-09 09:41:40 -06:00
ausboss 2edd8c0859 Camera Move: move the camera of your own clip
A new node, Camera Move. Press Show my clip and it shows the clip in 3D on
the node: drag to turn the camera, scroll or pinch to pull back or push in,
and click the time bar to set keys. It turns the clip's depth (core MoGe
Inference) into a guide clip where what the old camera never saw is pink,
for MiniMax H3 with the CrossView-Warp LoRA to paint in.

The face also works on a subgraph box: once the camera input is brought up
to the box, the box shows the shot, its settings card and the LoRA rows.

Adds the MiniMax H3 Camera Move example (Load Video, one box, Save Video),
the help page, the Chinese names, tests for the camera maths, and the
panel and wheel guards for the new face.
2026-11-07 17:45:19 -06:00
ausboss d6f7ddc5a3 Turn Fill small holes off by default on Mask by Touch (#307)
It quietly filled holes I wanted to keep, and it's folded under More so
you don't see it's on. New nodes now start with it off. Saved workflows
keep whatever they had. The Mask by Touch example is saved with it off.
2026-11-07 17:11:08 -06:00
ausboss 8f4e6f20fd H3 Prompt Writer: only a real voice makes a swapped person speak (#305) 2026-11-07 16:58:11 -06:00
ausboss 8bc2d183c9 Mask by Touch: show a changed picture without unplugging it (#304) 2026-11-07 16:07:39 -06:00
ausboss 421849cae0 Save Image and Save Video: call the Steps switch Preview steps (#303) 2026-11-07 11:37:47 -06:00
ausboss d7415f7863 Release 2.21.2 (#302) 2026-11-07 09:52:10 -06:00
ausboss 6e48ca10d8 H3 Prompt Writer: plain swap sentences, the new person from the first frame, a looks line written from clip (#300)
A sentence like "Replace the man in the red shirt with @1" now names only the person who goes. Every swap with a picture asks for the replacement from the first frame. A text model on clip writes the @1 looks like line for hair and face when none is typed.
2026-11-07 09:49:14 -06:00
ausboss 0b9259f86b Fix Workflow Note Tile growing in classic canvas (#298)
Suppress ComfyUI's core image preview on Workflow Note panels so stale previews cannot enlarge compact tiles.
2026-11-07 02:37:50 -06:00
ausboss 0386e9c416 Keep node IDs unique across frontend smoke probes (#297) 2026-11-06 20:11:28 -06:00
ausboss 95a259cc69 Keep a new Workflow Note Tile compact in the classic canvas (#294) 2026-11-06 19:10:07 -06:00
43 changed files with 5831 additions and 48 deletions
+41
View File
@@ -4,6 +4,47 @@ All notable changes to ComfyUI-AusBoss are documented here.
## Unreleased
- **New node: Camera Move 🆎.** Moves the camera of your own clip. Drag the
picture on the node to turn the camera, pull back or push in, and set keys
on a time bar. What the old camera never saw comes out pink, ready for
MiniMax H3 with the CrossView-Warp LoRA to paint in. The new MiniMax H3
Camera Move example puts it all in one box between Load Video and Save
Video.
- **Mask by Touch 🆎 no longer fills holes unless you ask.** Fill small
holes, under More, now starts off, so a hole you pick stays a hole. Saved
workflows keep their setting.
- **H3 Prompt Writer: a swap no longer makes the new person talk to noise.** With
**Keep the clip's sound** on and no words typed, the writer told H3 the new
person speaks as in the video whenever the clip's sound was loud, so a train
or a beat made their mouth move. It now does that only when it hears a real
voice: half a second or more of clear speech.
- **Mask by Touch shows a changed picture without unplugging it.** When the
picture coming in changes and the run stops because the picks were made on
the old one, the new picture now shows on the node, ready to pick from. A
new reload button beside Undo shows it any time, without running the
workflow.
- **Save Image and Save Video: the Steps switch is now called Preview steps.** It
shows the picture or the clip's first frame while it is being made, and "Steps"
read like a sampler's step count. It works the same, and saved workflows keep
their setting.
## 2.21.2 - 2026-11-07
- **H3 Prompt Writer reads a plain swap sentence.** "Replace the man in the red
shirt with @1", "Swap the woman for @1" and similar phrasings now name only the
person who goes, instead of putting the whole sentence into the prompt.
- **A swapped-in person is there from the first frame.** Swaps with a picture
always ask for the replacement throughout, with no transition or morph.
- **A text model on H3 Prompt Writer's `clip` input writes the `@1 looks like`
line in a swap.** It describes the person's hair and face from the picture
when no line is typed. Without a model on `clip`, nothing changes.
- **Workflow Note tiles start compact in the classic canvas again.** A leftover
image preview filed under the node's id made the classic canvas grow a new
tile to hold the picture; notes now stand that preview down.
- **Colored link buttons and pop-out notes keep their tile layout when shared.**
Callout Tile and Workflow Note Tile save as distinct node types, so an older
install can report what is missing. Tiles have larger editors and visible
+7 -1
View File
@@ -167,6 +167,12 @@ Upload a sound file, or pick one from the Source list, and the node shows it as
It returns the `audio` you kept and its length in `seconds`. Only that part is decoded. [Help page](js/docs/AUSBOSS_NODES_LoadAudio.md).
### Camera Move
Moves the camera of your own clip. Press **Show my clip** and the node shows your clip in 3D. Drag the picture to turn the camera, scroll or pinch to pull back or push in, and click the time bar to set keys. What the old camera never saw comes out pink, and MiniMax H3 with Cseti's CrossView-Warp LoRA paints it in. Stay within about 45° left or right, 30° up and 20° down. Your keys add to any move your clip's camera already makes.
It needs depth from the core **MoGe Inference** node and returns the `guide` clip for H3 and a `holes` mask. [Help page](js/docs/AUSBOSS_NODES_CameraMove.md). [Open the MiniMax H3 Camera Move workflow →](example_workflows/MiniMax%20H3%20Camera%20Move%20%28AusBoss%29.json)
### Video Crop + Rotate + Pad → Frame
Picks one frame from a video, then gives you the same rotate, crop and pad controls as the image node, plus an optional megapixel resize. Scrub to the frame with the playhead on the node; the editor adds frame-by-frame stepping and playback. It returns the transformed `image`, a `mask` of the added area, a `stitcher`, the chosen `original` frame, and the output `width` and `height`. For a whole clip, use the Clip version below.
@@ -199,7 +205,7 @@ It returns the new frames and their fps. It doesn't touch audio, so carry the so
Saves a frame batch or a core `VIDEO` as MP4 (H.264 or H.265), WebM (VP9 or AV1), ProRes MOV, lossless FFV1 MKV, GIF or WebP. Which encoders you get depends on your FFmpeg and PyAV build, and H.264 and H.265 come in CPU and NVENC versions. The videos carry color tags, and optionally audio and the workflow.
**Ping-pong** plays forward and then back without repeating the turnaround frames. A wired `VIDEO` brings its own frame rate; otherwise wire fps from Load Video or Frame Interpolate. The player on the node shows the saved file, and **STEPS** shows the clip's first frame while it is being made, with a **STOP** button. Drop a saved video back on the canvas to load the workflow inside it, for the formats that can hold one. The [help page](js/docs/AUSBOSS_NODES_SaveVideo.md) has the format and audio details.
**Ping-pong** plays forward and then back without repeating the turnaround frames. A wired `VIDEO` brings its own frame rate; otherwise wire fps from Load Video or Frame Interpolate. The player on the node shows the saved file, and **Preview steps** shows the clip's first frame while it is being made, with a **STOP** button. Drop a saved video back on the canvas to load the workflow inside it, for the formats that can hold one. The [help page](js/docs/AUSBOSS_NODES_SaveVideo.md) has the format and audio details.
## Mask and inpaint nodes
+1
View File
@@ -20,6 +20,7 @@ NODE_MODULES = [
"node_align_image",
"node_batch_utils",
"node_callout",
"node_camera_move",
"node_color_match",
"node_compare",
"node_frame_interpolate",
+3 -3
View File
@@ -1,9 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" width="105" height="20" role="img" aria-label="release: v2.21.1">
<title>release: v2.21.1</title>
<svg xmlns="http://www.w3.org/2000/svg" width="105" height="20" role="img" aria-label="release: v2.21.2">
<title>release: v2.21.2</title>
<path fill="#555" d="M0 0h48v20H0z"/>
<path fill="#00b4aa" d="M48 0h57v20H48z"/>
<g fill="#fff" text-anchor="middle" font-family="Verdana,Geneva,DejaVu Sans,sans-serif" font-size="11">
<text x="24" y="14">release</text>
<text x="76.5" y="14">v2.21.1</text>
<text x="76.5" y="14">v2.21.2</text>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 425 B

After

Width:  |  Height:  |  Size: 425 B

@@ -439,7 +439,7 @@
0,
0,
0,
true,
false,
"stop the run",
false,
false,
@@ -451,7 +451,7 @@
"grow": 0,
"soften": 0,
"smooth": 0,
"fill_holes": true,
"fill_holes": false,
"if_nothing": "stop the run",
"free_vram": false,
"invert": false,
Binary file not shown.

After

Width:  |  Height:  |  Size: 118 KiB

File diff suppressed because it is too large Load Diff
+92
View File
@@ -0,0 +1,92 @@
// The settings brought up to a Camera Move box, as one compact card above
// the shot: what people change per clip on top (size, seed), the model files
// folded under "Models", since they are set once. A LoRA Loader's rows keep
// their own panel, with its strength bars; the camera keys are the shot.
//
// A box row shares its options with the setting inside the box, and Nodes 2.0
// reads "hidden" from there, so a row the card takes over is hidden inside the
// box too in that renderer (the classic canvas still shows it). Its value is
// set from the box either way.
import { mountWidgetCard } from "../shared/widget_card.mjs";
import { boxRowInputs } from "../shared/lora_box.mjs";
import { hideWidget } from "../shared/widget_visibility.mjs";
const STASH = "__ausbossVisibilityStash"; // widget_visibility.mjs: a row already hidden
const MODEL_FILE = /\.(safetensors|sft|ckpt|pt|pth|bin|gguf|onnx)$/i;
const SUFFIX = { megapixels: "MP" };
const ROW_TYPES = new Set(["number", "slider", "combo", "toggle", "text"]);
const isLoraLoader = (node) => node?.type === "AUSBOSS_NODES_LoraLoader" || node?.comfyClass === "AUSBOSS_NODES_LoraLoader";
const isModelRow = (widget) => {
if (widget?.type !== "combo") return false;
if (MODEL_FILE.test(String(widget.value ?? ""))) return true;
const values = typeof widget.options?.values === "function" ? [] : widget.options?.values;
return Array.isArray(values) && values.some((value) => MODEL_FILE.test(String(value)));
};
function labelOf(box, widget) {
const input = (box.inputs ?? []).find((slot) => slot?.widget?.name === widget.name);
const text = String(input?.label || widget.label || widget.name).replace(/_/g, " ");
return text.charAt(0).toUpperCase() + text.slice(1);
}
// The rows a box's card shows: every setting brought up to it except `skip`
// (the camera keys), a LoRA Loader's rows and anything with its own panel.
export function boxCardRows(box, skip) {
const loraRows = new Set(boxRowInputs(box, isLoraLoader).map((entry) => entry.name));
const top = [], models = [];
for (const widget of box.widgets ?? []) {
if (!widget || widget === skip || widget.element || !ROW_TYPES.has(String(widget.type)) || loraRows.has(widget.name)) continue;
if (widget[STASH] && !widget.__ausbossCameraCard) continue;
const row = { widget: widget.name, label: labelOf(box, widget) };
if (isModelRow(widget)) models.push({ ...row, kind: "select", group: "models" });
else top.push(SUFFIX[innerName(box, widget)] ? { ...row, suffix: SUFFIX[innerName(box, widget)] } : row);
}
return models.length ? [...top, { group: "models", label: `Models · ${models.length} files` }, ...models] : top;
}
// The name of the setting inside the box that a box row stands for.
function innerName(box, widget) {
const input = (box.subgraph?.inputs ?? []).find((slot) => slot?.name === widget.name);
for (const id of input?.linkIds ?? []) {
const link = box.subgraph.getLink?.(id) ?? box.subgraph.links?.get?.(id) ?? box.subgraph.links?.[id];
const node = link && (box.subgraph.getNodeById?.(link.target_id));
const name = node?.inputs?.[link.target_slot]?.widget?.name ?? node?.inputs?.[link.target_slot]?.name;
if (name) return name;
}
return widget.name;
}
// The frontend now and then makes a box's rows afresh (a new object under
// the same name), so the card hides its rows again whenever it finds one
// showing. Cheap: a handful of names.
export function rehideBoxCard(box) {
let changed = false;
for (const name of box.__ausbossCameraCardRows ?? []) {
const widget = (box.widgets ?? []).find((w) => w?.name === name);
if (!widget || widget[STASH]) continue;
hideWidget(widget);
widget.__ausbossCameraCard = true;
changed = true;
}
return changed;
}
// Mount the card once per box. Settings brought up later show as plain rows.
// `tidy` puts the box's panels back in order and says whether it moved one.
export function mountBoxCard(box, skip, tidy) {
if (box.__ausbossCameraCard) { rehideBoxCard(box); return false; }
const rows = boxCardRows(box, skip);
if (!rows.length) return false;
box.__ausbossCameraCardRows = rows.filter((row) => row.widget).map((row) => row.widget);
rehideBoxCard(box);
mountWidgetCard(box, { rows, minWidth: 300, first: true });
box.__ausbossCameraCard = true;
const draw = box.onDrawForeground;
box.onDrawForeground = function (...args) {
if (rehideBoxCard(this) | Boolean(tidy?.(this))) this.setDirtyCanvas?.(true, true);
return draw?.apply(this, args);
};
return true;
}
+183
View File
@@ -0,0 +1,183 @@
// Camera Move: the camera maths the face shares with the node.
//
// The same keys, pose and easing as nodes/_camera_move_helpers.py, so the
// picture on the node is the shot the warp makes. No DOM and no app import:
// tests/camera_move.test.mjs runs it under node:test.
//
// Axes are OpenCV's: x right, y down, z into the picture. A key is
// { t, az, el, back, px, py }: t from 0 to 1 over the clip, turn and rise in
// degrees, back and the pan in distances to the pivot.
export const LIM = { az: [-90, 90], el: [-30, 60], back: [-0.6, 2], px: [-1, 1], py: [-1, 1] };
export const K0 = { az: 0, el: 0, back: 0, px: 0, py: 0 };
export const FIELDS = Object.keys(K0);
// Two keys this close on the clip are the same key.
export const SAME_KEY = 0.015;
export const MAX_KEYS = 64;
export const LORA_RANGE = { az: 45, up: 30, down: -20 };
export const DEFAULT_KEYS = [{ t: 0, ...K0 }, { t: 1, ...K0, az: 30, el: 5 }];
export const clamp = (v, a, b) => Math.max(a, Math.min(b, v));
const num = (v, fallback = 0) => {
const n = Number(v);
return Number.isFinite(n) ? n : fallback;
};
export function cleanKey(raw) {
const key = { t: clamp(num(raw?.t), 0, 1) };
for (const name of FIELDS) key[name] = clamp(num(raw?.[name]), ...LIM[name]);
return key;
}
// The camera a node holds: { keys, pivot }. Unreadable text gives the default move.
export function parseCamera(text) {
let data = null;
if (text && typeof text === "object") data = text;
else if (typeof text === "string" && text.trim()) {
try { data = JSON.parse(text); } catch { data = null; }
}
const fallback = !data || !Array.isArray(data.keys) || !data.keys.length;
if (fallback) data = { keys: DEFAULT_KEYS, pivot: null };
const keys = [];
for (const raw of data.keys.slice(0, MAX_KEYS)) {
const key = cleanKey(raw);
const same = keys.find((k) => Math.abs(k.t - key.t) < SAME_KEY);
if (same) Object.assign(same, key);
else keys.push(key);
}
keys.sort((a, b) => a.t - b.t);
const p = data.pivot;
const pivot = p && typeof p === "object" && "u" in p && "v" in p
? { u: clamp(num(p.u, 0.5), 0, 1), v: clamp(num(p.v, 0.5), 0, 1), t: clamp(num(p.t), 0, 1) }
: null;
return { keys, pivot, fallback };
}
const round = (v, places) => Math.round(v * 10 ** places) / 10 ** places;
export function cameraText({ keys, pivot }) {
const out = {
v: 1,
keys: keys.map((k) => ({ t: round(k.t, 4), az: round(k.az, 2), el: round(k.el, 2), back: round(k.back, 3), px: round(k.px, 3), py: round(k.py, 3) })),
pivot: pivot ? { u: round(pivot.u, 4), v: round(pivot.v, 4), t: round(pivot.t, 4) } : null,
};
return JSON.stringify(out);
}
// How fast one field moves as it passes each key: zero at the first and last
// key (a gentle start and end) and where the field turns back or holds (it
// never runs past a key); elsewhere it keeps its speed through the key, so a
// move made of several keys does not slow down at each one.
function keySlopes(ts, ys) {
const n = ys.length, m = new Array(n).fill(0);
for (let k = 1; k < n - 1; k++) {
const h0 = ts[k] - ts[k - 1], h1 = ts[k + 1] - ts[k];
if (h0 <= 0 || h1 <= 0) continue;
const d0 = (ys[k] - ys[k - 1]) / h0, d1 = (ys[k + 1] - ys[k]) / h1;
if (d0 * d1 > 0) m[k] = 3 * (h0 + h1) / ((2 * h1 + h0) / d0 + (h1 + 2 * h0) / d1);
}
return m;
}
// Where the camera is at moment t: through every key, gently at the ends.
// Same maths as camera_at in nodes/_camera_move_helpers.py.
export function camAt(keys, t) {
if (!keys.length) return { ...K0 };
const n = keys.length;
const pick = (k) => { const o = {}; for (const f of FIELDS) o[f] = k[f] || 0; return o; };
if (n === 1 || t <= keys[0].t) return pick(keys[0]);
if (t >= keys[n - 1].t) return pick(keys[n - 1]);
const ts = keys.map((k) => k.t);
for (let i = 0; i < n - 1; i++) {
const a = keys[i], b = keys[i + 1];
if (t >= a.t && t <= b.t) {
const h = Math.max(1e-6, b.t - a.t), s = (t - a.t) / h;
const h00 = 2 * s ** 3 - 3 * s ** 2 + 1, h10 = s ** 3 - 2 * s ** 2 + s, h01 = -2 * s ** 3 + 3 * s ** 2, h11 = s ** 3 - s ** 2;
const o = {};
for (const f of FIELDS) {
const m = keySlopes(ts, keys.map((k) => k[f] || 0));
o[f] = h00 * (a[f] || 0) + h10 * h * m[i] + h01 * (b[f] || 0) + h11 * h * m[i + 1];
}
return o;
}
}
return pick(keys[n - 1]);
}
export const keyAt = (keys, t) => keys.findIndex((k) => Math.abs(k.t - t) < SAME_KEY);
// Moving the camera at the time bar sets the key there, or adds one.
export function setKeyAt(keys, t, cam) {
const c = {};
for (const f of FIELDS) c[f] = clamp(num(cam[f]), ...LIM[f]);
const i = keyAt(keys, t);
if (i >= 0) { Object.assign(keys[i], c); return keys; }
if (keys.length >= MAX_KEYS) return keys;
keys.push({ t: clamp(t, 0, 1), ...c });
keys.sort((a, b) => a.t - b.t);
return keys;
}
export const pastLoraRange = (c) => Math.abs(c.az) > LORA_RANGE.az || c.el > LORA_RANGE.up || c.el < LORA_RANGE.down;
// R = Ry(-az) Rx(-el), row-major.
export function rot(az, el) {
const th = -az * Math.PI / 180, ph = -el * Math.PI / 180;
const ct = Math.cos(th), st = Math.sin(th), cp = Math.cos(ph), sp = Math.sin(ph);
return [ct, st * sp, st * cp, 0, cp, -sp, -st, ct * sp, ct * cp];
}
export const mv = (R, v) => [R[0] * v[0] + R[1] * v[1] + R[2] * v[2], R[3] * v[0] + R[4] * v[1] + R[5] * v[2], R[6] * v[0] + R[7] * v[1] + R[8] * v[2]];
// The moved camera's rotation R and centre C. Pan slides the camera in its own frame.
export function pose(cam, pivot) {
const R = rot(cam.az, cam.el), p = pivot, Rp = mv(R, p), d = Math.hypot(p[0], p[1], p[2]);
const b = mv(R, [(cam.px || 0) * d, (cam.py || 0) * d, -(cam.back || 0) * d]);
return { R, C: [p[0] - Rp[0] + b[0], p[1] - Rp[1] + b[1], p[2] - Rp[2] + b[2]] };
}
// A world point in the moved camera: R^T (P - C).
export function toCam(R, C, P) {
const d = [P[0] - C[0], P[1] - C[1], P[2] - C[2]];
return [R[0] * d[0] + R[3] * d[1] + R[6] * d[2], R[1] * d[0] + R[4] * d[1] + R[7] * d[2], R[2] * d[0] + R[5] * d[1] + R[8] * d[2]];
}
// The node's depth picture: red is the high byte, green the low one, on a log
// scale from zmin to zmax; 0 is no depth (sky), which sits at `far`.
export function unpackDepth(rgba, count, zmin, zmax, far) {
const z = new Float32Array(count), valid = new Uint8Array(count);
const lmin = Math.log(zmin), lr = Math.log(zmax) - lmin;
for (let i = 0; i < count; i++) {
const q = rgba[i * 4] * 256 + rgba[i * 4 + 1];
if (q === 0) z[i] = far;
else { z[i] = Math.exp(lmin + (q - 1) / 65534 * lr); valid[i] = 1; }
}
return { z, valid };
}
// The 3D spot under a pixel of a frame: the middle of the nearest depth found
// within three pixels, as the node does it. Null over sky.
export function pivotFromPixel(frame, u, v) {
const { W, H, z, valid, fx, fy, cx, cy } = frame;
const x = clamp(Math.floor(u * W), 0, W - 1), y = clamp(Math.floor(v * H), 0, H - 1);
for (let r = 0; r <= 3; r++) {
const found = [];
for (let yy = Math.max(0, y - r); yy <= Math.min(H - 1, y + r); yy++) {
for (let xx = Math.max(0, x - r); xx <= Math.min(W - 1, x + r); xx++) if (valid[yy * W + xx]) found.push(z[yy * W + xx]);
}
if (found.length) {
found.sort((a, b) => a - b);
const depth = found.length % 2 ? found[found.length >> 1] : (found[found.length / 2 - 1] + found[found.length / 2]) / 2;
return [(x + 0.5 - cx) / fx * depth, (y + 0.5 - cy) / fy * depth, depth];
}
}
return null;
}
// Words for where the camera is.
export function fmtAz(v) { const r = Math.round(v); return r === 0 ? "0°" : `${Math.abs(r)}° ${r > 0 ? "right" : "left"}`; }
export function fmtEl(v) { const r = Math.round(v); return r === 0 ? "0°" : `${Math.abs(r)}° ${r > 0 ? "up" : "down"}`; }
export function describe(c) {
const part = (v, neg, pos) => (Math.abs(v) < 0.005 ? "" : `${Math.abs(v).toFixed(2)} ${v > 0 ? pos : neg}`);
const pan = [part(c.px, "left", "right"), part(c.py, "up", "down")].filter(Boolean).join(", ");
return `Turn ${fmtAz(c.az)} · Rise ${fmtEl(c.el)} · ${c.back < -0.004 ? "In" : "Back"} ${Math.abs(c.back).toFixed(2)}×${pan ? ` · Pan ${pan}` : ""}`;
}
+900
View File
@@ -0,0 +1,900 @@
// Camera Move's face: the shot on the node, steered by grabbing it.
//
// Drag the picture to turn the camera round, pinch or scroll to pull back,
// two fingers or Shift to pan, tap a spot to circle round it. Top right, as
// in Blender: an axis ball, a magnifier, a hand, and a camera that puts you
// back on your own shot. Under it the time bar with its keys.
//
// No app import: index.js hands in where the camera is kept, how to run the
// node and how to reach a file, so the same face sits on the node or on the
// box that holds it.
//
// The stage is the one part of the node that keeps the mouse wheel (it zooms
// the camera, as in Blender); its root carries no "ausboss-" class, so the
// pack's wheel hand-off leaves it alone. The controls under it do carry one,
// and the wheel there zooms the graph as everywhere else.
import { makeScrubInput } from "../shared/scrub_input.mjs";
import {
K0, LIM, SAME_KEY, camAt, cameraText, clamp, describe, keyAt, mv, parseCamera, pastLoraRange,
pivotFromPixel, pose, rot, setKeyAt, toCam, unpackDepth,
} from "./camera_math.mjs";
export const MIN_WIDTH = 380; // the play row on one line, with "243 / 243" frames
export const MIN_STAGE = 184;
export const CONTROLS_HEIGHT = 168;
const FPS = 24; // H3 renders at 24 frames a second; only playback speed uses it
const MAG = 0xFFFF00FF;
const CSS = `
.acm-face{position:relative;display:flex;flex-direction:column;width:100%;height:100%;box-sizing:border-box;
--acm-teal:#4fb3a9;--acm-teal-dim:#2b605b;--acm-line:#34413f;--acm-fg:#e3e9e8;--acm-muted:#92a2a1;--acm-amber:#e2a94c;
--acm-stage:#0c0f10;--acm-panel:#1f2629;--acm-head:#283236;color:var(--acm-fg);font:12.5px system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
.acm-face *{box-sizing:border-box}
.acm-stage{position:relative;flex:1 1 auto;min-height:${MIN_STAGE}px;background:var(--acm-stage);border-radius:6px;overflow:hidden;
touch-action:none;user-select:none;-webkit-user-select:none}
.acm-stage>canvas.acm-pic{position:absolute;display:block;cursor:grab}
.acm-stage>canvas.acm-pic:active{cursor:grabbing}
.acm-pivot{position:absolute;width:18px;height:18px;margin:-9px 0 0 -9px;border:2px solid var(--acm-teal);border-radius:50%;pointer-events:none;box-shadow:0 0 0 1px rgba(0,0,0,.6)}
.acm-pivot::after{content:"";position:absolute;left:50%;top:50%;width:3px;height:3px;margin:-1.5px;background:var(--acm-teal);border-radius:50%}
.acm-nav{position:absolute;top:6px;right:6px;display:flex;flex-direction:column;align-items:center;gap:5px}
.acm-nav canvas{width:64px;height:64px;cursor:grab;touch-action:none;margin-bottom:1px}
.acm-nav button,.acm-helpbtn{width:30px;height:30px;border-radius:50%;border:1px solid rgba(146,162,161,.4);background:rgba(12,15,16,.62);
color:var(--acm-fg);padding:0;display:grid;place-items:center;touch-action:none;box-shadow:0 1px 6px rgba(0,0,0,.45);cursor:pointer}
.acm-nav button svg{width:18px;height:18px;fill:none;stroke:currentColor;stroke-width:1.8;stroke-linecap:round;stroke-linejoin:round}
.acm-nav button:hover,.acm-helpbtn:hover{border-color:var(--acm-teal)}
.acm-nav button.on,.acm-helpbtn[aria-expanded="true"]{background:var(--acm-teal-dim);border-color:var(--acm-teal)}
.acm-nav .pan{cursor:move}.acm-nav .zoom{cursor:ns-resize}
.acm-helpbtn{position:absolute;top:6px;left:6px;width:24px;height:24px;font:600 12px system-ui,sans-serif}
.acm-inset{position:absolute;left:6px;bottom:6px;width:36%;border:1px solid var(--acm-line);border-radius:5px;overflow:hidden;background:var(--acm-stage);
box-shadow:0 2px 10px rgba(0,0,0,.5);pointer-events:none}
.acm-inset canvas{display:block;width:100%;height:auto}
.acm-tag{position:absolute;top:4px;left:5px;font-size:10px;color:var(--acm-muted);background:rgba(12,15,16,.7);padding:1px 5px;border-radius:4px;pointer-events:none}
.acm-empty{position:absolute;inset:0;display:flex;flex-direction:column;align-items:center;justify-content:center;gap:8px;padding:16px;text-align:center;color:var(--acm-muted);line-height:1.4}
.acm-empty b{color:var(--acm-fg);font-weight:600;font-size:13px}
.ausboss-camera-move-controls{flex:0 0 auto;display:flex;flex-direction:column;gap:6px;padding:7px 0 2px}
.acm-readout{display:flex;justify-content:space-between;gap:8px;font-size:11.5px;color:var(--acm-fg);font-variant-numeric:tabular-nums;min-height:15px}
.acm-readout .acm-where{flex:1 1 auto;min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
.acm-readout .acm-pink{color:var(--acm-muted);flex:0 0 auto}
.acm-warn{color:var(--acm-amber);font-size:11.5px;line-height:15px;height:15px;white-space:nowrap;overflow:hidden;text-overflow:ellipsis}
.acm-back{display:flex;align-items:center;gap:8px}
.acm-back>span{width:40px;color:var(--acm-muted)}
.acm-track{position:relative;height:34px;background:var(--acm-stage);border:1px solid var(--acm-line);border-radius:6px;touch-action:none;cursor:pointer}
.acm-track .acm-fill{position:absolute;top:15px;height:2px;left:8px;right:8px;background:var(--acm-line)}
.acm-track .acm-inner{position:absolute;left:8px;right:8px;top:0;bottom:0}
.acm-track .acm-key{position:absolute;top:10px;width:12px;height:12px;margin-left:-6px;transform:rotate(45deg);background:var(--acm-teal);border:1px solid #0c0f10;cursor:ew-resize}
.acm-track .acm-key.on{background:#bfe9e4}
.acm-track .acm-head{position:absolute;top:2px;bottom:2px;width:2px;margin-left:-1px;background:var(--acm-fg);pointer-events:none}
.acm-track .acm-head::before{content:"";position:absolute;top:-2px;left:-4px;width:10px;height:6px;background:var(--acm-fg);border-radius:2px}
.acm-row{display:flex;align-items:center;gap:6px}
.acm-row>.acm-keynav,.acm-row>.acm-btn{flex:none;white-space:nowrap}
.acm-btn{border:1px solid var(--acm-line);background:var(--acm-head);color:var(--acm-fg);font:inherit;font-size:12.5px;padding:6px 10px;border-radius:6px;cursor:pointer;line-height:1.1}
.acm-btn:disabled{opacity:.4;cursor:default}
.acm-btn.primary{background:var(--acm-teal-dim);border-color:var(--acm-teal-dim)}
.acm-keynav{display:inline-flex;border:1px solid var(--acm-line);border-radius:6px;background:var(--acm-head);overflow:hidden}
.acm-keynav button{width:32px;height:29px;border:0;background:transparent;color:var(--acm-fg);padding:0;display:grid;place-items:center;cursor:pointer}
.acm-keynav button+button{border-left:1px solid var(--acm-line)}
.acm-keynav button:disabled{color:var(--acm-muted);opacity:.35;cursor:default}
.acm-keynav button:not(:disabled):hover{background:var(--acm-panel)}
.acm-keynav svg{width:14px;height:14px}
.acm-keynav .dia svg{width:16px;height:16px;fill:none;stroke:currentColor;stroke-width:1.6}
.acm-keynav .dia.on svg{fill:#bfe9e4;stroke:#bfe9e4}
.acm-keynav .arr svg{fill:currentColor}
.acm-keynav.transport .play{width:36px;background:var(--acm-teal-dim);font-size:12px}
.acm-keynav.transport .play:hover{background:var(--acm-teal)}
.acm-keynav.transport .loop{color:var(--acm-muted)}
.acm-keynav.transport .loop svg{width:16px;height:16px;fill:none;stroke:currentColor;stroke-width:1.5;stroke-linecap:round;stroke-linejoin:round}
.acm-keynav.transport .loop[aria-pressed="true"]{color:#bfe9e4;background:var(--acm-teal-dim)}
.acm-time{margin-left:auto;font-size:11.5px;color:var(--acm-muted);font-variant-numeric:tabular-nums;white-space:nowrap}
.acm-help{position:absolute;top:36px;left:6px;right:6px;z-index:5;background:#172023;border:1px solid var(--acm-line);border-radius:8px;
padding:10px 12px 12px;box-shadow:0 8px 28px rgba(0,0,0,.55);max-height:calc(100% - 44px);overflow:auto}
.acm-help-h{display:flex;justify-content:space-between;align-items:center;font-weight:600;font-size:13px;margin-bottom:6px}
.acm-help-x{border:0;background:transparent;color:var(--acm-muted);font-size:20px;line-height:1;cursor:pointer;padding:0 2px}
.acm-help dl{margin:0;display:grid;grid-template-columns:auto 1fr;gap:7px 12px;font-size:12.5px;line-height:1.4}
.acm-help dt{color:var(--acm-teal);font-weight:600}.acm-help dd{margin:0}
.acm-nav,.acm-keynav,.acm-btn,.acm-helpbtn{-webkit-touch-callout:none;-webkit-user-select:none;user-select:none}
.acm-tip{position:fixed;z-index:10000;max-width:250px;background:#0a0d0e;color:#e3e9e8;border:1px solid #34413f;border-radius:6px;
padding:5px 8px;font:12px/1.35 system-ui,sans-serif;pointer-events:none;box-shadow:0 4px 14px rgba(0,0,0,.5)}
.acm-face [hidden],.acm-tip[hidden]{display:none!important}
`;
function ensureCss() {
if (document.getElementById("ausboss-camera-move-css")) return;
const style = document.createElement("style");
style.id = "ausboss-camera-move-css";
style.textContent = CSS;
document.head.append(style);
}
const el = (tag, cls, attrs = {}) => {
const e = document.createElement(tag);
if (cls) e.className = cls;
for (const [k, v] of Object.entries(attrs)) {
if (k === "text") e.textContent = v;
else if (k === "html") e.innerHTML = v;
else e.setAttribute(k, v);
}
return e;
};
const NAV_ICONS = {
zoom: '<svg viewBox="0 0 24 24"><circle cx="10.5" cy="10.5" r="6"/><path d="M15 15l5 5"/></svg>',
pan: '<svg viewBox="0 0 24 24"><path d="M7.5 12.5V6.3a1.25 1.25 0 0 1 2.5 0v5M10 11.3V4.8a1.25 1.25 0 0 1 2.5 0v6.5M12.5 11.3V5.6a1.25 1.25 0 0 1 2.5 0v6M15 11.8V8a1.25 1.25 0 0 1 2.5 0v5.5c0 3.9-2.6 6.8-6.3 6.8h-.6c-2.1 0-3.6-.9-4.8-2.6l-2.4-3.5a1.35 1.35 0 0 1 2.1-1.7l2 2.3"/></svg>',
shot: '<svg viewBox="0 0 24 24"><rect x="2.8" y="7.3" width="12.6" height="9.4" rx="1.8"/><path d="M15.4 10.6 21 7.8v8.4l-5.6-2.8"/></svg>',
};
const NAV_TIPS = { zoom: "Zoom: drag up to push in, down to pull back", pan: "Pan: drag to slide the camera", shot: "Back to your own shot" };
const GZ = 64;
// Blender's names for the scene's directions: X to the right, Y into the picture, Z up.
const AXES = [
{ n: "X", v: [1, 0, 0], col: "#e05561", pos: { az: 90, el: 0 }, neg: { az: -90, el: 0 } },
{ n: "Y", v: [0, 0, 1], col: "#86b84a", pos: null, neg: { az: 0, el: 0 } }, // nothing to see from behind the scene
{ n: "Z", v: [0, -1, 0], col: "#4b8fdf", pos: { az: 0, el: LIM.el[1] }, neg: { az: 0, el: LIM.el[0] } },
];
const SUB = [[-0.25, -0.25], [0.25, -0.25], [-0.25, 0.25], [0.25, 0.25]];
const VIEW = { az: -58, el: 28 }; // the corner map looks at the scene from the left
// ---------- tooltips: hover on a computer, press and hold a button on a phone ----------
let tipEl = null, tipTimer = 0, tipFor = null, tipXY = [0, 0], tipHeld = false;
function tip() {
if (!tipEl || !tipEl.isConnected) {
tipEl = el("div", "acm-tip", { role: "tooltip" });
tipEl.hidden = true;
document.body.append(tipEl);
}
return tipEl;
}
function placeTip(target) {
const t = target.dataset.tip, box = tip();
if (!t || !target.isConnected) return hideTip();
box.textContent = t;
box.hidden = false;
const r = target.getBoundingClientRect(), tw = box.offsetWidth, th = box.offsetHeight, atPtr = target.dataset.tipAt === "pointer";
if (target.closest(".acm-nav")) { // the column top right: the tip opens to its left and covers no other button
box.style.left = `${Math.max(8, r.left - tw - 8)}px`;
box.style.top = `${r.top + r.height / 2 - th / 2}px`;
return;
}
const ax = atPtr ? tipXY[0] : r.left + r.width / 2;
const below = atPtr ? tipXY[1] + 22 : r.bottom + 8, above = atPtr ? tipXY[1] - th - 12 : r.top - th - 8;
box.style.left = `${clamp(ax - tw / 2, 8, innerWidth - tw - 8)}px`;
box.style.top = `${below + th > innerHeight - 8 ? above : below}px`;
}
function hideTip() { clearTimeout(tipTimer); if (tipEl) tipEl.hidden = true; tipFor = null; }
const armTip = (target, ms) => { clearTimeout(tipTimer); tipFor = target; tipTimer = setTimeout(() => placeTip(target), ms); };
function wireTips(root) {
const targetOf = (e) => e.target?.closest?.("[data-tip]") ?? null;
root.addEventListener("pointermove", (e) => {
if (e.pointerType !== "mouse") return;
tipXY = [e.clientX, e.clientY];
if (e.buttons) { if (tipFor) hideTip(); return; }
const t = targetOf(e);
if (!t) { if (tipFor) hideTip(); return; }
if (t !== tipFor) { hideTip(); armTip(t, t.dataset.tipAt === "pointer" ? 900 : 450); }
else if (t.dataset.tipAt === "pointer") { hideTip(); armTip(t, 900); } // over a picture the tip waits until the mouse rests
}, true);
root.addEventListener("pointerleave", hideTip);
root.addEventListener("pointerdown", (e) => {
hideTip();
if (e.pointerType === "mouse") return;
const t = targetOf(e);
if (!t || t.dataset.tipAt === "pointer") return;
const x0 = e.clientX, y0 = e.clientY;
armTip(t, 500);
const done = () => { removeEventListener("pointermove", move, true); removeEventListener("pointerup", up, true); removeEventListener("pointercancel", up, true); };
const move = (m) => { if (Math.hypot(m.clientX - x0, m.clientY - y0) > 8) { clearTimeout(tipTimer); done(); } };
const up = () => {
if (tipEl && !tipEl.hidden) { tipHeld = true; setTimeout(() => { tipHeld = false; }, 400); setTimeout(hideTip, 1600); } else clearTimeout(tipTimer);
done();
};
addEventListener("pointermove", move, true); addEventListener("pointerup", up, true); addEventListener("pointercancel", up, true);
}, true);
// A hold that showed a tip is not a click.
root.addEventListener("click", (e) => { if (tipHeld) { tipHeld = false; e.preventDefault(); e.stopPropagation(); } }, true);
root.addEventListener("contextmenu", (e) => { if (e.target.closest?.(".acm-nav, .acm-keynav, .acm-btn")) e.preventDefault(); });
}
// ---------- one frame of the clip, as points ----------
async function loadFrame(entry, info, view, blobOf) {
const W = info.w, H = info.h;
const picture = new Image();
picture.decoding = "async";
const pictureReady = new Promise((ok, fail) => { picture.onload = ok; picture.onerror = () => fail(new Error("picture")); });
picture.src = view(entry.color);
const depthBlob = await blobOf(entry.depth);
const depthBitmap = await createImageBitmap(depthBlob, { colorSpaceConversion: "none", premultiplyAlpha: "none" });
await pictureReady;
const canvas = document.createElement("canvas");
canvas.width = W; canvas.height = H;
const g = canvas.getContext("2d", { willReadFrequently: true });
g.drawImage(picture, 0, 0, W, H);
const rgba = new Uint32Array(g.getImageData(0, 0, W, H).data.buffer.slice(0));
g.clearRect(0, 0, W, H);
g.drawImage(depthBitmap, 0, 0, W, H);
const { z, valid } = unpackDepth(g.getImageData(0, 0, W, H).data, W * H, info.zmin, info.zmax, info.far);
// the corner map leaves out what is very far away
const zs = [];
for (let i = 0; i < W * H; i += 5) if (valid[i]) zs.push(z[i]);
zs.sort((a, b) => a - b);
const zcap = Math.min(zs[Math.floor(zs.length * 0.97)] || info.far, (zs[zs.length >> 1] || 1) * 3.5);
return { ...entry, W, H, z, valid, rgba, zcap, picture };
}
// ---------- the face ----------
// opts: read() -> camera text, write(text), run() -> Promise<boolean>,
// view(fileEntry) -> url, onSize() when the floor changes.
export function mountCameraFace(opts) {
ensureCss();
const root = el("div", "acm-face");
const stage = el("div", "acm-stage");
const pic = el("canvas", "acm-pic", { "data-tip": "Drag to turn · Shift-drag to pan · wheel to zoom · click a spot to circle round it", "data-tip-at": "pointer" });
const picCtx = pic.getContext("2d");
const pivotMark = el("div", "acm-pivot");
const nav = el("div", "acm-nav");
const gizmo = el("canvas", "", { "data-tip": "Drag to turn. Click an axis to look from that side.", role: "img", "aria-label": "Axis ball: drag to turn the camera, tap an axis to look from that side" });
gizmo.width = GZ * 2; gizmo.height = GZ * 2;
const gctx = gizmo.getContext("2d");
nav.append(gizmo);
const inset = el("div", "acm-inset");
const out = el("canvas");
out.width = 360; out.height = 225;
const outCtx = out.getContext("2d");
const outImg = outCtx.createImageData(out.width, out.height);
const outZ = new Float32Array(out.width * out.height);
inset.append(out, el("span", "acm-tag", { text: "where the camera is" }));
const helpBtn = el("button", "acm-helpbtn", { text: "?", "aria-label": "How to steer", "aria-expanded": "false", "data-tip": "How to steer" });
const empty = el("div", "acm-empty");
stage.append(pic, pivotMark, inset, nav, helpBtn, empty);
const controls = el("div", "ausboss-camera-move-controls");
const readout = el("div", "acm-readout");
const where = el("span", "acm-where");
const pink = el("span", "acm-pink", { "data-tip": "How much of the frame the camera never saw. The LoRA paints it in." });
readout.append(where, pink);
// Always one line tall, empty or not, so a warning never pushes the controls down.
const warn = el("div", "acm-warn");
const backRow = el("div", "acm-back");
const back = makeScrubInput({
value: 0, min: LIM.back[0], max: LIM.back[1], step: 0.05, fineStep: 0.01, decimals: 2, unit: "×", unitWidth: 14, width: 132,
title: "How far the camera pulls back from what it circles. Below 0 pushes in.",
onChange: (v) => { stopPlay(); setCam({ ...camAt(keys, p), back: v }); },
});
backRow.append(el("span", "", { text: "Back" }), back.root);
const track = el("div", "acm-track", { "data-tip": "Click to put a key here. Drag to move the time bar, or drag a key to change its time.", "data-tip-at": "pointer" });
const inner = el("div", "acm-inner");
const head = el("div", "acm-head");
inner.append(head);
track.append(el("div", "acm-fill"), inner);
const row = el("div", "acm-row");
const transport = el("span", "acm-keynav transport", { role: "group", "aria-label": "Playback" });
const toStartBtn = el("button", "arr", { "aria-label": "Back to the start", "data-tip": "Back to the start", html: '<svg viewBox="0 0 14 14"><path d="M2.5 2.5h1.6v9H2.5zM11.5 2.5v9L5 7z"/></svg>' });
const play = el("button", "play", { text: "▶", "aria-label": "Play", "data-tip": "Play" });
const loop = el("button", "loop", { "aria-label": "Play on a loop", "aria-pressed": "false", "data-tip": "Play on a loop", html: '<svg viewBox="0 0 16 16"><path d="M3 9.5V7a2.5 2.5 0 0 1 2.5-2.5H12M10 2.5l2 2-2 2M13 6.5V9a2.5 2.5 0 0 1-2.5 2.5H4M6 13.5l-2-2 2-2"/></svg>' });
transport.append(toStartBtn, play, loop);
const keynav = el("span", "acm-keynav", { role: "group", "aria-label": "Keys" });
const prevKey = el("button", "arr", { "aria-label": "Previous key", "data-tip": "Previous key", html: '<svg viewBox="0 0 14 14"><path d="M10 2.5v9L3.5 7z"/></svg>' });
const toggleKey = el("button", "dia", { "aria-label": "Add a key here", "data-tip": "Add a key here", html: '<svg viewBox="0 0 16 16"><path d="M8 1.8 14.2 8 8 14.2 1.8 8z"/></svg>' });
const nextKey = el("button", "arr", { "aria-label": "Next key", "data-tip": "Next key", html: '<svg viewBox="0 0 14 14"><path d="M4 2.5v9L10.5 7z"/></svg>' });
keynav.append(prevKey, toggleKey, nextKey);
const reset = el("button", "acm-btn", { text: "Start over", "data-tip": "One key again, on your own shot" });
const time = el("span", "acm-time", { "data-tip": "Frame at the time bar" });
row.append(transport, keynav, reset, time);
controls.append(readout, warn, backRow, track, row);
const help = el("div", "acm-help");
help.hidden = true;
root.append(stage, controls, help);
wireTips(root);
// ---------- state ----------
let keys = [], pivotSpec = null, lastText = null, p = 0, playing = false, lastT = 0, dirty = true, alive = true;
let info = null, frames = [], shown = null, pivot = [0, 0, 2], pinkPct = 0, status = "idle", statusNote = "";
let flashText = "", flashUntil = 0, gBalls = [], gHot = false, hovered = false, overFace = false, looping = false, needsWrite = false;
let buf = null; // z, colour and source pixel of the moved view
const fallbackLens = { W: 416, H: 234, fx: 380, fy: 380, cx: 208, cy: 117 };
function readCamera() {
const text = opts.read();
if (text === lastText) return;
lastText = text;
const parsed = parseCamera(text);
keys = parsed.keys;
pivotSpec = parsed.pivot;
resolvePivot();
dirty = true;
}
function writeSoon() { needsWrite = true; dirty = true; }
function flushWrite() {
if (!needsWrite) return;
needsWrite = false;
const text = cameraText({ keys, pivot: pivotSpec });
lastText = text;
if (text !== opts.read()) opts.write(text);
}
function setCam(cam) { setKeyAt(keys, p, cam); writeSoon(); }
function lens() { return shown ?? (info ? { ...fallbackLens, ...info.frames[0], W: info.w, H: info.h } : fallbackLens); }
function resolvePivot() {
if (!info) { pivot = [0, 0, 2]; return; }
if (!pivotSpec) { pivot = info.pivot_auto?.slice() ?? [0, 0, 2]; return; }
if (info.pivot_from && Math.abs(info.pivot_from.u - pivotSpec.u) < 1e-4 && Math.abs(info.pivot_from.v - pivotSpec.v) < 1e-4
&& Math.abs(info.pivot_from.t - pivotSpec.t) < 1e-4) { pivot = info.pivot.slice(); return; }
const f = nearestLoaded(pivotSpec.t);
const found = f && pivotFromPixel(f, pivotSpec.u, pivotSpec.v);
if (found) pivot = found;
}
// ---------- frames ----------
function nearestFrame(t) {
let best = null;
for (const f of frames) if (!best || Math.abs(f.t - t) < Math.abs(best.t - t)) best = f;
return best;
}
function nearestLoaded(t) {
let best = null;
for (const f of frames) if (f.data && (!best || Math.abs(f.t - t) < Math.abs(best.t - t))) best = f;
return best?.data ?? null;
}
async function loadAll(payload) {
const order = [...frames].sort((a, b) => Math.abs(a.t - p) - Math.abs(b.t - p));
let failed = 0, next = 0;
// Four at a time, the frames nearest the time bar first.
const worker = async () => {
while (next < order.length) {
const f = order[next++];
if (info !== payload) return;
try { f.data = await loadFrame(f.entry, payload, opts.view, opts.blob); } catch { failed++; }
if (info !== payload) return;
if (f.data && status !== "ready") { status = "ready"; resolvePivot(); }
dirty = true;
}
};
await Promise.all([worker(), worker(), worker(), worker()]);
if (info !== payload) return;
if (failed === frames.length) { info = null; frames = []; shown = null; status = "idle"; statusNote = "The last run's pictures are gone. Run it again to see your clip."; dirty = true; }
}
// ---------- the camera's view ----------
function fitPicture() {
const L = lens(), sw = stage.clientWidth, sh = stage.clientHeight;
if (!sw || !sh) return null;
const s = Math.min(sw / L.W, sh / L.H), w = L.W * s, h = L.H * s, x = (sw - w) / 2, y = (sh - h) / 2;
Object.assign(pic.style, { left: `${x}px`, top: `${y}px`, width: `${w}px`, height: `${h}px` });
inset.hidden = !shown || sw < 300 || sh < MIN_STAGE;
return { x, y, w, h };
}
function renderPreview(cam, fit) {
const f = shown;
if (!f) { pic.hidden = true; pivotMark.hidden = true; return; }
pic.hidden = false;
const { W, H, z, rgba, fx, fy, cx, cy } = f;
if (!buf || buf.W !== W || buf.H !== H) {
buf = { W, H, zbuf: new Float32Array(W * H), cbuf: new Uint32Array(W * H), ibuf: new Int32Array(W * H), fill: new Uint8Array(W * H), img: picCtx.createImageData(W, H) };
pic.width = W; pic.height = H;
buf.img = picCtx.createImageData(W, H);
}
const { zbuf, cbuf, ibuf, fill, img } = buf;
const { R, C } = pose(cam, pivot);
const [r0, r1, r2, r3, r4, r5, r6, r7, r8] = R, [Cx, Cy, Cz] = C;
zbuf.fill(Infinity); ibuf.fill(-1);
for (let y = 0, i = 0; y < H; y++) for (let x = 0; x < W; x++, i++) {
const zz = z[i], c = rgba[i];
for (let s = 0; s < 4; s++) {
const X = (x + 0.5 + SUB[s][0] - cx) / fx * zz - Cx, Y = (y + 0.5 + SUB[s][1] - cy) / fy * zz - Cy, Z = zz - Cz;
const zc = r2 * X + r5 * Y + r8 * Z;
if (zc <= 1e-3) continue;
const u = fx * (r0 * X + r3 * Y + r6 * Z) / zc + cx, v = fy * (r1 * X + r4 * Y + r7 * Z) / zc + cy;
if (u < 0 || v < 0 || u >= W || v >= H) continue;
const idx = (v | 0) * W + (u | 0);
if (zc < zbuf[idx]) { zbuf[idx] = zc; cbuf[idx] = c; ibuf[idx] = i; }
}
}
for (let i = 0; i < W * H; i++) fill[i] = zbuf[i] < Infinity ? 1 : 0;
// fill one-pixel cracks: a gap mostly surrounded by picture takes their average (two passes, like the node)
for (let pass = 0; pass < 2; pass++) {
const was = fill.slice(), src = cbuf.slice();
let any = false;
for (let y = 1; y < H - 1; y++) for (let x = 1; x < W - 1; x++) {
const i = y * W + x;
if (was[i]) continue;
let n = 0, r = 0, g = 0, b = 0;
for (let oy = -1; oy <= 1; oy++) for (let ox = -1; ox <= 1; ox++) {
const q = i + oy * W + ox;
if (!was[q]) continue;
const c = src[q]; n++; r += c & 255; g += (c >>> 8) & 255; b += (c >>> 16) & 255;
}
if (n >= 5) { cbuf[i] = 0xFF000000 | ((b / n) << 16) | ((g / n) << 8) | (r / n); fill[i] = 1; any = true; }
}
if (!any) break;
}
const o = new Uint32Array(img.data.buffer);
let holes = 0;
for (let i = 0; i < W * H; i++) { if (fill[i]) o[i] = cbuf[i]; else { o[i] = MAG; holes++; } }
picCtx.putImageData(img, 0, 0);
pinkPct = 100 * holes / (W * H);
const q = toCam(R, C, pivot);
if (fit && q[2] > 1e-3) {
pivotMark.hidden = false;
pivotMark.style.left = `${fit.x + fit.w * (fx * q[0] / q[2] + cx) / W}px`;
pivotMark.style.top = `${fit.y + fit.h * (fy * q[1] / q[2] + cy) / H}px`;
} else pivotMark.hidden = true;
}
// ---------- the corner map: your scene from the side, and the camera in it ----------
function viewer() {
const P = pivot, d = Math.hypot(P[0], P[1], P[2]), T = [P[0] * 0.55, P[1] * 0.55, P[2] * 0.55];
const R = rot(VIEW.az, VIEW.el), b = mv(R, [0, 0, -d * 2.3]);
return { R, C: [T[0] + b[0], T[1] + b[1], T[2] + b[2]], f: 0.95 * out.width };
}
function proj(V, P) {
const q = toCam(V.R, V.C, P);
return q[2] <= 1e-3 ? null : [V.f * q[0] / q[2] + out.width / 2, V.f * q[1] / q[2] + out.height / 2, q[2]];
}
function frustum(V, cam, color, width) {
const L = lens(), { R, C } = pose(cam, pivot), s = Math.hypot(...pivot) * 0.3;
const corners = [[0, 0], [L.W, 0], [L.W, L.H], [0, L.H]].map(([x, y]) => {
const l = mv(R, [(x - L.cx) / L.fx * s, (y - L.cy) / L.fy * s, s]);
return proj(V, [C[0] + l[0], C[1] + l[1], C[2] + l[2]]);
});
const a = proj(V, C);
if (!a || corners.some((c) => !c)) return;
outCtx.strokeStyle = color; outCtx.lineWidth = width; outCtx.beginPath();
for (const c of corners) { outCtx.moveTo(a[0], a[1]); outCtx.lineTo(c[0], c[1]); }
outCtx.moveTo(corners[0][0], corners[0][1]);
for (const c of [...corners.slice(1), corners[0]]) outCtx.lineTo(c[0], c[1]);
outCtx.stroke();
// a small roof on the top edge, so up and down never mix
outCtx.fillStyle = color; outCtx.beginPath(); outCtx.moveTo(corners[0][0], corners[0][1]); outCtx.lineTo(corners[1][0], corners[1][1]);
const up = mv(R, [0, -L.H / 2 / L.fy * s * 1.35, s]), ut = proj(V, [C[0] + up[0], C[1] + up[1], C[2] + up[2]]);
if (ut) { outCtx.lineTo(ut[0], ut[1]); outCtx.closePath(); outCtx.globalAlpha = 0.6; outCtx.fill(); outCtx.globalAlpha = 1; }
}
function renderOutside(cam) {
if (inset.hidden) return;
const w = out.width, h = out.height, V = viewer(), o = new Uint32Array(outImg.data.buffer);
o.fill(0xFF100D0C); outZ.fill(Infinity);
const f = shown;
if (f) {
const { W, H, z, valid, rgba, zcap, fx, fy, cx, cy } = f, [r0, r1, r2, r3, r4, r5, r6, r7, r8] = V.R, [Cx, Cy, Cz] = V.C;
for (let y = 0; y < H; y += 2) for (let x = 0; x < W; x += 2) {
const i = y * W + x, zz = z[i];
if (!valid[i] || zz > zcap) continue;
const dx = (x + 0.5 - cx) / fx * zz - Cx, dy = (y + 0.5 - cy) / fy * zz - Cy, dz = zz - Cz, zc = r2 * dx + r5 * dy + r8 * dz;
if (zc <= 1e-3) continue;
const u = V.f * (r0 * dx + r3 * dy + r6 * dz) / zc + w / 2, v = V.f * (r1 * dx + r4 * dy + r7 * dz) / zc + h / 2;
const sz = clamp(Math.round(V.f * (zz / fx * 2.2) / zc), 1, 4), x0 = (u - sz / 2) | 0, y0 = (v - sz / 2) | 0;
for (let yy = y0; yy < y0 + sz; yy++) {
if (yy < 0 || yy >= h) continue;
for (let xx = x0; xx < x0 + sz; xx++) { if (xx < 0 || xx >= w) continue; const q = yy * w + xx; if (zc < outZ[q]) { outZ[q] = zc; o[q] = rgba[i]; } }
}
}
}
outCtx.putImageData(outImg, 0, 0);
frustum(V, { ...K0 }, "rgba(200,210,210,.55)", 1.5);
const project = (c) => proj(V, pose(c, pivot).C);
outCtx.strokeStyle = "rgba(79,179,169,.9)"; outCtx.lineWidth = 2; outCtx.setLineDash([5, 4]); outCtx.beginPath();
let first = true;
for (let i = 0; i <= 48; i++) {
const q = project(camAt(keys, i / 48));
if (!q) { first = true; continue; }
if (first) outCtx.moveTo(q[0], q[1]); else outCtx.lineTo(q[0], q[1]);
first = false;
}
outCtx.stroke(); outCtx.setLineDash([]);
for (const k of keys) {
const q = project(k);
if (!q) continue;
outCtx.save(); outCtx.translate(q[0], q[1]); outCtx.rotate(Math.PI / 4);
outCtx.fillStyle = Math.abs(k.t - p) < SAME_KEY ? "#bfe9e4" : "#4fb3a9"; outCtx.strokeStyle = "#0c0f10"; outCtx.lineWidth = 1.5;
outCtx.fillRect(-5, -5, 10, 10); outCtx.strokeRect(-5, -5, 10, 10); outCtx.restore();
}
const pv = proj(V, pivot), cc = project(cam);
if (pv && cc) { outCtx.strokeStyle = "rgba(79,179,169,.45)"; outCtx.lineWidth = 1; outCtx.beginPath(); outCtx.moveTo(cc[0], cc[1]); outCtx.lineTo(pv[0], pv[1]); outCtx.stroke(); }
frustum(V, cam, "#4fb3a9", 2.5);
}
// ---------- the axis ball ----------
function drawGizmo(cam) {
const g = gctx, c = GZ / 2, L = 21, R = rot(cam.az, cam.el);
g.setTransform(2, 0, 0, 2, 0, 0); g.clearRect(0, 0, GZ, GZ);
g.fillStyle = gHot ? "rgba(146,162,161,.26)" : "rgba(12,15,16,.42)"; g.beginPath(); g.arc(c, c, c - 1, 0, 7); g.fill();
gBalls = [];
for (const a of AXES) for (const sg of [1, -1]) {
const w = [a.v[0] * sg, a.v[1] * sg, a.v[2] * sg]; // the direction as the camera sees it
const q = [R[0] * w[0] + R[3] * w[1] + R[6] * w[2], R[1] * w[0] + R[4] * w[1] + R[7] * w[2], R[2] * w[0] + R[5] * w[1] + R[8] * w[2]];
gBalls.push({ a, sg, x: c + q[0] * L, y: c + q[1] * L, z: q[2], view: sg > 0 ? a.pos : a.neg });
}
gBalls.sort((m, n) => n.z - m.z); // far balls first, near ones on top
for (const b of gBalls) {
if (b.sg > 0) { g.strokeStyle = b.a.col; g.lineWidth = 2; g.beginPath(); g.moveTo(c, c); g.lineTo(b.x, b.y); g.stroke(); }
g.beginPath(); g.arc(b.x, b.y, b.sg > 0 ? 7 : 6, 0, 7);
if (b.sg > 0) {
g.fillStyle = b.a.col; g.fill();
g.fillStyle = "#0c0f10"; g.font = "bold 9px system-ui, sans-serif"; g.textAlign = "center"; g.textBaseline = "middle"; g.fillText(b.a.n, b.x, b.y + 0.5);
} else { g.globalAlpha = 0.35; g.fillStyle = b.a.col; g.fill(); g.globalAlpha = 1; g.strokeStyle = b.a.col; g.lineWidth = 1.2; g.stroke(); }
}
}
// ---------- gestures ----------
const picRect = () => pic.getBoundingClientRect();
function panBy(cam, dx, dy) { // the spot you circle stays under your finger
const L = lens(), r = picRect(), d = Math.hypot(...pivot), zz = Math.max(0.05 * d, pivot[2] + cam.back * d);
const du = dx * L.W / (r.width || 1), dv = dy * L.H / (r.height || 1);
return { px: (cam.px || 0) - du * zz / (L.fx * d), py: (cam.py || 0) - dv * zz / (L.fy * d) };
}
function dragTo(kind, cam, dx, dy) {
const r = picRect(), w = r.width || stage.clientWidth || 1, h = r.height || stage.clientHeight || 1;
if (kind === "orbit") setCam({ ...cam, az: cam.az - dx * 90 / w, el: cam.el + dy * 90 / w }); // the picture follows your finger
else if (kind === "pan") setCam({ ...cam, ...panBy(cam, dx, dy) });
else if (kind === "zoom") setCam({ ...cam, back: cam.back + dy / h * 2.4 }); // up = in, down = back
}
const kindFor = (e) => (e.shiftKey ? "pan" : (e.ctrlKey || e.metaKey) ? "zoom" : "orbit");
(() => {
const pts = new Map();
let g = null;
const begin = (kind, carry) => {
const v = [...pts.values()], cam = camAt(keys, p);
if (v.length >= 2) {
const [a, b] = v;
g = { kind: "pinch", cam, d: Math.max(8, Math.hypot(a[0] - b[0], a[1] - b[1])), mx: (a[0] + b[0]) / 2, my: (a[1] + b[1]) / 2, moved: true, multi: true };
} else if (v.length === 1) g = { kind, cam, x: v[0][0], y: v[0][1], moved: Boolean(carry), multi: Boolean(carry) };
else g = null;
};
pic.addEventListener("mousedown", (e) => { if (e.button === 1) e.preventDefault(); }); // no auto-scroll on middle click
pic.addEventListener("pointerdown", (e) => {
if (e.button > 1) return;
e.preventDefault(); e.stopPropagation();
pic.setPointerCapture(e.pointerId); pts.set(e.pointerId, [e.clientX, e.clientY]); stopPlay();
begin(kindFor(e));
});
pic.addEventListener("pointermove", (e) => {
if (!pts.has(e.pointerId) || !g) return;
pts.set(e.pointerId, [e.clientX, e.clientY]);
if (g.kind === "pinch") {
const [a, b] = [...pts.values()], d = Math.max(8, Math.hypot(a[0] - b[0], a[1] - b[1]));
const c = { ...g.cam, back: g.cam.back - Math.log(d / g.d) * 0.9 };
Object.assign(c, panBy(g.cam, (a[0] + b[0]) / 2 - g.mx, (a[1] + b[1]) / 2 - g.my));
setCam(c);
return;
}
const dx = e.clientX - g.x, dy = e.clientY - g.y;
if (!g.moved && Math.hypot(dx, dy) < 5) return;
g.moved = true;
dragTo(g.kind, g.cam, dx, dy);
});
const up = (e) => {
if (!pts.has(e.pointerId)) return;
pts.delete(e.pointerId);
if (pts.size === 0) {
if (g && !g.moved && !g.multi && g.kind === "orbit" && e.type === "pointerup" && e.button === 0) pickPivot(e);
g = null;
return;
}
begin("orbit", true); // a finger left: carry on from here
};
pic.addEventListener("pointerup", up);
pic.addEventListener("pointercancel", up);
})();
function pickPivot(e) { // tap: circle round what you tapped
const f = shown, r = picRect();
if (!f || !buf || !r.width) return;
const x = ((e.clientX - r.left) / r.width * f.W) | 0, y = ((e.clientY - r.top) / r.height * f.H) | 0;
if (x < 0 || y < 0 || x >= f.W || y >= f.H) return;
let best = -1;
for (let rr = 0; rr <= 3 && best < 0; rr++) for (let oy = -rr; oy <= rr && best < 0; oy++) for (let ox = -rr; ox <= rr; ox++) {
const xx = x + ox, yy = y + oy;
if (xx < 0 || yy < 0 || xx >= f.W || yy >= f.H) continue;
const i = buf.ibuf[yy * f.W + xx];
if (i >= 0) { best = i; break; }
}
if (best < 0 || !f.valid[best]) return; // sky: nothing to circle
const spec = { u: ((best % f.W) + 0.5) / f.W, v: (Math.floor(best / f.W) + 0.5) / f.H, t: f.t };
const found = pivotFromPixel(f, spec.u, spec.v);
if (!found) return;
pivotSpec = spec; pivot = found;
writeSoon();
}
stage.addEventListener("wheel", (e) => {
e.preventDefault(); e.stopPropagation(); stopPlay();
const unit = e.deltaMode === 1 ? 33 : e.deltaMode === 2 ? 400 : 1, c = camAt(keys, p);
if (e.shiftKey) { const a = (e.deltaY || e.deltaX) * unit; setCam({ ...c, ...panBy(c, 0, -a * 0.4) }); return; } // Shift + wheel pans up and down
setCam({ ...c, back: c.back + e.deltaY * unit * (e.ctrlKey ? 0.01 : 0.0008) }); // wheel, and a trackpad pinch (Ctrl), zoom
}, { passive: false });
stage.addEventListener("gesturestart", (e) => e.preventDefault()); // Safari: never zoom the page instead
stage.addEventListener("pointerdown", (e) => { if (e.button === 1) { e.preventDefault(); e.stopPropagation(); } });
stage.addEventListener("pointerenter", () => { hovered = true; });
stage.addEventListener("pointerleave", () => { hovered = false; });
// Blender's numpad, while the pointer is over the picture.
const onKey = (e) => {
if (!hovered || !alive || e.altKey || e.target?.closest?.("input, textarea, select, [contenteditable]")) return;
const c = camAt(keys, p), w = picRect().width || 300, step = 15, pan = e.ctrlKey || e.metaKey;
const go = { Numpad4: [-1, 0], Numpad6: [1, 0], Numpad8: [0, 1], Numpad2: [0, -1] }[e.code];
if (go) setCam(pan ? { ...c, ...panBy(c, -go[0] * w * 0.1, go[1] * w * 0.1) } : { ...c, az: c.az + go[0] * step, el: c.el + go[1] * step });
else if (e.code === "NumpadAdd" || e.code === "NumpadSubtract") setCam({ ...c, back: c.back + (e.code === "NumpadAdd" ? -0.1 : 0.1) });
else if (e.code === "Numpad1" && !pan) setCam({ ...c, az: 0, el: 0 }); // front
else if (e.code === "Numpad3") setCam({ ...c, az: pan ? -90 : 90, el: 0 }); // right, Ctrl: left
else if (e.code === "Numpad7") setCam({ ...c, az: 0, el: pan ? LIM.el[0] : LIM.el[1] }); // top, Ctrl: bottom
else if (e.code === "Numpad0") setCam({ ...K0 }); // your own shot
else if (e.key === "Escape" && !help.hidden) { openHelp(false); }
else return;
e.preventDefault(); e.stopPropagation(); stopPlay();
};
window.addEventListener("keydown", onKey, true);
// Space plays and pauses, Home goes to the start, while the pointer is over the face.
// Held there, Space would otherwise start a canvas pan.
root.addEventListener("pointerenter", () => { overFace = true; });
root.addEventListener("pointerleave", () => { overFace = false; });
const onPlayKey = (e) => {
if (!overFace || !alive || e.altKey || e.ctrlKey || e.metaKey || e.repeat && e.code === "Space") return;
if (e.target?.closest?.("input, textarea, select, button, [contenteditable]")) return;
if (e.key === "Home") toStart();
else if (e.code === "Space") play.click();
else return;
e.preventDefault(); e.stopPropagation();
};
window.addEventListener("keydown", onPlayKey, true);
// the axis ball: drag turns, a tap on an axis looks from that side
(() => {
let g = null;
gizmo.addEventListener("pointerenter", () => { gHot = true; dirty = true; });
gizmo.addEventListener("pointerleave", () => { if (!g) { gHot = false; dirty = true; } });
gizmo.addEventListener("pointerdown", (e) => {
if (e.button > 1) return;
e.preventDefault(); e.stopPropagation();
gizmo.setPointerCapture(e.pointerId); stopPlay(); gHot = true; dirty = true;
g = { id: e.pointerId, x: e.clientX, y: e.clientY, cam: camAt(keys, p), moved: false };
});
gizmo.addEventListener("pointermove", (e) => {
if (!g || e.pointerId !== g.id) return;
const dx = e.clientX - g.x, dy = e.clientY - g.y;
if (!g.moved && Math.hypot(dx, dy) < 4) return;
g.moved = true;
dragTo("orbit", g.cam, dx * 1.5, dy * 1.5);
});
const end = (e) => {
if (!g || e.pointerId !== g.id) return;
if (!g.moved && e.type === "pointerup" && (e.pointerType === "mouse" || !tipEl || tipEl.hidden)) {
const r = gizmo.getBoundingClientRect(), x = (e.clientX - r.left) * GZ / r.width, y = (e.clientY - r.top) * GZ / r.height;
let hit = null;
for (const b of gBalls) if (Math.hypot(b.x - x, b.y - y) <= 9) hit = b;
if (hit?.view) setCam({ ...camAt(keys, p), ...hit.view });
else if (hit) flash("Nothing behind: the camera never saw the back.");
}
g = null;
if (e.pointerType !== "mouse") gHot = false;
dirty = true;
};
gizmo.addEventListener("pointerup", end);
gizmo.addEventListener("pointercancel", end);
})();
for (const kind of ["zoom", "pan", "shot"]) {
const b = el("button", kind, { html: NAV_ICONS[kind], "data-tip": NAV_TIPS[kind], "aria-label": NAV_TIPS[kind] });
nav.append(b);
if (kind === "shot") { b.onclick = (e) => { e.stopPropagation(); stopPlay(); setCam({ ...K0 }); }; continue; }
let g = null;
b.addEventListener("pointerdown", (e) => {
if (e.button > 1) return;
e.preventDefault(); e.stopPropagation();
b.setPointerCapture(e.pointerId); stopPlay(); b.classList.add("on");
g = { id: e.pointerId, x: e.clientX, y: e.clientY, cam: camAt(keys, p) };
});
b.addEventListener("pointermove", (e) => {
if (!g || e.pointerId !== g.id) return;
const m = kind === "pan" ? 1 : 1.5;
dragTo(kind, g.cam, (e.clientX - g.x) * m, (e.clientY - g.y) * m);
});
const end = (e) => { if (g && e.pointerId === g.id) { g = null; b.classList.remove("on"); } };
b.addEventListener("pointerup", end);
b.addEventListener("pointercancel", end);
}
function flash(text) { flashText = text; flashUntil = performance.now() + 2600; dirty = true; setTimeout(() => { dirty = true; }, 2700); }
// ---------- the ? : the whole how-to, out of the way until asked for ----------
function buildHelp() {
help.innerHTML = "";
const h = el("div", "acm-help-h", { text: "How to steer" });
const x = el("button", "acm-help-x", { text: "×", "aria-label": "Close" });
x.onclick = () => openHelp(false);
h.append(x);
const dl = el("dl");
const rows = [
["Picture", "Drag to turn the camera round. Pinch or scroll to pull back or push in; two fingers or Shift pan. Tap a spot to circle round it."],
["Top right", "Drag the axis ball to turn, or tap an axis to look from that side. Drag the magnifier to zoom and the hand to pan. The camera puts you back on your own shot."],
["Keys", "Click the time bar to put a key there, or drag the time bar and move the camera. ◆ adds or removes a key, ◀ ▶ jump between keys. Drag a key along the bar to change its time."],
["Play", "⏮ goes back to the start, ▶ plays, and the loop button plays it round and round."],
["Mouse", "Like Blender: middle-drag turns, Shift pans, Ctrl zooms, the wheel zooms. Numpad 4 6 8 2 turn, + − zoom, 1 3 7 front, right, top, 0 your own shot. Space plays, Home goes to the start."],
["Pink", "What the camera never saw. The CrossView-Warp LoRA paints it in."],
];
for (const [k, v] of rows) dl.append(el("dt", "", { text: k }), el("dd", "", { text: v }));
help.append(h, dl);
}
function openHelp(open) { if (open) buildHelp(); help.hidden = !open; helpBtn.setAttribute("aria-expanded", String(open)); }
helpBtn.addEventListener("pointerdown", (e) => e.stopPropagation());
helpBtn.onclick = (e) => { e.stopPropagation(); openHelp(help.hidden); };
const closeHelp = (e) => { if (!help.hidden && !help.contains(e.target) && !helpBtn.contains(e.target)) openHelp(false); };
document.addEventListener("pointerdown", closeHelp, true);
// ---------- the time bar ----------
let keyEls = [];
function drawTrack() {
for (const k of keyEls) k.remove();
keyEls = [];
for (const k of keys) {
const d = el("div", `acm-key${Math.abs(k.t - p) < SAME_KEY ? " on" : ""}`);
d.style.left = `${k.t * 100}%`;
inner.insertBefore(d, head);
keyEls.push(d);
}
head.style.left = `${p * 100}%`;
const count = info?.count ?? 0;
time.textContent = count ? `${Math.round(p * (count - 1)) + 1} / ${count}` : `${Math.round(p * 100)}%`;
// the diamond is filled while the time bar sits on a key, hollow when it does not
const on = keyAt(keys, p) >= 0;
toggleKey.classList.toggle("on", on);
const tipText = !on ? "Add a key here" : keys.length > 1 ? "Remove this key" : "The last key stays";
toggleKey.dataset.tip = tipText; toggleKey.setAttribute("aria-label", tipText);
toggleKey.disabled = on && keys.length < 2;
prevKey.disabled = !keys.some((k) => k.t < p - SAME_KEY);
nextKey.disabled = !keys.some((k) => k.t > p + SAME_KEY);
}
(() => {
let drag = null;
const tAt = (e) => { const r = inner.getBoundingClientRect(); return clamp((e.clientX - r.left) / (r.width || 1), 0, 1); };
const set = (e) => { let t = tAt(e); for (const k of keys) if (Math.abs(k.t - t) < 0.03) t = k.t; p = t; dirty = true; };
track.addEventListener("pointerdown", (e) => {
if (e.button !== 0) return;
e.stopPropagation();
track.setPointerCapture(e.pointerId); stopPlay();
const r = inner.getBoundingClientRect();
let hit = null, best = 11; // press on a key to grab it; anywhere else moves the time bar
for (const k of keys) { const d = Math.abs(r.left + k.t * r.width - e.clientX); if (d < best) { best = d; hit = k; } }
drag = { key: hit, x: e.clientX, moved: false };
if (hit) { p = hit.t; dirty = true; } else set(e);
});
track.addEventListener("pointermove", (e) => {
if (!drag) return;
if (!drag.key) { if (Math.abs(e.clientX - drag.x) >= 4) drag.moved = true; set(e); return; }
if (!drag.moved && Math.abs(e.clientX - drag.x) < 4) return;
drag.moved = true; // a key slides between its neighbours and never jumps over one
const i = keys.indexOf(drag.key), lo = i > 0 ? keys[i - 1].t + 0.02 : 0, hi = i < keys.length - 1 ? keys[i + 1].t - 0.02 : 1;
drag.key.t = clamp(tAt(e), lo, hi); p = drag.key.t;
writeSoon();
});
// A click on an empty spot of the bar puts a key there; a drag only moves the time bar.
const end = (e) => {
if (drag && !drag.key && !drag.moved && e.type === "pointerup" && keyAt(keys, p) < 0) setCam(camAt(keys, p));
drag = null;
};
track.addEventListener("pointerup", end);
track.addEventListener("pointercancel", end);
})();
const playLabel = (on) => { play.textContent = on ? "❚❚" : "▶"; play.dataset.tip = on ? "Pause" : "Play"; play.setAttribute("aria-label", play.dataset.tip); };
function stopPlay() { if (playing) { playing = false; playLabel(false); } }
play.onclick = () => {
if (playing) { stopPlay(); return; }
if (p >= 0.999) p = 0;
playing = true; lastT = performance.now(); playLabel(true);
};
loop.onclick = () => {
looping = !looping;
loop.setAttribute("aria-pressed", String(looping));
loop.dataset.tip = looping ? "Looping: click to play once" : "Play on a loop";
loop.setAttribute("aria-label", loop.dataset.tip);
};
const toStart = () => { p = 0; lastT = performance.now(); dirty = true; }; // playing carries on from the start
toStartBtn.onclick = toStart;
toggleKey.onclick = () => { // add a key at the time bar, or remove the one it sits on
stopPlay();
const i = keyAt(keys, p);
if (i < 0) setCam(camAt(keys, p));
else if (keys.length > 1) { keys.splice(i, 1); writeSoon(); }
};
const jump = (dir) => {
stopPlay();
const k = dir < 0 ? [...keys].reverse().find((x) => x.t < p - SAME_KEY) : keys.find((x) => x.t > p + SAME_KEY);
if (k) { p = k.t; dirty = true; }
};
prevKey.onclick = () => jump(-1);
nextKey.onclick = () => jump(1);
reset.onclick = () => {
stopPlay();
keys.length = 0; keys.push({ t: 0, ...K0 });
pivotSpec = null; resolvePivot(); p = 0;
writeSoon();
};
for (const b of [play, prevKey, toggleKey, nextKey, reset]) b.addEventListener("pointerdown", (e) => e.stopPropagation());
// ---------- before there is a clip to look at ----------
function drawEmpty() {
const show = status !== "ready";
empty.hidden = !show;
if (!show) return;
const key = `${status}|${statusNote}`;
if (empty.dataset.key === key) return;
empty.dataset.key = key;
empty.innerHTML = "";
if (status === "running") {
empty.append(el("b", "", { text: "Reading the depth…" }), el("span", "", { text: "MoGe looks at every frame once. A long clip takes a while." }));
return;
}
if (status === "loading") { empty.append(el("b", "", { text: "Loading your clip…" })); return; }
const go = el("button", "acm-btn primary", { text: "Show my clip", "data-tip": "Runs this node and the depth before it, not the whole workflow" });
go.addEventListener("pointerdown", (e) => e.stopPropagation());
go.onclick = async (e) => {
e.stopPropagation();
status = "running"; statusNote = ""; dirty = true;
let ok = false;
try { ok = await opts.run(); } catch { ok = false; }
if (!ok && status === "running") { status = "idle"; statusNote = "It did not run. Pick a clip in Load Video, and check MoGe Inference is connected."; dirty = true; }
};
empty.append(el("b", "", { text: "Steer the camera on your clip" }), el("span", "", { text: statusNote || "Run the depth once to see your first frame here." }), go);
}
// ---------- loop ----------
function frame(now) {
if (!alive) return;
requestAnimationFrame(frame);
if (!root.isConnected) return;
readCamera();
if (playing) {
const dur = Math.max(1, (info?.count ?? 121) / FPS);
p += (now - lastT) / 1000 / dur; lastT = now; dirty = true;
if (p >= 1) { if (looping) p %= 1; else { p = 1; stopPlay(); } }
}
flushWrite();
if (!dirty) return;
if (root.offsetParent === null && !root.getClientRects().length) return;
dirty = false;
const cam = camAt(keys, p);
const want = nearestLoaded(p);
// a new run's frames take over as they load; until then the last ones stay up
if (want && want !== shown) shown = want;
if (!info) shown = null;
const fit = fitPicture();
drawEmpty();
renderPreview(cam, fit);
drawGizmo(cam);
renderOutside(cam);
drawTrack();
where.textContent = describe(cam);
pink.textContent = shown ? `pink ${pinkPct.toFixed(0)}%` : "";
const over = pastLoraRange(cam), fl = performance.now() < flashUntil;
const warnText = fl ? flashText : over ? "Past the LoRA's range: it learned up to 45° of turn, 30° up and 20° down." : "";
if (warn.textContent !== warnText) { warn.textContent = warnText; warn.dataset.tip = warnText; }
if (!warnText) delete warn.dataset.tip;
back.set(cam.back);
}
const resize = new ResizeObserver(() => { dirty = true; });
resize.observe(stage);
requestAnimationFrame(frame);
return {
root,
// What a run of the node sent back: the frames to look at.
takeScene(payload) {
if (!payload || !Array.isArray(payload.frames) || !payload.frames.length) return;
info = payload;
frames = payload.frames.map((entry) => ({ t: entry.t, entry, data: null }));
shown = null;
if (status !== "ready") status = "loading";
resolvePivot();
dirty = true;
loadAll(payload);
},
runFailed(note) {
if (status !== "running") return;
status = "idle"; statusNote = note || "It did not run. Pick a clip in Load Video, and check MoGe Inference is connected."; dirty = true;
},
running() { if (status !== "ready") { status = "running"; dirty = true; } },
refresh() { lastText = null; dirty = true; },
get hasScene() { return Boolean(info); },
// The least height the face needs: the picture's least, and the controls as they measure.
floor() { return MIN_STAGE + (controls.offsetHeight || CONTROLS_HEIGHT); },
destroy() {
alive = false;
resize.disconnect();
window.removeEventListener("keydown", onKey, true);
window.removeEventListener("keydown", onPlayKey, true);
document.removeEventListener("pointerdown", closeHelp, true);
hideTip();
},
};
}
+331
View File
@@ -0,0 +1,331 @@
// Camera Move: the shot on the node, and on the box (subgraph node) that holds it.
//
// The face itself is face.mjs. This file mounts it, keeps it pointed at
// wherever the camera keys are kept, runs the node alone when the face asks
// for the clip, and hands it what a run sent back.
//
// Where the keys are kept:
// - on the node, in its own `camera` setting;
// - once `camera` is brought up to a box (a subgraph input linked to it),
// in the box's own copy of the setting, one copy per box. The face then
// sits on the box too, and the node inside shows that box's keys.
import { api } from "../../../scripts/api.js";
import { app } from "../../../scripts/app.js";
import { chainCallback, keepDomWidgetWidthAuto, notifyAusbossChange, showToast } from "../shared/index.mjs";
import { executionIdOf, isSwitchedOff } from "../shared/graph_ids.mjs";
import { SUBGRAPH_INPUT_ID, boxesOf, hostBox, workflowNodes } from "../shared/lora_box.mjs";
import { WIDGET_FRAME, ensureNodeMinHeight, fillNodeHeight } from "../shared/panel_layout.mjs";
import { hideWidget } from "../shared/widget_visibility.mjs";
import { CONTROLS_HEIGHT, MIN_STAGE, MIN_WIDTH, mountCameraFace } from "./face.mjs";
import { mountBoxCard } from "./box_card.mjs";
import { CARD_WIDGET } from "../shared/widget_card.mjs";
const NODE_CLASS = "AUSBOSS_NODES_CameraMove";
const CAMERA = "camera";
const UI_KEY = "ausboss_camera_scene";
const FACE_WIDGET = "ausboss_camera_move";
const OPENING_WIDTH = 460;
const OPENING_ROOM = 90; // a new node opens this much taller than its floor, so a wide clip fills the picture
const MIN_HEIGHT = MIN_STAGE + CONTROLS_HEIGHT + WIDGET_FRAME;
const mounts = new Set();
const rootGraph = () => app.rootGraph ?? app.graph;
const isMove = (node) => node?.type === NODE_CLASS || node?.comfyClass === NODE_CLASS;
const same = (a, b) => a != null && b != null && String(a) === String(b);
const linkOf = (graph, id) => (id == null || !graph ? null : graph.getLink?.(id) ?? graph.links?.get?.(id) ?? graph.links?.[id] ?? null);
const nodeOf = (graph, id) => graph?.getNodeById?.(id) ?? (graph?.nodes ?? graph?._nodes ?? []).find((node) => same(node?.id, id)) ?? null;
const ownCamera = (node) => node?.widgets?.find((widget) => widget?.name === CAMERA) ?? null;
const boxCopy = (box, name) => (box?.widgets ?? []).find((widget) => widget?.name === name && typeof widget.value === "string") ?? null;
// The subgraph input that brings a node's camera in from its box, or null.
function cameraInputName(node) {
const graph = node?.graph;
const index = (node?.inputs ?? []).findIndex((input) => input?.name === CAMERA);
if (!graph || index < 0) return null;
const link = linkOf(graph, node.inputs[index].link);
if (!link || !same(link.origin_id, SUBGRAPH_INPUT_ID)) return null;
return graph.inputs?.[link.origin_slot]?.name ?? null;
}
// The first Camera Move inside a box whose camera comes up to the box: { name, inner }.
function boxCamera(box) {
const subgraph = box?.subgraph;
for (const input of subgraph?.inputs ?? []) {
for (const id of input?.linkIds ?? []) {
const link = linkOf(subgraph, id);
const inner = link && nodeOf(subgraph, link.target_id);
if (isMove(inner) && inner.inputs?.[link.target_slot]?.name === CAMERA) return { name: input.name, inner };
}
}
return null;
}
// Where a face reads and writes the keys, and which run of the node it shows.
function sourceOf(mount) {
const root = rootGraph();
if (mount.box) {
const widget = boxCopy(mount.box, mount.inputName);
if (widget) hideWidget(widget);
const at = executionIdOf(root, mount.box);
return { widget, id: at ? `${at}:${mount.inner.id}` : null, holder: mount.box };
}
const node = mount.inner;
const fed = cameraInputName(node);
if (fed) {
const box = hostBox(node.graph, boxesOf(node.graph, workflowNodes(root)), null);
const widget = box && boxCopy(box, fed);
const at = box && executionIdOf(root, box);
if (widget) return { widget, id: at ? `${at}:${node.id}` : executionIdOf(root, node), holder: box };
}
return { widget: ownCamera(node), id: executionIdOf(root, node), holder: node };
}
const viewRoute = (file) => `/view?${new URLSearchParams({ filename: file.filename, type: file.type || "temp", subfolder: file.subfolder || "" })}`;
const viewUrl = (file) => api.apiURL(viewRoute(file));
// The depth picture's exact bytes, through ComfyUI's own request helper.
const viewBlob = async (file) => {
const response = await api.fetchApi(viewRoute(file));
if (!response.ok) throw new Error(`depth ${response.status}`);
return response.blob();
};
function locatorOf(node) {
const graph = node?.graph;
if (!graph || node.id == null) return null;
const root = graph.rootGraph ?? graph;
return graph !== root && graph.id ? `${graph.id}:${node.id}` : String(node.id);
}
function restore(mount) {
if (mount.face.hasScene) return;
const scene = app.nodeOutputs?.[locatorOf(mount.inner)]?.[UI_KEY]?.[0];
if (scene) mount.face.takeScene(scene);
}
function mount(host, { box = null, inputName = null, inner = host } = {}) {
const entry = { host, box, inputName, inner, face: null, widget: null };
const face = mountCameraFace({
read: () => String(sourceOf(entry).widget?.value ?? ""),
write: (text) => {
const widget = sourceOf(entry).widget;
if (!widget) return;
widget.value = text;
app.canvas?.setDirty?.(true, false);
notifyAusbossChange();
},
view: viewUrl,
blob: viewBlob,
run: async () => {
const { id, holder } = sourceOf(entry);
if (!id || typeof app.queuePrompt !== "function") return false;
if (isSwitchedOff(rootGraph(), holder) || isSwitchedOff(rootGraph(), inner)) {
showToast({ severity: "warn", summary: "Camera Move", detail: "This node is switched off (bypassed or muted), so it cannot run." });
return false;
}
entry.askedAt = performance.now();
// At the front of the queue: the clip shows up once the job running now is done.
const queued = await app.queuePrompt(-1, 1, [id]);
// Refused: an empty loader, a missing link, or a check that stopped it (it says why in a note).
// Nothing was queued, so the face must not wait for a picture that never comes.
if (queued === false) return false;
return true;
},
});
entry.face = face;
const widget = host.addDOMWidget(box ? `${FACE_WIDGET}:${inputName}` : FACE_WIDGET, FACE_WIDGET, face.root, { serialize: false, hideOnZoom: false });
keepDomWidgetWidthAuto(widget);
// Never saved with the workflow: options.serialize only keeps it out of the prompt.
widget.serialize = false;
// The floor follows the controls as they measure on this screen (fonts differ between phones).
fillNodeHeight(widget, { minWidth: MIN_WIDTH, minHeight: () => face.floor() + WIDGET_FRAME, minNodeSize: [MIN_WIDTH, MIN_HEIGHT], exactMinWidth: true });
entry.widget = widget;
mounts.add(entry);
restore(entry);
holdFloor(host);
return entry;
}
// A node or box kept shorter than the face needs (a saved size, a font that
// runs taller on this screen) would cut the controls off at its bottom edge.
function holdFloor(host) {
requestAnimationFrame(() => requestAnimationFrame(() => {
if (ensureNodeMinHeight(host)) host.setDirtyCanvas?.(true, true);
}));
}
function unmount(entry) {
mounts.delete(entry);
entry.face.destroy();
try { entry.host.removeWidget?.(entry.widget); } catch { entry.widget?.onRemove?.(); }
}
// A new node or box opens with room for the picture; a saved size wins.
function openRoomy(host) {
const floor = Number(host.computeSize?.()?.[1]);
if (!Number.isFinite(floor) || host.size?.[1] > floor + 40) return;
host.setSize?.([Math.max(host.size?.[0] ?? 0, OPENING_WIDTH), floor + OPENING_ROOM]);
}
function build(node) {
if (node.__ausbossCameraMove) return node.__ausbossCameraMove;
hideWidget(ownCamera(node));
node.__ausbossCameraMove = mount(node);
return node.__ausbossCameraMove;
}
// The settings card and any LoRA rows sit on top; the shot comes last and
// takes the rest of the box, so its buttons stay on the bottom edge.
function faceLast(box) {
// A box makes `widgets` afresh on every read (its settings, then the panels
// added to it), so the panels are put in order in their own list: any other
// panel (a LoRA Loader's rows), the settings card, then the face. Nodes 2.0
// lays them out in the order they were made, which is the same.
const widgets = Array.isArray(box._extraWidgets) ? box._extraWidgets : box.widgets ?? [];
const order = [widgets.find((w) => w?.name === CARD_WIDGET), box.__ausbossCameraBox?.widget].filter(Boolean);
const want = [...widgets.filter((w) => !order.includes(w)), ...order];
if (want.every((w, i) => widgets[i] === w)) return false;
widgets.splice(0, widgets.length, ...want);
box.setDirtyCanvas?.(true, true);
return true;
}
// A box row reads its options from the widget inside. A node with a card
// (Image Resize) hides its own rows by setting options.hidden there, and
// Nodes 2.0 would then hide the box's row too. The box keeps every setting
// brought up to it in view, except the camera text the face stands in for.
const SHOWN = new WeakMap();
function shownOptions(options) {
if (!options || typeof options !== "object" || !options.hidden) return options;
let view = SHOWN.get(options);
if (!view) {
view = new Proxy(options, { get: (target, key) => (key === "hidden" ? false : Reflect.get(target, key)) });
SHOWN.set(options, view);
}
return view;
}
function showSettings(box, keep) {
for (const widget of box.widgets ?? []) {
if (widget === keep || widget.__ausbossCameraShown || !widget.options?.hidden) continue;
const own = Object.getOwnPropertyDescriptor(widget, "options");
if (!own?.get || !own.configurable || widget.__ausbossVisibilityStash) continue;
Object.defineProperty(widget, "options", { ...own, get() { return shownOptions(own.get.call(this)); } });
widget.__ausbossCameraShown = true;
}
}
// A box shows the face while the camera of a Camera Move inside comes up to it.
function syncBox(box) {
if (!box?.isSubgraphNode?.() || !box.graph) return;
const found = boxCamera(box);
const have = box.__ausbossCameraBox;
if (have && (!found || have.inner !== found.inner || have.inputName !== found.name || !boxCopy(box, found.name))) {
unmount(have);
box.__ausbossCameraBox = null;
box.setSize?.([box.size[0], box.computeSize?.()[1] ?? box.size[1]]);
}
if (box.__ausbossCameraBox) {
if (mountBoxCard(box, boxCopy(box, box.__ausbossCameraBox.inputName), faceLast)) holdFloor(box);
faceLast(box);
showSettings(box, boxCopy(box, box.__ausbossCameraBox.inputName));
return;
}
if (!found || !boxCopy(box, found.name)) return;
hideWidget(boxCopy(box, found.name));
// The card before the face: Nodes 2.0 lays a box's panels out in the order they were made.
mountBoxCard(box, boxCopy(box, found.name), faceLast);
box.__ausbossCameraBox = mount(box, { box, inputName: found.name, inner: found.inner });
faceLast(box);
showSettings(box, boxCopy(box, found.name));
if (!box.__ausbossCameraSized) openRoomy(box);
box.__ausbossCameraSized = true;
box.setDirtyCanvas?.(true, true);
}
function watchBox(box) {
if (!box?.isSubgraphNode?.() || box.__ausbossCameraWatch) return;
const abort = new AbortController();
box.__ausbossCameraWatch = abort;
const later = () => requestAnimationFrame(() => syncBox(box));
for (const type of ["widget-promoted", "widget-demoted", "input-added", "removing-input"]) {
box.subgraph?.events?.addEventListener?.(type, later, { signal: abort.signal });
}
chainCallback(box, "onConfigure", (info) => { if (info?.size) box.__ausbossCameraSized = true; later(); holdFloor(box); });
chainCallback(box, "onRemoved", () => {
abort.abort();
if (box.__ausbossCameraBox) unmount(box.__ausbossCameraBox);
box.__ausbossCameraBox = null;
});
later();
}
// What a run sent back reaches every face that shows that run of the node.
api.addEventListener("executed", ({ detail }) => {
try {
const scene = detail?.output?.[UI_KEY]?.[0];
if (!scene) return;
const ids = new Set([String(detail.node ?? ""), String(detail.display_node ?? "")]);
for (const entry of mounts) if (ids.has(String(sourceOf(entry).id))) entry.face.takeScene(scene);
} catch {
// Never in the way of a run.
}
});
const stopped = (note) => { for (const entry of mounts) entry.face.runFailed(note); };
api.addEventListener("execution_error", ({ detail }) => {
if (/OutOfMemory/i.test(String(detail?.exception_type ?? ""))) {
stopped("The graphics card is full. Close other jobs using it, then try again.");
return;
}
const why = String(detail?.exception_message ?? "").split("\n")[0].slice(0, 160);
stopped(why ? `The run stopped: ${why}` : null);
});
api.addEventListener("execution_interrupted", () => stopped("The run was stopped."));
// A run that ended without sending the clip (it never reached the node)
// would leave a face waiting for good. Once the queue is empty, it stops.
api.addEventListener("status", ({ detail }) => {
if (detail?.exec_info?.queue_remaining !== 0) return;
const now = performance.now();
for (const entry of mounts) if (now - (entry.askedAt ?? 0) > 1500) entry.face.runFailed();
});
app.registerExtension({
name: "ausboss.camera_move",
onNodeOutputsUpdated() {
for (const entry of mounts) restore(entry);
},
nodeCreated(node) {
if (node?.isSubgraphNode?.()) watchBox(node);
},
loadedGraphNode(node) {
if (node?.isSubgraphNode?.()) watchBox(node);
},
beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData?.name !== NODE_CLASS) return;
chainCallback(nodeType.prototype, "onNodeCreated", function () {
build(this);
});
chainCallback(nodeType.prototype, "onAdded", function () {
if (this.__ausbossCameraSized) return;
this.__ausbossCameraSized = true;
openRoomy(this);
});
chainCallback(nodeType.prototype, "onConfigure", function (info) {
if (info?.size) this.__ausbossCameraSized = true;
const entry = build(this);
hideWidget(ownCamera(this));
queueMicrotask(() => { entry.face.refresh(); restore(entry); });
holdFloor(this);
});
chainCallback(nodeType.prototype, "onConnectionsChange", function () {
this.__ausbossCameraMove?.face.refresh();
// The camera may have just come up to a box, or gone back inside.
for (const box of boxesOf(this.graph, workflowNodes(rootGraph()))) requestAnimationFrame(() => syncBox(box));
});
chainCallback(nodeType.prototype, "onRemoved", function () {
const entry = this.__ausbossCameraMove;
if (entry) unmount(entry);
this.__ausbossCameraMove = null;
});
},
});
+82
View File
@@ -0,0 +1,82 @@
# Camera Move
Moves the camera of your clip. You steer it on the node's picture, the way
you turn a model in a 3D viewer, and set keys on the time bar. What the old
camera never saw comes out pink, and the CrossView-Warp LoRA for MiniMax H3
paints it in.
## How to use it
1. Load your clip, scale it to the size you will render, and run it through
**Load MoGe Model** and **MoGe Inference** (core ComfyUI nodes).
2. Connect MoGe Inference's geometry to Camera Move.
3. Press **Show my clip** on the node. It runs the depth once and shows your
first frame.
4. Steer:
- Drag the picture to turn the camera round.
- Pinch or scroll to pull back or push in. Two fingers, or Shift-drag, pan.
- Tap a spot to circle round it.
- Top right: drag the axis ball to turn, or tap an axis to look from that
side. Drag the magnifier to zoom and the hand to pan. The camera button
puts you back on your own shot.
5. Keys: click the time bar to put a key there, using the camera at that
moment. Or drag the time bar, then move the camera, and a key appears
there. The diamond adds or removes a key; the arrows either side jump
between keys. Drag a key along the bar to change its time.
⏮ goes back to the start, ▶ plays, and the loop button plays it round
and round.
6. Run the workflow. `guide` goes to the H3 guide at frame 0.
The keys move the camera from where your clip's own camera was. If your
clip's camera already moves (walks down an aisle, for example), that move
stays, and your keys add to it. After the last key the camera holds that
turn, but it still travels along with your clip.
The line under the picture turns amber when a key goes past where the LoRA
was trained: 45° of turn, 30° up, 20° down. Past that it may leave pink.
### Mouse and keys (like Blender)
Middle-drag turns, Shift + middle-drag pans, Ctrl + middle-drag zooms, and
the wheel zooms. On the numpad: 4 6 8 2 turn, Ctrl with them pans, + and −
zoom, 1 is the front, 3 the right side (Ctrl: left), 7 the top (Ctrl:
bottom), 0 your own shot. Space plays and pauses and Home goes back to the
start, while the pointer is over the node.
The wheel over the picture zooms the camera, not the graph. Everywhere else
on the node it zooms the graph as usual.
## Inputs and outputs
| | |
|---|---|
| `moge_geometry` | Depth from MoGe Inference, one per frame. |
| `camera` | The keys, written by the node's picture. You never type here. |
| `guide` | Your clip from the moved camera, pink where nothing was seen. |
| `holes` | White where the guide is pink. |
## Technical details
- Each key holds a moment `t` (0 to 1 over the clip), a turn `az` and a rise
`el` in degrees, `back` (how far the camera pulls back, in distances to the
pivot; below 0 pushes in), and a pan `px`, `py` (same unit). The camera
passes through each key without slowing down, and only eases at the first
and last key. Each value follows a monotone cubic through the keys
(Fritsch-Butland slopes), so it never runs past a key: a value that turns
back or holds at a key stops there.
- The camera turns round the pivot: a spot you tap (saved as a place on one
frame, `pivot: {u, v, t}`) or, by default, the nearest fifth of what sits in
the middle of the first frame, never further than 3 m.
- `R = Ry(-az) · Rx(-el)`, `C = pivot − R·pivot + R·[px·d, py·d, −back·d]`
with `d = |pivot|`, in OpenCV axes (x right, y down, z forward).
- MoGe's depth scale and lens wobble from frame to frame, which shook the
moved picture. Every frame is scaled so its median depth follows a
smoothed line (15 frames wide) through all of them, and every frame uses
the clip's median lens.
- Each pixel is drawn as 4 points, nearest wins, and gaps mostly surrounded
by picture (5 of 8 neighbours, two passes) are filled. Everything else
stays magenta (1, 0, 1).
- The node is an output node so its picture can run it on its own. Each run
writes up to 61 small frames (416 px on the long side) with their depth to
ComfyUI's temp folder; the depth is a PNG with a log scale in red (high
byte) and green (low byte), 0 for no depth.
+10 -2
View File
@@ -120,6 +120,8 @@ or an area you painted.
## The tools on the node
- **Reload** (the round arrow) shows the picture coming in now: see
**Which picture the picks belong to**, below.
- **Undo** takes back the last change, or a Clear. Ctrl+Z works as well: it
is ComfyUI's own undo, and every change counts.
- **Clear** picks nothing.
@@ -152,7 +154,8 @@ or an area you painted.
- **More → Smooth** (`smooth`): rounds off a ragged outline by this many
pixels.
- **More → Fill small holes** (`fill_holes`): closes pinholes inside the
picked area. Larger openings stay open, like a mug's handle.
picked area. Larger openings stay open, like a mug's handle. Off by
default, so a hole you pick stays a hole.
- **More → Invert** (`invert`): turns the mask round. On, the mask is
everything except what you picked.
- **More → If nothing** (`if_nothing`): **stop the run** or **empty mask**,
@@ -209,7 +212,12 @@ nothing instead.
Picks belong to the picture they were made on. Load another picture and the
node shows it with nothing picked, and says the old picks are not used.
If the workflow runs before the node has shown the new picture, the run
stops and says the picks were made on another picture.
stops and says the picks were made on another picture. The new picture
shows on the node then, ready to pick from.
- **The reload button** (the round arrow beside Undo) shows the picture
coming in now. Use it after you change the picture before this node, for
example with an edit. It runs only this node and what comes before it.
- With a Load Image wired straight in, or through Reroute nodes, the
picture shows as soon as you choose a file.
+1 -1
View File
@@ -52,7 +52,7 @@ file's name. A caption on the `caption_text` input writes the paired `.txt`.
still lists it. When a run saves more than one image (a batch, or a list
such as a folder of images), arrows beside the switch (‹ 2 / 5 ›) step
through them.
- **Steps** (`step_preview`, off by default): the switch left of
- **Preview steps** (`step_preview`, off by default): the switch left of
**preview** shows the picture while it is being made, one step at a time,
where the saved image goes, with a **STOP** button on the bar. Stop a run
that is going wrong after the first steps instead of waiting for the end;
+1 -1
View File
@@ -9,7 +9,7 @@ duration, shows a tall video whole, and has a **LOOP** control. `gif` and
`webp` are shown as pictures with just the file name, since a video player
cannot play either.
**Steps** above the player (`step_preview`, off by default) shows the clip's first frame in
**Preview steps** above the player (`step_preview`, off by default) shows the clip's first frame in
the player while it is being made, one step at a time, with a **STOP**
button on the player. Stop a clip that is going wrong after the first steps
instead of waiting for the end; a stopped run saves nothing. The saved
+1 -1
View File
@@ -334,7 +334,7 @@ function installStepsSwitch(state, signal) {
if (!widget) return;
state.stepsWidget = widget;
hideWidget(widget);
const button = panelSwitch("Steps", "Show the picture here while it is being made, one step at a time, with a Stop button. Off by default.");
const button = panelSwitch("Preview steps", "Show the picture here while it is being made, one step at a time, with a Stop button. Off by default.");
button.classList.add("steps");
button.addEventListener("pointerdown", (event) => event.stopPropagation(), { signal });
button.addEventListener("click", (event) => {
+6 -1
View File
@@ -168,13 +168,18 @@ api.addEventListener("execution_error", ({ detail }) => {
api.addEventListener("execution_interrupted", () => settleAll());
// A look is one long, quiet run. The node sends what it has found every
// second or two, so things light up while it is still looking.
// A run that stops at the node (nothing picked on this picture) sends the
// picture that came in first, so a picture that changed upstream shows at
// once. It is shown like a kept note: it starts no run of its own.
api.addEventListener(LOOK_EVENT, ({ detail }) => {
try {
if (!faces.size || !detail?.note) return;
const id = String(detail.node ?? "");
const root = app.rootGraph ?? app.graph;
for (const face of faces) {
if (face.alive && face.inFlight && executionIdOf(root, face.node) === id) face.takeNote(detail.note, { interim: true });
if (!face.alive || executionIdOf(root, face.node) !== id) continue;
if (detail.stopped) face.takeNote(detail.note, { restored: true });
else if (face.inFlight) face.takeNote(detail.note, { interim: true });
}
} catch {
// Never in the way of a run.
+2 -2
View File
@@ -39,7 +39,7 @@ function getLoopEnabled(node) {
return !!node.properties.ausboss_save_video_loop;
}
// The STEPS button: the hidden `step_preview` widget (the card hides it).
// The Preview steps switch: the hidden `step_preview` widget (the card hides it).
// On, the player shows each step of the clip's first frame while it is
// being made, and a STOP button joins the tools for the run.
function stepsOn(state) {
@@ -131,7 +131,7 @@ function buildPreview(node) {
loopButton.classList.add("ausboss-steps-loop");
const stopButton = makeToolButton("STOP", "Stop this run now. The next run in the queue still starts.");
stopButton.classList.add("ausboss-steps-stop");
const stepsButton = panelSwitch("Steps", "Show the clip's first frame here while it is being made, one step at a time, with a Stop button. Off by default.");
const stepsButton = panelSwitch("Preview steps", "Show the clip's first frame here while it is being made, one step at a time, with a Stop button. Off by default.");
tools.classList.add("ausboss-steps-tools");
tools.append(stopButton, loopButton);
stage.append(video, image, status, tools);
+1 -1
View File
@@ -9,7 +9,7 @@ import { app } from "../../../scripts/app.js";
// Must match `version` in pyproject.toml — scripts/release_preflight.py
// enforces it. The backend serves its copy at /ausboss/pack_version; a
// mismatch means the browser cached JavaScript from an older install.
export const AUSBOSS_JS_VERSION = "2.21.1";
export const AUSBOSS_JS_VERSION = "2.21.2";
export const BRAND = "#00b4aa"; // AusBoss teal — keep node accents consistent.
export const BRAND_DARK = "#007f78";
+19 -9
View File
@@ -95,8 +95,9 @@ export const NOTE_KEY = "ausboss_mask_by_touch";
const TOOLS_HEIGHT = 24;
const BAR_HEIGHT = 20;
const STAGE_MIN = 150;
// Wide enough for the bar under the picture: the arrows, a pick's name, Subject and Background.
export const PANEL_MIN_WIDTH = 400;
// Wide enough for the tools over the picture (Reload, Undo, Clear and the eye
// included) and the bar under it: the arrows, a pick's name, Subject and Background.
export const PANEL_MIN_WIDTH = 428;
// What the panel's element needs at its smallest: tools, stage and bar, two
// gaps and the padding under them.
const CHIPS_HEIGHT = 22;
@@ -104,9 +105,9 @@ const CHIPS_HEIGHT = 22;
const CHIP_ROWS = 3;
// A node this wide has room for the names in front of the brush's two
// numbers. A narrower one shows the numbers alone.
const LABELS_FROM = 440;
const LABELS_FROM = 470;
// The width a new node opens with: the names fit.
export const OPENING_WIDTH = 460;
export const OPENING_WIDTH = 490;
const PANEL_NEEDS = TOOLS_HEIGHT + STAGE_MIN + BAR_HEIGHT + CHIPS_HEIGHT + 12 + 6;
// Saved with the workflow: the brush size once it was set by hand, the snap
// switch, and how the picked area is shown.
@@ -134,6 +135,8 @@ const EYE_TITLES = {
// The eye beside Undo and Clear, drawn here: an almond and its pupil.
const EYE_ICON = '<svg width="16" height="16" viewBox="-8 -8 16 16" fill="none" stroke="currentColor" stroke-width="1.5" stroke-linejoin="round" aria-hidden="true">'
+ '<path d="M-6.6 0Q0 -6 6.6 0Q0 6 -6.6 0Z"/><circle r="2" fill="currentColor" stroke="none"/></svg>';
const RELOAD_ICON = '<svg width="14" height="14" viewBox="-8 -8 16 16" fill="none" stroke="currentColor" stroke-width="1.6" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true">'
+ '<path d="M5.6 -2.4A6 6 0 1 0 5.2 3"/><path d="M6 -6.2V-2.2H2"/></svg>';
// The brush size, across the picture, in picture pixels.
const BRUSH_SIZE = [4, 1000];
// A brush mark counts as touching an area within this many pixels of the
@@ -241,7 +244,7 @@ function ensureCss() {
.ausboss-touch.narrow .ausboss-touch-tools{gap:3px;}
.ausboss-touch.narrow .ausboss-touch-field.size .ausboss-scrub{width:56px !important;}
.ausboss-touch.narrow .ausboss-touch-field.soft .ausboss-scrub{width:48px !important;}
.ausboss-touch-btn.eye{display:inline-flex;align-items:center;justify-content:center;width:26px;min-width:26px;padding:0;}
.ausboss-touch-btn.eye,.ausboss-touch-btn.reload{display:inline-flex;align-items:center;justify-content:center;width:26px;min-width:26px;padding:0;}
.ausboss-touch-btn.eye.on{border-color:${BRAND};background:${BRAND};color:#04201f;}
.ausboss-touch-btn.arrow{min-width:20px;width:20px;padding:0;font-size:14px;line-height:16px;}
.ausboss-touch-swatch{box-sizing:border-box;flex:none;width:${BAR_HEIGHT}px;height:${BAR_HEIGHT}px;border-radius:4px;background:#0b1113;border:1px solid rgba(0,180,170,.3);pointer-events:auto;}
@@ -742,7 +745,7 @@ function settings(face) {
grow: Math.round(number("grow")),
soften: Math.max(0, number("soften")),
smooth: Math.max(0, Math.round(number("smooth"))),
fill: widgetByName(face.node, "fill_holes")?.value !== false,
fill: widgetByName(face.node, "fill_holes")?.value === true,
invert: widgetByName(face.node, "invert")?.value === true,
fine: fineChoice(widgetByName(face.node, "fine_edges")?.value),
};
@@ -1403,6 +1406,7 @@ function syncTools(face) {
const waiting = live && face.pending.length > 0;
face.undo.disabled = !live || (!some && !face.past.length && !waiting);
face.clear.disabled = !some && !waiting;
face.reload.disabled = !live || face.inFlight;
if (face.lookButton) {
const offer = live && face.snap && !face.held && !face.inFlight && !face.pic.scoped && Boolean(face.host.lookOnButton?.());
face.lookButton.style.display = offer ? "" : "none";
@@ -2877,9 +2881,9 @@ export function mountTouchStage(node, host) {
};
readProperties(face);
const showPicture = () => { if (!face.inFlight) { face.retries = 0; if (face.pic) face.pic.paused = false; requestRun(face, "scope"); } };
face.start = button("ausboss-touch-start", "Show picture",
"Runs only this node, so the picture shows here and its pieces light up under the pointer.",
() => { if (!face.inFlight) { face.retries = 0; if (face.pic) face.pic.paused = false; requestRun(face, "scope"); } }, signal);
"Runs only this node, so the picture shows here and its pieces light up under the pointer.", showPicture, signal);
wake.append(face.start, hint);
// Tools: what a press does, the brush, undo and clear; on the right, how
@@ -2976,7 +2980,13 @@ export function mountTouchStage(node, host) {
if (event.shiftKey) face.setView(face.view === "cut" ? "mask" : face.view === "mask" ? "cut" : face.result === "cut" ? "mask" : "cut");
else face.setView(face.view === "tint" ? face.result : "tint");
}, { signal });
tools.append(face.toolSwitch.root, face.sizeField, face.softField, el("div", "ausboss-touch-gap"), face.undo, face.clear, face.eye);
// The picture coming in changed (an edit made upstream): this shows the
// new one, the same way Show picture does.
face.reload = button("ausboss-touch-btn reload", "",
"Reload the picture: runs only this node, so a picture that changed upstream shows here.", showPicture, signal);
face.reload.innerHTML = RELOAD_ICON;
face.reload.setAttribute("aria-label", "Reload the picture");
tools.append(face.toolSwitch.root, face.sizeField, face.softField, el("div", "ausboss-touch-gap"), face.reload, face.undo, face.clear, face.eye);
newButton.addEventListener("click", (event) => {
event.preventDefault();
event.stopPropagation();
+8
View File
@@ -166,6 +166,14 @@ export function noteIsEmpty(note) {
);
}
// A note shows its tile face when it is in tile layout and has anything to
// show. The node is then just a button: the note itself opens in a dialog,
// so there is no card inside the node whose height needs fitting.
export function noteShowsTileFace(note) {
if (!note || typeof note !== "object") return false;
return note.layout === "tile" && !noteIsEmpty(note);
}
// "ComfyUI/models/loras" - the folder hint under each group of rows.
export function folderPath(dir) {
if (!dir) return "ComfyUI/models";
+1 -1
View File
@@ -416,7 +416,7 @@ const CARDS = {
],
},
AUSBOSS_NODES_SaveVideo: {
// step_preview is the STEPS button on the player (js/save_video).
// step_preview is the Preview steps switch on the player (js/save_video).
minWidth: 320, first: true, hide: ["step_preview"],
rows: [
{ widget: "filename_prefix", label: "Prefix", placeholder: "video" },
+13
View File
@@ -21,6 +21,7 @@ import { copyToClipboard } from "../shared/clipboard.mjs";
import { confirmDiscard } from "../shared/discard_prompt.mjs";
import { WIDGET_FRAME, fillNodeHeight, holdNodeMinHeight } from "../shared/panel_layout.mjs";
import { hideInputsInDef, hideWidget } from "../shared/widget_visibility.mjs";
import { suppressCoreImagePreview } from "../shared/core_preview.mjs";
import {
COMMON_FOLDERS,
INPUT_FOLDER,
@@ -34,6 +35,7 @@ import {
matchInstalled,
noteFit,
noteIsEmpty,
noteShowsTileFace,
normalizeNote,
packsFromGraph,
parseMarkdown,
@@ -709,6 +711,12 @@ function naturalHeight(state, width) {
// empty space until someone drags it shorter.
function fitNote(state) {
if (!classicCanvas()) return;
// A tile face is a button, not a card: fitting it would measure the note
// behind the button and grow a new tile straight back to full height.
if (noteShowsTileFace(state.note)) {
state.fitHeight = undefined;
return;
}
const { node, root, domWidget } = state;
const width = node.size[0] - WIDGET_FRAME;
if (!(width > 0)) return;
@@ -1113,6 +1121,11 @@ function buildPanel(node) {
const widget = node.widgets?.find((item) => item.name === "note");
if (!widget) return null;
hideWidget(widget);
// A note never has output images of its own. The frontend files previews
// by node id, so one left over from another node can still land here, and
// in the classic canvas its widget asks for the picture's full height:
// the node grows to hold it and a tile stops starting as a tile.
suppressCoreImagePreview(node);
const root = el("div", "ausboss-note");
const card = el("div", "ausboss-note-card");
+20
View File
@@ -1591,5 +1591,25 @@
"AUSBOSS_NODES_WorkflowNoteTile": {
"display_name": "工作流说明卡片 🆎",
"description": "点击打开完整说明的小按钮。点击铅笔编辑,拖动角落调整大小。旧版节点包会将它显示为缺失节点,请更新 ComfyUI-AusBoss。"
},
"AUSBOSS_NODES_CameraMove": {
"display_name": "镜头移动 🆎",
"description": "移动视频的镜头。在节点画面上拖动来转动镜头,捏合或滚轮拉远推近,并在时间轴上设置关键帧。原镜头从未拍到的地方显示为粉色,交给 CrossView-Warp LoRA 补画。需要 MoGe Inference 的深度。",
"inputs": {
"moge_geometry": {
"name": "MoGe 几何"
},
"camera": {
"name": "镜头关键帧"
}
},
"outputs": {
"0": {
"name": "引导"
},
"1": {
"name": "空洞"
}
}
}
}
+428
View File
@@ -0,0 +1,428 @@
"""Camera Move: the camera keys, the camera's pose and the depth warp.
The keys are what the node's face writes: a list of moments on the clip
(t from 0 to 1) and where the camera is at each one. The camera passes
through every key without slowing down, and starts and ends gently. The face draws its picture with the
same maths, so what you steer on the node is what the warp makes.
Axes are OpenCV's: x right, y down, z into the picture. Each frame's points
stay where the original camera saw them; only the camera moves.
Technical details
-----------------
A key holds az (turn, degrees, right is positive), el (rise, degrees, up is
positive), back (pull back, in distances to the pivot; negative pushes in),
px and py (slide right and down, in distances to the pivot).
R = Ry(-az) . Rx(-el)
C = pivot - R.pivot + R.[px.d, py.d, -back.d] d = |pivot|
A point P lands in the new camera at R^T (P - C). The pivot is the spot the
camera circles: picked by a tap on the face, else what sits in the middle
of the first frame.
"""
from __future__ import annotations
import json
import math
LIMITS = {"az": (-90.0, 90.0), "el": (-30.0, 60.0), "back": (-0.6, 2.0), "px": (-1.0, 1.0), "py": (-1.0, 1.0)}
FIELDS = tuple(LIMITS)
HOME = {name: 0.0 for name in FIELDS}
# Where the LoRA was trained to work well. Past this the face warns.
LORA_RANGE = {"az": 45.0, "el_up": 30.0, "el_down": -20.0}
# Two keys within this much of the clip are the same key.
SAME_KEY = 0.015
MAX_KEYS = 64
MAGENTA = (1.0, 0.0, 1.0)
# The nearest thing in the middle of the first frame, never further than this
# (in the depth model's metres): the move stays a move, not a flight.
PIVOT_CAP = 3.0
SCENE_LONG_SIDE = 416
# Every second frame of a 5-second clip, so the face plays smoothly (about 190 KB a frame).
SCENE_FRAMES = 61
DEFAULT_CAMERA = {
"v": 1,
"keys": [{"t": 0.0, **HOME}, {"t": 1.0, **HOME, "az": 30.0, "el": 5.0}],
"pivot": None,
}
def default_camera_text() -> str:
return json.dumps(DEFAULT_CAMERA, separators=(",", ":"))
def _number(value, fallback=0.0) -> float:
try:
number = float(value)
except (TypeError, ValueError):
return fallback
return number if math.isfinite(number) else fallback
def _clamp(value, low, high):
return max(low, min(high, value))
def clean_key(raw) -> dict:
"""One key with every field present, inside its limits."""
raw = raw if isinstance(raw, dict) else {}
key = {"t": _clamp(_number(raw.get("t")), 0.0, 1.0)}
for name, (low, high) in LIMITS.items():
key[name] = _clamp(_number(raw.get(name)), low, high)
return key
def parse_camera(text) -> dict:
"""The camera the face wrote. Anything unreadable falls back to the default move.
Returns {"keys": [...], "pivot": None | {"u", "v", "t"}, "fallback": bool}.
Keys come back sorted by time, with keys at the same moment merged (the
later one wins), and never empty.
"""
data = None
if isinstance(text, dict):
data = text
elif isinstance(text, str) and text.strip():
try:
data = json.loads(text)
except (ValueError, RecursionError):
data = None
fallback = not isinstance(data, dict) or not isinstance(data.get("keys"), list) or not data["keys"]
if fallback:
data = DEFAULT_CAMERA
keys = []
for raw in data["keys"][:MAX_KEYS]:
key = clean_key(raw)
same = next((k for k in keys if abs(k["t"] - key["t"]) < SAME_KEY), None)
if same is not None:
same.update(key)
else:
keys.append(key)
keys.sort(key=lambda k: k["t"])
pivot = data.get("pivot")
if isinstance(pivot, dict) and all(name in pivot for name in ("u", "v")):
pivot = {
"u": _clamp(_number(pivot.get("u"), 0.5), 0.0, 1.0),
"v": _clamp(_number(pivot.get("v"), 0.5), 0.0, 1.0),
"t": _clamp(_number(pivot.get("t"), 0.0), 0.0, 1.0),
}
else:
pivot = None
return {"keys": keys, "pivot": pivot, "fallback": fallback}
def _key_slopes(ts, ys):
"""How fast one field moves as it passes each key.
Zero at the first and last key, so the move starts and ends gently. Zero
at a key where the field turns back or holds, so it never runs past a
key. Elsewhere it keeps its speed through the key (Fritsch-Butland), so
a long move made of several keys does not slow down at each one.
"""
n = len(ys)
slopes = [0.0] * n
for k in range(1, n - 1):
h0, h1 = ts[k] - ts[k - 1], ts[k + 1] - ts[k]
if h0 <= 0 or h1 <= 0:
continue
d0, d1 = (ys[k] - ys[k - 1]) / h0, (ys[k + 1] - ys[k]) / h1
if d0 * d1 > 0:
slopes[k] = 3.0 * (h0 + h1) / ((2.0 * h1 + h0) / d0 + (h1 + 2.0 * h0) / d1)
return slopes
def camera_at(keys, t: float) -> dict:
"""Where the camera is at moment t (0..1): through every key, gently at the ends."""
if not keys:
return dict(HOME)
first, last = keys[0], keys[-1]
if len(keys) == 1 or t <= first["t"]:
return {name: first[name] for name in FIELDS}
if t >= last["t"]:
return {name: last[name] for name in FIELDS}
ts = [k["t"] for k in keys]
for i in range(len(keys) - 1):
a, b = keys[i], keys[i + 1]
if a["t"] <= t <= b["t"]:
h = max(1e-6, b["t"] - a["t"])
s = (t - a["t"]) / h
h00, h10, h01, h11 = 2 * s**3 - 3 * s**2 + 1, s**3 - 2 * s**2 + s, -2 * s**3 + 3 * s**2, s**3 - s**2
out = {}
for name in FIELDS:
m = _key_slopes(ts, [k[name] for k in keys])
out[name] = h00 * a[name] + h10 * h * m[i] + h01 * b[name] + h11 * h * m[i + 1]
return out
return {name: last[name] for name in FIELDS}
def frame_moments(count: int) -> list[float]:
"""The moment (0..1) of each frame. One frame sits at the start."""
if count <= 1:
return [0.0] * max(0, count)
return [i / (count - 1) for i in range(count)]
def rotation(az: float, el: float) -> list[list[float]]:
"""R = Ry(-az) . Rx(-el): az > 0 swings the camera right, el > 0 lifts it."""
th, ph = math.radians(-az), math.radians(-el)
ct, st, cp, sp = math.cos(th), math.sin(th), math.cos(ph), math.sin(ph)
return [[ct, st * sp, st * cp], [0.0, cp, -sp], [-st, ct * sp, ct * cp]]
def _mv(R, v):
return [sum(R[r][c] * v[c] for c in range(3)) for r in range(3)]
def pose(cam: dict, pivot) -> tuple[list[list[float]], list[float]]:
"""The moved camera's rotation R and centre C, for a key and a pivot point."""
R = rotation(cam["az"], cam["el"])
d = math.sqrt(sum(x * x for x in pivot))
Rp = _mv(R, pivot)
b = _mv(R, [cam.get("px", 0.0) * d, cam.get("py", 0.0) * d, -cam["back"] * d])
return R, [pivot[i] - Rp[i] + b[i] for i in range(3)]
def past_lora_range(cam: dict) -> bool:
return abs(cam["az"]) > LORA_RANGE["az"] or cam["el"] > LORA_RANGE["el_up"] or cam["el"] < LORA_RANGE["el_down"]
def range_report(keys) -> str:
"""One plain line when a key goes past where the LoRA was trained, else empty."""
over = [k for k in keys if past_lora_range(k)]
if not over:
return ""
when = ", ".join(f"{k['t']:.2f}" for k in over[:4])
return (
f"Camera Move: {len(over)} key(s) past the LoRA's range (at {when} of the clip). "
"It learned up to 45 degrees of turn, 30 up and 20 down; more may leave pink."
)
def scene_frame_indices(count: int, most: int = SCENE_FRAMES) -> list[int]:
"""Which frames the face gets to look at: evenly spread, first and last included."""
if count <= 0:
return []
if count <= most:
return list(range(count))
return sorted({round(i * (count - 1) / (most - 1)) for i in range(most)})
def scene_size(width: int, height: int, long_side: int = SCENE_LONG_SIDE) -> tuple[int, int]:
scale = min(1.0, long_side / max(width, height))
return max(1, round(width * scale)), max(1, round(height * scale))
def pack_depth(z, valid, zmin: float, zmax: float):
"""Depth as two bytes a pixel (red high, green low) on a log scale; 0 is no depth.
z and valid are numpy arrays. Returns an (H, W, 3) uint8 array.
"""
import numpy as np
lo, hi = math.log(max(zmin, 1e-6)), math.log(max(zmax, zmin * 1.0001, 1e-6))
q = np.zeros(z.shape, dtype=np.uint32)
if valid.any():
zz = np.clip(z[valid], math.exp(lo), math.exp(hi))
q[valid] = 1 + np.round((np.log(zz) - lo) / max(1e-9, hi - lo) * 65534).astype(np.uint32)
out = np.zeros(z.shape + (3,), dtype=np.uint8)
out[..., 0] = (q >> 8) & 255
out[..., 1] = q & 255
return out
def unpack_depth(rgb, zmin: float, zmax: float, far: float):
"""The face's reading of pack_depth, for tests."""
import numpy as np
q = rgb[..., 0].astype(np.uint32) * 256 + rgb[..., 1].astype(np.uint32)
lo, hi = math.log(zmin), math.log(zmax)
z = np.exp(lo + (q.astype(np.float64) - 1) / 65534 * (hi - lo))
return np.where(q == 0, far, z), q > 0
# -- the warp (torch) ---------------------------------------------------------
def pixel_intrinsics(K, width: int, height: int):
"""MoGe's lens as pixels; it may come normalised to the picture size."""
K = K.clone().float()
if K[0, 2] < 4:
K[0, :] *= width
K[1, :] *= height
return K
def steady_depth(z, valid, image, radius: int = 4, colour_tol: float = 0.05):
"""Calm the depth of things that hold still.
The depth model looks at every frame afresh, so a wall that never moves
still gets a slightly different depth each frame, and the moved camera
shakes. A spot whose colour stays the same a few frames either side has
its depth averaged over those frames. Anything that changes in that
window, such as a person moving, keeps its own depth.
"""
import torch
count = z.shape[0]
if count < 2 or radius < 1:
return z
logz = torch.where(valid, z.clamp(min=1e-6).log(), torch.zeros_like(z))
out = z.clone()
inv = 1.0 / (2.0 * colour_tol * colour_tol)
for i in range(count):
lo, hi = max(0, i - radius), min(count, i + radius + 1)
change = ((image[lo:hi] - image[i]) ** 2).sum(-1).amax(0)
held = valid[lo:hi].all(0)
still = torch.exp(-change * inv) * held
mean = logz[lo:hi].sum(0) / (hi - lo)
calm = (still * mean + (1 - still) * logz[i]).exp()
out[i] = torch.where(valid[i] & held, calm, z[i])
return out
def prepare_geometry(geometry: dict, device=None):
"""Points, colours and lens from a MOGE_GEOMETRY, with every frame held to one depth scale.
The depth model's scale and lens wobble from frame to frame, which shakes
the moved picture. Each frame is scaled so its median depth follows a
smoothed line through all of them, every frame gets the clip's median
lens, and the depth of whatever holds still is calmed (steady_depth).
"""
import torch
import torch.nn.functional as F
points = geometry.get("points")
intrinsics = geometry.get("intrinsics")
if points is None or intrinsics is None:
raise ValueError(
"Camera Move needs depth from MoGe Inference with its camera lens. "
"Connect MoGe Inference's geometry, not a panorama."
)
points = points.float()
count, height, width, _ = points.shape
image = geometry["image"][..., :3].float()
if image.shape[1:3] != (height, width):
image = F.interpolate(image.movedim(-1, 1), size=(height, width), mode="bilinear", align_corners=False).movedim(1, -1)
z = points[..., 2]
valid = torch.isfinite(points).all(-1) & (z > 0)
mask = geometry.get("mask")
if mask is not None:
valid &= mask.bool().to(valid.device)
z = torch.where(valid, z, torch.zeros_like(z))
meds = torch.stack([z[i][valid[i]].median() if valid[i].any() else torch.tensor(1.0) for i in range(count)])
win = min(15, count) | 1
padded = F.pad(meds.log()[None, None], (win // 2, win // 2), mode="replicate")
smooth = F.avg_pool1d(padded, win, 1)[0, 0]
z = z * (smooth - meds.log()).exp()[:, None, None]
if device is not None:
z = steady_depth(z.to(device), valid.to(device), image.to(device)).to(z.device)
else:
z = steady_depth(z, valid, image)
# The depth model guesses the lens again for every frame and the guess
# wobbles a few percent, which shakes the warp. One lens for the clip.
K = torch.stack([pixel_intrinsics(intrinsics[i], width, height) for i in range(count)]).median(dim=0).values
Ks = K[None].expand(count, 3, 3).clone()
v0 = valid[0]
far = float(torch.quantile(z[0][v0][::7], 0.98)) * 2.5 if v0.any() else float(meds.median()) * 10.0
return {"z": z, "valid": valid, "image": image, "K": Ks, "far": far, "count": count, "width": width, "height": height}
def unproject(K, x: float, y: float, depth: float) -> list[float]:
fx, fy, cx, cy = float(K[0, 0]), float(K[1, 1]), float(K[0, 2]), float(K[1, 2])
return [(x - cx) / fx * depth, (y - cy) / fy * depth, depth]
def auto_pivot(scene) -> list[float]:
"""What sits in the middle of the first frame: the nearest fifth of the centre box."""
import torch
z0, v0, H, W = scene["z"][0], scene["valid"][0], scene["height"], scene["width"]
box = (slice(int(H * 0.35), int(H * 0.65)), slice(int(W * 0.4), int(W * 0.6)))
vb = v0[box]
if vb.any():
depth = float(torch.quantile(z0[box][vb].float(), 0.2))
elif v0.any():
depth = float(z0[v0].median())
else:
depth = 1.0
return [0.0, 0.0, min(depth, PIVOT_CAP)]
def resolve_pivot(scene, pivot) -> list[float]:
"""The 3D spot the camera circles: the tapped one, else the automatic one."""
if not pivot:
return auto_pivot(scene)
count, H, W = scene["count"], scene["height"], scene["width"]
i = min(count - 1, max(0, round(pivot["t"] * (count - 1))))
x = min(W - 1, max(0, int(pivot["u"] * W)))
y = min(H - 1, max(0, int(pivot["v"] * H)))
z, valid = scene["z"][i], scene["valid"][i]
for r in range(0, 4):
y0, y1, x0, x1 = max(0, y - r), min(H, y + r + 1), max(0, x - r), min(W, x + r + 1)
patch = valid[y0:y1, x0:x1]
if patch.any():
depth = float(z[y0:y1, x0:x1][patch].median())
return unproject(scene["K"][i], x + 0.5, y + 0.5, depth)
return auto_pivot(scene)
def _fill_cracks(img, filled, passes=2):
"""Fill one-pixel gaps that are mostly surrounded by picture."""
import torch
import torch.nn.functional as F
k = torch.ones(1, 1, 3, 3, device=img.device)
for _ in range(passes):
f = filled.float()[None, None]
cnt = F.conv2d(f, k, padding=1)[0, 0]
s = F.conv2d((img * filled.float())[:, None], k, padding=1)[:, 0]
take = (~filled) & (cnt >= 5)
if not take.any():
break
img = torch.where(take[None], s / cnt.clamp(min=1), img)
filled = filled | take
return img, filled
def warp_frame(scene, i: int, cam: dict, pivot, device):
"""One frame seen from the moved camera. Returns (colour (H,W,3), hole (H,W) bool) on the CPU."""
import torch
H, W = scene["height"], scene["width"]
K = scene["K"][i].to(device)
Kinv = torch.linalg.inv(K)
valid = scene["valid"][i].to(device)
zi = torch.where(valid, scene["z"][i].to(device), torch.full((H, W), scene["far"], device=device))
col = scene["image"][i].to(device)
sub = torch.tensor([[-0.25, -0.25], [0.25, -0.25], [-0.25, 0.25], [0.25, 0.25]], device=device)
vv, uu = torch.meshgrid(torch.arange(H, device=device, dtype=torch.float32), torch.arange(W, device=device, dtype=torch.float32), indexing="ij")
u = (uu[None] + 0.5 + sub[:, 0, None, None]).reshape(-1)
v = (vv[None] + 0.5 + sub[:, 1, None, None]).reshape(-1)
zz = zi[None].expand(4, H, W).reshape(-1)
cc = col[None].expand(4, H, W, 3).reshape(-1, 3)
P = (torch.stack([u, v, torch.ones_like(u)], -1) @ Kinv.T) * zz[:, None]
R_list, C_list = pose(cam, pivot)
R = torch.tensor(R_list, dtype=torch.float32, device=device)
C = torch.tensor(C_list, dtype=torch.float32, device=device)
Pc = (P - C) @ R # R^T (P - C), row vectors
zc = Pc[:, 2]
ok = zc > 1e-3
uv = Pc[:, :2] / zc.clamp(min=1e-3)[:, None]
up = K[0, 0] * uv[:, 0] + K[0, 2]
vp = K[1, 1] * uv[:, 1] + K[1, 2]
ui, vi = up.floor().long(), vp.floor().long()
ok &= (ui >= 0) & (ui < W) & (vi >= 0) & (vi < H)
lin, zc, cc = vi[ok] * W + ui[ok], zc[ok], cc[ok]
zbuf = torch.full((H * W,), float("inf"), device=device)
zbuf.scatter_reduce_(0, lin, zc, reduce="amin", include_self=True)
win = zc <= zbuf[lin] * (1 + 1e-5)
magenta = torch.tensor(MAGENTA, device=device).view(3, 1, 1)
frame = magenta.expand(3, H, W).clone()
frame.view(3, -1)[:, lin[win]] = cc[win].T
filled = torch.isfinite(zbuf).view(H, W)
frame, filled = _fill_cracks(frame, filled)
frame = torch.where(filled[None], frame, magenta)
return frame.movedim(0, -1).clamp(0, 1).cpu(), (~filled).cpu()
+51 -2
View File
@@ -102,6 +102,33 @@ ONE_GOES = "the original person"
APPEARANCE_LINE = re.compile(r'^\s*@([1-4])\s+looks like\s+([^"\n]+?)\s*$', re.I)
# Hair and face only, no clothes, expression or background. When the line names clothes that differ from what the
# person in the clip wears, H3 tends to open on the picture's own backdrop before cutting to the clip.
DESCRIBE_PERSON = (
"Describe only the hair and face of the person in this picture, in one short line for a video prompt: hair colour "
"and style, and glasses, a beard or other face details you can see. No clothes, no expression, no background. "
"Start with 'a man' or 'a woman'."
)
def add_looks(text, jobs, describe):
"""A swap's script with an @N looks like line for each new person who has none, written by describe(slot), a
small picture-reading model shown that person's picture. Without such a line H3 kept the original person for the
first seconds (2026-11-07), so a connected model writes it when the user didn't."""
typed = {int(match.group(1)) for match in map(APPEARANCE_LINE.match, text.splitlines()) if match}
for slot in sorted(jobs):
if jobs[slot] not in (PERSON, SHEET) or slot in typed:
continue
words = next((line for line in str(describe(slot) or "").splitlines() if line.strip()), "")
words = re.sub(r'["\u201c\u201d]', "", words).strip(" .,;:")
words = re.sub(r"^(?:the\s+)?(?:person|subject)\s+(?:in\s+the\s+picture\s+)?(?:is|looks\s+like)\s+", "", words, flags=re.I)
# "and no visible beard": naming what isn't there can put it in the picture.
words = re.sub(r",?\s*(?:and\s+|with\s+)*(?:no|without)\s+(?:visible\s+)?[^,.;]*", "", words, flags=re.I).strip(" .,;:")
if words:
text = f"{text.rstrip()}\n@{slot} looks like {words[:1].lower() + words[1:]}"
return text
def swap_appearances(text, jobs, video):
"""An optional separate @1 looks like line describes a replacement, not who goes.
@@ -143,6 +170,21 @@ REPLACES = re.compile(
r"|swaps?\s+(?:in\s+)?for|goes\s+in\s+for|stands?\s+in\s+for)\s+",
re.I,
)
# A plain sentence: "Replace the man in red with @1", "swap the guy for @1", "make the dancer look like @1",
# "swap @1 in for the singer". The words caught are who goes.
ASKS = re.compile(
r"^\s*(?:please\s+)?(?:replace|swap(?:\s+out)?|switch(?:\s+out)?|change|turn|make)\s+(?!@\d)(.+?)\s+"
r"(?:with|for|to|into|by|look\s+like|become)\s+@\d\b"
r"|^\s*(?:please\s+)?(?:swap|put|place|drop|switch)\s+@\d\s+(?:in\s+)?(?:with|for|over|on|onto|in\s+place\s+of)\s+(.+)",
re.I,
)
# "the cashier is now @1", "the man on the left becomes @1".
BECOMES = re.compile(
r"^\s*(.+?)\s+(?:is\s+now|becomes|turns\s+into|is\s+replaced\s+(?:by|with)|gets\s+replaced\s+(?:by|with)"
r"|is\s+swapped\s+(?:for|with|to)|gets\s+swapped\s+(?:for|with)|->|\u2192)\s*@\d\b",
re.I,
)
IN_THE_CLIP = re.compile(r"\s+(?:in|from|of)\s+the\s+(?:clip|video|original(?:\s+(?:clip|video))?)\b", re.I)
# "Elon is @1," names the new person, not the one who goes.
# "the man on the left, Dario is" ends where the next person's name starts.
NAMES_NEXT = re.compile(r",?\s*(?:[A-Z][\w'.-]*\s+)+(?:is|as|plays)\s*$")
@@ -189,9 +231,16 @@ REVOICE_KEPT = (
def who_goes_in(text):
"""Who a typed swap line says goes, without the rest of the line: "Elon is @1, he replaces the man at the
counter" is the man at the counter."""
counter" is the man at the counter, and so is "Replace the man at the counter with @1."."""
replaces = REPLACES.search(text)
text = text[replaces.end() :] if replaces else NAMES_THE_NEW.sub("", text)
plain = None if replaces else next((match for match in (ASKS.search(text), BECOMES.search(text)) if match), None)
if replaces:
text = text[replaces.end() :]
elif plain:
text = next(group for group in plain.groups() if group)
else:
text = NAMES_THE_NEW.sub("", text)
text = IN_THE_CLIP.sub("", text)
text = re.split(r"[.;!?]", text.strip(" .;!?"), maxsplit=1)[0]
text = NAMES_NEXT.sub("", text)
text = SAYS_WHO.sub("", TOKEN.sub("", text).strip(" .,;")).strip(" .,;")
+12 -2
View File
@@ -340,14 +340,16 @@ def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="E
cast.clip_sound = True
# With no words typed, the clip's own speech is still heard, so the new person says it as the video does.
cast.typed_lines = any(moment.speech for moment in scene.moments)
cast.speaks_as_video = bool(speech) and not cast.typed_lines
# Only a voice makes them speak: a train or a beat is loud in the voice band too, but has no clear pitch,
# and the line made the new person mouth along to it (subway clip, 2026-11-07: 0.0 s of clear voice).
cast.speaks_as_video = heard_a_voice(speech) and not cast.typed_lines
time_from_sound(scene.moments, speech)
scene.moments = [moment for moment in scene.moments if not (moment.voice or moment.other)]
if not scene.moments:
scene.moments.append(Moment(ACTION_GOES_ON if video == KEEP else ""))
if voice and not any(moment.speech for moment in scene.moments):
raise ValueError(VOICE_NO_WORDS)
if ask is not None:
if ask is not None and video != SWAP: # A swap keeps the clip's own action; its model only writes the looks.
add_moments(scene, seconds, ask, jobs)
add_sound(scene, ask, jobs)
audio += soundtrack and video == KEEP
@@ -438,6 +440,14 @@ LONGEST_LINE = 12 # Stretches of speech one line can take.
GAP_WEIGHT, INNER_GAP_WEIGHT, NOISE_WEIGHT, PITCH_WEIGHT = 1.0, 1.0, 1.5, 6.0
SPEAKING_VOICE = 0.5 # Seconds of clear voice before the clip counts as someone talking.
def heard_a_voice(speech):
"""Whether find_speech heard someone talk: enough clear voice, not just loud sound in the voice band."""
return sum(stretch[3] for stretch in speech or ()) >= SPEAKING_VOICE
def find_speech(samples, rate):
"""[(start, end, pitch, voiced), ...]: the stretches of a mono track loud
enough in the voice band to be words, in seconds, with each stretch's
+8 -3
View File
@@ -727,11 +727,16 @@ def before_the_action(number, text, cast, wearers):
for _, who, new in cast.swaps:
picture = next((p for p in cast.swappers if subject_label(p.subject) == new), None)
appearance = cast.appearances.get(picture.slot) if picture else None
if appearance:
if picture:
# Every swap gets the firm wording. Without it H3 kept the original person for the first seconds,
# or for the whole clip at Fast preview; with it the new
# person is there from the first frame. An @1 looks like line only adds its own words.
looks = (f"has the defined appearance from {picture_label(picture.picture)}: {appearance}" if appearance
else f"has the face, hair and clothes of the person in {picture_label(picture.picture)}")
notes.append(
f"{new} occupies the starting position of {who} and follows the same route, movements, timing and pace. "
f"{new} has the defined appearance from {picture_label(picture.picture)}: {appearance}. "
f"The replacement stays present throughout, ending at the original performance's final position. "
f"{new} {looks}. "
f"The replacement stays present throughout, from the first frame, ending at the original performance's final position. "
f"Do not show {who} in that role at any time; there is no transition or morph into the replacement."
)
else:
+170
View File
@@ -0,0 +1,170 @@
"""Camera Move 🆎: move the camera of a clip, steered with keys on the node."""
from __future__ import annotations
from pathlib import Path
from ._camera_move_helpers import (
camera_at,
default_camera_text,
frame_moments,
pack_depth,
parse_camera,
prepare_geometry,
range_report,
resolve_pivot,
auto_pivot,
scene_frame_indices,
scene_size,
warp_frame,
)
UI_KEY = "ausboss_camera_scene"
def _device():
try:
import comfy.model_management as mm
return mm.get_torch_device()
except Exception:
import torch
return torch.device("cuda" if torch.cuda.is_available() else "cpu")
class AusBossCameraMove:
CATEGORY = "🆎 AusBoss/Video"
DESCRIPTION = (
"Moves the camera of your clip. Steer it on the node: drag the picture "
"to turn, pinch or scroll to pull back, and set keys on the time bar. "
"Places the old camera never saw come out pink, ready for the "
"CrossView-Warp LoRA to paint in. Needs depth from MoGe Inference."
)
SEARCH_ALIASES = [
"camera move",
"move camera",
"orbit camera",
"camera orbit video",
"crossview warp",
"novel view",
"dolly",
"camera keyframes",
"ausboss",
]
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"moge_geometry": (
"MOGE_GEOMETRY",
{"tooltip": "Depth of your clip from MoGe Inference. Every frame needs it."},
),
"camera": (
"STRING",
{
"default": default_camera_text(),
"multiline": False,
"tooltip": "The camera keys. Set them on the node's picture; you never type here.",
},
),
}
}
RETURN_TYPES = ("IMAGE", "MASK")
RETURN_NAMES = ("guide", "holes")
OUTPUT_TOOLTIPS = (
"Your clip seen from the moved camera. Pink is what the old camera never saw.",
"White where the guide is pink: the parts to paint in.",
)
OUTPUT_NODE = True
FUNCTION = "run"
def run(self, moge_geometry, camera):
import torch
parsed = parse_camera(camera)
if parsed["fallback"]:
print("[AusBoss] Camera Move: could not read the camera keys, using the default move.")
report = range_report(parsed["keys"])
if report:
print("[AusBoss] " + report)
device = _device()
scene = prepare_geometry(moge_geometry, device)
pivot = resolve_pivot(scene, parsed["pivot"])
try:
import comfy.utils
bar = comfy.utils.ProgressBar(scene["count"])
except Exception:
bar = None
frames, holes = [], []
with torch.no_grad():
for i, t in enumerate(frame_moments(scene["count"])):
frame, hole = warp_frame(scene, i, camera_at(parsed["keys"], t), pivot, device)
frames.append(frame)
holes.append(hole)
if bar is not None:
bar.update(1)
guide = torch.stack(frames)
mask = torch.stack(holes).float()
info = self._scene(scene, pivot)
info["pivot_from"] = parsed["pivot"]
return {"ui": {UI_KEY: [info]}, "result": (guide, mask)}
# -- what the node's face looks at ----------------------------------------
def _scene(self, scene, pivot):
"""A few frames, small, with their depth, written to ComfyUI's temp folder."""
import folder_paths
import numpy as np
from PIL import Image
W, H, count = scene["width"], scene["height"], scene["count"]
w, h = scene_size(W, H)
ys = np.clip(((np.arange(h) + 0.5) * H / h).astype(int), 0, H - 1)
xs = np.clip(((np.arange(w) + 0.5) * W / w).astype(int), 0, W - 1)
picks = scene_frame_indices(count)
small = []
for i in picks:
z = scene["z"][i].cpu().numpy()[ys][:, xs]
valid = scene["valid"][i].cpu().numpy()[ys][:, xs]
small.append((i, z, valid))
every = np.concatenate([z[v] for _, z, v in small if v.any()]) if any(v.any() for _, _, v in small) else np.array([1.0])
zmin, zmax = float(every.min()), float(max(every.max(), every.min() * 1.001))
folder, name, counter, subfolder, _ = folder_paths.get_save_image_path(
"ausboss_camera_move", folder_paths.get_temp_directory(), w, h
)
frames = []
for j, (i, z, valid) in enumerate(small):
colour = (scene["image"][i].cpu().numpy().clip(0, 1) * 255).astype(np.uint8)
stem = f"{name}_{counter:05}_{j:02}"
Image.fromarray(colour).resize((w, h), Image.LANCZOS).save(Path(folder) / f"{stem}_c.jpg", quality=88)
Image.fromarray(pack_depth(z, valid, zmin, zmax)).save(Path(folder) / f"{stem}_d.png", compress_level=1)
K = scene["K"][i]
frames.append({
"i": i,
"t": i / (count - 1) if count > 1 else 0.0,
"color": {"filename": f"{stem}_c.jpg", "subfolder": subfolder, "type": "temp"},
"depth": {"filename": f"{stem}_d.png", "subfolder": subfolder, "type": "temp"},
"fx": float(K[0, 0]) * w / W,
"fy": float(K[1, 1]) * h / H,
"cx": float(K[0, 2]) * w / W,
"cy": float(K[1, 2]) * h / H,
})
return {
"w": w,
"h": h,
"count": count,
"zmin": zmin,
"zmax": zmax,
"far": scene["far"],
"pivot": [float(x) for x in pivot],
"pivot_auto": [float(x) for x in auto_pivot(scene)],
"frames": frames,
}
NODE_CLASS_MAPPINGS = {"AUSBOSS_NODES_CameraMove": AusBossCameraMove}
NODE_DISPLAY_NAME_MAPPINGS = {"AUSBOSS_NODES_CameraMove": "Camera Move 🆎"}
+6 -2
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
import sys
from ._h3_cast_helpers import EDIT, JOBS, KEEP, KEEPS_ITS_LENGTH, PERSON, PLACE, START, SWAP, THING, VIDEO_JOBS
from ._h3_cast_helpers import DESCRIBE_PERSON, add_looks, EDIT, JOBS, KEEP, KEEPS_ITS_LENGTH, PERSON, PLACE, START, SWAP, THING, VIDEO_JOBS
from ._h3_prompt_helpers import find_beats, find_speech, split_sections, write_prompt
from ._h3_script_helpers import CAMERAS, LOOKS, script_report
from ._route_errors import safe_route_error
@@ -177,7 +177,8 @@ class AusBossH3PromptWriter:
{
"tooltip": (
"Optional: a small text model such as Qwen3-VL 8B. It adds "
"moments after a one-line idea and matches the sound to the place."
"moments after a one-line idea and matches the sound to the place. "
"In a swap it writes what the new person looks like, from their picture."
),
},
),
@@ -335,6 +336,9 @@ class AusBossH3PromptWriter:
kinds = {1: job_1, 2: job_2, 3: job_3, 4: job_4}
jobs = {slot: kinds[slot] for slot in given}
video_job = "" if video is None else video_job
if clip is not None and video_job == SWAP:
# The model on clip writes each new person's @N looks like line from their picture, when none is typed.
script = add_looks(script, jobs, lambda slot: model_asker(clip, given[slot])(DESCRIBE_PERSON, 60))
frames = clip_frames(seconds)
if video_job in KEEPS_ITS_LENGTH:
frames = video_frames(video.shape[0])
+35 -9
View File
@@ -212,19 +212,19 @@ def _server():
return None
def _tell(node_id, note: dict, end: bool) -> bool:
def _tell(node_id, note: dict, end: bool, stopped: bool = False) -> bool:
"""Tell the node's face what a look has found, over ComfyUI's own channel
to its page. It goes to the page that asked for the run. True when it
was sent; a failure never stops the look."""
was sent; a failure never stops the look. `stopped`: the run stops at
this node, and the face is only shown the picture that came in."""
server = _server()
if server is None or node_id is None:
return False
event = {"node": str(node_id), "note": note, "end": bool(end)}
if stopped:
event["stopped"] = True
try:
server.send_sync(
LOOK_EVENT,
{"node": str(node_id), "note": note, "end": bool(end)},
getattr(server, "client_id", None),
)
server.send_sync(LOOK_EVENT, event, getattr(server, "client_id", None))
except Exception: # noqa: BLE001
return False
return True
@@ -720,10 +720,10 @@ class AusBossMaskByTouch:
"fill_holes": (
"BOOLEAN",
{
"default": True,
"default": False,
"tooltip": (
"Closes pinholes inside the picked area. Larger openings "
"stay open, like a mug's handle."
"stay open, like a mug's handle. Off keeps every hole you pick."
),
},
),
@@ -1105,6 +1105,28 @@ class AusBossMaskByTouch:
entry = self._keep(("picture", key), self._write(picture, "picture", "jpg", format="JPEG", quality=90))
return entry
def _show_picture(self, session, image, key: str, unique_id) -> bool:
"""Send the face the picture that came in, with nothing picked on it,
for a run that stops at this node. A failure never changes why the
run stops."""
if unique_id is None or _server() is None:
return False
try:
height, width = int(image.shape[1]), int(image.shape[2])
small_width, small_height, scale = small_size(width, height)
picture = self._picture_file(image, key, small_width, small_height)
if picture is None:
return False
note = {
"key": key, "width": width, "height": height, "scale": scale, "stamp": "",
"picture": picture, "selection": None, "items": [], "share": 0.0,
"pieces": self._pieces_note(_MEMORY.get_pieces((session.token, session.reader, key))),
"fine": None, "areas": None,
}
except Exception: # noqa: BLE001
return False
return _tell(unique_id, note, True, stopped=True)
def _pieces_note(self, pieces):
if pieces is None:
return None
@@ -1239,6 +1261,10 @@ class AusBossMaskByTouch:
raw = torch.stack(selections)
soft = raw.dtype != torch.bool
if not bool(raw.any()) and not asked and str(if_nothing) != IF_NOTHING[1]:
# The run stops here, but the face is shown the picture that
# came in first: a picture that changed upstream is there to
# pick from at once, without unplugging it.
self._show_picture(session, image, key, unique_id)
# Picks the face may still show, made before the picture
# changed: say so, not that nothing is picked.
raise RuntimeError(OTHER_PICTURE if elsewhere else NOTHING_PICKED)
+1 -1
View File
@@ -3,7 +3,7 @@ name = "ausboss-nodes"
# Manager/registry search matches against this text, so it names the actual
# nodes instead of describing the collection abstractly.
description = "ComfyUI-AusBoss (AusBoss nodes): image and video nodes with compact controls: Image and Video Crop + Rotate + Pad (frame or clip), Load Image + Pad, Image Folder, Image Resize, Align Image, Image Size, Color Match, Image Compare A/B, Realign to Source (experimental), Save Image (PNG, WebP/JXL, captions), Load Video, Load Audio, Save Video, Select Frame, Frame Interpolate, Mask Refine, Mask by Name and Mask by Touch (type a name or click a thing, get its mask), LaMa Inpaint, Crop For Inpaint, Stitch Inpaint, Tiled Upscale and Tiled Upscale Stitch, LoRA Loader, Krea 2 Encode and Outpaint Model Patch, Latent Size, Layout Boxes, Seed, Select Every Nth, Split Batch, Merge Batches, Text, H3 Prompt Writer, Integer, Float, Math Expression, Show Text, Free Memory, Workflow Note, Workflow Note Tile, Callout, Callout Tile, Workflow Switches, Run Timer, and Step Preview (stop bad runs). Example workflows for Qwen Image 2.1, Krea 2, Klein 9B and LTX 2.3."
version = "2.21.1"
version = "2.21.2"
# comfy-cli serializes this to JSON verbatim, so `file` publishes the literal
# string {"file": "LICENSE"} to the registry listing. `text` puts a readable
# name there instead. Both forms are supported; the LICENSE file is unchanged.
+4 -1
View File
@@ -94,7 +94,10 @@ for (const type of types) {
const seen = await evalJs(`(async () => {
const { app } = window;
app.canvas.linkConnector?.reset?.();
app.graph.clear(); app.graph.last_node_id += 50;
// clear() resets the counter, but frontend preview caches can outlive
// the graph. Reserve fresh ids before clearing so probes stay isolated.
const nextNodeId = app.graph.last_node_id + 50;
app.graph.clear(); app.graph.last_node_id = nextNodeId;
let node;
try { node = window.LiteGraph.createNode(${JSON.stringify(type)}); } catch (error) { return { error: "createNode threw: " + (error?.message ?? error) }; }
if (!node) return { error: "createNode gave nothing" };
+1
View File
@@ -52,6 +52,7 @@ PUBLIC_NODE_IDS = {
"AUSBOSS_NODES_AlignImage",
"AUSBOSS_NODES_Callout",
"AUSBOSS_NODES_CalloutTile",
"AUSBOSS_NODES_CameraMove",
"AUSBOSS_NODES_ColorMatch",
"AUSBOSS_NODES_Compare",
"AUSBOSS_NODES_CropForInpaint",
+94
View File
@@ -0,0 +1,94 @@
// Camera Move's face maths must match the node's (nodes/_camera_move_helpers.py).
import assert from "node:assert/strict";
import test from "node:test";
import {
DEFAULT_KEYS, K0, LIM, camAt, cameraText, keyAt, parseCamera, pastLoraRange, pivotFromPixel, pose, setKeyAt, toCam, unpackDepth,
} from "../js/camera_move/camera_math.mjs";
const close = (a, b, eps = 1e-9) => assert.ok(Math.abs(a - b) < eps, `${a} != ${b}`);
test("pose matches the node's numbers", () => {
// From nodes/_camera_move_helpers.pose with the same key and pivot.
const { R, C } = pose({ az: 33, el: -12, back: 0.4, px: 0.2, py: -0.1 }, [0.3, -0.2, 2.0]);
const R_py = [0.838670567945424, -0.11323682265532711, -0.532737365365924, 0, 0.9781476007338057, -0.20791169081775934,
0.5446390350150271, 0.17436941582062362, 0.8203436038418747];
R.forEach((v, i) => close(v, R_py[i]));
[1.8881747404128695, 0.3816804134905334, -0.250127706778845].forEach((v, i) => close(C[i], v));
});
test("easing between keys matches the node's", () => {
const { keys } = parseCamera({ keys: [{ t: 0 }, { t: 0.4, az: 20, el: 10, back: 0.5 }, { t: 1, az: -10, px: 0.3 }] });
const c = camAt(keys, 0.3);
close(c.az, 16.875); close(c.el, 8.4375); close(c.back, 0.421875);
});
test("the camera keeps moving through a key, the same as the node", () => {
const { keys } = parseCamera({ keys: [{ t: 0 }, { t: 0.3, az: 12 }, { t: 0.6, az: 25, el: 5 }, { t: 1, az: 40, el: 5, back: 0.4 }] });
// From nodes/_camera_move_helpers.camera_at with the same keys.
close(camAt(keys, 0.45).az, 18.547068965517244); close(camAt(keys, 0.8).az, 34.51724137931035);
close(camAt(keys, 0.8).back, 0.2);
const steady = parseCamera({ keys: [{ t: 0 }, { t: 0.5, az: 20 }, { t: 1, az: 40 }] }).keys;
assert.ok((camAt(steady, 0.51).az - camAt(steady, 0.49).az) / 0.02 > 35);
});
test("the pivot stays where it was on screen", () => {
const pivot = [0.3, -0.2, 2], d = Math.hypot(...pivot);
const cam = { az: 40, el: 15, back: 0.5, px: 0, py: 0 };
const { R, C } = pose(cam, pivot);
const q = toCam(R, C, pivot);
close(q[0], 0.3); close(q[1], -0.2); close(q[2], 2 + 0.5 * d);
});
test("unreadable camera text falls back to the default move", () => {
for (const text of ["", "nope", "[]", '{"keys":[]}', null]) {
const parsed = parseCamera(text);
assert.equal(parsed.fallback, true);
assert.equal(parsed.keys.length, DEFAULT_KEYS.length);
}
});
test("camera text round-trips, clamped and sorted", () => {
const parsed = parseCamera({ keys: [{ t: 0.9, az: 300 }, { t: 0.1, el: -90, back: 5 }], pivot: { u: 0.25, v: 2, t: 0.5 } });
assert.deepEqual(parsed.keys.map((k) => k.t), [0.1, 0.9]);
assert.equal(parsed.keys[1].az, LIM.az[1]);
assert.equal(parsed.keys[0].el, LIM.el[0]);
assert.deepEqual(parsed.pivot, { u: 0.25, v: 1, t: 0.5 });
const again = parseCamera(cameraText(parsed));
assert.deepEqual(again.keys, parsed.keys);
assert.deepEqual(again.pivot, parsed.pivot);
});
test("moving the camera at the time bar sets or adds a key", () => {
const keys = [{ t: 0, ...K0 }, { t: 1, ...K0 }];
setKeyAt(keys, 0.5, { az: 10 });
assert.equal(keys.length, 3);
assert.equal(keys[1].t, 0.5);
setKeyAt(keys, 0.505, { az: 99 });
assert.equal(keys.length, 3);
assert.equal(keys[1].az, 90);
assert.equal(keyAt(keys, 0.51), 1);
assert.equal(keyAt(keys, 0.6), -1);
});
test("the LoRA's range", () => {
assert.equal(pastLoraRange({ az: 45, el: 30 }), false);
assert.equal(pastLoraRange({ az: 46, el: 0 }), true);
assert.equal(pastLoraRange({ az: 0, el: -21 }), true);
});
test("the depth picture reads back as the node wrote it", () => {
// q = 1 is zmin, q = 65535 is zmax, 0 is sky.
const rgba = new Uint8ClampedArray([0, 1, 0, 255, 255, 255, 0, 255, 0, 0, 0, 255]);
const { z, valid } = unpackDepth(rgba, 3, 0.5, 40, 99);
close(z[0], 0.5, 1e-6); close(z[1], 40, 1e-4); assert.equal(z[2], 99);
assert.deepEqual([...valid], [1, 1, 0]);
});
test("a tapped pixel becomes a point in front of the camera", () => {
const W = 4, H = 2, z = new Float32Array([2, 2, 2, 2, 2, 2, 2, 2]), valid = new Uint8Array([0, 0, 0, 0, 0, 1, 0, 0]);
const frame = { W, H, z, valid, fx: 2, fy: 2, cx: 2, cy: 1 };
const p = pivotFromPixel(frame, 0.1, 0.9);
assert.ok(p);
close(p[2], 2);
assert.equal(pivotFromPixel({ ...frame, valid: new Uint8Array(8) }, 0.5, 0.5), null);
});
+1
View File
@@ -108,6 +108,7 @@ test("only the listed parts of the pack listen for the wheel", () => {
const js = join(dirname(fileURLToPath(import.meta.url)), "..", "js");
const listed = [
"callout/index.js wrap", // the text box while a note is being edited
"camera_move/face.mjs stage", // the shot: the wheel pulls the camera back or in
"canvas_passthrough/index.js window", // hands the wheel to the graph
"image_folder/index.js root", // the copy question, a window over the page
"image_folder/index.js root", // the folder list, a window over the page
+28
View File
@@ -0,0 +1,28 @@
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import test from "node:test";
import vm from "node:vm";
test("face smoke gives each probe a fresh node id after clearing the graph", () => {
const source = readFileSync(new URL("../scripts/dev/face_smoke.mjs", import.meta.url), "utf8");
// Run the browser setup used by every probe. ComfyUI clears the node-id
// counter with the graph, but output/preview caches can retain those ids.
const setup = source.match(/const seen = await evalJs\(`\(async \(\) => \{([\s\S]*?)\n let node;/)?.[1];
assert.ok(setup, "the browser probe setup must be exercised");
const previews = new Map();
const graph = { last_node_id: 0, clear() { this.last_node_id = 0; } };
let resets = 0;
const context = vm.createContext({ window: { app: {
graph, canvas: { linkConnector: { reset() { resets++; } } },
} } });
const ids = [];
for (let probe = 0; probe < 48; probe++) {
vm.runInContext(`(() => { ${setup} })()`, context);
const id = ++graph.last_node_id;
assert.equal(previews.has(id), false, "a later node must not inherit an earlier node's preview");
previews.set(id, `preview-${probe}`);
ids.push(id);
}
assert.equal(new Set(ids).size, 48);
assert.equal(resets, 48);
});
+17
View File
@@ -40,6 +40,23 @@
"optional": [],
"outputs": []
},
"AUSBOSS_NODES_CameraMove": {
"required": [
"moge_geometry",
"camera"
],
"optional": [],
"outputs": [
[
"IMAGE",
"guide"
],
[
"MASK",
"holes"
]
]
},
"AUSBOSS_NODES_ColorMatch": {
"required": [
"image",
+6 -1
View File
@@ -28,6 +28,8 @@ const JS_ROOT = join(dirname(fileURLToPath(import.meta.url)), "..", "js");
// A panel whose DOM is built by a shared module, so a second view can reuse
// it, is audited where it is built.
const SHARED_PANELS = { mask_by_touch: join("shared", "touch_stage.mjs") };
// A panel whose face lives in its own module beside index.js is audited with it.
const PANEL_PARTS = { camera_move: [join("camera_move", "face.mjs")] };
function domWidgetEntries() {
const entries = [];
@@ -38,7 +40,8 @@ function domWidgetEntries() {
} catch {
continue;
}
const source = readFileSync(path, "utf-8");
const source = [path, ...(PANEL_PARTS[name] ?? []).map((part) => join(JS_ROOT, part))]
.map((file) => readFileSync(file, "utf-8")).join("\n");
if (source.includes("addDOMWidget")) entries.push({ name, source });
}
return entries;
@@ -110,6 +113,7 @@ test("a panel that should follow the node's height never declares computeSize",
// is this a stage that should grow, or a fixed row?
const mustGrow = new Set([
"callout",
"camera_move",
"resolution",
"compare",
"image_crop_rotate_pad",
@@ -397,6 +401,7 @@ test("every panel root class carries border-box and an overflow clip", () => {
show_text: ".ausboss-show-text{",
step_preview: ".ausboss-steppv{",
callout: ".ausboss-callout{",
camera_move: ".acm-face{",
workflow_note: ".ausboss-note{",
seed: ".ausboss-seed{",
run_timer: ".ausboss-timer{",
+319
View File
@@ -0,0 +1,319 @@
"""Offline tests for Camera Move: keys, pose, depth packing and the warp."""
from __future__ import annotations
import json
import math
import sys
import tempfile
import types
import unittest
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
del sys.modules["nodes"]
from nodes._camera_move_helpers import (
DEFAULT_CAMERA,
LIMITS,
camera_at,
default_camera_text,
frame_moments,
pack_depth,
parse_camera,
past_lora_range,
pose,
range_report,
rotation,
scene_frame_indices,
scene_size,
unpack_depth,
)
try:
import numpy as np
import torch
except ImportError: # pragma: no cover - the warp tests need ComfyUI's Python
np = torch = None
def _apply(R, v):
return [sum(R[r][c] * v[c] for c in range(3)) for r in range(3)]
def _apply_t(R, v):
return [sum(R[c][r] * v[c] for c in range(3)) for r in range(3)]
class ParseCameraTests(unittest.TestCase):
def test_default_text_reads_back_as_the_default(self):
parsed = parse_camera(default_camera_text())
self.assertFalse(parsed["fallback"])
self.assertEqual([k["t"] for k in parsed["keys"]], [0.0, 1.0])
self.assertEqual(parsed["keys"][1]["az"], DEFAULT_CAMERA["keys"][1]["az"])
self.assertIsNone(parsed["pivot"])
def test_unreadable_text_falls_back_to_the_default_move(self):
for text in ("", "not json", "[]", '{"keys": []}', '{"keys": "x"}', None, 12):
parsed = parse_camera(text)
self.assertTrue(parsed["fallback"], text)
self.assertEqual(len(parsed["keys"]), 2)
def test_keys_are_clamped_sorted_and_merged(self):
text = json.dumps({"keys": [
{"t": 0.9, "az": 500, "el": -99, "back": 9},
{"t": 0.1, "az": "12", "px": -5},
{"t": 0.905, "az": 20},
{"t": 7, "el": float("nan")},
]})
keys = parse_camera(text)["keys"]
self.assertEqual([round(k["t"], 3) for k in keys], [0.1, 0.905, 1.0])
self.assertEqual(keys[0]["az"], 12.0)
self.assertEqual(keys[0]["px"], LIMITS["px"][0])
# The later key at the same moment wins.
self.assertEqual(keys[1]["az"], 20.0)
self.assertEqual(keys[2]["el"], 0.0)
for key in keys:
for name, (low, high) in LIMITS.items():
self.assertTrue(low <= key[name] <= high)
def test_pivot_is_kept_inside_the_picture(self):
parsed = parse_camera({"keys": [{"t": 0}], "pivot": {"u": 1.4, "v": -1, "t": 0.5}})
self.assertEqual(parsed["pivot"], {"u": 1.0, "v": 0.0, "t": 0.5})
self.assertIsNone(parse_camera({"keys": [{"t": 0}], "pivot": {"u": 0.2}})["pivot"])
class CameraAtTests(unittest.TestCase):
keys = [{"t": 0.0, "az": 0, "el": 0, "back": 0, "px": 0, "py": 0},
{"t": 0.5, "az": 40, "el": 10, "back": 1, "px": 0, "py": 0},
{"t": 1.0, "az": 40, "el": 10, "back": 1, "px": 0.5, "py": 0}]
def test_holds_before_the_first_and_after_the_last_key(self):
self.assertEqual(camera_at(self.keys, -1)["az"], 0)
self.assertEqual(camera_at(self.keys, 2)["px"], 0.5)
def test_eases_between_keys(self):
self.assertAlmostEqual(camera_at(self.keys, 0.25)["az"], 20.0)
self.assertAlmostEqual(camera_at(self.keys, 0.125)["az"], 40 * (0.25 ** 2 * 2.5))
self.assertEqual(camera_at(self.keys, 0.5)["az"], 40)
def test_one_key_holds_still(self):
self.assertEqual(camera_at(self.keys[1:2], 0.0)["back"], 1)
def test_keeps_moving_through_a_key(self):
keys = parse_camera({"keys": [{"t": 0}, {"t": 0.5, "az": 20}, {"t": 1, "az": 40}]})["keys"]
speed = (camera_at(keys, 0.51)["az"] - camera_at(keys, 0.49)["az"]) / 0.02
self.assertGreater(speed, 35) # the steady speed is 40 a clip; easing at each key gave about 0
self.assertAlmostEqual(camera_at(keys, 0.5)["az"], 20)
def test_never_runs_past_a_key(self):
keys = parse_camera({"keys": [{"t": 0}, {"t": 0.3, "az": 30}, {"t": 0.5, "az": 30}, {"t": 1, "az": -20}]})["keys"]
values = [camera_at(keys, i / 200)["az"] for i in range(201)]
self.assertLessEqual(max(values), 30 + 1e-9)
self.assertGreaterEqual(min(values), -20 - 1e-9)
self.assertTrue(all(abs(camera_at(keys, t)["az"] - 30) < 1e-9 for t in (0.3, 0.4, 0.5))) # two equal keys hold still
def test_matches_the_face(self):
keys = parse_camera({"keys": [{"t": 0}, {"t": 0.3, "az": 12}, {"t": 0.6, "az": 25, "el": 5},
{"t": 1, "az": 40, "el": 5, "back": 0.4}]})["keys"]
self.assertAlmostEqual(camera_at(keys, 0.45)["az"], 18.547068965517244) # tests/camera_move.test.mjs
self.assertAlmostEqual(camera_at(keys, 0.8)["az"], 34.51724137931035)
def test_frame_moments_run_from_start_to_end(self):
self.assertEqual(frame_moments(1), [0.0])
self.assertEqual(frame_moments(3), [0.0, 0.5, 1.0])
class PoseTests(unittest.TestCase):
pivot = [0.3, -0.2, 2.0]
def test_rotation_is_a_rotation(self):
R = rotation(33, -17)
for r in range(3):
for c in range(3):
dot = sum(R[k][r] * R[k][c] for k in range(3))
self.assertAlmostEqual(dot, 1.0 if r == c else 0.0)
def test_turning_right_moves_the_camera_right(self):
_, C = pose({"az": 30, "el": 0, "back": 0, "px": 0, "py": 0}, [0, 0, 2])
self.assertGreater(C[0], 0)
_, C = pose({"az": 0, "el": 20, "back": 0, "px": 0, "py": 0}, [0, 0, 2])
self.assertLess(C[1], 0) # y is down: rising is negative y
def test_home_is_the_original_camera(self):
R, C = pose({"az": 0, "el": 0, "back": 0, "px": 0, "py": 0}, self.pivot)
self.assertEqual([[round(x, 9) for x in row] for row in R], [[1, 0, 0], [0, 1, 0], [0, 0, 1]])
for x in C:
self.assertAlmostEqual(x, 0.0)
def test_the_pivot_stays_where_it_was_on_screen(self):
d = math.sqrt(sum(x * x for x in self.pivot))
for cam in ({"az": 40, "el": 15, "back": 0.5}, {"az": -70, "el": -25, "back": -0.4}):
cam = {**cam, "px": 0, "py": 0}
R, C = pose(cam, self.pivot)
seen = _apply_t(R, [self.pivot[i] - C[i] for i in range(3)])
# Where the original camera saw it, pushed back along the view by back * d.
self.assertAlmostEqual(seen[0], self.pivot[0])
self.assertAlmostEqual(seen[1], self.pivot[1])
self.assertAlmostEqual(seen[2], self.pivot[2] + cam["back"] * d)
def test_pan_slides_the_camera_in_its_own_frame(self):
d = math.sqrt(sum(x * x for x in self.pivot))
R, C0 = pose({"az": 25, "el": 5, "back": 0, "px": 0, "py": 0}, self.pivot)
_, C1 = pose({"az": 25, "el": 5, "back": 0, "px": 0.5, "py": 0}, self.pivot)
moved = _apply_t(R, [C1[i] - C0[i] for i in range(3)])
self.assertAlmostEqual(moved[0], 0.5 * d)
self.assertAlmostEqual(moved[1], 0.0)
self.assertAlmostEqual(moved[2], 0.0)
def test_lora_range(self):
self.assertFalse(past_lora_range({"az": 45, "el": 30}))
self.assertTrue(past_lora_range({"az": -46, "el": 0}))
self.assertTrue(past_lora_range({"az": 0, "el": -21}))
self.assertEqual(range_report(CameraAtTests.keys), "")
report = range_report([{"t": 0.5, "az": 60, "el": 0}])
self.assertIn("past the LoRA's range", report)
report.encode("ascii")
class SceneTests(unittest.TestCase):
def test_frames_for_the_face(self):
self.assertEqual(scene_frame_indices(0), [])
self.assertEqual(scene_frame_indices(5), [0, 1, 2, 3, 4])
picks = scene_frame_indices(121)
self.assertEqual((picks[0], picks[-1], len(picks)), (0, 120, 61))
self.assertEqual(picks[:3], [0, 2, 4])
def test_scene_size_keeps_the_shape(self):
self.assertEqual(scene_size(832, 480), (416, 240))
self.assertEqual(scene_size(300, 200), (300, 200))
@unittest.skipIf(np is None, "needs numpy")
def test_depth_packing_round_trips(self):
rng = np.random.default_rng(1)
z = rng.uniform(0.4, 60.0, (24, 32))
valid = rng.uniform(size=z.shape) > 0.2
packed = pack_depth(z, valid, float(z[valid].min()), float(z[valid].max()))
self.assertEqual(packed.dtype, np.uint8)
back, ok = unpack_depth(packed, float(z[valid].min()), float(z[valid].max()), 999.0)
self.assertTrue((ok == valid).all())
self.assertLess(float(np.abs(back[valid] / z[valid] - 1).max()), 1e-4)
self.assertTrue((back[~valid] == 999.0).all())
def _wall_and_box(frames=3, H=48, W=64):
"""A wall 4 m away with a box 2 m away in the middle, as MoGe would hand it over."""
f = 60.0
K = torch.tensor([[f, 0, W / 2], [0, f, H / 2], [0, 0, 1.0]])
vv, uu = torch.meshgrid(torch.arange(H) + 0.5, torch.arange(W) + 0.5, indexing="ij")
z = torch.full((H, W), 4.0)
z[H // 3: 2 * H // 3, W // 3: 2 * W // 3] = 2.0
pts = torch.stack([(uu - W / 2) / f * z, (vv - H / 2) / f * z, z], -1)
image = torch.zeros(H, W, 3)
image[..., 0] = uu / W
image[..., 1] = vv / H
image[..., 2] = 0.2
return {
"points": pts[None].expand(frames, H, W, 3).clone(),
"intrinsics": K[None].expand(frames, 3, 3).clone(),
"image": image[None].expand(frames, H, W, 3).clone(),
"mask": torch.ones(frames, H, W, dtype=torch.bool),
}
@unittest.skipIf(torch is None, "needs torch")
class WarpTests(unittest.TestCase):
def setUp(self):
from nodes._camera_move_helpers import auto_pivot, prepare_geometry, resolve_pivot, warp_frame
self.prepare, self.auto, self.resolve, self.warp = prepare_geometry, auto_pivot, resolve_pivot, warp_frame
self.scene = prepare_geometry(_wall_and_box())
def test_home_gives_back_the_clip(self):
home = {"az": 0, "el": 0, "back": 0, "px": 0, "py": 0}
frame, hole = self.warp(self.scene, 0, home, self.auto(self.scene), torch.device("cpu"))
self.assertEqual(int(hole.sum()), 0)
self.assertLess(float((frame - self.scene["image"][0]).abs().max()), 1e-4)
def test_a_turn_opens_pink_holes_beside_the_box(self):
cam = {"az": 25, "el": 0, "back": 0, "px": 0, "py": 0}
frame, hole = self.warp(self.scene, 0, cam, self.auto(self.scene), torch.device("cpu"))
self.assertGreater(int(hole.sum()), 10)
pink = frame[hole]
self.assertTrue(torch.allclose(pink, torch.tensor([1.0, 0.0, 1.0]).expand_as(pink)))
def test_auto_pivot_is_the_box(self):
self.assertAlmostEqual(self.auto(self.scene)[2], 2.0, places=4)
def test_a_tapped_pivot_is_the_spot_tapped(self):
on_wall = self.resolve(self.scene, {"u": 0.05, "v": 0.05, "t": 0.0})
self.assertAlmostEqual(on_wall[2], 4.0, places=4)
self.assertLess(on_wall[0], 0)
self.assertLess(on_wall[1], 0)
def test_one_steady_lens_and_depth_for_the_clip(self):
geometry = _wall_and_box(frames=12)
wobble = torch.tensor([1.0, 1.06, 0.95, 1.04, 0.97, 1.05, 0.94, 1.03, 0.96, 1.05, 0.95, 1.0])
geometry["points"] = geometry["points"] * wobble[:, None, None, None]
geometry["intrinsics"] = geometry["intrinsics"].clone()
geometry["intrinsics"][:, 0, 0] *= wobble
scene = self.prepare(geometry)
self.assertTrue(torch.equal(scene["K"][0], scene["K"][7]))
meds = torch.stack([scene["z"][i][scene["valid"][i]].median() for i in range(12)])
jumps = (meds[1:] / meds[:-1] - 1).abs()
self.assertLess(float(jumps.max()), 0.02) # the clip wobbles up to 11% frame to frame
def test_calms_still_depth_and_leaves_a_mover_alone(self):
from nodes._camera_move_helpers import steady_depth
frames, H, W = 9, 8, 8
z = torch.full((frames, H, W), 4.0)
z[:, :, :4] *= torch.tensor([1.0, 1.08, 0.93, 1.05, 0.95, 1.07, 0.94, 1.04, 1.0])[:, None, None] # wall flicker
image = torch.full((frames, H, W, 3), 0.5)
image[4, :, 6:] = torch.tensor([0.9, 0.2, 0.1]) # someone passes on the right in frame 4
z[4, :, 6:] = 2.0
out = steady_depth(z, torch.ones(frames, H, W, dtype=torch.bool), image)
self.assertLess(float((out[4, :, :4] / out[3, :, :4] - 1).abs().max()), 0.03) # was 12% between frames
self.assertTrue(torch.allclose(out[4, :, 6:], torch.tensor(2.0)))
def test_needs_the_lens(self):
with self.assertRaises(ValueError):
self.prepare({"image": torch.zeros(1, 4, 4, 3), "depth": torch.ones(1, 4, 4)})
@unittest.skipIf(torch is None, "needs torch")
class NodeTests(unittest.TestCase):
def test_run_returns_guide_holes_and_the_scene(self):
temp = tempfile.mkdtemp()
stub = types.ModuleType("folder_paths")
stub.get_temp_directory = lambda: temp
def get_save_image_path(prefix, folder, w, h):
return folder, prefix, 1, "", prefix
stub.get_save_image_path = get_save_image_path
sys.modules["folder_paths"] = stub
import nodes.node_camera_move as node_module
from nodes.node_camera_move import UI_KEY, AusBossCameraMove
node_module._device = lambda: torch.device("cpu")
out = AusBossCameraMove().run(_wall_and_box(frames=4), default_camera_text())
guide, holes = out["result"]
self.assertEqual(tuple(guide.shape), (4, 48, 64, 3))
self.assertEqual(tuple(holes.shape), (4, 48, 64))
self.assertEqual(float(holes[0].sum()), 0.0) # the first key is your own shot
self.assertGreater(float(holes[-1].sum()), 0.0)
info = out["ui"][UI_KEY][0]
self.assertEqual((info["w"], info["h"], info["count"]), (64, 48, 4))
self.assertEqual(len(info["frames"]), 4)
for frame in info["frames"]:
self.assertTrue((Path(temp) / frame["color"]["filename"]).is_file())
self.assertTrue((Path(temp) / frame["depth"]["filename"]).is_file())
json.dumps(info)
if __name__ == "__main__":
unittest.main()
+69 -1
View File
@@ -733,6 +733,63 @@ class VideoJobTests(unittest.TestCase):
self.assertIn("<Video 1>: partially_preserved", parts["retention_analysis"])
self.assertIn("the background of <Picture 1> is not transferred", parts["retention_analysis"])
def test_a_connected_model_writes_the_new_persons_looks_in_a_swap(self):
from nodes._h3_cast_helpers import add_looks
asked = []
def describe(slot):
asked.append(slot)
return '"A man with curly brown hair, wearing glasses."\nExtra line.'
script = add_looks("Replace the guy with @1.", {1: "person"}, describe)
self.assertEqual(script, "Replace the guy with @1.\n@1 looks like a man with curly brown hair, wearing glasses")
self.assertEqual(asked, [1])
text = write_prompt(script, seconds=8.0, jobs={1: "person"}, video="swap the person")
self.assertIn("<Subject 2> has the defined appearance from <Picture 1>: a man with curly brown hair", text)
# A typed line wins, places and empty answers add nothing.
typed = "Replace the guy with @1.\n@1 looks like a tall woman."
self.assertEqual(add_looks(typed, {1: "person"}, describe), typed)
self.assertEqual(add_looks("Swap him for @1.", {1: "place"}, describe), "Swap him for @1.")
self.assertEqual(add_looks("Swap him for @1.", {1: "person"}, lambda slot: ""), "Swap him for @1.")
said = "a man with curly brown hair, wearing dark-framed glasses, and no visible beard or facial hair."
self.assertEqual(add_looks("Swap him for @1.", {1: "person"}, lambda slot: said),
"Swap him for @1.\n@1 looks like a man with curly brown hair, wearing dark-framed glasses")
def test_a_swap_gets_the_firm_wording_without_a_looks_like_line(self):
text = write_prompt("Replace the guy with @1.", seconds=8.0, jobs={1: "person"}, video="swap the person")
self.assertIn("<Subject 2> has the face, hair and clothes of the person in <Picture 1>", text)
self.assertIn("stays present throughout, from the first frame", text)
self.assertIn("Do not show the guy in that role at any time", text)
def test_a_swap_with_a_model_adds_no_moments(self):
node = AusBossH3PromptWriter()
calls = []
import nodes.node_h3_prompt_writer as writer
real = writer.model_asker
writer.model_asker = lambda clip, image: (lambda question, longest: calls.append(question) or "a woman with a dark bob")
try:
text = node.write("Swap the woman for @1.", 6, clip=object(), picture_1=torch.zeros(1, 8, 8, 3),
video=torch.zeros(40, 8, 8, 3), video_job="swap the person")[0]
finally:
writer.model_asker = real
self.assertEqual(len(calls), 1)
self.assertIn("a woman with a dark bob", text)
self.assertNotIn("[Shot 2]", text)
def test_a_plain_swap_sentence_names_only_who_goes(self):
for script, who in (
("Replace the man in the red shirt with @1.", "the man in the red shirt"),
("swap the guy with @1", "the guy"),
("swap out the guy in the hat for @1 in the clip", "the guy in the hat"),
("the cashier is now @1", "the cashier"),
("make the dancer look like @1", "the dancer"),
("swap @1 in for the drummer", "the drummer"),
("Replace the woman with black hair with @1.\n@1 looks like a woman with platinum hair.", "the woman with black hair"),
):
text = write_prompt(script, seconds=8.0, jobs={1: "person"}, video="swap the person")
self.assertIn(f"<Subject 1> is {who} in <Video 1>, selected for replacement.", text)
self.assertNotIn("@1", text)
def test_typed_words_in_a_swap_say_who_goes(self):
parts = parts_of("The child in the seat.", jobs={1: "person"}, video="swap the person")
self.assertIn("<Subject 1> is the child in the seat in <Video 1>", parts["subject_definitions"])
@@ -748,7 +805,8 @@ class VideoJobTests(unittest.TestCase):
self.assertIn("<Subject 4> is the person in <Picture 2>", parts["subject_definitions"])
self.assertIn("replace the person on the right with <Subject 4>", parts["retention_analysis"])
self.assertIn("change only the people", parts["retention_analysis"])
self.assertIn("<Subject 4> takes the place of the person on the right", parts["detailed_description"])
self.assertIn("<Subject 4> occupies the starting position of the person on the right", parts["detailed_description"])
self.assertIn("<Subject 4> has the face, hair and clothes of the person in <Picture 2>", parts["detailed_description"])
self.assertNotIn("Show one person", parts["detailed_description"])
def test_a_swap_with_two_people_reads_who_goes_from_the_script(self):
@@ -1420,6 +1478,16 @@ class ClipSoundTests(unittest.TestCase):
self.assertNotIn("lips", line)
self.assertNotIn("speaks as in", line)
def test_loud_sound_with_no_voice_does_not_make_them_speak(self):
# The subway clip: one long stretch of train noise, loud in the voice band, no clear pitch.
noise = heard([(1.1, 5.42, None, 0.0)])
line = parts_of("@1 replaces the woman on the platform.", clip_sound=True, speech=noise, **SWAP_JOBS)["detailed_description"]
self.assertNotIn("speaks as in", line)
self.assertNotIn("lips", line)
a_word = heard([(1.0, 1.3, 150.0, 0.2)]) # A cough or one short sound: under half a second of voice.
line = parts_of("@1 replaces the woman on the platform.", clip_sound=True, speech=a_word, **SWAP_JOBS)["detailed_description"]
self.assertNotIn("speaks as in", line)
def test_a_typed_sound_wins_over_the_clips_sound(self):
parts = parts_of("@1 replaces the man in the hat.\nsound: rain on the roof", clip_sound=True, speech=plain([(0.4, 1.9)]), **SWAP_JOBS)
self.assertIn("rain on the roof", parts["overall_soundscape"].lower())
+31 -1
View File
@@ -1775,6 +1775,36 @@ class TellTests(TouchCase):
self.swap("_server", lambda: None)
self.assertEqual(self.note(self.run_node("", ask="scope", unique_id=7))["pieces"]["list"], told["pieces"]["list"])
def test_a_run_that_stops_still_shows_the_face_the_picture_that_came_in(self):
# Issue 301: the picture changed upstream, the run stopped, and the
# face kept the old one until the image was unplugged and plugged in.
other = touches(tap(75, 34), key="160x120-0000000000000000")
for text, message in ((other, OTHER_PICTURE), ("", NOTHING_PICKED)):
self.sent.clear()
with self.assertRaises(RuntimeError) as stopped:
self.run_node(text, unique_id=7)
self.assertEqual(str(stopped.exception), message, "the run still stops, for the same reason")
[(event, data, sid)] = self.sent
self.assertEqual((event, data["node"], data["end"], data["stopped"], sid), (node_module.LOOK_EVENT, "7", True, True, "the page that asked"))
note = data["note"]
self.assertEqual((note["key"], note["width"], note["height"]), (picture_key(self.picture), W, H))
self.assertEqual((note["items"], note["selection"], note["stamp"]), ([], None, ""), "nothing picked on it")
self.assertEqual(self.opened(note["picture"]).size[0] > 0, True)
self.assertEqual(FakeSam.calls, [], "showing the picture asks SAM 3 nothing")
def test_a_stop_that_cannot_tell_the_face_stops_for_the_same_reason(self):
def broken(*_args):
raise ConnectionError("the page is gone")
self.server.send_sync = broken
with self.assertRaises(RuntimeError) as stopped:
self.run_node("", unique_id=7)
self.assertEqual(str(stopped.exception), NOTHING_PICKED)
self.server.send_sync = lambda *args: self.sent.append(args)
with self.assertRaises(RuntimeError):
self.run_node("")
self.assertEqual(self.sent, [], "no id: nobody to tell")
def test_a_page_that_cannot_be_told_never_stops_the_look(self):
def broken(*_args):
raise ConnectionError("the page is gone")
@@ -2048,7 +2078,7 @@ class ContractTests(unittest.TestCase):
self.assertEqual((required["soften"][0], soften["default"], soften["min"], soften["max"], soften["step"]), ("FLOAT", 0.0, 0.0, 100.0, 0.1))
smooth = required["smooth"][1]
self.assertEqual((required["smooth"][0], smooth["default"], smooth["min"], smooth["max"]), ("INT", 0, 0, 64))
self.assertEqual((required["fill_holes"][0], required["fill_holes"][1]["default"]), ("BOOLEAN", True))
self.assertEqual((required["fill_holes"][0], required["fill_holes"][1]["default"]), ("BOOLEAN", False))
self.assertEqual((required["if_nothing"][0], required["if_nothing"][1]["default"]), (list(IF_NOTHING), "stop the run"))
self.assertEqual((optional["ask"][0], optional["ask"][1]["default"]), ("STRING", ""))
self.assertEqual((optional["invert"][0], optional["invert"][1]["default"]), ("BOOLEAN", False))
+24
View File
@@ -15,6 +15,7 @@ import {
matchInstalled,
noteFit,
noteIsEmpty,
noteShowsTileFace,
normalizeNote,
packsFromGraph,
parseInline,
@@ -198,6 +199,23 @@ test("noteFit ignores a card it could not measure", () => {
assert.deepEqual(noteFit({ natural: 794.125, nodeHeight: undefined, chrome: 22 }), { fit: null, grow: false });
});
test("a tile face shows only for a tile layout note that has content", () => {
assert.equal(noteShowsTileFace(normalizeNote('{"title":"Workflow info","layout":"tile"}')), true);
assert.equal(noteShowsTileFace(normalizeNote('{"title":"Workflow info","layout":"card"}')), false);
assert.equal(noteShowsTileFace(normalizeNote('{"layout":"tile"}')), false);
assert.equal(noteShowsTileFace(normalizeNote("")), false);
assert.equal(noteShowsTileFace(null), false);
});
test("the classic fit leaves a tile face compact", () => {
const source = readFileSync(new URL("../js/workflow_note/index.js", import.meta.url), "utf-8");
assert.match(
source,
/function fitNote\(state\) \{\n if \(!classicCanvas\(\)\) return;\n \/\/ A tile face is a button, not a card: fitting it would measure the note\n \/\/ behind the button and grow a new tile straight back to full height\.\n if \(noteShowsTileFace\(state\.note\)\) \{\n state\.fitHeight = undefined;\n return;\n \}/,
"fitNote skips a note that shows its tile face, so a new tile keeps its compact start",
);
});
test("parseInline handles bold, italic, code and safe links", () => {
assert.deepEqual(parseInline("a **b** *c* `d` [e](https://f.g) [h](javascript:x) i"), [
{ type: "text", text: "a " },
@@ -308,3 +326,9 @@ test("note formatting leaves old prose, unsafe markup and code literal", () => {
assert.equal(parseInline("`[amber]warning[/amber]`")[0].type, "code");
assert.equal(parseMarkdown("```\n|a|\n|---|\n```")[0].type, "code");
});
test("note panels stand down the frontend image preview", () => {
const source = readFileSync(new URL("../js/workflow_note/index.js", import.meta.url), "utf-8");
assert.match(source, /import \{ suppressCoreImagePreview \} from "\.\.\/shared\/core_preview\.mjs";/, "the note imports the pack's preview suppression");
assert.match(source, /suppressCoreImagePreview\(node\);/, "building a note panel stands the core image preview down");
});