Compare commits

..
Author SHA1 Message Date
ausboss b8c872ade0 H3 Prompt Writer: fixes from a full review
Thirteen fixes with tests: keep-going endings and audio, timing,
script parsing, undo, stale timeline updates, thumbnails, bypassed
inputs and wrapped picture chips.
2026-11-04 01:54:49 -06:00
ausboss c37386c1d4 Seed: add a compact look, picked from the gear (#274) 2026-11-03 23:59:16 -06:00
ausboss 7cb0abaacf Add Load Audio node: wave, trim, preview and max length (#272) 2026-11-03 23:58:16 -06:00
41 changed files with 1787 additions and 393 deletions
+5
View File
@@ -4,6 +4,11 @@ All notable changes to ComfyUI-AusBoss are documented here.
## Unreleased
- **New node: Load Audio.** Load a sound file and keep only the part you
want. The node shows the sound as a wave: drag the IN and OUT handles to
trim it, and press play to hear just that part. Set Max length to the
longest sound the next step takes, and anything longer shows in red as
cut off, so you see exactly what comes out before you run.
- **Resizing one node of a selection now resizes them all.** With two or more
nodes selected, pull the edge or corner of one: nodes stacked in a column
get its width, nodes side by side in a row get its height, and a grid gets
+6
View File
@@ -162,6 +162,12 @@ Upload a video, or pick one from the Source list, which plays the clip under the
It returns the frames, audio, frame count, fps, width, height, duration and a core `VIDEO`. Only the part you kept is decoded, and it checks memory before a large load. When you save the frames, wire its fps into the save node.
### Load Audio
Upload a sound file, or pick one from the Source list, and the node shows it as a wave. Drag the IN and OUT handles to keep only the part you want, or type the times. Press play to hear just that part. Set **Max length** to the longest sound the next step can take, such as a video's length: anything past it turns red and is cut off, so you see what comes out before you run. Click the red length next to OUT to move OUT to the cut.
It returns the `audio` you kept and its length in `seconds`. Only that part is decoded. [Help page](js/docs/AUSBOSS_NODES_LoadAudio.md).
### Video Crop + Rotate + Pad → Frame
Picks one frame from a video, then gives you the same rotate, crop and pad controls as the image node, plus an optional megapixel resize. Scrub to the frame with the playhead on the node; the editor adds frame-by-frame stepping and playback. It returns the transformed `image`, a `mask` of the added area, a `stitcher`, the chosen `original` frame, and the output `width` and `height`. For a whole clip, use the Clip version below.
+1
View File
@@ -36,6 +36,7 @@ NODE_MODULES = [
"node_layout_boxes",
"node_literals",
"node_load_image_pad",
"node_load_audio",
"node_load_video",
"node_lora_loader",
"node_mask_by_name",
+52
View File
@@ -0,0 +1,52 @@
# Load Audio
Loads a sound file and hands on only the part you pick. Use it for a voice
clip, a song or any sound a video model or a save node takes, when you want
one part of the file and not the whole thing.
The node shows the whole file as a wave, so you can see where the loud parts
are. The part that comes out is bright. Drag the white **IN** and **OUT**
handles to set where it starts and stops, drag the bright part to slide it,
or click somewhere else on the wave to move it there. You can also type a time
under the wave, in seconds (`12.5`) or minutes and seconds (`1:02.5`).
Press **▶** to hear only the part that comes out. **LOOP** plays it again and
again.
## Max length
Set **Max length** to the longest sound the next step can take, such as the
length of the video you are making. The start stays where it is, and anything
past the limit turns red with a dashed line where it is cut. Next to OUT,
the node shows how long the sound that comes out is, such as `15.0 s`. It
turns red when part of your pick is cut; hold the mouse over it to see how
much, and click it to move OUT to the cut. Play stops at the cut, so what you
hear is what comes out. `0` means no limit.
Max length can be wired. When the wire comes from a number node such as
Float 🆎, the wave shows the cut for that number; otherwise the length's tooltip says
`max wired` and the run still cuts the sound to the wired length.
## Controls
- **Source** (`audio`): a sound file in ComfyUI's input folder. **Upload**
adds one. Videos are listed as well, for their sound.
- **Max length** (`max_seconds`): the longest the sound may be, in seconds.
`0` means no limit.
- **IN** (`start_seconds`): where the sound starts.
- **OUT** (`end_seconds`): where the sound stops. `0` means the end of the
file.
## Outputs
- `audio`: the part you picked, for any node that takes `AUDIO`.
- `seconds`: how long that sound is.
## Technical details
- Only the picked part is decoded: decoding stops once the window is full,
so a short pick from a long recording stays quick.
- The sound keeps its own sample rate and channels.
- A start past the end of the file stops the run with a message that says so.
- The wave is drawn in the browser from the file itself; nothing extra is
stored with the workflow besides the four values above.
+9
View File
@@ -27,6 +27,15 @@ needs.
- The line underneath shows the last run's seed. The history saves with the
workflow.
## Compact look
The **gear** beside the number picks the look. **Compact** is one short
row: the number, a mode chip and a dice. Tap the chip to go Random → Fixed →
+1 → −1. The dice rolls a new seed and pins it. A long seed shrinks to fit.
Hover the number to see the last run's seed. The look is saved on the node,
and new Seed nodes start in the look you picked last. **Full** brings back
the whole card, with the seeds it ran with.
## Output
- **seed**: The seed, unchanged.
+1
View File
@@ -33,6 +33,7 @@ help panel.
| Apply a stack of LoRAs | LoRA Loader (`AUSBOSS_NODES_LoraLoader`) |
| Krea 2 prompt, and Krea 2 outpainting | Krea 2 Encode (`AUSBOSS_NODES_Krea2Encode`), Krea 2 Outpaint Model Patch (`AUSBOSS_NODES_Krea2OutpaintModelPatch`) |
| Load or save a video | Load Video (`AUSBOSS_NODES_LoadVideo`), Save Video (`AUSBOSS_NODES_SaveVideo`) |
| Load a sound file and keep only part of it, up to a set length | Load Audio (`AUSBOSS_NODES_LoadAudio`) |
| Turn, crop and pad a video: one frame, or the whole clip | Video Crop + Rotate + Pad → Frame (`AUSBOSS_NODES_VideoCropRotatePad`), → Clip (`AUSBOSS_NODES_VideoCropRotatePadClip`) |
| Change a clip's frame rate | Frame Interpolate (`AUSBOSS_NODES_FrameInterpolate`) |
| Work with frame batches | Select Frame, Select Every Nth, Split Batch, Merge Batches (`AUSBOSS_NODES_SelectFrame`, `_SelectEveryNth`, `_SplitBatch`, `_MergeBatches`) |
+84
View File
@@ -0,0 +1,84 @@
// Load Audio's window rules for the face, mirroring audio_window in
// nodes/_audio_trim_helpers.py so the wave shows exactly what one Run hands
// on. Pure: tests/load_audio_window.test.mjs runs it without a browser.
function finite(value, fallback = 0) {
const number = Number(value);
return Number.isFinite(number) ? number : fallback;
}
// The pick, and what comes out of it. `end` 0 means "to the end of the
// file". A max above 0 keeps the start and cuts off anything past
// start + max. All times in seconds; duration is the file's length.
export function outputWindow(durationValue, startValue, endValue, maxValue = 0) {
const duration = Math.max(0, finite(durationValue));
const start = Math.min(Math.max(0, finite(startValue)), duration);
const pickedEnd = finite(endValue) > 0 ? Math.min(finite(endValue), duration) : duration;
const end = Math.max(start, pickedEnd);
const max = Math.max(0, finite(maxValue));
const keepEnd = max > 0 && end - start > max ? Math.min(duration, start + max) : end;
return { start, end, keepEnd, length: keepEnd - start, cut: Math.max(0, end - keepEnd) };
}
// "0:12.0" style, the way the IN and OUT fields write a time.
export function secondsLabel(value) {
const seconds = Math.max(0, finite(value));
if (seconds < 60) return `${seconds.toFixed(1)} s`;
const minutes = Math.floor(seconds / 60);
return `${minutes}:${(seconds - minutes * 60).toFixed(1).padStart(4, "0")}`;
}
// The line between IN and OUT: what comes out, and what the max cuts off.
export function outputSummary(window, { maxFromLink = false } = {}) {
if (!window || window.length <= 0) return "";
const parts = [secondsLabel(window.length)];
if (window.cut > 0.005) parts.push(`cut ${secondsLabel(window.cut)}`);
if (maxFromLink) parts.push("max wired");
return parts.join(" · ");
}
// Loud and quiet across the file: for each of `buckets` slices, the lowest
// and highest sample of the channels mixed down. Channel data is a list of
// Float32Arrays (AudioBuffer.getChannelData). Returns a Float32Array of
// [min0, max0, min1, max1, ...].
export function wavePeaks(channels, bucketsValue = 1200) {
const list = Array.isArray(channels) ? channels.filter(Boolean) : [];
const buckets = Math.max(1, Math.floor(finite(bucketsValue, 1200)));
const peaks = new Float32Array(buckets * 2);
const length = list.length ? list[0].length : 0;
if (!length) return peaks;
const count = list.length;
for (let bucket = 0; bucket < buckets; bucket += 1) {
const from = Math.floor((bucket * length) / buckets);
const to = Math.max(from + 1, Math.floor(((bucket + 1) * length) / buckets));
let low = 0;
let high = 0;
for (let index = from; index < to && index < length; index += 1) {
let sample = 0;
for (let channel = 0; channel < count; channel += 1) sample += list[channel][index] || 0;
sample /= count;
if (sample < low) low = sample;
if (sample > high) high = sample;
}
peaks[bucket * 2] = low;
peaks[bucket * 2 + 1] = high;
}
return peaks;
}
// Which part of the wave a press at `fraction` (0..1 across it) grabs: a
// handle within `hitPixels` of it, the inside of the pick to slide it, or
// "jump" to move the pick there.
export function grabZone(window, durationValue, fraction, widthValue, hitPixels = 10) {
const duration = Math.max(0, finite(durationValue));
const width = Math.max(0, finite(widthValue));
if (duration <= 0 || width <= 0) return "none";
const x = Math.min(1, Math.max(0, finite(fraction))) * width;
const startX = (window.start / duration) * width;
const endX = (window.end / duration) * width;
const toStart = Math.abs(x - startX);
const toEnd = Math.abs(x - endX);
if (Math.min(toStart, toEnd) <= hitPixels) return toStart <= toEnd ? "start" : "end";
if (x > startX && x < endX) return "move";
return "jump";
}
+563
View File
@@ -0,0 +1,563 @@
// Load Audio 🆎 - the wave, the trim and the player.
//
// The card above (js/widget_cards) holds Source and Max length. This panel
// draws the whole file as a wave: the part that comes out is bright, the
// part a Max length cuts off is red, and the rest is dim. Drag a handle to
// move IN or OUT, drag the middle to slide the pick, or click elsewhere to
// move it there. Play plays only what comes out. The widgets underneath
// (start_seconds, end_seconds, max_seconds) stay the source of truth.
import { api } from "/scripts/api.js";
import { app } from "/scripts/app.js";
import { BRAND, chainCallback, keepDomWidgetWidthAuto, notifyAusbossChange } from "../shared/index.mjs";
import { keepKeyInField } from "../shared/canvas_passthrough.mjs";
import { WIDGET_FRAME, fillNodeHeight } from "../shared/panel_layout.mjs";
import { formatTimecode, parseTimecode } from "../shared/timecode.mjs";
import { parseImageReference } from "../shared/pad_canvas.mjs";
import { mediaViewQuery, splitMediaName } from "../shared/video_preview.mjs";
import { ensureVideoCss, hideCanvasWidget, makeToolButton } from "../shared/video_ui.mjs";
import { MIN_TRIM_SECONDS, dragTrimHandle, slideTrimWindow } from "../load_video/trim_preview.mjs";
import { grabZone, outputSummary, outputWindow, secondsLabel, wavePeaks } from "./audio_window.mjs";
const NODE_NAME = "AUSBOSS_NODES_LoadAudio";
const PANEL_WIDGET = "ausboss_load_audio_wave";
// The frontend's own player for an audio_upload input. The wave replaces it.
const CORE_AUDIO_WIDGET = "audioUI";
const MIN_WIDTH = 280;
// 8px root padding + the wave's 28px floor + 6px gap + the 22px play, IN/OUT and loop row.
const PANEL_MIN_HEIGHT = 64;
// Opens small and neat: 300 px wide, with a wave about as tall as a field
// (32 px). Drag the corner down for a taller wave.
const DEFAULT_WIDTH = 300;
const DEFAULT_WAVE_EXTRA = 4;
const CUT_RED = "#e0525a";
const PEAK_BUCKETS = 1600;
const END_EPSILON = 0.03;
// Decoded once per file and kept small: only the peaks and the length.
const peakCache = new Map();
function findWidget(node, name) {
return node.widgets?.find((widget) => widget.name === name);
}
function roundHundredth(value) {
return Math.round(Number(value || 0) * 100) / 100;
}
function hideCoreAudio(node) {
for (const widget of node.widgets ?? []) {
if (widget?.name === CORE_AUDIO_WIDGET) hideCanvasWidget(widget, widget.element);
}
}
// A wired Max length: the number the wire carries, when the far end is a
// node that shows it (Float, Integer, a Primitive). null when it can't be known.
function wiredMax(node) {
const input = node.inputs?.find((slot) => slot?.widget?.name === "max_seconds");
if (!input || input.link == null) return { wired: false, value: null };
const graph = node.graph;
const link = graph?.getLink?.(input.link) ?? graph?.links?.get?.(input.link) ?? graph?.links?.[input.link];
const origin = link ? graph?.getNodeById?.(link.origin_id) : null;
const shown = origin?.widgets?.find((widget) => typeof widget?.value === "number");
return { wired: true, value: shown ? Number(shown.value) : null };
}
function maxSeconds(state) {
const wired = wiredMax(state.node);
if (wired.wired) return { value: wired.value ?? 0, fromLink: true, known: wired.value != null };
return { value: Number(state.maxWidget?.value) || 0, fromLink: false, known: true };
}
function currentWindow(state) {
return outputWindow(state.duration, state.startWidget.value, state.endWidget.value, maxSeconds(state).value);
}
async function loadPeaks(name, route) {
const key = name;
if (peakCache.has(key)) return peakCache.get(key);
const job = (async () => {
const response = await api.fetchApi(route, { cache: "no-store" });
if (!response.ok) throw new Error(`the file could not be read (${response.status})`);
const bytes = await response.arrayBuffer();
const Context = window.AudioContext || window.webkitAudioContext;
const context = new Context();
try {
const buffer = await context.decodeAudioData(bytes);
const channels = [];
for (let index = 0; index < buffer.numberOfChannels; index += 1) channels.push(buffer.getChannelData(index));
return { duration: buffer.duration, peaks: wavePeaks(channels, PEAK_BUCKETS) };
} finally {
context.close?.();
}
})();
peakCache.set(key, job);
job.catch(() => peakCache.delete(key));
return job;
}
function ensureAudioCss() {
ensureVideoCss();
if (document.getElementById("ausboss-load-audio-css")) return;
const style = document.createElement("style");
style.id = "ausboss-load-audio-css";
style.textContent = `
.ausboss-audio-root .ausboss-video-stage{min-height:28px;background:#050b0b;cursor:pointer;touch-action:none;}
.ausboss-audio-root canvas.ausboss-audio-wave{position:absolute;inset:0;width:100%;height:100%;display:block;}
.ausboss-audio-root .ausboss-video-stage:not(.is-empty) .ausboss-video-status{display:none;}
.ausboss-audio-root .ausboss-audio-row{display:flex;align-items:center;gap:6px;flex:none;padding:0 2px;}
.ausboss-audio-root .ausboss-audio-row .ausboss-video-values{flex:1 1 auto;min-width:0;}
.ausboss-audio-root .ausboss-audio-play{min-width:30px;font-size:10px;}
.ausboss-audio-root .ausboss-audio-length{flex:none;padding:0 2px;color:#7fd9d2;font-size:10px;font-variant-numeric:tabular-nums;white-space:nowrap;cursor:default;}
.ausboss-audio-root .ausboss-audio-length:empty{display:none;}
.ausboss-audio-root .ausboss-audio-length.has-cut{color:#ff9aa0;cursor:pointer;text-decoration:underline dotted rgba(224,82,90,.8);text-underline-offset:2px;}
.ausboss-audio-root .ausboss-audio-row .ausboss-video-tool{flex:none;}
.ausboss-audio-root .ausboss-video-values{display:flex;gap:8px;}
.ausboss-audio-root .ausboss-video-value input{width:48px;}
.ausboss-audio-root .ausboss-audio-length{margin-left:auto;}
`;
document.head.appendChild(style);
}
function draw(state) {
const { canvas } = state;
const rect = canvas.getBoundingClientRect();
const scale = window.devicePixelRatio || 1;
const width = Math.max(1, Math.round(rect.width * scale));
const height = Math.max(1, Math.round(rect.height * scale));
if (canvas.width !== width || canvas.height !== height) {
canvas.width = width;
canvas.height = height;
}
const ctx = canvas.getContext("2d");
ctx.clearRect(0, 0, width, height);
if (!state.peaks || state.duration <= 0) return;
const win = currentWindow(state);
const xOf = (seconds) => (seconds / state.duration) * width;
const startX = xOf(win.start);
const keepX = xOf(win.keepEnd);
const endX = xOf(win.end);
// Backdrops: the part that comes out, then the part cut off.
ctx.fillStyle = "rgba(0,180,170,0.13)";
ctx.fillRect(startX, 0, keepX - startX, height);
if (endX > keepX) {
ctx.fillStyle = "rgba(224,82,90,0.16)";
ctx.fillRect(keepX, 0, endX - keepX, height);
}
// The wave, one bar per pixel column.
const mid = height / 2;
const reach = mid * 0.92;
const buckets = state.peaks.length / 2;
for (let x = 0; x < width; x += 1) {
const from = Math.floor((x / width) * buckets);
const to = Math.max(from + 1, Math.floor(((x + 1) / width) * buckets));
let low = 0;
let high = 0;
for (let bucket = from; bucket < to && bucket < buckets; bucket += 1) {
low = Math.min(low, state.peaks[bucket * 2]);
high = Math.max(high, state.peaks[bucket * 2 + 1]);
}
if (x >= startX && x < keepX) ctx.fillStyle = BRAND;
else if (x >= keepX && x < endX) ctx.fillStyle = CUT_RED;
else ctx.fillStyle = "#3a4948";
const top = mid - high * reach;
ctx.fillRect(x, top, 1, Math.max(1, (high - low) * reach));
}
// The handles: IN and OUT, and a dashed line where the max cuts off.
// The grips stay small on a short wave so they don't hide it.
const grip = Math.round(Math.max(3, Math.min(7, rect.height * 0.12)) * scale);
const handle = (x, color) => {
ctx.fillStyle = color;
ctx.fillRect(Math.round(x) - scale, 0, 2 * scale, height);
ctx.fillRect(Math.round(x) - 4 * scale, 0, 8 * scale, grip);
ctx.fillRect(Math.round(x) - 4 * scale, height - grip, 8 * scale, grip);
};
handle(startX, "#f4ffff");
handle(endX, "#f4ffff");
if (endX > keepX + 1) {
ctx.strokeStyle = CUT_RED;
ctx.lineWidth = 2 * scale;
ctx.setLineDash([4 * scale, 3 * scale]);
ctx.beginPath();
ctx.moveTo(keepX, 0);
ctx.lineTo(keepX, height);
ctx.stroke();
ctx.setLineDash([]);
}
// The playhead while it plays, or where it paused inside the pick.
if (state.audio && !state.audio.paused) {
const x = xOf(state.audio.currentTime);
ctx.fillStyle = "#ffffff";
ctx.fillRect(x - scale / 2, 0, scale, height);
}
}
function updateFace(state) {
const win = currentWindow(state);
const max = maxSeconds(state);
if (document.activeElement !== state.startInput) state.startInput.value = formatTimecode(win.start);
if (document.activeElement !== state.endInput) state.endInput.value = formatTimecode(win.end);
if (state.duration > 0) {
const summary = outputSummary(win, { maxFromLink: max.fromLink && !max.known });
state.summary.textContent = win.length > 0 ? secondsLabel(win.length) : "";
state.summary.classList.toggle("has-cut", win.cut > 0.005);
state.summary.title = win.cut > 0.005
? `${summary}. Max length is ${max.value} s, so ${formatTimecode(win.keepEnd)} to ${formatTimecode(win.end)} is cut off (red). Click to move OUT to ${formatTimecode(win.keepEnd)}.`
: `${summary}: ${formatTimecode(win.start)} to ${formatTimecode(win.keepEnd)} comes out.`;
} else {
state.summary.textContent = "";
}
draw(state);
}
function writeWidget(widget, value) {
const next = roundHundredth(value);
if (Math.abs(Number(widget.value || 0) - next) < 0.0001) return;
widget.value = next;
widget.callback?.(next);
}
function writeTrim(state, bounds) {
state.writing = true;
try {
writeWidget(state.startWidget, bounds.start);
// OUT at the very end stays 0, the widget's own "to the end".
writeWidget(state.endWidget, bounds.end >= state.duration - 0.005 ? 0 : bounds.end);
} finally {
state.writing = false;
}
updateFace(state);
state.node.setDirtyCanvas?.(true, true);
}
function installDrag(state) {
const surface = state.stage;
const infoAt = (clientX) => {
const rect = state.canvas.getBoundingClientRect();
const fraction = rect.width ? Math.min(1, Math.max(0, (clientX - rect.left) / rect.width)) : 0;
const win = currentWindow(state);
return { fraction, win, zone: grabZone(win, state.duration, fraction, rect.width) };
};
surface.addEventListener("pointermove", (event) => {
if (state.dragging || state.duration <= 0) return;
const { zone } = infoAt(event.clientX);
surface.style.cursor = zone === "start" || zone === "end" ? "ew-resize" : zone === "move" ? "grab" : "pointer";
});
surface.addEventListener("pointerdown", (event) => {
if (event.button !== 0 || state.dragging || state.duration <= 0) return;
event.preventDefault();
event.stopPropagation();
const initial = infoAt(event.clientX);
const bounds = { start: initial.win.start, end: initial.win.end };
const anchor = initial.fraction * state.duration;
const downX = event.clientX;
let moved = false;
state.dragging = true;
try { surface.setPointerCapture(event.pointerId); } catch (_error) { /* mouse fallback */ }
const centred = (fraction) => {
const span = bounds.end - bounds.start;
return slideTrimWindow(state.duration, bounds, fraction * state.duration - span / 2 - bounds.start);
};
const apply = (clientX) => {
const next = infoAt(clientX);
if (initial.zone === "start" || initial.zone === "end") {
const now = currentWindow(state);
writeTrim(state, dragTrimHandle(state.duration, { start: now.start, end: now.end }, initial.zone, next.fraction, MIN_TRIM_SECONDS));
} else if (initial.zone === "move") {
writeTrim(state, slideTrimWindow(state.duration, bounds, next.fraction * state.duration - anchor));
} else {
writeTrim(state, centred(next.fraction));
}
};
const move = (moveEvent) => {
if (!(moveEvent.buttons & 1)) { finish(moveEvent); return; }
if (!moved && Math.abs(moveEvent.clientX - downX) < 3) return;
moved = true;
apply(moveEvent.clientX);
};
const finish = (finishEvent) => {
if (!state.dragging) return;
state.dragging = false;
surface.removeEventListener("pointermove", move);
try { surface.releasePointerCapture(event.pointerId); } catch (_error) { /* already released */ }
if (!moved && initial.zone === "jump") writeTrim(state, centred(infoAt(finishEvent?.clientX ?? downX).fraction));
if (moved || initial.zone === "jump") {
if (state.audio && !state.audio.paused) state.audio.currentTime = currentWindow(state).start;
notifyAusbossChange();
}
};
surface.addEventListener("pointermove", move);
surface.addEventListener("pointerup", finish, { once: true });
surface.addEventListener("pointercancel", finish, { once: true });
surface.addEventListener("lostpointercapture", finish, { once: true });
});
}
function loopEnabled(node) {
node.properties ??= {};
return !!node.properties.ausboss_load_audio_loop;
}
function installPlayer(state) {
const audio = state.audio;
const stop = () => {
if (state.raf) cancelAnimationFrame(state.raf);
state.raf = 0;
state.playButton.textContent = "▶";
state.playButton.classList.remove("active");
draw(state);
};
const tick = () => {
if (audio.paused) { stop(); return; }
const win = currentWindow(state);
if (audio.currentTime >= win.keepEnd - END_EPSILON || audio.currentTime < win.start - 0.25) {
if (loopEnabled(state.node)) audio.currentTime = win.start;
else { audio.pause(); audio.currentTime = win.start; stop(); return; }
}
draw(state);
state.raf = requestAnimationFrame(tick);
};
audio.addEventListener("play", () => {
state.playButton.textContent = "❚❚";
state.playButton.classList.add("active");
if (state.raf) cancelAnimationFrame(state.raf);
state.raf = requestAnimationFrame(tick);
}, { signal: state.abort.signal });
audio.addEventListener("pause", stop, { signal: state.abort.signal });
state.playButton.addEventListener("click", (event) => {
event.preventDefault();
event.stopPropagation();
if (state.duration <= 0) return;
if (!audio.paused) { audio.pause(); return; }
const win = currentWindow(state);
if (audio.currentTime < win.start || audio.currentTime >= win.keepEnd - END_EPSILON) audio.currentTime = win.start;
audio.play().catch(() => stop());
});
state.stopPlayer = () => { audio.pause(); stop(); };
}
function commitTimeInput(state, edge, input) {
if (state.duration <= 0) return;
const seconds = parseTimecode(input.value);
const win = currentWindow(state);
if (seconds === null) {
input.value = formatTimecode(edge === "start" ? win.start : win.end);
return;
}
writeTrim(state, dragTrimHandle(state.duration, { start: win.start, end: win.end }, edge, seconds / state.duration, MIN_TRIM_SECONDS));
notifyAusbossChange();
input.blur();
}
function watchWidget(state, widget) {
if (!widget || widget.__ausbossAudioWatch) return;
widget.__ausbossAudioWatch = true;
const prior = widget.callback;
widget.callback = function (...args) {
const result = prior?.apply(this, args);
if (!state.writing) updateFace(state);
return result;
};
}
function buildPanel(node) {
if (node.__ausbossLoadAudio) return node.__ausbossLoadAudio;
const audioWidget = findWidget(node, "audio");
const startWidget = findWidget(node, "start_seconds");
const endWidget = findWidget(node, "end_seconds");
if (!audioWidget || !startWidget || !endWidget) return null;
ensureAudioCss();
hideCoreAudio(node);
hideCanvasWidget(startWidget);
hideCanvasWidget(endWidget);
const root = document.createElement("div");
root.className = "ausboss-video-root ausboss-audio-root";
const stage = document.createElement("div");
stage.className = "ausboss-video-stage is-empty";
stage.title = "Drag a handle to set where the sound starts or stops. Drag the middle to slide it.";
const canvas = document.createElement("canvas");
canvas.className = "ausboss-audio-wave";
const status = document.createElement("div");
status.className = "ausboss-video-status";
status.textContent = "Choose or upload a sound file";
const playButton = makeToolButton("▶", "Play the part that comes out");
playButton.classList.add("ausboss-audio-play");
const loopButton = makeToolButton("LOOP", "Play it again and again");
stage.append(canvas, status);
const values = document.createElement("div");
values.className = "ausboss-video-values";
const makeValue = (labelText, title) => {
const wrap = document.createElement("div");
wrap.className = "ausboss-video-value";
const label = document.createElement("label");
label.textContent = labelText;
const input = document.createElement("input");
input.type = "text";
input.inputMode = "decimal";
input.placeholder = "m:ss.s";
input.title = title;
input.addEventListener("keydown", (event) => {
keepKeyInField(event);
if (event.key === "Enter") input.dispatchEvent(new Event("change"));
});
wrap.append(label, input);
return { wrap, input };
};
const startValue = makeValue("IN", "Where the sound starts. Type seconds (12.5) or a time (0:12.5).");
const endValue = makeValue("OUT", "Where the sound stops. Type seconds (24) or a time (0:24.0).");
// How long the output is, beside OUT and off the wave. Red when Max
// length cuts the pick short; the tooltip says how much.
const summary = document.createElement("div");
summary.className = "ausboss-audio-length";
values.append(startValue.wrap, endValue.wrap);
const row = document.createElement("div");
row.className = "ausboss-audio-row";
row.append(playButton, values, summary, loopButton);
root.append(stage, row);
const widget = node.addDOMWidget(PANEL_WIDGET, "ausboss_audio", root, {
serialize: false,
hideOnZoom: false,
getMinHeight: () => PANEL_MIN_HEIGHT + WIDGET_FRAME,
});
keepDomWidgetWidthAuto(widget);
widget.serialize = false;
fillNodeHeight(widget, {
minWidth: MIN_WIDTH,
minHeight: PANEL_MIN_HEIGHT + WIDGET_FRAME,
minNodeSize: [MIN_WIDTH, 200],
exactMinWidth: true,
});
const audio = new Audio();
audio.preload = "auto";
const abort = new AbortController();
const state = node.__ausbossLoadAudio = {
node, root, stage, canvas, status, playButton, loopButton, summary, audio,
startInput: startValue.input, endInput: endValue.input,
audioWidget, startWidget, endWidget, maxWidget: findWidget(node, "max_seconds"),
duration: 0, peaks: null, abort, writing: false, dragging: false, raf: 0, loading: 0,
};
// Clicking the red length moves OUT to where the sound really ends.
summary.addEventListener("click", (event) => {
const win = currentWindow(state);
if (win.cut <= 0.005) return;
event.preventDefault();
event.stopPropagation();
writeTrim(state, { start: win.start, end: win.keepEnd });
notifyAusbossChange();
});
const updateLoop = () => loopButton.classList.toggle("active", loopEnabled(node));
updateLoop();
loopButton.addEventListener("click", (event) => {
event.preventDefault();
event.stopPropagation();
node.properties.ausboss_load_audio_loop = !loopEnabled(node);
updateLoop();
notifyAusbossChange();
});
const refresh = async () => {
state.stopPlayer?.();
const name = audioWidget.value;
const ticket = ++state.loading;
state.duration = 0;
state.peaks = null;
if (!name || typeof name !== "string") {
audio.removeAttribute("src");
stage.classList.add("is-empty");
status.textContent = "Choose or upload a sound file";
updateFace(state);
return;
}
const parts = splitMediaName(name);
stage.classList.add("is-empty");
status.textContent = `Loading ${parts.filename}…`;
const type = parseImageReference(name)?.type ?? "input";
const route = `/view?${mediaViewQuery({ ...parts, type }, "input")}`;
audio.src = api.apiURL(route);
try {
const result = await loadPeaks(name, route);
if (ticket !== state.loading) return;
state.duration = result.duration;
state.peaks = result.peaks;
stage.classList.remove("is-empty");
updateFace(state);
} catch (error) {
if (ticket !== state.loading) return;
status.textContent = `This sound could not be shown: ${error?.message || error}`;
}
};
state.refresh = refresh;
installDrag(state);
installPlayer(state);
watchWidget(state, startWidget);
watchWidget(state, endWidget);
watchWidget(state, state.maxWidget);
startValue.input.addEventListener("change", () => commitTimeInput(state, "start", startValue.input));
endValue.input.addEventListener("change", () => commitTimeInput(state, "end", endValue.input));
const observer = new ResizeObserver(() => draw(state));
observer.observe(stage);
state.observer = observer;
const priorAudioCallback = audioWidget.callback;
audioWidget.callback = function (...args) {
const result = priorAudioCallback?.apply(this, args);
peakCache.delete(audioWidget.value);
refresh();
return result;
};
setTimeout(refresh, 0);
return state;
}
app.registerExtension({
name: "ausboss.load_audio",
beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData?.name !== NODE_NAME) return;
// The frontend's Upload button for an audio_upload input looks for its
// player widget and fails without one, and it adds that player only to
// its own audio nodes. Ask for it here, ahead of the button; the panel
// hides it, since the wave plays the sound instead.
const required = nodeData.input?.required;
if (required && !required.audioUI) {
const entries = Object.entries(required);
const at = entries.findIndex(([name]) => name === "audio") + 1;
entries.splice(at, 0, ["audioUI", ["AUDIO_UI", {}]]);
nodeData.input.required = Object.fromEntries(entries);
}
chainCallback(nodeType.prototype, "onNodeCreated", function () {
buildPanel(this);
// Only for a genuinely new node: onConfigure restores a saved size
// after this runs, so a workflow's own size still wins.
const floor = Number(this.computeSize?.()?.[1]) || 0;
if (floor > 0) this.setSize?.([DEFAULT_WIDTH, floor + DEFAULT_WAVE_EXTRA]);
});
chainCallback(nodeType.prototype, "onConfigure", function () {
queueMicrotask(() => {
const state = buildPanel(this);
if (state) {
hideCoreAudio(this);
state.refresh?.();
}
});
});
// A wire into Max length changes what comes out; redraw when one comes or goes.
chainCallback(nodeType.prototype, "onConnectionsChange", function () {
const state = this.__ausbossLoadAudio;
if (state) queueMicrotask(() => updateFace(state));
});
chainCallback(nodeType.prototype, "onRemoved", function () {
const state = this.__ausbossLoadAudio;
state?.stopPlayer?.();
state?.abort?.abort();
state?.observer?.disconnect();
if (state) state.audio.removeAttribute("src");
this.__ausbossLoadAudio = null;
});
},
});
+123 -12
View File
@@ -19,6 +19,8 @@ import { copyToClipboard } from "../shared/clipboard.mjs";
import { hideInputsInDef, hideWidget } from "../shared/widget_visibility.mjs";
import { holdNodeMinWidth, pinVuePanelHeight } from "../shared/panel_layout.mjs";
import { formatSeed, pushSeed, seedFromExecuted } from "../shared/seed_history.mjs";
import { SEED_LOOK_MENU, SEED_LOOK_SCOPE, controlChip, nextControl, seedLook, setSeedLook } from "../shared/seed_look.mjs";
import { gearIconSvg, loadSettings, openSettingsMenu } from "../shared/settings_menu.mjs";
const NODE_CLASS = "AUSBOSS_NODES_Seed";
const CSS_ID = "ausboss-seed-css";
@@ -28,8 +30,19 @@ const PROPERTY = "ausboss_seed_history"; // [seed, newest first]
// in a double; a fresh seed rolled here obeys the same ceiling.
const ROLL_MAX = 0x4000000000000;
const CARD_HEIGHT = 158;
const PANEL_DECLARED_HEIGHT = CARD_HEIGHT + 20; // + the frontend's wrapper insets
// The compact look is one row: 8px card padding, a 32px row, 8px padding.
const COMPACT_CARD_HEIGHT = 50;
const PANEL_INSETS = 20; // the frontend's wrapper insets
const NODE_WIDTH = 300;
const COMPACT_NODE_WIDTH = 290;
function panelHeight(node) {
return (seedLook(node) === "compact" ? COMPACT_CARD_HEIGHT : CARD_HEIGHT) + PANEL_INSETS;
}
function nodeWidthFloor(node) {
return seedLook(node) === "compact" ? COMPACT_NODE_WIDTH : NODE_WIDTH;
}
let openMenu = null;
@@ -59,6 +72,21 @@ function ensureCss() {
.ausboss-seed-btn:disabled{opacity:.4;cursor:default;pointer-events:none;}
.ausboss-seed-hint{overflow:hidden;white-space:nowrap;text-overflow:ellipsis;color:#6f8886;font-size:10.5px;text-align:center;}
.ausboss-seed-hint b{color:#9fe3dc;font-weight:600;font-family:ui-monospace,"SF Mono",Menlo,Consolas,monospace;}
.ausboss-seed-top{display:flex;align-items:center;gap:6px;min-width:0;}
.ausboss-seed-top .ausboss-seed-field{flex:1 1 auto;min-width:0;}
.ausboss-seed-gear{display:grid;place-items:center;}
.ausboss-seed-chip,.ausboss-seed-dice{display:none;flex:none;height:32px;border:1px solid #2a3437;border-radius:7px;background:#0f1516;color:#d8ecea;font:600 11.5px/1 -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;cursor:pointer;pointer-events:auto;}
.ausboss-seed-chip{min-width:56px;padding:0 7px;}
.ausboss-seed-chip.random{border-color:rgba(0,180,170,.55);color:${BRAND};}
.ausboss-seed-dice{width:32px;font-size:15px;}
.ausboss-seed-chip:hover,.ausboss-seed-dice:hover{border-color:${BRAND};color:#fff;}
.ausboss-seed-card.compact{gap:0;}
.ausboss-seed-card.compact .ausboss-seed-seg,.ausboss-seed-card.compact .ausboss-seed-row,.ausboss-seed-card.compact .ausboss-seed-hint{display:none;}
.ausboss-seed-card.compact .ausboss-seed-chip,.ausboss-seed-card.compact .ausboss-seed-dice{display:block;}
.ausboss-seed-card.compact .ausboss-seed-field{height:32px;padding:0 3px 0 9px;}
.ausboss-seed-card.compact .ausboss-seed-field input{font-size:14px;}
.ausboss-seed-card.compact .ausboss-seed-tag,.ausboss-seed-card.compact .ausboss-seed-icon:not(.ausboss-seed-gear){display:none;}
.ausboss-seed-card.compact .ausboss-seed-field input{font-size:13.5px;letter-spacing:0;}
.ausboss-seed-menu{position:fixed;z-index:10000;min-width:230px;padding:4px;border:1px solid #3a4047;border-radius:7px;background:#1c1f23;box-shadow:0 8px 28px rgba(0,0,0,.5);font:12px/1.3 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace;color:#d7dde2;}
.ausboss-seed-menu-head{padding:5px 8px 6px;border-bottom:1px solid #2c3238;color:#78908e;font:10px/1 -apple-system,BlinkMacSystemFont,"Segoe UI",sans-serif;letter-spacing:.08em;text-transform:uppercase;}
.ausboss-seed-menu-item{display:block;width:100%;padding:6px 8px;border:none;border-radius:5px;background:transparent;color:inherit;font:inherit;text-align:left;cursor:pointer;}
@@ -124,6 +152,7 @@ function render(state) {
const linked = seedLinked(node);
const value = seedWidget(node)?.value;
if (document.activeElement !== state.input) state.input.value = formatSeed(Number(value));
fitCompactNumber(state);
state.input.disabled = linked;
state.field.classList.toggle("linked", linked);
state.field.title = linked ? "The seed comes from its input link" : "Click to type a seed";
@@ -136,6 +165,11 @@ function render(state) {
state.segStep.title = stepping
? "Click again to step the other way"
: "Increment the seed after each queue (click again for decrement)";
const chip = controlChip(control);
state.chip.textContent = chip.text;
state.chip.title = chip.title;
state.chip.classList.toggle("random", control === "randomize");
state.card.classList.toggle("compact", seedLook(node) === "compact");
state.useLast.disabled = !seeds.length;
state.more.disabled = seeds.length < 2;
state.hint.textContent = "";
@@ -145,6 +179,8 @@ function render(state) {
b.textContent = formatSeed(last);
state.hint.append(b);
if (control === "randomize") state.hint.append(" · rolls a new seed each queue");
// The compact look has no hint line: the last run shows on the number.
if (seedLook(node) === "compact") state.field.title = `${state.field.title}. Last run: ${formatSeed(last)}`;
} else {
state.hint.textContent = control === "randomize"
? "rolls a new seed each queue"
@@ -152,6 +188,16 @@ function render(state) {
}
}
// The compact row is narrow, so a long seed gets smaller digits instead of
// being cut off: monospace digits are about 0.6 of the font size wide.
function fitCompactNumber(state) {
const { input } = state;
if (seedLook(state.node) !== "compact") { input.style.fontSize = ""; return; }
const digits = Math.max(1, String(input.value || "").length);
const room = input.clientWidth || 100;
input.style.fontSize = `${Math.max(8, Math.min(13.5, room / (0.64 * digits))).toFixed(1)}px`;
}
function showMenu(state) {
closeMenu();
const seeds = history(state.node);
@@ -241,7 +287,9 @@ function buildPanel(node) {
input.inputMode = "numeric";
input.spellcheck = false;
const copy = button("ausboss-seed-icon", "⧉", "Copy the seed");
field.append(tag, input, copy);
const gear = button("ausboss-seed-icon ausboss-seed-gear", "", "Seed settings: full or compact look");
gear.innerHTML = gearIconSvg(12);
field.append(tag, input, copy, gear);
field.addEventListener("pointerdown", (event) => event.stopPropagation());
const seg = el("div", "ausboss-seed-seg");
@@ -257,11 +305,16 @@ function buildPanel(node) {
row.append(newSeed, useLast, more);
const hint = el("div", "ausboss-seed-hint");
card.append(field, seg, row, hint);
// Compact look: the mode chip and a dice sit on the number's row.
const chip = button("ausboss-seed-chip", "Fixed");
const dice = button("ausboss-seed-dice", "🎲", "Roll a fresh seed and pin it (Fixed)");
const top = el("div", "ausboss-seed-top");
top.append(field, chip, dice);
card.append(top, seg, row, hint);
root.append(card);
const state = (node.__ausbossSeed = {
node, root, field, tag, input, copy, segRandom, segFixed, segStep, newSeed, useLast, more, hint,
node, root, card, field, tag, input, copy, gear, chip, dice, segRandom, segFixed, segStep, newSeed, useLast, more, hint,
});
const commitTyped = () => {
@@ -304,6 +357,31 @@ function buildPanel(node) {
notifyAusbossChange();
});
newSeed.addEventListener("click", () => { setSeed(node, rollSeed()); render(state); });
dice.addEventListener("click", () => { setSeed(node, rollSeed()); render(state); });
// The chip walks the same modes as the full card's buttons, and starts a
// new run the same way: into Random rolls now, Fixed into +1 steps now.
chip.addEventListener("click", () => {
const current = controlWidget(node)?.value ?? "fixed";
const next = nextControl(current);
setControl(node, next);
if (!seedLinked(node)) {
const value = Number(seed.value) || 0;
if (next === "randomize") setSeed(node, rollSeed(), { pin: false });
else if (current === "fixed" && next === "increment") setSeed(node, value + 1, { pin: false });
}
render(state);
notifyAusbossChange();
});
gear.addEventListener("click", () => {
openSettingsMenu({
scope: SEED_LOOK_SCOPE,
schema: SEED_LOOK_MENU,
anchor: gear.getBoundingClientRect(),
title: "Seed settings",
initial: { look: seedLook(node) },
onChange: (values) => applyLook(state, values.look),
});
});
useLast.addEventListener("click", () => {
const seeds = history(node);
if (seeds.length) { setSeed(node, seeds[0]); render(state); }
@@ -318,7 +396,7 @@ function buildPanel(node) {
const widget = node.addDOMWidget(WIDGET_NAME, "ausboss_seed", root, {
serialize: false,
hideOnZoom: false,
getMinHeight: () => PANEL_DECLARED_HEIGHT,
getMinHeight: () => panelHeight(node),
});
keepDomWidgetWidthAuto(widget);
// Not saved with the workflow either: options.serialize only keeps it out
@@ -327,18 +405,20 @@ function buildPanel(node) {
// A constant-height card: pinned on purpose (see
// tests/panel_guards.test.mjs, fixedByDesign).
widget.computeSize = (width) => [
Math.max(NODE_WIDTH, Number(width || node.size?.[0] || NODE_WIDTH)),
PANEL_DECLARED_HEIGHT,
Math.max(nodeWidthFloor(node), Number(width || node.size?.[0] || nodeWidthFloor(node))),
panelHeight(node),
];
// Pinned in Nodes 2.0 as well, where a node can be made taller than its card.
pinVuePanelHeight(widget);
widget.options.minNodeSize = [NODE_WIDTH, 120];
widget.options.minNodeSize = [nodeWidthFloor(node), 60];
// The frontend skips a pinned card's width, so its floor is held here.
holdNodeMinWidth(widget, NODE_WIDTH);
holdNodeMinWidth(widget, nodeWidthFloor(node));
state.widget = widget;
new ResizeObserver(() => fitCompactNumber(state)).observe(input);
render(state);
node.setSize?.([
Math.max(node.size?.[0] ?? 0, NODE_WIDTH),
Math.max(node.size?.[0] ?? 0, nodeWidthFloor(node)),
node.computeSize?.()[1] ?? node.size?.[1],
]);
// The card is a fixed-height row: width may grow, but a corner drag
@@ -356,18 +436,49 @@ function buildPanel(node) {
return state;
}
// Switch looks: the card, the node's floor and its height follow. Going
// compact also makes the node as narrow as the compact row allows, and
// going full widens it back to the full card's floor.
function applyLook(state, look) {
const { node } = state;
const before = seedLook(node);
const after = setSeedLook(node, look);
if (before === after) return;
render(state);
if (state.widget) {
state.widget.options.minNodeSize = [nodeWidthFloor(node), 60];
holdNodeMinWidth(state.widget, nodeWidthFloor(node));
}
const width = after === "compact" ? COMPACT_NODE_WIDTH : Math.max(node.size?.[0] ?? 0, NODE_WIDTH);
node.setSize?.([width, node.computeSize?.()[1] ?? node.size?.[1]]);
node.setDirtyCanvas?.(true, true);
notifyAusbossChange();
}
app.registerExtension({
name: "AusBoss.Seed",
beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData?.name !== NODE_CLASS) return;
hideInputsInDef(nodeData, ["seed"]);
chainCallback(nodeType.prototype, "onNodeCreated", function () {
buildPanel(this);
// A new node starts in the look last picked from any Seed's gear. A
// loaded one keeps its own: onConfigure restores its properties after.
if (loadSettings(SEED_LOOK_SCOPE, SEED_LOOK_MENU).look === "compact") setSeedLook(this, "compact");
const state = buildPanel(this);
if (state && seedLook(this) === "compact") {
this.setSize?.([COMPACT_NODE_WIDTH, this.computeSize?.()[1] ?? this.size?.[1]]);
}
});
chainCallback(nodeType.prototype, "onConfigure", function () {
queueMicrotask(() => {
const state = buildPanel(this);
if (state) render(state);
if (!state) return;
render(state);
// A saved compact Seed: lower the floor to the compact row's.
if (state.widget) {
state.widget.options.minNodeSize = [nodeWidthFloor(this), 60];
holdNodeMinWidth(state.widget, nodeWidthFloor(this));
}
});
});
chainCallback(nodeType.prototype, "onConnectionsChange", function () {
+26
View File
@@ -11,6 +11,32 @@ export function lookupLink(graph, linkId) {
return graph?.links?.get?.(linkId) ?? graph?.links?.[linkId] ?? null;
}
// The live source of a named input. Bypass forwards a compatible input;
// mute, broken links and cycles provide no value.
export function linkSource(node, name) {
const graph = node.graph;
let input = node.inputs?.find((slot) => slot?.widget?.name === name || slot?.name === name);
const seen = new Set();
const compatible = (a, b) => !a || !b || a === "*" || b === "*"
|| String(a).split(",").some((type) => String(b).split(",").includes(type));
while (input?.link !== null && input?.link !== undefined) {
const link = lookupLink(graph, input.link) ?? graph?._links?.get?.(input.link);
if (!link || seen.has(link)) return null;
seen.add(link);
const source = graph?.getNodeById?.(link.origin_id);
if (!source || source.mode === 2) return null;
if (source.mode !== 4) return source;
const slot = link.origin_slot ?? 0;
const outputType = source.outputs?.[slot]?.type;
const type = input.type ?? outputType;
const inputs = source.inputs ?? [];
const matches = (entry) => entry && compatible(entry.type, outputType) && compatible(entry.type, type);
if (type === "*" || type === "") input = inputs[slot] ?? inputs[0];
else input = matches(inputs[slot]) ? inputs[slot] : inputs.find((entry) => entry.type === type) ?? inputs.find(matches);
}
return null;
}
// Wire origin's output slot to target's input slot, both by index. Returns
// the new link, or null when it could not be made.
//
+6 -11
View File
@@ -3,7 +3,8 @@
// dashed chip, so it reads as optional. The widgets underneath stay the
// source of truth: a chip only changes their value.
import { app } from "/scripts/app.js";
import { BRAND, chainCallback } from "./index.mjs";
import { BRAND, chainCallback, notifyAusbossChange } from "./index.mjs";
import { linkSource } from "./graph_links.mjs";
import { liftSocket } from "./widget_card.mjs";
import { commitWidgetValue, comboValues } from "./widget_card_math.mjs";
@@ -31,15 +32,6 @@ function ensureCss() {
document.head.append(style);
}
function linkSource(node, name) {
const slot = node.inputs?.find((input) => input?.widget?.name === name || input?.name === name);
if (slot?.link === null || slot?.link === undefined) return null;
const graph = node.graph;
const link = graph?.links?.[slot.link] ?? graph?._links?.get?.(slot.link);
const source = graph?.getNodeById?.(link?.origin_id);
return (source?.mode ?? 0) === 0 ? source ?? {} : null;
}
const widgetOf = (node, name) => node.widgets?.find((widget) => widget.name === name) ?? null;
function chip(className = "") {
@@ -63,7 +55,10 @@ function choice(node, name, into) {
select.value = widget?.value ?? "";
select.title = widget?.options?.tooltip ?? "";
select.addEventListener("change", () => {
if (widget) commitWidgetValue(node, widget, select.value, app.canvas);
if (widget && widget.value !== select.value) {
commitWidgetValue(node, widget, select.value, app.canvas);
notifyAusbossChange();
}
node.setDirtyCanvas?.(true, true);
});
return (values) => { if (document.activeElement !== select) select.value = values[name] ?? ""; };
+16 -13
View File
@@ -4,7 +4,8 @@
// words the node reads as instructions (@1, 3s:, a cut) stand out.
import { api } from "/scripts/api.js";
import { app } from "/scripts/app.js";
import { BRAND, chainCallback } from "./index.mjs";
import { BRAND, chainCallback, notifyAusbossChange } from "./index.mjs";
import { linkSource } from "./graph_links.mjs";
import { dragRange, moveMoment, retime, rulerSeconds, stripCaption, stripRows, toggleCut, wiredJobs } from "./moment_strip_math.mjs";
import { attachScriptAssist } from "./script_assist.mjs";
import { attachScriptColours } from "./script_colours.mjs";
@@ -47,16 +48,6 @@ function ensureCss() {
document.head.append(style);
}
// A bypassed or muted source doesn't reach the node when it runs, so it doesn't count.
function linkSource(node, name) {
const slot = node.inputs?.find((input) => input?.widget?.name === name || input?.name === name);
if (slot?.link === null || slot?.link === undefined) return null;
const graph = node.graph;
const link = graph?.links?.[slot.link] ?? graph?._links?.get?.(slot.link);
const source = graph?.getNodeById?.(link?.origin_id);
return (source?.mode ?? 0) === 0 ? source ?? {} : null;
}
function wired(node, name) {
return linkSource(node, name) !== null;
}
@@ -106,6 +97,7 @@ export function momentStrip(node) {
const next = change(String(widget.value ?? ""));
if (next === widget.value) return;
commitWidgetValue(node, widget, next, app.canvas);
notifyAusbossChange();
sync({ ...lastValues, script: widget.value });
};
// The left edge of a moment that opens a typed line: drag it to set when
@@ -266,6 +258,7 @@ export function momentStrip(node) {
let shown = null;
let timer = null;
let asked = 0;
let removed = false;
const ask = async (script, seconds, jobs, video) => {
const turn = ++asked;
try {
@@ -283,6 +276,7 @@ export function momentStrip(node) {
let latest = null;
const sync = (values) => {
if (removed) return;
latest = values;
lastValues = values;
colours.sync(values, [1, 2, 3, 4].filter((number) => wired(node, `picture_${number}`)));
@@ -302,13 +296,22 @@ export function momentStrip(node) {
lastSeconds = seconds;
const jobs = wiredJobs(values, (name) => wired(node, name));
const video = wired(node, "video") ? values.video_job : "";
const key = JSON.stringify([seconds, script, jobs, video]);
const pictures = [1, 2, 3, 4].map((number) => pictureUrl(node, number));
const key = JSON.stringify([seconds, script, jobs, video, pictures]);
if (key === shown) return;
shown = key;
clearTimeout(timer);
asked += 1;
timer = setTimeout(() => ask(script, seconds, jobs, video), WAIT);
};
const watch = setInterval(() => latest && sync(latest), WATCH);
chainCallback(node, "onRemoved", () => { clearInterval(watch); colours.dispose(); assist.dispose(); });
chainCallback(node, "onRemoved", () => {
removed = true;
clearTimeout(timer);
asked += 1;
clearInterval(watch);
colours.dispose();
assist.dispose();
});
return { element, sync };
}
+2 -2
View File
@@ -31,8 +31,8 @@ export function stripRows(report) {
}));
}
// A time typed at the start of a line: 3s:, 3 s, at 3 seconds, [3s].
const TYPED_TIME = /^\[?(?:at\s+)?\d+(?:\.\d+)?\s*(?:seconds?|secs?|s)\b\]?\s*[:,\-]?\s*/i;
// The timestamp grammar in _h3_script_helpers.py, including clocks and ranges.
const TYPED_TIME = /^\[?(?:at\s+)?(?:\d+:\d\d(?:\.\d+)?|\d+(?:\.\d+)?\s*(?:(?:seconds?|secs?|s)\b|(?=[\-–—]\s*\d)))(?:\s*[\-–—]\s*\d+(?:\.\d+)?\s*(?:(?:seconds?|secs?|s)\b)?)?\]?\s*[:,\-]?\s*/i;
// The script with one line set to start at a second: its old time, if any,
// gives way to the new one. line counts typed lines from 0.
+4 -1
View File
@@ -20,6 +20,7 @@ const emptyWidget = (inputs, name) => !linked(inputs?.[name]) && blank(inputs?.[
const picture = (name) => (inputs) => (emptyWidget(inputs, name) ? { input: name, kind: "picture" } : null);
const video = (name) => (inputs) => (emptyWidget(inputs, name) ? { input: name, kind: "video" } : null);
const sound = (name) => (inputs) => (emptyWidget(inputs, name) ? { input: name, kind: "sound file" } : null);
function videoSource(inputs) {
const mode = inputs?.source_mode;
@@ -38,6 +39,7 @@ export const SOURCE_LOADERS = Object.freeze({
AUSBOSS_NODES_ImageCropRotatePad: picture("image"),
// A wired source_image replaces the file, so the file may stay blank.
AUSBOSS_NODES_LoadImagePad: (inputs) => (linked(inputs?.source_image) ? null : picture("image")(inputs)),
AUSBOSS_NODES_LoadAudio: sound("audio"),
AUSBOSS_NODES_LoadVideo: video("video"),
AUSBOSS_NODES_VideoCropRotatePad: videoSource,
AUSBOSS_NODES_VideoCropRotatePadClip: videoSource,
@@ -110,7 +112,8 @@ export function emptySourceMessage(found) {
if (!first) return "";
const more = found.length - 1;
const tail = more > 0 ? ` (and ${more} more node${more === 1 ? "" : "s"})` : "";
return `Load a ${first.kind === "video" ? "video" : "picture"} first: ${first.title}${tail}`;
const noun = first.kind === "video" || first.kind === "sound file" ? first.kind : "picture";
return `Load a ${noun} first: ${first.title}${tail}`;
}
// "Save Image: exact_name may not ..." names the input it is about, after
+61
View File
@@ -0,0 +1,61 @@
// Seed 🆎's two looks, picked from the gear on its card: the full card
// (the default), or "compact", one short row with the number, a mode chip
// and a new-seed button. The look is kept in the node's properties, so the
// node's inputs, saved values and the prompt it sends are the same in both,
// and the last look picked is what new Seed nodes start with.
//
// Data and decisions only (tests/seed_look.test.mjs); js/seed/index.js
// draws the card.
export const SEED_LOOK_PROPERTY = "ausboss_seed_look";
export const SEED_LOOKS = ["full", "compact"];
export const SEED_LOOK_SCOPE = "seed_look";
export function seedLook(node) {
return node?.properties?.[SEED_LOOK_PROPERTY] === "compact" ? "compact" : "full";
}
// The full card leaves no property behind, so a node set back to it is
// saved exactly as one that was never changed.
export function setSeedLook(node, look) {
if (!node) return "full";
node.properties ??= {};
if (look === "compact") node.properties[SEED_LOOK_PROPERTY] = "compact";
else delete node.properties[SEED_LOOK_PROPERTY];
return seedLook(node);
}
// The gear menu (js/shared/settings_menu.mjs). The pick is also kept as the
// look new Seed nodes start with.
export const SEED_LOOK_MENU = [
{
key: "look",
label: "Look",
type: "choice",
options: SEED_LOOKS,
default: "full",
hint: "How this Seed node looks. New Seed nodes start the same way.",
optionHints: {
full: "the whole card: mode buttons, New seed, Use last run, and the seeds it ran with.",
compact: "one short row: the number, a mode chip you tap to change, and a dice for a new seed.",
},
},
];
// The compact row's chip: tap to go Random -> Fixed -> Step +1 -> Step -1
// -> Random. Values are the frontend's control_after_generate choices.
const CYCLE = ["randomize", "fixed", "increment", "decrement"];
export function nextControl(control) {
const index = CYCLE.indexOf(control);
return CYCLE[(index + 1) % CYCLE.length] ?? "fixed";
}
export function controlChip(control) {
switch (control) {
case "randomize": return { text: "Random", title: "Rolls a new seed after each run. Tap for Fixed." };
case "increment": return { text: "+1", title: "Adds 1 after each run. Tap for -1." };
case "decrement": return { text: "−1", title: "Takes 1 off after each run. Tap for Random." };
default: return { text: "Fixed", title: "Keeps this seed for every run. Tap for +1." };
}
}
+11 -1
View File
@@ -344,7 +344,7 @@ const CARDS = {
{ widget: "sound", label: "Sound", placeholder: "rain on a window" },
{ widget: "music", label: "Music", placeholder: "slow piano: music makes it louder" },
{ section: "Pictures and settings" },
{ panel: castChips, height: 30 },
{ panel: castChips, height: 82 },
{ widget: "seconds", label: "Length", suffix: "s" },
{ widget: "camera", label: "Camera", kind: "select" },
{ widget: "look", label: "Look", kind: "select" },
@@ -384,6 +384,16 @@ const CARDS = {
{ widget: "max_frames", label: "Limit" },
],
},
AUSBOSS_NODES_LoadAudio: {
// The wave, IN and OUT live in the panel under the card (js/load_audio).
minWidth: 280, first: true, hide: ["upload"],
rows: [
{ widget: "audio", label: "Source", kind: "select",
button: { text: "Upload", title: "Upload a sound file into ComfyUI's input folder", onClick: (node) => node.widgets?.find((w) => w.name === "upload")?.callback?.() } },
{ widget: "max_seconds", label: "Max length", suffix: "s", decimals: 1, trimZeros: true,
title: "The longest the sound may be. Anything past it is cut off and shows in red. 0 means no limit." },
],
},
AUSBOSS_NODES_LoadImagePad: {
minWidth: 340, first: true, hide: ["upload"],
rows: [
+26
View File
@@ -1557,5 +1557,31 @@
"AUSBOSS_NODES_WorkflowSwitches": {
"display_name": "工作流开关 🆎",
"description": "用一张小卡片开关工作流的各个部分。选中组成一个部分的节点,例如一个框和它的保存节点,按 + Switch:它们会得到一个 off | on 开关,名字由你定。每个组也可以有自己的开关。off 会旁路这些节点,图片仍然传到下一步(也可以设置为静音)。on 让它们重新运行。开关也可以改设置,而不是关闭节点:分别保存节点在 on 和 off 时的值,点一下就能把 Steps 设为 8 并打开一行 LoRA。按 Done,节点会缩成只有标签和开关。它从不运行,也没有连线。"
},
"AUSBOSS_NODES_LoadAudio": {
"display_name": "加载音频 🆎",
"description": "加载声音文件,只输出你选中的部分。节点把声音画成波形:拖动两个手柄设置起点和终点,按播放只听这一段。把最大长度设为下一步能接收的最长声音,超出的部分会显示为红色并被剪掉,运行前就能看到输出的内容。",
"inputs": {
"audio": {
"name": "音频"
},
"start_seconds": {
"name": "起始秒数"
},
"end_seconds": {
"name": "结束秒数"
},
"max_seconds": {
"name": "最大秒数"
}
},
"outputs": {
"0": {
"name": "音频"
},
"1": {
"name": "秒数"
}
}
}
}
+162
View File
@@ -0,0 +1,162 @@
"""Trimmed audio decoding for Load Audio.
The window rules live in ``audio_window`` so the node and its tests agree on
what one Run hands on; ``js/load_audio/audio_window.mjs`` mirrors them for the
face. Decoding counts samples from the start of the sound track and stops as
soon as the window is full, so a short pick from a long file never decodes
the whole file.
"""
from __future__ import annotations
import math
from pathlib import Path
import av
import numpy as np
import torch
from ._execution_helpers import raise_if_interrupted
try:
import folder_paths
except ImportError: # offline tests
folder_paths = None
AUDIO_EXTENSIONS = {
".aac",
".aif",
".aiff",
".flac",
".m4a",
".mp3",
".oga",
".ogg",
".opus",
".wav",
".webm",
".wma",
}
# Video files carry sound too: a song is often only on hand inside a clip.
SOUND_IN_VIDEO_EXTENSIONS = {".mkv", ".mov", ".mp4", ".m4v"}
FALLBACK_SAMPLE_RATE = 44100
# Shorter than this is no window at all; the face keeps its handles this far apart.
MIN_WINDOW_SECONDS = 0.1
def _number(value, default: float = 0.0) -> float:
try:
number = float(value)
except (TypeError, ValueError):
return default
return number if math.isfinite(number) else default
def list_input_audio() -> list[str]:
"""Sound files (and videos, for their sound) in ComfyUI's input folder."""
if folder_paths is None:
return [""]
root = Path(folder_paths.get_input_directory())
kinds = AUDIO_EXTENSIONS | SOUND_IN_VIDEO_EXTENSIONS
results = [
path.relative_to(root).as_posix()
for path in root.rglob("*")
if path.is_file() and not path.name.startswith(".") and path.suffix.lower() in kinds
]
return sorted(results, key=str.casefold) or [""]
def audio_window(start_seconds, end_seconds, max_seconds=0.0, duration=None) -> dict:
"""The part of a sound file one Run hands on.
end 0 means "to the end of the file". max_seconds above 0 caps the
window's length: the pick keeps its start and anything past
start + max_seconds is cut off. ``duration`` is the file's length when it
is known; None leaves an open end open.
Returns start, end (None when open), the end the person picked
(``picked_end``) and how many seconds the cap cut off (``cut``).
"""
start = max(0.0, _number(start_seconds))
picked_end = _number(end_seconds)
end = picked_end if picked_end > 0 else None
if duration is not None:
length = max(0.0, _number(duration))
start = min(start, length)
end = length if end is None else min(end, length)
if end is not None:
end = max(start, end)
picked = end
limit = max(0.0, _number(max_seconds))
if limit > 0 and (end is None or end - start > limit):
end = start + limit
if duration is not None:
end = min(end, max(0.0, _number(duration)))
cut = 0.0
if picked is not None and end is not None:
cut = max(0.0, picked - end)
return {"start": start, "end": end, "picked_end": picked, "cut": cut}
def decode_audio_window(path: Path, start_seconds, end_seconds, max_seconds=0.0) -> dict:
"""ComfyUI AUDIO for the window ``audio_window`` describes.
Raises a plain ValueError for a file with no sound, or a start at or past
the end of the sound.
"""
window = audio_window(start_seconds, end_seconds, max_seconds)
start = window["start"]
with av.open(str(path)) as container:
stream = next((candidate for candidate in container.streams if candidate.type == "audio"), None)
if stream is None:
raise ValueError(f"{path.name} has no sound in it.")
rate = int(stream.rate or FALLBACK_SAMPLE_RATE)
first = round(start * rate)
last = None if window["end"] is None else round(window["end"] * rate)
resampler = av.AudioResampler(format="fltp", layout=stream.layout, rate=rate)
chunks: list[np.ndarray] = []
position = 0
def take(frame) -> None:
nonlocal position
for resampled in resampler.resample(frame):
data = resampled.to_ndarray()
begin = position
position += data.shape[1]
if position <= first:
continue
low = max(0, first - begin)
high = data.shape[1] if last is None else min(data.shape[1], last - begin)
if high > low:
chunks.append(data[:, low:high])
for frame in container.decode(stream):
raise_if_interrupted()
take(frame)
if last is not None and position >= last:
break
else:
take(None) # the resampler hands back what it still holds
if not chunks:
if position == 0:
raise ValueError(f"{path.name} has no sound in it.")
raise ValueError(
f"The start ({start:.2f} s) is past the end of {path.name}, "
f"which is {position / rate:.2f} s long. Drag the start back into the sound."
)
data = np.concatenate(chunks, axis=1).astype(np.float32, copy=False)
return {
"waveform": torch.from_numpy(np.ascontiguousarray(data)).unsqueeze(0),
"sample_rate": rate,
}
__all__ = [
"AUDIO_EXTENSIONS",
"MIN_WINDOW_SECONDS",
"SOUND_IN_VIDEO_EXTENSIONS",
"audio_window",
"decode_audio_window",
"list_input_audio",
]
+27 -14
View File
@@ -37,7 +37,7 @@ def split_sections(text):
"""The six parts of a prompt by name, or None when one is missing."""
found = {}
current = None
for line in text.splitlines():
for line in text.split("\n"):
name = line.strip().rstrip(":")
if line.strip().endswith(":") and name in SECTIONS:
current = found.setdefault(name, [])
@@ -100,17 +100,29 @@ def spoken_line(speaker, words, language, voice_label, on_screen, number=1, lips
return f"{speaker} (S{number}) says in an off-screen voiceover, {line}{closed}{timbre}"
def snap_cuts(detailed, beats, gap=0.4):
def snap_cuts(detailed, beats, gap=0.4, seconds=None):
"""Moves every timed moment to the nearest beat, keeping them in order and
at least `gap` seconds apart."""
`gap` seconds apart, or closer if needed to fit inside the clip."""
moments = list(CUT.finditer(detailed))
if not moments or not beats:
return detailed
limit = seconds if seconds is not None else max(
*beats, *(int(match.group(2)) * 60 + float(match.group(3)) for match in moments)
)
gap = min(gap, limit / len(moments))
last = 0.0
remaining = len(moments)
def move(match):
nonlocal last
nonlocal last, remaining
remaining -= 1
time = int(match.group(2)) * 60 + float(match.group(3))
options = [beat for beat in beats if beat >= last + gap]
low, high = last + gap, limit - remaining * gap
options = [beat for beat in beats if low <= beat <= high]
if options:
time = min(options, key=lambda beat: abs(beat - time))
else:
time = min(high, max(low, time))
last = time
minutes, seconds = divmod(time, 60)
return f"{match.group(1)}{int(minutes):02d}:{seconds:06.3f}"
@@ -131,14 +143,14 @@ def add_voice(parts, label, speaker=None):
)
def cut_on_beat(parts, beats):
def cut_on_beat(parts, beats, seconds=None):
if beats:
parts["detailed_description"] = snap_cuts(parts["detailed_description"], beats).replace(
parts["detailed_description"] = snap_cuts(parts["detailed_description"], beats, seconds=seconds).replace(
"the camera cuts to", "the camera cuts on the beat to"
)
def add_song(parts, label, beats):
def add_song(parts, label, beats, seconds=None):
parts["subject_definitions"] += f"\n{label} is the complete source soundtrack reused in the target video."
parts["summary"] = add_task(parts["summary"], "audio reuse")
if beats:
@@ -148,14 +160,14 @@ def add_song(parts, label, beats):
)
parts["overall_soundscape"] = "N/A"
parts["non_diegetic_music"] = f"{label} is directly reused as the complete audience-only score."
cut_on_beat(parts, beats)
cut_on_beat(parts, beats, seconds)
def add_song_under(parts, beats):
def add_song_under(parts, beats, seconds=None):
"""A song to lay under the finished clip. H3 drops a song it is asked to
keep when someone also speaks, so it hears none and makes no music."""
parts["non_diegetic_music"] = "N/A"
cut_on_beat(parts, beats)
cut_on_beat(parts, beats, seconds)
def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="English", voice=False, song_beats=None,
@@ -213,9 +225,9 @@ def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="E
if voice:
add_voice(parts, voice_label, speaker)
if song_beats is not None and voice:
add_song_under(parts, song_beats)
add_song_under(parts, song_beats, seconds)
elif song_beats is not None:
add_song(parts, f"<Audio {audio}>", song_beats)
add_song(parts, f"<Audio {audio}>", song_beats, seconds)
return re.sub(r"[ \t]{2,}", " ", join_sections(parts)).replace(" \n", "\n").strip()
@@ -237,7 +249,8 @@ def find_beats(samples, rate, seconds=30.0):
[np.abs(np.fft.rfft(samples[index * hop : index * hop + window] * taper)) for index in range(frames)]
)
rise = np.maximum(np.diff(np.log1p(1000 * spectra), axis=0), 0).sum(axis=1)
rise = np.maximum(rise - np.convolve(rise, np.ones(43) / 43, "same"), 0)
smooth = min(43, len(rise))
rise = np.maximum(rise - np.convolve(rise, np.ones(smooth) / smooth, "same"), 0)
if rise.max() <= 0:
return []
rise = rise / rise.max()
+3 -1
View File
@@ -298,6 +298,7 @@ def take_label(line, script):
def read_notes(text, script):
text, _ = mask_quotes(text)
if NO_SOUND.search(text):
script.silent = True
if NO_MUSIC.search(text):
@@ -385,7 +386,7 @@ def read_script(text):
if not line or take_label(line, script):
continue
opens = block_line + offset
for piece in split_line(NUMBER.sub("", line).strip()):
for piece in split_line(line if TIMED.match(line) else NUMBER.sub("", line).strip()):
note = SIDE_NOTE.match(piece)
if note: # "no music, just rain" on its own is a note, not a moment.
read_notes(piece, script)
@@ -408,6 +409,7 @@ def read_script(text):
put_in_order(script)
if script.one_take or not any(moment.cut for moment in script.moments):
for moment in script.moments:
moment.cut = False
script.camera, script.sound = moment.camera or script.camera, moment.sound or script.sound
moment.camera = moment.sound = ""
return script
+3 -4
View File
@@ -177,10 +177,9 @@ def round_half_away_from_zero(value: float) -> int:
Python's round() rounds halves to even (2.5 -> 2), which reads as a bug
when the number is on screen next to the float it came from.
"""
fraction, whole = math.modf(value)
# Compare the fraction directly: adding 0.5 can round the float before
# we round it, including changing an already whole large number.
return int(whole) + (1 if fraction >= 0.5 else -1 if fraction <= -0.5 else 0)
if value >= 0:
return int(math.floor(value + 0.5))
return int(math.ceil(value - 0.5))
__all__ = [
+2 -3
View File
@@ -847,13 +847,12 @@ def _safe_route_error(exc: Exception) -> str:
def _query_int(request, name: str, default: int) -> int:
try:
return int(float(request.query.get(name, default)))
except (TypeError, ValueError, OverflowError):
except (TypeError, ValueError):
return default
def _query_float(request, name: str, default: float) -> float:
try:
value = float(request.query.get(name, default))
return value if math.isfinite(value) else default
return float(request.query.get(name, default))
except (TypeError, ValueError):
return default
+8 -4
View File
@@ -88,8 +88,8 @@ def clip_frames(seconds):
def video_frames(frames):
"""A clip's own length, on H3's grid and within what it can make."""
longest = LONGEST_SECONDS * FRAME_RATE
return max(1, (min(frames, longest) - 5) // 17) * 17 + 5
longest = clip_frames(LONGEST_SECONDS)
return max(0, (min(frames, longest) - 5) // 17) * 17 + 5
def cut_audio(audio, start, seconds, name):
@@ -104,10 +104,14 @@ def cut_audio(audio, start, seconds, name):
def last_of_clip(video, sound):
"""The end of a clip, picture and sound, to carry on from."""
frames = min(len(video), KEEP_FRAMES)
if frames >= 5:
frames = video_frames(frames)
if sound is not None:
rate = int(sound["sample_rate"])
sound = {"waveform": sound["waveform"][..., -int(round(KEEP_FRAMES / FRAME_RATE * rate)) :], "sample_rate": rate}
return video[-KEEP_FRAMES:], sound
count = int(round(frames / FRAME_RATE * rate))
sound = {"waveform": sound["waveform"][..., -count:] if count else sound["waveform"][..., :0], "sample_rate": rate}
return video[-frames:] if frames else video[:0], sound
def under_voice(song):
+126
View File
@@ -0,0 +1,126 @@
"""Load Audio 🆎."""
from __future__ import annotations
import asyncio
from ._audio_trim_helpers import decode_audio_window, list_input_audio
from ._execution_helpers import ignore_extra_inputs
from ._media_helpers import resolve_input_path
class AusBossLoadAudio:
CATEGORY = "🆎 AusBoss/Video"
DESCRIPTION = (
"Loads a sound file and hands on only the part you pick. The node shows the "
"sound as a wave, so you can see where the loud parts are. Drag the two "
"handles to set where it starts and stops, and press play to hear just that "
"part. Set Max length to the longest sound the next step takes: anything "
"longer shows in red as cut off, so you see exactly what comes out before "
"you run anything."
)
SEARCH_ALIASES = [
"load audio",
"audio loader",
"trim audio",
"cut audio",
"song",
"voice",
"music",
"waveform",
"ausboss",
]
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"audio": (
list_input_audio(),
{
"audio_upload": True,
"tooltip": "A sound file in ComfyUI's input folder. Use Upload to add one.",
},
),
"start_seconds": (
"FLOAT",
{
"default": 0.0,
"min": 0.0,
"max": 86400.0,
"step": 0.01,
"tooltip": "Where the sound starts. Drag the wave's IN handle or type a time.",
},
),
"end_seconds": (
"FLOAT",
{
"default": 0.0,
"min": 0.0,
"max": 86400.0,
"step": 0.01,
"tooltip": "Where the sound stops. 0 plays to the end. Drag the OUT handle or type a time.",
},
),
},
"optional": {
"max_seconds": (
"FLOAT",
{
"default": 0.0,
"min": 0.0,
"max": 86400.0,
"step": 0.1,
"tooltip": (
"The longest the sound may be. Anything past it is cut off and shows "
"in red. 0 means no limit."
),
},
),
},
}
RETURN_TYPES = ("AUDIO", "FLOAT")
RETURN_NAMES = ("audio", "seconds")
OUTPUT_TOOLTIPS = (
"The part of the sound you picked, ready for any node that takes audio.",
"How long that sound is, in seconds.",
)
FUNCTION = "load_audio"
async def load_audio(self, audio, start_seconds, end_seconds, max_seconds=0.0):
path = resolve_input_path(audio)
# Decoding blocks for as long as the pick is; a worker thread keeps
# ComfyUI's event loop answering, and carries the context ComfyUI
# reads to attribute an interrupt to this node.
sound = await asyncio.to_thread(
decode_audio_window, path, start_seconds, end_seconds, max_seconds
)
seconds = sound["waveform"].shape[-1] / float(sound["sample_rate"])
return (sound, float(seconds))
@classmethod
def IS_CHANGED(cls, audio, **_values):
try:
stat = resolve_input_path(audio).stat()
except Exception:
return float("nan")
return f"{audio}:{stat.st_mtime_ns}:{stat.st_size}"
@classmethod
@ignore_extra_inputs
def VALIDATE_INPUTS(cls, audio):
# Only the source is named: ComfyUI skips its own range checks for
# every input named here. resolve_input_path takes a file inside the
# input folder, or one named "[output]" or "[temp]", and nothing else.
try:
resolve_input_path(audio)
except Exception as exc:
return f"Load Audio: {exc}"
return True
NODE_CLASS_MAPPINGS = {"AUSBOSS_NODES_LoadAudio": AusBossLoadAudio}
NODE_DISPLAY_NAME_MAPPINGS = {"AUSBOSS_NODES_LoadAudio": "Load Audio 🆎"}
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
+1
View File
@@ -67,6 +67,7 @@ PUBLIC_NODE_IDS = {
"AUSBOSS_NODES_Krea2OutpaintModelPatch",
"AUSBOSS_NODES_LaMaInpaint",
"AUSBOSS_NODES_LayoutBoxes",
"AUSBOSS_NODES_LoadAudio",
"AUSBOSS_NODES_LoadImagePad",
"AUSBOSS_NODES_LoadVideo",
"AUSBOSS_NODES_LoraLoader",
+20
View File
@@ -490,6 +490,26 @@
]
]
},
"AUSBOSS_NODES_LoadAudio": {
"required": [
"audio",
"start_seconds",
"end_seconds"
],
"optional": [
"max_seconds"
],
"outputs": [
[
"AUDIO",
"audio"
],
[
"FLOAT",
"seconds"
]
]
},
"AUSBOSS_NODES_LoadImagePad": {
"required": [
"image",
+34
View File
@@ -3,6 +3,7 @@ import test from "node:test";
import {
linkSlots,
linkSource,
lookupLink,
restoreOutputLinks,
snapshotLinks,
@@ -62,6 +63,39 @@ test("lookupLink reads Map and plain-object link tables", () => {
assert.equal(lookupLink(null, 7), null);
});
test("linkSource follows compatible bypass chains and rejects mute, missing links and cycles", () => {
const loader = { id: 1, mode: 0 };
const processor = { id: 2, mode: 4, inputs: [{ type: "MASK", link: null }, { type: "IMAGE", link: 2 }], outputs: [{ type: "IMAGE" }] };
const pass = { id: 3, mode: 4, inputs: [{ type: "IMAGE", link: 3 }], outputs: [{ type: "IMAGE" }] };
const nodes = new Map([[1, loader], [2, processor], [3, pass]]);
const graph = { links: new Map([[1, { origin_id: 3, origin_slot: 0 }], [2, { origin_id: 1, origin_slot: 0 }], [3, { origin_id: 2, origin_slot: 0 }]]), getNodeById: (id) => nodes.get(id) };
const target = { graph, inputs: [{ name: "picture_1", type: "IMAGE", link: 1 }] };
assert.equal(linkSource(target, "picture_1"), loader);
loader.mode = 2;
assert.equal(linkSource(target, "picture_1"), null);
loader.mode = 0;
processor.inputs[1].link = null;
assert.equal(linkSource(target, "picture_1"), null);
processor.inputs[1].link = 1;
assert.equal(linkSource(target, "picture_1"), null);
processor.inputs[1] = { type: "AUDIO", link: 2 };
assert.equal(linkSource(target, "picture_1"), null);
target.inputs[0].link = 99;
assert.equal(linkSource(target, "picture_1"), null);
});
test("linkSource prefers the matching input at the output's index", () => {
const first = { id: 1 }, second = { id: 2 };
const bypass = { id: 3, mode: 4, inputs: [{ type: "IMAGE", link: 2 }, { type: "IMAGE", link: 3 }], outputs: [{ type: "IMAGE" }, { type: "IMAGE" }] };
const graph = { _links: new Map([[1, { origin_id: 3, origin_slot: 1 }], [2, { origin_id: 1 }], [3, { origin_id: 2 }]]), getNodeById: (id) => [null, first, second, bypass][id] };
assert.equal(linkSource({ graph, inputs: [{ name: "picture_1", type: "IMAGE", link: 1 }] }, "picture_1"), second);
bypass.inputs[1].type = "MASK";
assert.equal(linkSource({ graph, inputs: [{ name: "picture_1", type: "IMAGE", link: 1 }] }, "picture_1"), first);
for (const type of ["*", ""]) {
assert.equal(linkSource({ graph, inputs: [{ name: "value", type, link: 1 }] }, "value"), second);
}
});
test("linkSlots hands the resolved slot objects to connectSlots", () => {
const { makeNode } = makeGraph();
const source = makeNode(1, [], ["IMAGE", "MASK"]);
+178
View File
@@ -0,0 +1,178 @@
import assert from "node:assert/strict";
import test from "node:test";
import { readFileSync } from "node:fs";
import vm from "node:vm";
import * as math from "../js/shared/moment_strip_math.mjs";
import * as links from "../js/shared/graph_links.mjs";
import { commitWidgetValue, comboValues } from "../js/shared/widget_card_math.mjs";
class Element {
constructor(tag) {
this.tagName = tag.toUpperCase();
this.children = [];
this.listeners = {};
this.style = {};
this.className = "";
this.value = "";
this.classList = {
add: (name) => { if (!this.classList.contains(name)) this.className += ` ${name}`; },
remove: (name) => { this.className = this.className.split(" ").filter((part) => part !== name).join(" "); },
contains: (name) => this.className.split(" ").includes(name),
toggle: (name, on) => on ? this.classList.add(name) : this.classList.remove(name),
};
}
append(...children) { for (const child of children) { child.parentElement = this; this.children.push(child); } }
set textContent(text) { this.text = text; this.children = []; }
get textContent() { return (this.text ?? "") + this.children.map((child) => child.textContent).join(""); }
get childElementCount() { return this.children.length; }
addEventListener(name, fn) { (this.listeners[name] ??= []).push(fn); }
removeEventListener(name, fn) { this.listeners[name] = (this.listeners[name] ?? []).filter((entry) => entry !== fn); }
dispatch(name, event = {}) {
for (const fn of [...(this.listeners[name] ?? [])]) fn({ button: 0, clientX: 0, preventDefault() {}, stopPropagation() {}, ...event });
}
getBoundingClientRect() { return { left: 0, width: 100 }; }
remove() { if (this.parentElement) this.parentElement.children = this.parentElement.children.filter((child) => child !== this); }
}
function page() {
let now = 0, next = 0, captures = 0;
const timers = new Map(), intervals = new Map(), pending = [];
const document = { head: new Element("head"), createElement: (tag) => new Element(tag), getElementById: () => null };
const app = { canvas: {}, extensionManager: { workflow: { activeWorkflow: { changeTracker: { captureCanvasState() { captures++; } } } } } };
const globals = {
document, app, console, ...math, ...links, commitWidgetValue, comboValues,
api: { fetchApi: (url, options) => new Promise((resolve) => pending.push({ body: JSON.parse(options.body), resolve })) },
setTimeout: (fn, delay) => { const id = ++next; timers.set(id, { fn, at: now + delay }); return id; },
clearTimeout: (id) => timers.delete(id),
setInterval: (fn) => { const id = ++next; intervals.set(id, fn); return id; },
clearInterval: (id) => intervals.delete(id),
queueMicrotask: (fn) => fn(),
liftSocket() {},
attachScriptColours: () => ({ sync() {}, dispose() {} }),
attachScriptAssist: () => ({ dispose() {} }),
};
// Only browser imports are replaced; the controls and history notifier run as shipped.
const load = (file, names) => {
const source = readFileSync(new URL(`../js/shared/${file}`, import.meta.url), "utf8")
.replace(/^import .*;\n/gm, "").replace(/\bexport (?=(?:function|const)\b)/g, "");
return vm.runInNewContext(`${source}\n;({${names.join(",")}})`, { ...globals }, { filename: file });
};
Object.assign(globals, load("index.mjs", ["BRAND", "chainCallback", "notifyAusbossChange"]));
return {
...load("h3_cast_chips.mjs", ["castChips"]), ...load("moment_strip.mjs", ["momentStrip"]),
pending, intervals, timers, get captures() { return captures; },
advance(ms) {
now += ms;
for (const [id, timer] of [...timers]) if (timer.at <= now) { timers.delete(id); timer.fn(); }
},
};
}
const find = (element, predicate) => predicate(element) ? element : element.children.map((child) => find(child, predicate)).find(Boolean);
const named = (name) => (element) => element.classList.contains(name);
const makeNode = (script = "She walks.\nShe stops.") => ({ widgets: [{ name: "script", value: script }], inputs: [], graph: {}, setDirtyCanvas() {} });
const values = (node) => ({ script: node.widgets[0].value, seconds: 8, job_1: "person" });
const report = (text = "line", moments = [0, 1]) => ({ ok: true, json: async () => ({ seconds: 8, moments: moments.map((line, index) => ({ start: index * 4, end: (index + 1) * 4, text: `${text}${index}`, line })) }) });
const flush = async () => { for (let i = 0; i < 6; i++) await Promise.resolve(); };
function pictureGraph(bypass = false) {
const loader = { id: 10, mode: 0, imgs: [{ src: "test.png" }], outputs: [{ type: "IMAGE" }] };
const processor = { id: 11, mode: 4, inputs: [{ type: "IMAGE", link: 2 }], outputs: [{ type: "IMAGE" }] };
const graph = { links: { 1: { origin_id: bypass ? 11 : 10, origin_slot: 0 }, 2: { origin_id: 10, origin_slot: 0 } }, getNodeById: (id) => id === 10 ? loader : id === 11 ? processor : null };
return { graph, loader, processor };
}
test("a chip job change captures history once, after the widget write", () => {
const p = page(), { graph } = pictureGraph();
const widget = { name: "job_1", value: "person", options: { values: ["person", "place"] } };
const node = { graph, widgets: [widget], inputs: [{ name: "picture_1", link: 1 }] };
const panel = p.castChips(node);
panel.sync({ job_1: "person" });
p.advance(120);
assert.equal(p.captures, 0, "sync does not dirty the workflow");
const select = find(panel.element, (element) => element.tagName === "SELECT");
select.value = "place"; select.dispatch("change");
assert.equal(widget.value, "place");
assert.equal(p.captures, 0, "capture waits for the completed write");
p.advance(120);
assert.equal(p.captures, 1);
select.dispatch("change"); p.advance(120);
assert.equal(p.captures, 1, "an unchanged job stays clean");
node.onRemoved();
});
test("a completed timeline cut captures history once", async () => {
const p = page(), node = makeNode(), panel = p.momentStrip(node);
panel.sync(values(node)); p.advance(200);
p.pending[0].resolve(report()); await flush();
const handle = find(panel.element, named("ausboss-strip-handle"));
handle.dispatch("pointerdown"); handle.dispatch("pointerup");
assert.equal(node.widgets[0].value, "She walks.\n\nShe stops.");
p.advance(120);
assert.equal(p.captures, 1);
node.onRemoved();
});
test("editing invalidates an old report before the next debounce fires", async () => {
const p = page(), node = makeNode(), panel = p.momentStrip(node);
panel.sync(values(node)); p.advance(200);
node.widgets[0].value = "new"; panel.sync(values(node));
p.pending[0].resolve(report("OLD")); await flush();
assert.doesNotMatch(panel.element.textContent, /OLD/);
p.advance(200);
p.pending[1].resolve(report("NEW", [0])); await flush();
assert.match(panel.element.textContent, /NEW/);
node.onRemoved();
});
test("removal cancels debounce, polling, and in-flight repaint", async () => {
const p = page(), node = makeNode(), panel = p.momentStrip(node);
panel.sync(values(node)); node.onRemoved(); p.advance(200);
assert.equal(p.pending.length, 0, "no request starts after removal");
assert.equal(p.intervals.size, 0);
const other = makeNode(), otherPanel = p.momentStrip(other);
otherPanel.sync(values(other)); p.advance(200); other.onRemoved();
p.pending.at(-1).resolve(report("REMOVED")); await flush();
assert.doesNotMatch(otherPanel.element.textContent, /REMOVED/);
otherPanel.sync({ script: "later", seconds: 8 }); p.advance(200);
assert.equal(p.pending.length, 1, "a detached panel stays stopped");
});
test("replacing a loader image refreshes the timeline thumbnail", async () => {
const p = page(), node = makeNode("@1 runs."), { graph, loader } = pictureGraph();
node.graph = graph; node.inputs = [{ name: "picture_1", link: 1 }];
const panel = p.momentStrip(node);
panel.sync(values(node)); p.advance(200);
p.pending.at(-1).resolve(report("@1 runs", [0])); await flush();
assert.equal(find(panel.element, (element) => element.tagName === "IMG").src, "test.png");
loader.imgs[0].src = "replacement.png";
panel.sync(values(node)); p.advance(200);
p.pending.at(-1).resolve(report("@1 runs", [0])); await flush();
assert.equal(find(panel.element, (element) => element.tagName === "IMG").src, "replacement.png");
node.onRemoved();
});
test("bypassed image processors forward pictures to chips and timeline", async () => {
const p = page(), node = makeNode("@1 runs."), { graph, processor } = pictureGraph(true);
node.graph = graph; node.inputs = [{ name: "picture_1", link: 1 }];
node.widgets.push({ name: "job_1", value: "person", options: { values: ["person"] } });
const chips = p.castChips(node), strip = p.momentStrip(node);
chips.sync(values(node)); strip.sync(values(node)); p.advance(200);
assert.equal(find(chips.element, (element) => element.tagName === "IMG")?.src, "test.png");
assert.deepEqual(p.pending.at(-1).body.jobs, { 1: "person" });
p.pending.at(-1).resolve(report("@1 runs", [0])); await flush();
assert.equal(find(strip.element, (element) => element.tagName === "IMG")?.src, "test.png");
processor.mode = 2; chips.sync(values(node)); strip.sync(values(node)); p.advance(200);
assert.match(chips.element.textContent, /\+ @1/);
assert.deepEqual(p.pending.at(-1).body.jobs, {});
node.onRemoved();
assert.equal(p.intervals.size, 0);
});
test("the chips row reserves room for three wrapped rows at minimum width", () => {
const cards = readFileSync(new URL("../js/widget_cards/index.js", import.meta.url), "utf8");
const css = readFileSync(new URL("../js/shared/h3_cast_chips.mjs", import.meta.url), "utf8");
const height = Number(cards.match(/panel: castChips, height: (\d+)/)[1]);
const chipHeight = Number(css.match(/\.ausboss-chip\{[^}]*height:(\d+)px/)[1]);
const gap = Number(css.match(/\.ausboss-chips\{[^}]*gap:(\d+)px/)[1]);
assert.ok(height >= 3 * chipHeight + 2 * gap, `${height}px cannot hold three chip rows`);
});
+38
View File
@@ -0,0 +1,38 @@
import assert from "node:assert/strict";
import test from "node:test";
import { grabZone, outputSummary, outputWindow, wavePeaks } from "../js/load_audio/audio_window.mjs";
test("the face shows what the node hands on", () => {
// Mirrors audio_window in nodes/_audio_trim_helpers.py.
assert.deepEqual(outputWindow(40, 8, 30, 15), { start: 8, end: 30, keepEnd: 23, length: 15, cut: 7 });
assert.deepEqual(outputWindow(40, 8, 12, 15), { start: 8, end: 12, keepEnd: 12, length: 4, cut: 0 });
assert.equal(outputWindow(40, 5, 0, 0).end, 40, "OUT 0 means the end of the file");
assert.equal(outputWindow(40, 30, 0, 20).keepEnd, 40, "a max past the end stops at the end");
assert.equal(outputWindow(10, 50, 0).start, 10, "a start past the end is held at the end");
assert.equal(outputWindow(10, "x", null, NaN).length, 10);
});
test("the line between IN and OUT says how long and what is cut", () => {
assert.equal(outputSummary(outputWindow(40, 8, 30, 15)), "15.0 s · cut 7.0 s");
assert.equal(outputSummary(outputWindow(200, 0, 0, 0)), "3:20.0");
assert.equal(outputSummary(outputWindow(40, 8, 30, 0), { maxFromLink: true }), "22.0 s · max wired");
assert.equal(outputSummary(outputWindow(0, 0, 0)), "");
});
test("a press grabs the nearest handle, the middle, or jumps", () => {
const win = outputWindow(40, 10, 30);
assert.equal(grabZone(win, 40, 0.25, 400), "start");
assert.equal(grabZone(win, 40, 0.75, 400), "end");
assert.equal(grabZone(win, 40, 0.5, 400), "move");
assert.equal(grabZone(win, 40, 0.05, 400), "jump");
assert.equal(grabZone(win, 0, 0.5, 400), "none");
});
test("peaks keep the loudest sample of each slice, channels mixed", () => {
const left = Float32Array.from([0, 0.5, -0.5, 0, 1, -1]);
const right = Float32Array.from([0, 0.5, -0.5, 0, 1, -1]);
const peaks = wavePeaks([left, right], 2);
assert.deepEqual(Array.from(peaks), [-0.5, 0.5, -1, 1]);
assert.equal(wavePeaks([], 3).length, 6);
});
+11 -1
View File
@@ -1,7 +1,7 @@
import assert from "node:assert/strict";
import test from "node:test";
import { stripCaption, stripRows, wiredJobs } from "../js/shared/moment_strip_math.mjs";
import { moveMoment, retime, stripCaption, stripRows, wiredJobs } from "../js/shared/moment_strip_math.mjs";
const row = (start, end, extra = {}) => ({ start, end, text: "She runs", cut: false, speech: false, ...extra });
@@ -43,3 +43,13 @@ test("only connected pictures send their job", () => {
test("a warning goes first", () => {
assert.deepEqual(stripCaption({ warning: "Too many.", moments: [row(0, 5)] }), { text: "Too many.", warn: true });
});
test("retiming replaces every timestamp format the script accepts", () => {
for (const prefix of ["0:03:", "0:03.500:", "3-6s:", "3s-6s:", "[at 3 seconds]", "at 3 secs,", "2.5s:", "3–6s:"]) {
assert.equal(retime(`${prefix} She waves.`, 0, 5), "5s: She waves.", prefix);
}
});
test("moving moments leaves clocks and time ranges in their original slots", () => {
assert.equal(moveMoment("0:03: She waves.\n3-6s: She runs.", [0, 1], 1, 0), "0:03: She runs.\n3-6s: She waves.");
});
+1
View File
@@ -116,6 +116,7 @@ test("a panel that should follow the node's height never declares computeSize",
"image_folder",
"input_preview",
"layout_boxes",
"load_audio",
"load_image_pad",
"load_video",
"lora_loader",
+15
View File
@@ -184,3 +184,18 @@ test("different problems, other kinds of error and other packs' nodes are left a
assert.equal(errors[2].errors.length, 2);
assert.equal(collapseRepeatedErrors(null), null);
});
test("an empty Load Audio asks for a sound file", () => {
const outputs = (classType) => (classType === "SaveAudio"
? { output_node: true, input: { required: { audio: ["AUDIO"] } } }
: definitionOf(classType));
const prompt = {
1: node("AUSBOSS_NODES_LoadAudio", { audio: "" }, "Song"),
2: node("SaveAudio", { audio: ["1", 0] }),
};
const found = findEmptySources(prompt, outputs);
assert.deepEqual(found.map((entry) => entry.input), ["audio"]);
assert.equal(emptySourceMessage(found), "Load a sound file first: Song");
prompt[1].inputs.audio = "song.mp3";
assert.deepEqual(findEmptySources(prompt, outputs), []);
});
+26
View File
@@ -0,0 +1,26 @@
import assert from "node:assert/strict";
import test from "node:test";
import { SEED_LOOK_PROPERTY, controlChip, nextControl, seedLook, setSeedLook } from "../js/shared/seed_look.mjs";
test("the look lives on the node, and full leaves nothing behind", () => {
const node = { properties: {} };
assert.equal(seedLook(node), "full");
assert.equal(setSeedLook(node, "compact"), "compact");
assert.equal(node.properties[SEED_LOOK_PROPERTY], "compact");
assert.equal(setSeedLook(node, "full"), "full");
assert.deepEqual(node.properties, {});
assert.equal(seedLook(null), "full");
assert.equal(seedLook({ properties: { [SEED_LOOK_PROPERTY]: "huge" } }), "full");
});
test("the chip walks Random, Fixed, +1, -1 and back", () => {
assert.equal(nextControl("randomize"), "fixed");
assert.equal(nextControl("fixed"), "increment");
assert.equal(nextControl("increment"), "decrement");
assert.equal(nextControl("decrement"), "randomize");
assert.equal(nextControl("something else"), "randomize");
assert.equal(controlChip("randomize").text, "Random");
assert.equal(controlChip("fixed").text, "Fixed");
assert.equal(controlChip(undefined).text, "Fixed");
});
+90
View File
@@ -0,0 +1,90 @@
from __future__ import annotations
import sys
import tempfile
import unittest
from pathlib import Path
import av
import numpy as np
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
del sys.modules["nodes"]
from nodes._audio_trim_helpers import audio_window, decode_audio_window
RATE = 8000
def write_ramp(path: Path, seconds: float) -> None:
"""A mono wav whose sample value tells the second it was written at."""
samples = int(seconds * RATE)
data = (np.arange(samples, dtype=np.float32) / RATE / 100.0).reshape(1, -1)
with av.open(str(path), "w") as container:
stream = container.add_stream("pcm_f32le", rate=RATE, layout="mono")
frame = av.AudioFrame.from_ndarray(data, format="flt", layout="mono")
frame.sample_rate = RATE
for packet in stream.encode(frame):
container.mux(packet)
for packet in stream.encode(None):
container.mux(packet)
class AudioWindowTests(unittest.TestCase):
def test_open_end_stays_open_until_the_length_is_known(self):
self.assertEqual(audio_window(2, 0)["end"], None)
self.assertEqual(audio_window(2, 0, duration=40)["end"], 40)
def test_max_keeps_the_start_and_cuts_the_rest(self):
window = audio_window(8, 30, 15, duration=40)
self.assertEqual((window["start"], window["end"], window["cut"]), (8, 23, 7))
def test_max_longer_than_the_pick_changes_nothing(self):
window = audio_window(8, 12, 15, duration=40)
self.assertEqual((window["end"], window["cut"]), (12, 0))
def test_max_with_an_open_end_caps_it(self):
self.assertEqual(audio_window(5, 0, 10)["end"], 15)
def test_bad_values_fall_back(self):
window = audio_window("x", None, float("nan"), duration=10)
self.assertEqual((window["start"], window["end"], window["cut"]), (0, 10, 0))
def test_end_before_start_is_an_empty_pick(self):
window = audio_window(6, 4, duration=10)
self.assertEqual(window["end"], 6)
class DecodeTests(unittest.TestCase):
def setUp(self):
self.folder = tempfile.TemporaryDirectory()
self.path = Path(self.folder.name, "ramp.wav")
write_ramp(self.path, 10.0)
def tearDown(self):
self.folder.cleanup()
def test_takes_exactly_the_pick(self):
audio = decode_audio_window(self.path, 2.0, 5.0)
waveform = audio["waveform"]
self.assertEqual(audio["sample_rate"], RATE)
self.assertEqual(tuple(waveform.shape), (1, 1, 3 * RATE))
self.assertAlmostEqual(float(waveform[0, 0, 0]) * 100, 2.0, places=3)
def test_max_cuts_the_end(self):
audio = decode_audio_window(self.path, 1.0, 9.0, 2.5)
self.assertEqual(audio["waveform"].shape[-1], int(2.5 * RATE))
def test_open_end_runs_to_the_end_of_the_file(self):
audio = decode_audio_window(self.path, 7.5, 0.0)
self.assertEqual(audio["waveform"].shape[-1], int(2.5 * RATE))
def test_start_past_the_end_says_so(self):
with self.assertRaisesRegex(ValueError, "past the end"):
decode_audio_window(self.path, 12.0, 0.0)
if __name__ == "__main__":
unittest.main()
+45 -31
View File
@@ -20,7 +20,6 @@ from nodes._h3_prompt_helpers import (
clean_words,
find_beats,
join_sections,
next_label,
snap_cuts,
split_sections,
write_prompt,
@@ -88,31 +87,6 @@ def count_moments(line):
return 1 + len(re.findall(r"\bAt \d\d:", line))
class PromptTextHelperTests(unittest.TestCase):
def test_sections_round_trip_unicode_and_windows_newlines(self):
parts = split_sections(SIX)
parts["summary"] = '[reference generation] Café 雨 — "stay."'
parts["detailed_description"] += "\n\nA sign reads: café."
text = join_sections(parts)
self.assertEqual(split_sections(text), parts)
self.assertEqual(split_sections(text.replace("\n", "\r\n")), parts)
def test_incomplete_sections_are_not_mistaken_for_a_complete_prompt(self):
text = SIX.replace("non_diegetic_music:", "music:")
self.assertIsNone(split_sections(text))
def test_next_label_uses_the_highest_number_of_the_requested_kind(self):
text = "<Audio 2> <Audio 9> <Audio 2> <Picture 30> <Audio 4>"
self.assertEqual(next_label(text, "Audio"), 10)
self.assertEqual(next_label(text, "Video"), 1)
def test_blank_words_and_punctuation_stay_well_formed(self):
for text, expected in ((None, ""), ("\t\n ", ""), ("<>", ""),
(" café\n 雨! ", "café 雨!"), ("go.", "go.")):
with self.subTest(text=text):
self.assertEqual(clean_words(text), expected)
class ReadingTests(unittest.TestCase):
def test_every_line_is_a_moment_in_the_order_typed(self):
self.assertEqual(moments(LINES), ["She lowers the camera", "She looks at us and smiles", "She raises the camera again and a bright flash fires"])
@@ -137,6 +111,26 @@ class ReadingTests(unittest.TestCase):
def test_lines_that_all_name_their_second_play_by_the_second(self):
self.assertEqual(moments("5s: she sits\n2s: she stands"), ["@0 holds still", "She stands", "She sits"])
def test_fractional_seconds_are_not_list_numbers(self):
for second in (0.5, 2.5):
script = read_script(f"She waits.\n{second}s: She waves.")
self.assertEqual(script.moments[1].at, second)
self.assertEqual(script.moments[1].text, "She waves")
def test_spoken_directives_do_not_change_the_clip(self):
script = read_script('She says "no cuts, no sound, no music."\n\nShe walks.')
self.assertFalse(script.one_take)
self.assertFalse(script.silent)
self.assertNotEqual(script.music, "N/A")
self.assertTrue(script.moments[1].cut)
self.assertTrue(parts_of('She says "no sound."\n\nsound: rain')["overall_soundscape"].startswith("Rain."))
def test_no_cuts_clears_effective_cuts_everywhere(self):
text = "no cuts\nIt starts to rain.\n\nShe stops."
self.assertFalse(any(moment.cut for moment in read_script(text).moments))
self.assertFalse(any(moment["cut"] for moment in script_report(text, 8)["moments"]))
self.assertIn("The rain keeps falling", write_prompt(text))
def test_an_opening_and_an_ending_go_where_they_belong(self):
script = read_script("ends on her crying\nstarts on a wide shot of the city\nshe walks out")
self.assertEqual([moment.text for moment in script.moments], ["A wide shot of the city", "She walks out", "Her crying"])
@@ -846,6 +840,17 @@ class SongTests(unittest.TestCase):
def test_minutes_carry_over(self):
self.assertIn("At 01:00.100,", snap_cuts("[Shot 2] At 00:59.900, a", [60.1]))
def test_beat_exhaustion_keeps_times_in_order_and_inside_the_clip(self):
detail = "\n".join(f"At 00:0{time:.3f}, a" for time in (4.0, 4.1, 4.2))
fallback = snap_cuts(detail, [1, 3, 4.5])
times = [float(second) for second in re.findall(r"At 00:(\d+\.\d+)", fallback)]
self.assertEqual(times, sorted(times))
text = snap_cuts(detail, [1, 3, 4.5], seconds=5)
times = [float(second) for second in re.findall(r"At 00:(\d+\.\d+)", text)]
self.assertEqual(len(times), 3)
self.assertTrue(all(b - a >= 0.399 for a, b in zip(times, times[1:])))
self.assertLessEqual(times[-1], 5)
def test_six_parts_take_a_song_and_a_voice_too(self):
for voice in (False, True):
for song in (None, [2.0, 4.0, 6.0]):
@@ -1044,9 +1049,10 @@ class NodeTests(unittest.TestCase):
sound = {"waveform": torch.arange(48000.0).reshape(1, 1, -1), "sample_rate": 12000}
output = AusBossH3PromptWriter().write("", 6, video=frames, video_sound=sound, video_job="keep going")
video, heard = output[5], output[6]
self.assertEqual(video.shape[0], 44)
self.assertEqual(video.shape[0], 39)
self.assertEqual(float(video[0, 0, 0, 0]), 61.0)
self.assertEqual(float(video[-1, 0, 0, 0]), 99.0)
self.assertEqual(heard["waveform"].shape[-1], 22000)
self.assertEqual(heard["waveform"].shape[-1], 19500)
self.assertEqual(float(heard["waveform"][0, 0, -1]), 47999.0)
def test_the_clip_takes_the_shape_of_the_video_or_else_the_first_picture(self):
@@ -1058,9 +1064,17 @@ class NodeTests(unittest.TestCase):
self.assertEqual(node.write("@2 waves.", 6, picture_2=wide, job_2="person")[-2:], (832, 480))
self.assertEqual(node.write("@1 dances.", 6, picture_1=tall, video=video)[-2:], (1280, 720))
def test_a_short_clip_keeps_all_it_has(self):
def test_a_short_clip_keeps_a_valid_tail_and_matching_sound(self):
frames = torch.zeros(20, 2, 2, 3)
self.assertEqual(AusBossH3PromptWriter().write("", 6, video=frames, video_job="keep going")[5].shape[0], 20)
output = AusBossH3PromptWriter().write("", 6, video=frames, video_sound=self.audio(), video_job="keep going")
self.assertEqual(output[5].shape[0], 5)
self.assertEqual(output[6]["waveform"].shape[-1], round(5 / 24 * 22050))
def test_a_short_song_segment_does_not_crash(self):
song = {"waveform": torch.from_numpy(clicks(100, seconds=10)).reshape(1, 1, -1), "sample_rate": 22050}
for duration in (0.15, 0.2, 0.3, 0.4, 0.55):
output = AusBossH3PromptWriter().write(LINES, 6, picture_1=PICTURE, song=song, song_start=10 - duration)
self.assertEqual(output[SONG]["waveform"].shape[-1], round(duration * 22050))
def test_a_swap_keeps_the_videos_length(self):
frames = torch.zeros(100, 8, 8, 3)
@@ -1069,7 +1083,7 @@ class NodeTests(unittest.TestCase):
self.assertEqual(output[FRAMES], 90)
def test_a_video_length_is_one_h3_can_make(self):
self.assertEqual([video_frames(count) for count in (124, 125, 141, 5000)], [124, 124, 141, 345])
self.assertEqual([video_frames(count) for count in (5, 124, 125, 141, 362, 5000)], [5, 124, 124, 141, 362, 362])
def test_cut_on_beat_off_keeps_the_times(self):
text = AusBossH3PromptWriter().write(LINES, 6, picture_1=PICTURE, song=self.audio(), cut_on_beat=False)[0]
-40
View File
@@ -1,7 +1,6 @@
from __future__ import annotations
from pathlib import Path
import math
import sys
import unittest
@@ -80,37 +79,10 @@ class EvaluateExpressionTests(unittest.TestCase):
with self.assertRaisesRegex(ValueError, "parsed"):
evaluate_expression("1 +")
def test_non_finite_results_are_refused(self):
for expression, variables in (
("1e309", None),
("1e308 * 2", None),
("a + 1", {"a": float("inf")}),
("a", {"a": float("nan")}),
):
with self.subTest(expression=expression, variables=variables):
with self.assertRaisesRegex(ValueError, "non-finite"):
evaluate_expression(expression, variables)
def test_negative_division_and_power_precedence(self):
for expression, expected in (("-7 // 2", -4), ("-7 % 2", 1),
("-2 ** 2", -4), ("2 ** 3 ** 2", 512)):
with self.subTest(expression=expression):
self.assertEqual(evaluate_expression(expression), expected)
class ExpressionSafetyTests(unittest.TestCase):
"""The expression is attacker-controlled; everything here must be refused."""
def test_non_text_input_is_refused(self):
for value in (None, 12, True, ["1 + 2"]):
with self.subTest(value=value), self.assertRaisesRegex(ValueError, "as text"):
evaluate_expression(value)
def test_bitwise_operators_and_assignment_are_refused(self):
for expression in ("1 << 2", "4 >> 1", "1 | 2", "1 & 2", "1 ^ 2", "~1", "(a := 1)"):
with self.subTest(expression=expression), self.assertRaises(ValueError):
evaluate_expression(expression)
def test_rejects_import(self):
with self.assertRaisesRegex(ValueError, "__import__"):
evaluate_expression("__import__('os')")
@@ -178,18 +150,6 @@ class ExpressionSafetyTests(unittest.TestCase):
class RoundHalfAwayTests(unittest.TestCase):
def test_values_immediately_below_a_half_do_not_round_up(self):
below = math.nextafter(0.5, 0.0)
for value in (below, -below):
with self.subTest(value=value):
self.assertEqual(round_half_away_from_zero(value), 0)
def test_large_exact_integers_are_unchanged(self):
# Adding 0.5 to an odd integer here rounds it to the next float.
for value in (2**52 + 1, -(2**52 + 1), 2**53, -(2**53)):
with self.subTest(value=value):
self.assertEqual(round_half_away_from_zero(float(value)), value)
def test_halves_round_away_from_zero(self):
self.assertEqual(round_half_away_from_zero(2.5), 3)
self.assertEqual(round_half_away_from_zero(-2.5), -3)
-39
View File
@@ -124,45 +124,6 @@ class TempFolders(unittest.TestCase):
class InputFolderTests(TempFolders):
def test_symlink_chains_stay_inside_the_selected_folder(self):
for kind in ("input", "output", "temp"):
root = getattr(self, kind)
target = self.touch(root / "target.png")
(root / "first.png").symlink_to(target)
(root / "second.png").symlink_to(root / "first.png")
for resolve in (resolve_input_path, resolve_server_file):
with self.subTest(folder=kind, resolver=resolve.__name__):
self.assertEqual(resolve(f"second.png [{kind}]"), target)
def test_similarly_named_sibling_and_cross_folder_links_are_refused(self):
sibling = self.root / "input-backup"
sibling.mkdir()
target = self.touch(sibling / "pic.png")
(self.input / "sibling.png").symlink_to(target)
(self.input / "output.png").symlink_to(self.made)
for resolve in (resolve_input_path, resolve_server_file):
for name in ("sibling.png", "output.png"):
with self.subTest(name=name, resolver=resolve.__name__):
with self.assertRaisesRegex(ValueError, WHOLE_PATH_REFUSAL):
resolve(name)
def test_annotated_backslash_traversal_is_refused_before_disk_access(self):
for resolve in (resolve_input_path, resolve_server_file):
for kind in ("input", "output", "temp"):
for name in (f"sub\\..\\pic.png [{kind}]", f"..\\secret.png [{kind}]",
f"sub/../pic.png [{kind}]"):
with self.subTest(name=name, resolver=resolve.__name__):
with patch.object(Path, "resolve") as disk:
with self.assertRaisesRegex(ValueError, WHOLE_PATH_REFUSAL):
resolve(name)
disk.assert_not_called()
def test_directories_cannot_be_selected_as_files(self):
for resolve in (resolve_input_path, resolve_server_file):
with self.subTest(resolver=resolve.__name__):
with self.assertRaisesRegex(ValueError, "no longer"):
resolve("sub [input]")
def test_files_inside_the_input_folder_resolve(self):
self.assertEqual(resolve_input_path("pic.png"), self.inside)
self.assertEqual(resolve_input_path("sub/pic.png"), self.nested)
-137
View File
@@ -1,137 +0,0 @@
"""Exercise preview handlers offline, with decoding replaced by small fakes."""
from __future__ import annotations
import asyncio
import json
from pathlib import Path
import sys
import tempfile
import types
import unittest
from unittest.mock import patch
from aiohttp import web
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
del sys.modules["nodes"]
from nodes import _media_helpers as media
class QueryNumberTests(unittest.TestCase):
def test_missing_and_malformed_values_use_the_default(self):
for parser, default in ((media._query_int, 17), (media._query_float, 2.5)):
for query in ({}, {"value": ""}, {"value": "words"}, {"value": None}):
with self.subTest(parser=parser.__name__, query=query):
self.assertEqual(parser(types.SimpleNamespace(query=query), "value", default), default)
def test_non_finite_numbers_use_the_default(self):
for parser, default in ((media._query_int, 17), (media._query_float, 2.5)):
for value in ("nan", "NaN", "inf", "-inf", "1e999"):
with self.subTest(parser=parser.__name__, value=value):
request = types.SimpleNamespace(query={"value": value})
self.assertEqual(parser(request, "value", default), default)
def test_finite_decimal_and_exponent_values_keep_existing_conversion(self):
request = types.SimpleNamespace(query={"value": " -1.25e1 "})
self.assertEqual(media._query_int(request, "value", 0), -12)
self.assertEqual(media._query_float(request, "value", 0.0), -12.5)
class VideoRouteTests(unittest.IsolatedAsyncioTestCase):
async def asyncSetUp(self):
# Keep the worker boundary awaitable without starting decoder threads.
async def run_worker(executor, function, *args):
return function(*args)
patcher = patch.object(asyncio.get_running_loop(), "run_in_executor", run_worker)
patcher.start()
self.addCleanup(patcher.stop)
def setUp(self):
tmp = tempfile.TemporaryDirectory()
self.addCleanup(tmp.cleanup)
self.root = Path(tmp.name)
self.dirs = {kind: self.root / kind for kind in ("input", "output", "temp")}
for root in self.dirs.values():
root.mkdir()
(root / "clip.mp4").write_bytes(b"placeholder")
folders = types.SimpleNamespace(**{
f"get_{kind}_directory": lambda root=root: str(root)
for kind, root in self.dirs.items()
})
patcher = patch.object(media, "folder_paths", folders)
patcher.start()
self.addCleanup(patcher.stop)
# Only collect route decorators; no server or listener is started.
instance = types.SimpleNamespace(routes=web.RouteTableDef())
server = types.ModuleType("server")
server.PromptServer = types.SimpleNamespace(instance=instance)
with patch.dict(sys.modules, {"server": server}):
media.register_video_routes()
self.handlers = {route.path: route.handler for route in instance.routes}
async def call(self, endpoint, **query):
request = types.SimpleNamespace(query=query)
return await self.handlers[f"/ausboss/transform/video/{endpoint}"](request)
async def test_annotated_names_select_each_allowed_root(self):
for kind, root in self.dirs.items():
with self.subTest(folder=kind):
with patch.object(media, "cached_video_metadata", return_value={"fps": 24}) as worker:
response = await self.call("metadata", video=f"clip.mp4 [{kind}]")
self.assertEqual(response.status, 200)
self.assertEqual(json.loads(response.text), {"fps": 24})
worker.assert_called_once_with(root / "clip.mp4")
async def test_invalid_sources_never_reach_preview_workers(self):
outside = self.root / "outside.mp4"
outside.write_bytes(b"keep")
(self.dirs["input"] / "escape.mp4").symlink_to(outside)
sources = (
{"video": "../outside.mp4"},
{"video": str(outside)},
{"video": "escape.mp4"},
{"source_mode": "local path", "local_path": "../outside.mp4 [output]"},
{"source_mode": "unexpected", "video": "clip.mp4"},
)
for endpoint, name in (("metadata", "cached_video_metadata"),
("frame", "cached_preview"), ("storyboard", "storyboard_payload")):
for query in sources:
with self.subTest(endpoint=endpoint, query=query):
with patch.object(media, name) as worker:
response = await self.call(endpoint, **query)
self.assertEqual(response.status, 400)
self.assertIn("error", json.loads(response.text))
self.assertNotIn(str(self.root), response.text)
worker.assert_not_called()
async def test_non_finite_frame_parameters_are_normalized_before_decoding(self):
with patch.object(media, "cached_preview", return_value=(b"jpeg", 0, 0.0)) as worker:
response = await self.call(
"frame", video="clip.mp4", frame_index="inf", frame_time="nan",
max_width="1e999", max_height="-inf",
)
self.assertEqual(response.status, 200, response.text)
worker.assert_called_once_with(
self.dirs["input"] / "clip.mp4", "frame index", 0, 0.0, 1600, 1600
)
self.assertEqual(response.body, b"jpeg")
async def test_decoder_errors_do_not_reveal_server_paths(self):
for endpoint, name in (("metadata", "cached_video_metadata"),
("frame", "cached_preview"), ("storyboard", "storyboard_payload")):
with self.subTest(endpoint=endpoint):
error = ValueError(f"Cannot decode '{self.dirs['input'] / 'clip.mp4'}'")
with patch.object(media, name, side_effect=error):
response = await self.call(endpoint, video="clip.mp4")
self.assertEqual(response.status, 400)
self.assertEqual(json.loads(response.text), {"error": "The media request failed."})
if __name__ == "__main__":
unittest.main()
-19
View File
@@ -36,25 +36,6 @@ PADS = (3, 5, 2, 4) # left, top, right, bottom
class MaskContractTests(unittest.TestCase):
def test_rgba_single_pixel_axes_allow_padding_larger_than_the_source(self):
for height, width in ((1, 1), (1, 3), (3, 1)):
colors = torch.tensor([[0.2, 0.4, 0.6, 0.25], [0.8, 0.6, 0.4, 0.75]])
image = colors[:, None, None, :].expand(2, height, width, 4).clone()
original = image.clone()
for mode in PAD_MODES:
with self.subTest(size=(height, width), mode=mode):
out, mask = pad_image(image, 4, 3, 5, 6, mode, backdrop_blur=0.0)
self.assertEqual(tuple(out.shape), (2, height + 9, width + 9, 4))
self.assertEqual(tuple(mask.shape), (2, height + 9, width + 9))
self.assertTrue(torch.equal(out[:, 3:3 + height, 4:4 + width], image))
expected_mask = torch.ones_like(mask)
expected_mask[:, 3:3 + height, 4:4 + width] = 0
self.assertTrue(torch.equal(mask, expected_mask))
self.assertTrue(torch.isfinite(out).all())
if mode != "color":
self.assertTrue(torch.allclose(out, colors[:, None, None, :].expand_as(out)))
self.assertTrue(torch.equal(image, original))
def test_every_mode_shares_the_geometry_and_mask_contract(self):
image = rand_image(2, 16, 20, seed=1)
left, top, right, bottom = PADS
-60
View File
@@ -184,18 +184,6 @@ class PlanResizeTests(unittest.TestCase):
class ResampleBatchTests(unittest.TestCase):
def test_rgba_and_single_pixel_axes_keep_frames_and_channels_separate(self):
for batch_size in (1, 3):
colors = torch.arange(batch_size * 4, dtype=torch.float32).reshape(batch_size, 1, 1, 4) / 12
for source_h, source_w in ((1, 7), (7, 1), (3, 5)):
# A strided view exercises both the PIL and torch paths.
image = colors.expand(batch_size, source_h, source_w * 2, 4)[:, :, ::2]
for mode in INTERPOLATION_MODES:
with self.subTest(batch=batch_size, source=(source_h, source_w), mode=mode):
out = resample_batch(image, 9, 5, mode)
self.assertEqual(tuple(out.shape), (batch_size, 5, 9, 4))
self.assertTrue(torch.allclose(out, colors.expand_as(out), atol=1e-6))
def test_every_mode_returns_the_requested_shape(self):
image = rand_image(2, 16, 16, seed=1)
for mode in INTERPOLATION_MODES:
@@ -269,54 +257,6 @@ def run_resize(image, mask=None, **overrides):
class ApplyResizeTests(unittest.TestCase):
def test_nearest_scaling_keeps_each_image_and_mask_pixel_together(self):
mask = torch.arange(30, dtype=torch.float32).reshape(2, 3, 5) / 30
image = torch.stack((mask, 1 - mask, mask / 2, mask), dim=-1)
out, out_mask, width, height = run_resize(
image, mask, width=9, height=5, keep_proportion="stretch"
)
# Nearest pixel centres on this 5x3 -> 9x5 resize, written out so
# the check does not reuse the resampling helper's geometry.
rows = [0, 0, 1, 2, 2]
cols = [0, 0, 1, 1, 2, 3, 3, 4, 4]
self.assertEqual((width, height), (9, 5))
self.assertTrue(torch.equal(out, image[:, rows][:, :, cols]))
self.assertTrue(torch.equal(out_mask, mask[:, rows][:, :, cols]))
def test_odd_center_crop_keeps_rgba_and_each_mask_aligned(self):
for batch_size in (1, 3):
values = torch.arange(batch_size * 5 * 8, dtype=torch.float32).reshape(batch_size, 5, 8)
mask = values / values.numel()
image = mask.unsqueeze(-1).expand(-1, -1, -1, 4).clone()
original = image.clone()
out, out_mask, width, height = run_resize(
image, mask, width=5, height=5, keep_proportion="cover_crop"
)
self.assertEqual((width, height), (5, 5))
# Three excess columns: one comes off the left, two off the right.
self.assertTrue(torch.equal(out, image[:, :, 1:6]))
self.assertTrue(torch.equal(out_mask, mask[:, :, 1:6]))
self.assertTrue(torch.equal(image, original))
def test_odd_padding_keeps_rgba_and_single_or_per_frame_masks(self):
image = torch.zeros((3, 3, 5, 4))
image[..., 3] = 0.25
for mask_batch in (1, 3):
mask = torch.arange(mask_batch * 15, dtype=torch.float32).reshape(mask_batch, 3, 5) / 45
original = mask.clone()
out, out_mask, width, height = run_resize(
image, mask, width=5, height=6, keep_proportion="pad", fill_color="#ff0000"
)
self.assertEqual((width, height), (5, 6))
self.assertEqual(tuple(out.shape), (3, 6, 5, 4))
self.assertEqual(tuple(out_mask.shape), (mask_batch, 6, 5))
self.assertTrue(torch.equal(out[:, 1:4], image))
self.assertTrue(torch.equal(out_mask[:, 1:4], mask))
for rows in (slice(0, 1), slice(4, 6)):
self.assertTrue(torch.equal(out[:, rows], torch.tensor([1., 0., 0., 1.]).expand_as(out[:, rows])))
self.assertTrue(torch.all(out_mask[:, rows] == 1))
self.assertTrue(torch.equal(mask, original))
def test_stretch_hits_the_exact_target_and_reports_it(self):
image = rand_image(2, 24, 32, seed=3)
out, mask, width, height = run_resize(
+1
View File
@@ -43,6 +43,7 @@ NAMED = {
"AusBossImageCropRotatePad": ["image"],
"AusBossImageFolder": ["source", "folder"],
"AusBossLaMaInpaint": ["model"],
"AusBossLoadAudio": ["audio"],
"AusBossLoadImagePad": ["image", "source_image"],
"AusBossLoadVideo": ["video"],
"AusBossLoraLoader": ["loras", "on_missing"],