Compare commits

..
Author SHA1 Message Date
ausboss 9a1096149a Merge pull request #345 from ausboss/release/2.25.2
Release 2.25.2
2026-11-13 10:12:14 -06:00
ausboss 0401d9dd34 Release 2.25.2 2026-11-13 10:11:16 -06:00
ausboss 8961827079 Merge pull request #343 from ausboss/fix/wheel-in-text-box
Zoom the graph over a text box that has nothing to scroll
2026-11-13 10:11:06 -06:00
ausboss dea2eac8cd Merge pull request #344 from ausboss/fix/trim-in-past-length
Let the clip's IN handle drag when the Length covers the whole clip
2026-11-13 10:10:51 -06:00
ausboss 3c1eeaa486 Let the clip's IN handle drag when the Length covers the whole clip
A Length as long as the clip or longer locked IN at frame 0, and the only
way to trim the start was to turn Length off. A workflow that sets a
Length as an upper limit (240 frames, say) hit this with every shorter
clip. Now such a Length only caps the clip: the clip's own end is OUT and
IN moves over the whole clip. A Length shorter than the clip still stops
IN where OUT meets the end, as before. The row reads "= the whole clip"
and the tooltips say what dragging does.

Every IN this allows was already reachable with Length off; the backend
runs each one (checked on a 116-frame clip with Length 240: IN at 0,
1.83 s, 4.5 s and the last frame all run).
2026-11-13 10:08:47 -06:00
ausboss cf0eec8d31 Zoom the graph over a text box that has nothing to scroll
A text box on one of the pack's nodes kept the mouse wheel whenever the
cursor was in it, even when it was empty, so the graph stopped zooming
until you clicked away. A box now keeps the wheel only while it can
still scroll that way, typed in or not, as ComfyUI's own text boxes do.

Ctrl + wheel and trackpad pinch always zoom the graph; over a long text
box or list they zoomed the whole page.

The Callout editor no longer stops the wheel; the shared rule already
lets its text scroll.
2026-11-13 02:48:54 -06:00
ausboss 8618d8ca56 Merge pull request #342 from ausboss/release/2.25.1
Release 2.25.1
2026-11-13 00:52:23 -06:00
ausboss 46dde36185 Release 2.25.1 2026-11-13 00:50:15 -06:00
ausboss a29ca76fe8 Merge pull request #341 from ausboss/feat/edit-lock-tint
Edit Lock Restore: take out the pink tint the edit model adds
2026-11-13 00:46:16 -06:00
ausboss 2908eac20c Edit Lock Restore: take out the pink tint the edit model adds
MiniMax H3 Image makes every edit a little pinker, bluer and lighter
over the whole picture (Lab a +2 to +7 at 16 MP). Edit Lock removed it
where it locked, but when it stepped aside (new hair color, new
background, rain) the tint stayed.

Edit Lock Restore gets an optional remove_tint switch, on by default.
It lines the decode up with the original, reads the even Lab shift on
the part the edit left alone, and subtracts it from the whole picture
before the paste. A shift is only taken out when it looks like the
model's tint, so golden hour, red light and restyles pass untouched.

On 21 real edits at 16 MP: taken out on the 5 tinted ones (a +2.6 to
+6.5 down to +0.4 or less), left alone on the other 16. A different
sampler or less compressed weights (bf16 DiT, int8 text encoder) did
not remove it.
2026-11-13 00:44:24 -06:00
23 changed files with 420 additions and 474 deletions
+17 -6
View File
@@ -2,13 +2,24 @@
All notable changes to ComfyUI-AusBoss are documented here.
## Unreleased
## 2.25.2 - 2026-11-13
- **Load Audio 🆎: Clean up.** Turn on **Clean up** for a voice recording
and it comes out as one channel, with the quiet bits at the start and end
cut, less hum and hiss, and an even volume that never clips. A video model
that copies the sound it is given copies its noise too, so this helps a
phone recording. It is off by default, so saved workflows are unchanged.
- **The video clip's IN handle always drags.** On Video Crop + Rotate + Pad
(Clip), a Length as long as the clip or longer used to lock IN, and the
only way to trim the start was to turn Length off. Now IN drags anyway:
the Length then only caps the clip, and the clip's own end is OUT. A
Length shorter than the clip works as before.
## 2.25.1 - 2026-11-13
- **Edit Lock Restore 🆎 takes out the pink tint.** MiniMax H3 Image gives
every edit a faint pink, slightly lighter tint over the whole picture.
Edit Lock already removed it wherever it locked, but when it stepped
aside (a new hair color, a new background, rain) the tint stayed. A new
**Remove tint** switch, on by default, reads the tint from the part of
the picture the edit left alone and takes it out of the whole picture.
Edits meant to recolor everything, such as golden hour or red light, are
left as they are. Turn it off to get the old behavior.
## 2.25.0 - 2026-11-12
+1 -1
View File
@@ -193,7 +193,7 @@ example_workflows/ # example workflows (regular workflow JSON, not API JSON)
drag). Messages to the user go through `showToast`, never `alert()`.
- The mouse wheel over a node zooms the graph, on every part of its face.
`js/canvas_passthrough/` hands the wheel on from the pack's panels, and
keeps it only for a box that can still scroll or a field being typed in,
keeps it only for a box that can still scroll, typed in or not,
so a panel adds no wheel listener of its own
(`tests/canvas_passthrough.test.mjs` lists the few that exist). After a
change to a node face, `scripts/dev/wheel_sweep.mjs` turns the wheel at
+3 -3
View File
@@ -1,9 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" width="105" height="20" role="img" aria-label="release: v2.25.0">
<title>release: v2.25.0</title>
<svg xmlns="http://www.w3.org/2000/svg" width="105" height="20" role="img" aria-label="release: v2.25.2">
<title>release: v2.25.2</title>
<path fill="#555" d="M0 0h48v20H0z"/>
<path fill="#00b4aa" d="M48 0h57v20H48z"/>
<g fill="#fff" text-anchor="middle" font-family="Verdana,Geneva,DejaVu Sans,sans-serif" font-size="11">
<text x="24" y="14">release</text>
<text x="76.5" y="14">v2.25.0</text>
<text x="76.5" y="14">v2.25.2</text>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 425 B

After

Width:  |  Height:  |  Size: 425 B

-1
View File
@@ -649,7 +649,6 @@ function startEdit(state) {
if (event.target !== area) event.preventDefault();
});
wrap.addEventListener("pointerdown", (event) => event.stopPropagation());
wrap.addEventListener("wheel", (event) => event.stopPropagation(), { passive: true });
// Clicking another node on the canvas takes the keyboard from the text box.
// That is how the link is picked, so the edit stays open then; a click on
// empty canvas, or on the note itself, ends it.
+2 -2
View File
@@ -5,7 +5,7 @@
// zooming stopped over Compare, the crop previews, the video viewers and every
// control on a card. One listener here hands those gestures back, the same way
// ComfyUI does for its own text boxes. A panel that wants the wheel for itself
// (a scrolling list, a focused text field, an editor) still keeps it.
// (a scrolling list, a long text box) keeps it while it can still scroll.
//
// A file dragged in from the desktop, or a picture's card from the Assets
// panel, has the same problem in the classic renderer: dropped on a node's
@@ -31,7 +31,7 @@ function onWheel(event) {
const root = panelRoot(target);
const lgCanvas = canvas();
if (!root || !lgCanvas || event.defaultPrevented) return;
if (panelKeepsWheel(target, event, { root, host: panelHost(target), styleOf: (el) => getComputedStyle(el), activeElement: document.activeElement })) return;
if (panelKeepsWheel(target, event, { root, host: panelHost(target), styleOf: (el) => getComputedStyle(el) })) return;
event.preventDefault();
lgCanvas.processMouseWheel(event);
}
+29 -4
View File
@@ -1,6 +1,7 @@
# Edit Lock Restore
Puts your picture's own pixels back in every part the edit did not change.
Puts your picture's own pixels back in every part the edit did not change,
and takes out the faint pink tint some edit models add to the whole picture.
**Edit Lock 🆎** keeps the untouched parts in place while the picture is
made. They still pass through the VAE, which softens fine detail a little.
@@ -22,6 +23,14 @@ your picture pixel for pixel. Only the edit is new.
- **Feather** (`feather`): how softly your original fades into the edit,
in pixels. 16 is the default. The fade lies outside the edit, so the
edit itself is never mixed with the old picture.
- **Remove tint** (`remove_tint`): on by default. MiniMax H3 Image makes
every edit a little pinker and lighter all over. This reads that tint
from the part of the picture the edit left alone and takes it out of the
whole picture, the edit included. It matters most when Edit Lock stepped
aside, such as for a new hair color, a new background or rain, because
then nothing else removes it. Edits meant to recolor everything (golden
hour, red light, a new art style) are left alone. Turn it off if a color
change you asked for comes out weaker.
## Outputs
@@ -30,13 +39,15 @@ your picture pixel for pixel. Only the edit is new.
what the lock decided, or for a later step that should only touch the
edit.
- **report**: one line, such as "Edit Lock kept 82% of your picture as it
was and left 18% to the edit".
was and left 18% to the edit". When the tint was taken out it adds a
sentence that says so.
## When nothing was locked
The picture passes through unchanged, the mask is all white, and the
Your original pixels are not pasted back, the mask is all white, and the
report says why: the edit changed the whole picture, the lock was off, or
the picture and the result were different sizes.
the picture and the result were different sizes. Remove tint still works
here, so the picture can come out with the tint taken out.
## Technical details
@@ -46,3 +57,17 @@ by the same amount, and used to blend the decoded picture with your
original. If the decoded picture is not the size of your original (after
an upscale), the original is resized to fit first, which costs it some
sharpness. That is why this node goes before an upscaler.
**Remove tint.** The edit is lined up with your original (edits often come
back about one percent larger or a little shifted), both are averaged
into coarse cells of about 8 px at 1 megapixel, and their difference is
taken in Lab color. The tint is the even shift most cells share: it starts
at the median and keeps only cells within 4 Lab units of it, five times
over, so a new background or new hair drops out of the count. It is taken
out only when it looks like the model's own tint: a between +0.5 and +10,
b between -8 and 0, L between -1 and +6, read on at least 15% of the
picture. On 21 test edits at 16 megapixels it took out the tint on the 5
it should (a +2.6 to +6.5 down to +0.4 or less) and left the other 16
alone, golden hour, red light and an anime restyle included. It cannot
tell this tint from an edit that asks for a slightly pinker, lighter look
over the whole picture; turn it off for one of those.
+1 -26
View File
@@ -27,22 +27,6 @@ Max length can be wired. When the wire comes from a number node such as
Float 🆎, the wave shows the cut for that number; otherwise the length's tooltip says
`max wired` and the run still cuts the sound to the wired length.
## Clean up
Turn **Clean up** on for a voice recording you want a video model to say,
such as a line recorded on a phone. The sound that comes out:
- is one channel,
- has the quiet bits at the start and end cut off, so the voice starts right
away,
- has less hum, rumble and hiss,
- is at an even volume, never so loud that it clips.
A model that copies the sound it is given copies its noise too, so a clean
recording gives a cleaner result. Clean up works on the part you picked, so
the sound can come out a little shorter than the length shown under the wave;
`seconds` always gives the real length. Leave it off for music.
## Controls
- **Source** (`audio`): a sound file in ComfyUI's input folder. **Upload**
@@ -52,8 +36,6 @@ the sound can come out a little shorter than the length shown under the wave;
- **IN** (`start_seconds`): where the sound starts.
- **OUT** (`end_seconds`): where the sound stops. `0` means the end of the
file.
- **Clean up** (`clean_up`): on cleans up a voice recording, as above. Off
hands on the sound as it is.
## Outputs
@@ -67,11 +49,4 @@ the sound can come out a little shorter than the length shown under the wave;
- The sound keeps its own sample rate and channels.
- A start past the end of the file stops the run with a message that says so.
- The wave is drawn in the browser from the file itself; nothing extra is
stored with the workflow besides the five values above.
- Clean up runs in this order: the channels are averaged into one; a
high-pass at 80 Hz takes out rumble and mains hum; a noise print is
taken from the quietest moments and whatever sits close to it is turned
down by up to 12 dB (hiss, fans, room tone); quiet ends are cut, keeping
0.1 s before the first sound and 0.2 s after the last; then the volume is
set to -18 LUFS (the loudness broadcast meters read) with a limiter that
keeps every peak under -1 dBFS. The sample rate stays the file's own.
stored with the workflow besides the four values above.
@@ -60,8 +60,9 @@ model that paints black regions.
sits that many frames after IN and follows it when you drag IN, and dragging OUT or
typing the number changes the Length. To move the whole clip, drag IN or the small
grip in the middle of the kept part (the grip also slides a trimmed window with
Length off). When the Length is the whole clip there is nowhere to move it, and the
row says so: "= the whole clip. Shorten it to move IN." The Length stays when you
Length off). When the Length is as long as the clip or longer, the row says
"= the whole clip": the clip's own end is OUT, and dragging IN starts the clip
later without changing the Length. The Length stays when you
swap the video, so a workflow built for 97 frames keeps taking 97. Turning it on or
off never moves anything: the count becomes the OUT, or the OUT the count. If the
source ends first, the footer says how many frames are left. **To start** moves IN
+7 -4
View File
@@ -9,7 +9,9 @@
// in js/canvas_passthrough/ wires them to the page.
// The wheel belongs to the panel while it can still scroll that way (a long
// text box, a list of rows), or while a text field has the keyboard.
// text box, a list of rows). A text box with the cursor in it is no
// exception: when it has nothing left to scroll, the wheel zooms the graph,
// as it does over ComfyUI's own text boxes.
export function canScrollFurther(element, delta, axis = "y") {
if (!element || !delta) return false;
const horizontal = axis === "x";
@@ -25,10 +27,11 @@ export function canScrollFurther(element, delta, axis = "y") {
// and the panel's own root? `styleOf` is getComputedStyle, passed in so this
// stays testable without a page. `host` is the frontend's wrapper around the
// panel: a pointer on the wrapper itself is on nothing inside the panel.
export function panelKeepsWheel(target, event, { root, host, styleOf, activeElement }) {
// Ctrl + wheel, which is also how a trackpad pinch arrives, always zooms the
// graph; left to a scrolling box it would zoom the whole page.
export function panelKeepsWheel(target, event, { root, host, styleOf }) {
if (event.ctrlKey || event.metaKey) return false;
for (let element = target; element && element !== root && element !== host; element = element.parentElement) {
const tag = String(element.tagName ?? "").toUpperCase();
if ((tag === "TEXTAREA" || tag === "INPUT") && element === activeElement) return true;
const style = styleOf(element);
const overflow = style?.overflowY;
if ((overflow === "auto" || overflow === "scroll") && canScrollFurther(element, event.deltaY)) return true;
+1 -1
View File
@@ -9,7 +9,7 @@ import { app } from "../../../scripts/app.js";
// Must match `version` in pyproject.toml — scripts/release_preflight.py
// enforces it. The backend serves its copy at /ausboss/pack_version; a
// mismatch means the browser cached JavaScript from an older install.
export const AUSBOSS_JS_VERSION = "2.25.0";
export const AUSBOSS_JS_VERSION = "2.25.2";
export const BRAND = "#00b4aa"; // AusBoss teal — keep node accents consistent.
export const BRAND_DARK = "#007f78";
+10
View File
@@ -217,6 +217,16 @@ export function latestFirstFor(info, frames, everyNth = 1) {
return Math.max(0, info.count - 1 - (lastFrameFor(0, frames, everyNth)));
}
// How far a drag can move IN while a Length is on. A Length that fits stops
// IN where OUT meets the source's end, so the count stays whole. A Length
// as long as the clip or longer cannot keep its count anywhere: it only caps
// the clip, the source's end is OUT, and IN moves over the whole clip.
export function latestInFor(info, frames, everyNth = 1) {
if (!info?.count) return 0;
const latest = latestFirstFor(info, frames, everyNth);
return latest > 0 ? latest : info.count - 1;
}
// Everything the rail draws for the clip node, from the stored values:
// mode which way sets the length (lengthMode)
// first IN, the first source frame the run keeps
+7 -6
View File
@@ -32,7 +32,7 @@ import {
framesThrough,
keyboardStep,
lastFrameFor,
latestFirstFor,
latestInFor,
railZone,
setTrimFrame,
slideRange,
@@ -235,8 +235,9 @@ export function mountTransformTrim({ get, set, has = () => true, driven = () =>
let parked;
if (edge === "start") {
if (current.mode === "length") {
// A Length drags the whole window, stopping where OUT meets the end.
parked = Math.max(0, Math.min(clampFrame(frame, clip), latestFirstFor(clip, current.requested ?? get("max_frames", 1), nth)));
// A Length drags the whole window, stopping where OUT meets the end;
// a Length past the clip's end only caps it, so IN moves freely.
parked = Math.max(0, Math.min(clampFrame(frame, clip), latestInFor(clip, current.requested ?? get("max_frames", 1), nth)));
set("end_seconds", 0);
} else if (current.mode === "wired" && current.latestStart != null && !countAtRunTime()) {
parked = Math.min(clampFrame(frame, clip), current.latestStart);
@@ -359,7 +360,7 @@ export function mountTransformTrim({ get, set, has = () => true, driven = () =>
handle.title = isLocked ? lockTip(edge)
: edge === "start"
? current.mode === "free" ? "Drag IN to trim; Shift + arrow moves one frame."
: fills ? "The Length is the whole clip, so IN has nowhere to go. Shorten the Length to move IN."
: fills ? "Drag IN to start later. The Length is as long as the clip, so the clip's end is OUT."
: "Drag IN to move the clip; OUT follows at the Length. Shift + arrow moves one frame."
: current.mode === "free" ? "Drag OUT to trim; it lands on the last frame the run keeps." : "Drag OUT to change the Length; IN stays.";
handle.setAttribute("aria-valuemin", "0");
@@ -606,9 +607,9 @@ export function mountTransformTrim({ get, set, has = () => true, driven = () =>
count.set(blocked ? (current.requested ?? current.frames ?? 0) : (current.requested ?? get("max_frames", 1)));
disableControl(count, blocked, lockTip("end"));
const fills = lengthFillsClip(info(), current, everyNth());
hint.textContent = blocked ? `from ${wired}` : current.mode === "fixed" ? "exact (older setting)" : fills ? "= the whole clip. Shorten it to move IN" : active ? "" : "OUT ends the clip";
hint.textContent = blocked ? `from ${wired}` : current.mode === "fixed" ? "exact (older setting)" : fills ? "= the whole clip" : active ? "" : "OUT ends the clip";
hint.classList.toggle("note", fills && !blocked);
hint.title = fills && !blocked ? `The Length (${current.requested ?? current.frames} frames) covers the whole clip, so IN and the clip cannot move. The bar still scrubs.` : "";
hint.title = fills && !blocked ? `The Length (${current.requested ?? current.frames} frames) is as long as the clip, so the clip's end is OUT. Drag IN to start later; drag OUT or type a smaller Length to end sooner.` : "";
};
}
+7 -5
View File
@@ -133,7 +133,13 @@ const CARDS = {
{ widget: "tolerance", label: "Tolerance", decimals: 2, suffix: "×" },
],
},
AUSBOSS_NODES_EditLockRestore: { minWidth: 300, rows: [{ widget: "feather", label: "Feather", suffix: "px" }] },
AUSBOSS_NODES_EditLockRestore: {
minWidth: 300,
rows: [
{ widget: "feather", label: "Feather", suffix: "px" },
{ widget: "remove_tint", label: "Remove tint" },
],
},
AUSBOSS_NODES_ColorMatch: {
minWidth: 300,
rows: [
@@ -431,15 +437,11 @@ const CARDS = {
AUSBOSS_NODES_LoadAudio: {
// The wave, IN and OUT live in the panel under the card (js/load_audio).
minWidth: 280, first: true, hide: ["upload"],
// Older saves hand clean_up the upload button's empty value.
resetUnknown: { clean_up: false },
rows: [
{ widget: "audio", label: "Source", kind: "select",
button: { text: PANEL_LABELS.load, title: "Upload a sound file into ComfyUI's input folder", onClick: (node) => node.widgets?.find((w) => w.name === "upload")?.callback?.() } },
{ widget: "max_seconds", label: "Max length", suffix: "s", decimals: 1, trimZeros: true,
title: "The longest the sound may be. Anything past it is cut off and shows in red. 0 means no limit." },
{ widget: "clean_up", label: "Clean up", onText: PANEL_LABELS.on, offText: PANEL_LABELS.off,
title: "For a voice recording. On cuts the quiet bits at the start and end, takes out hum and hiss, and evens out the volume. Off hands on the sound as it is." },
],
},
// Load Image: the file, then where it is in the folder and the size it
-212
View File
@@ -1,212 +0,0 @@
"""Voice cleanup for Load Audio's clean_up switch.
One pass, in this order, on the part of the file the node hands on:
1. one channel (the channels are averaged);
2. a high-pass at 80 Hz for rumble and mains hum (a man's voice starts
around 85 Hz, so it keeps its depth);
3. light hiss reduction: a noise print from the quietest moments, and every
sound close to it is turned down by at most ``HISS_REDUCTION_DB``;
4. quiet stretches at the start and the end are cut, keeping a short pad;
5. an even loudness (``TARGET_LUFS``, measured the way broadcast meters do),
with a limiter that keeps every peak under ``PEAK_CEILING_DB``.
Everything is torch on the CPU, at the file's own sample rate.
"""
from __future__ import annotations
import math
import torch
import torch.nn.functional as F
from ._execution_helpers import raise_if_interrupted
HIGH_PASS_HZ = 80.0
HISS_REDUCTION_DB = 12.0
# A sound counts as hiss below this many dB over the noise print.
GATE_MARGIN_DB = 6.0
TARGET_LUFS = -18.0
PEAK_CEILING_DB = -1.0
MAX_BOOST_DB = 30.0
# Quiet ends: what stays before the first and after the last sound.
LEAD_PAD_SECONDS = 0.1
TAIL_PAD_SECONDS = 0.2
# Quieter than this everywhere is treated as silence and left alone.
SILENCE_DB = -60.0
_EPS = 1e-10
def _db(value: torch.Tensor) -> torch.Tensor:
return 20.0 * torch.log10(value.clamp_min(_EPS))
def _shape_spectrum(signal: torch.Tensor, rate: int, gain_for_hz) -> torch.Tensor:
"""Multiply the whole signal's spectrum by ``gain_for_hz(frequencies)``.
Zero phase. The signal is padded first so the filter's ring at one end
does not wrap around into the other.
"""
length = signal.shape[-1]
pad = int(rate * 0.1)
padded = F.pad(signal, (pad, pad))
spectrum = torch.fft.rfft(padded.double())
hz = torch.fft.rfftfreq(padded.shape[-1], d=1.0 / rate, dtype=torch.float64)
shaped = torch.fft.irfft(spectrum * gain_for_hz(hz), n=padded.shape[-1])
return shaped[..., pad : pad + length].to(signal.dtype)
def _high_pass_gain(hz: torch.Tensor) -> torch.Tensor:
# A fourth-order Butterworth's magnitude: -3 dB at the corner, -10 dB at
# 60 Hz hum, -16 dB at 50 Hz, -24 dB an octave under. Zero at DC.
ratio = (HIGH_PASS_HZ / hz.clamp_min(1e-6)) ** 8
return torch.where(hz > 0, 1.0 / torch.sqrt(1.0 + ratio), torch.zeros_like(hz))
def _biquad_magnitude(b, a, hz: torch.Tensor, rate: int) -> torch.Tensor:
z = torch.exp(-1j * 2.0 * math.pi * hz / rate)
top = b[0] + b[1] * z + b[2] * z * z
bottom = a[0] + a[1] * z + a[2] * z * z
return (top / bottom).abs()
def _k_weighting_gain(rate: int):
"""The loudness meter's ear curve (ITU-R BS.1770) at any sample rate."""
# Stage 1: a high shelf, +4 dB above about 1.7 kHz.
f0, gain_db, q = 1681.974450955533, 3.999843853973347, 0.7071752369554196
k = math.tan(math.pi * f0 / rate)
vh = 10.0 ** (gain_db / 20.0)
vb = vh ** 0.4996667741545416
a0 = 1.0 + k / q + k * k
shelf_b = ((vh + vb * k / q + k * k) / a0, 2.0 * (k * k - vh) / a0, (vh - vb * k / q + k * k) / a0)
shelf_a = (1.0, 2.0 * (k * k - 1.0) / a0, (1.0 - k / q + k * k) / a0)
# Stage 2: a high-pass at about 38 Hz.
f0, q = 38.13547087602444, 0.5003270373238773
k = math.tan(math.pi * f0 / rate)
a0 = 1.0 + k / q + k * k
pass_b = (1.0, -2.0, 1.0)
pass_a = (1.0, 2.0 * (k * k - 1.0) / a0, (1.0 - k / q + k * k) / a0)
def gain(hz: torch.Tensor) -> torch.Tensor:
return _biquad_magnitude(shelf_b, shelf_a, hz, rate) * _biquad_magnitude(pass_b, pass_a, hz, rate)
return gain
def integrated_loudness(mono: torch.Tensor, rate: int) -> float:
"""Integrated loudness in LUFS of a 1-D signal, or -inf for silence.
400 ms blocks every 100 ms, gated at -70 LUFS and then 10 LU under the
mean, as broadcast meters measure it.
"""
weighted = _shape_spectrum(mono, rate, _k_weighting_gain(rate)).double()
block = int(round(0.4 * rate))
hop = int(round(0.1 * rate))
if weighted.shape[-1] < block:
blocks = weighted.square().mean().reshape(1)
else:
blocks = weighted.unfold(-1, block, hop).square().mean(dim=-1)
def lufs(power: torch.Tensor) -> torch.Tensor:
return -0.691 + 10.0 * torch.log10(power.clamp_min(1e-20))
kept = blocks[lufs(blocks) > -70.0]
if kept.numel() == 0:
return float("-inf")
relative = lufs(kept.mean()) - 10.0
kept = kept[lufs(kept) > relative]
return float(lufs(kept.mean()))
def _fft_size(rate: int) -> int:
# About 40 ms: fine enough to tell a voice's harmonics from hiss.
return max(256, 1 << int(round(math.log2(rate * 0.04))))
def _reduce_hiss(mono: torch.Tensor, rate: int) -> torch.Tensor:
"""Spectral gate: a noise print from the quietest frames, then a soft,
smoothed gain that turns down whatever sits close to it."""
n_fft = _fft_size(rate)
hop = n_fft // 4
if mono.shape[-1] < n_fft * 4:
return mono # too short to learn what the noise is
window = torch.hann_window(n_fft, dtype=mono.dtype)
spec = torch.stft(mono, n_fft, hop, window=window, center=True, return_complex=True)
level = _db(spec.abs()) # bins x frames
# The quietest tenth of the frames, by their total energy, is the print.
energy = spec.abs().square().sum(dim=0)
count = max(4, energy.shape[0] // 10)
quiet = torch.topk(energy, count, largest=False).indices
noise = level[:, quiet]
threshold = noise.mean(dim=1, keepdim=True) + noise.std(dim=1, keepdim=True) + GATE_MARGIN_DB
# 0 for hiss, 1 for anything well over it, a 3 dB soft edge between.
keep = torch.sigmoid((level - threshold) / 1.5)
# Smooth across nearby bins and frames so the gain does not twinkle.
kernel = torch.outer(torch.hann_window(7, periodic=False), torch.hann_window(5, periodic=False))
kernel = (kernel / kernel.sum()).to(keep.dtype)[None, None]
keep = F.conv2d(keep[None, None], kernel, padding=(3, 2))[0, 0]
floor = 10.0 ** (-HISS_REDUCTION_DB / 20.0)
gain = floor + (1.0 - floor) * keep
return torch.istft(spec * gain, n_fft, hop, window=window, center=True, length=mono.shape[-1])
def _sound_span(mono: torch.Tensor, rate: int) -> tuple[int, int]:
"""First and last sample worth keeping, quiet ends cut, pads kept."""
frame = max(1, int(rate * 0.02))
usable = (mono.shape[-1] // frame) * frame
if usable == 0:
return 0, mono.shape[-1]
levels = _db(mono[:usable].reshape(-1, frame).square().mean(dim=1).sqrt())
loudest = float(levels.max())
if loudest < SILENCE_DB:
return 0, mono.shape[-1]
floor = float(torch.quantile(levels, 0.1))
# 10 dB over the quiet floor, or halfway to the loudest moment when the
# noise sits closer than that, and never more than 50 dB under it.
threshold = max(loudest - 50.0, min(floor + 10.0, (floor + loudest) / 2.0))
loud = torch.nonzero(levels > threshold).flatten()
first = max(0, int(loud[0]) * frame - int(rate * LEAD_PAD_SECONDS))
last = min(mono.shape[-1], (int(loud[-1]) + 1) * frame + int(rate * TAIL_PAD_SECONDS))
return first, last
def _limit(mono: torch.Tensor, rate: int, ceiling: float) -> torch.Tensor:
"""Look-ahead peak limiter: the gain dips just before a peak and comes
back after it, so nothing passes the ceiling and nothing is clipped."""
needed = (ceiling / mono.abs().clamp_min(_EPS)).clamp(max=1.0)
if float(needed.min()) >= 1.0:
return mono
reach = max(1, int(rate * 0.005))
width = 2 * reach + 1
# The lowest gain any sample within reach needs, then averaged over the
# same reach: every average is at or under what its own sample needs.
lowest = -F.max_pool1d(-needed[None, None], width, stride=1, padding=reach)
smooth = F.avg_pool1d(lowest, width, stride=1, padding=reach, count_include_pad=False)[0, 0]
return (mono * smooth).clamp(-ceiling, ceiling)
def clean_up_voice(sound: dict) -> dict:
"""ComfyUI AUDIO in, the cleaned-up voice out as one channel."""
waveform = sound["waveform"]
rate = int(sound["sample_rate"])
mono = waveform[0].float().mean(dim=0)
if mono.numel() == 0:
return {"waveform": mono.reshape(1, 1, 0), "sample_rate": rate}
raise_if_interrupted()
mono = _shape_spectrum(mono, rate, _high_pass_gain)
raise_if_interrupted()
mono = _reduce_hiss(mono, rate)
raise_if_interrupted()
first, last = _sound_span(mono, rate)
mono = mono[first:last]
loudness = integrated_loudness(mono, rate)
if math.isfinite(loudness):
gain_db = min(TARGET_LUFS - loudness, MAX_BOOST_DB)
mono = mono * (10.0 ** (gain_db / 20.0))
mono = _limit(mono, rate, 10.0 ** (PEAK_CEILING_DB / 20.0))
return {"waveform": mono.contiguous().reshape(1, 1, -1), "sample_rate": rate}
__all__ = ["clean_up_voice", "integrated_loudness"]
+153
View File
@@ -0,0 +1,153 @@
"""Edit Lock Restore's tint fix: take out the even tint an edit model adds to the whole picture.
MiniMax H3 Image gives every edit one even color shift: a little lighter,
pinker and bluer (Lab L +1 to +3, a +2 to +7, b -1 to -6 at 16 MP). Edit
Lock removes it wherever it locks, because those parts are your picture
again. When the lock steps aside (new hair color, new background, rain) the
shift stays. Here it is read from the part of the picture the edit left
alone and subtracted from every pixel.
A shift is only taken out when it looks like that tint. An edit that is
meant to recolor everything (golden hour, red light, a restyle) moves the
colors much further or the other way, or leaves too little of the picture
alone to read anything, and passes through untouched.
Pure torch, no ComfyUI imports.
"""
from __future__ import annotations
import math
import torch
import torch.nn.functional as F
from ._realign_helpers import measure, warp_to_source
# The pictures are compared at this long side; the fit is sub-pixel there.
WORK = 1536
# Cells closer than this to the running estimate count as "left alone".
NEAR = 4.0
# The tint as measured on 57 edits: lighter, pinker, bluer, within these
# bounds (Lab units). Smaller than MIN_A is not worth touching.
MIN_A = 0.5
MAX_A = 10.0
MAX_B = 0.0
MIN_B = -8.0
MIN_L = -1.0
MAX_L = 6.0
# At least this share of the picture must be left alone to read the tint.
MIN_SHARE = 0.15
_D65 = (0.95047, 1.0, 1.08883)
_TO_XYZ = ((0.4124564, 0.3575761, 0.1804375),
(0.2126729, 0.7151522, 0.0721750),
(0.0193339, 0.1191920, 0.9503041))
_TO_RGB = ((3.2404542, -1.5371385, -0.4985314),
(-0.9692660, 1.8760108, 0.0415560),
(0.0556434, -0.2040259, 1.0572252))
def rgb_to_lab(rgb: torch.Tensor) -> torch.Tensor:
"""sRGB in 0..1, [..., 3] -> CIE Lab, float64."""
x = rgb.double().clamp(0.0, 1.0)
linear = torch.where(x > 0.04045, ((x + 0.055) / 1.055) ** 2.4, x / 12.92)
xyz = linear @ torch.tensor(_TO_XYZ, dtype=torch.float64).T / torch.tensor(_D65, dtype=torch.float64)
f = torch.where(xyz > 0.008856, xyz.clamp(min=1e-9) ** (1.0 / 3.0), 7.787 * xyz + 16.0 / 116.0)
return torch.stack([116.0 * f[..., 1] - 16.0, 500.0 * (f[..., 0] - f[..., 1]),
200.0 * (f[..., 1] - f[..., 2])], -1)
def lab_to_rgb(lab: torch.Tensor) -> torch.Tensor:
"""CIE Lab, [..., 3] -> sRGB in 0..1, float64."""
fy = (lab[..., 0] + 16.0) / 116.0
f = torch.stack([fy + lab[..., 1] / 500.0, fy, fy - lab[..., 2] / 200.0], -1)
xyz = torch.where(f > 0.206893, f ** 3, (f - 16.0 / 116.0) / 7.787) * torch.tensor(_D65, dtype=torch.float64)
linear = (xyz @ torch.tensor(_TO_RGB, dtype=torch.float64).T).clamp(0.0, 1.0)
return torch.where(linear > 0.0031308, 1.055 * linear ** (1.0 / 2.4) - 0.055, 12.92 * linear).clamp(0.0, 1.0)
def _fit(image: torch.Tensor, long_side: int) -> torch.Tensor:
"""HWC float at most long_side px on its long side."""
height, width = int(image.shape[0]), int(image.shape[1])
scale = long_side / max(height, width)
if scale >= 1.0:
return image.float()
size = (max(1, round(height * scale)), max(1, round(width * scale)))
return F.interpolate(image.movedim(-1, 0)[None].float(), size=size, mode="bilinear",
align_corners=False, antialias=True)[0].movedim(0, -1)
def measure_tint(result: torch.Tensor, source: torch.Tensor) -> tuple[list[float] | None, float]:
"""The even shift in (L, a, b) from source to result, read where the edit left the picture alone.
result, source: HWC float in 0..1, any sizes. Returns (shift or None, the
share of the picture it was read on). The result is lined up with the
source first, since edits often come back slightly zoomed or moved.
"""
src = _fit(source[..., :3], WORK)
edit = _fit(result[..., :3], WORK)
fit = measure(src, edit, mode="zoom + shift")
if not fit.reliable:
return None, 0.0
height, width = int(src.shape[0]), int(src.shape[1])
scaled = F.interpolate(edit.movedim(-1, 0)[None], size=(height, width), mode="bilinear",
align_corners=False)[0].movedim(0, -1)
try:
warped = warp_to_source(scaled, fit.matrix, width, height)
warped = warped[0] if isinstance(warped, tuple) else warped
except Exception:
warped = scaled
# coarse cells of about 8 px at 1 MP, so grain and small detail drop out
cell = max(4, round(8 / math.sqrt(1e6 / (width * height))))
def cells(img: torch.Tensor) -> torch.Tensor:
return F.avg_pool2d(img.movedim(-1, 0)[None], cell, stride=cell)[0].movedim(0, -1)
diff = rgb_to_lab(cells(warped)) - rgb_to_lab(cells(src))
margin = max(1, int(diff.shape[0]) // 12) # the edges carry the warp's filled strip
diff = diff[margin:-margin, margin:-margin].reshape(-1, 3)
if diff.shape[0] < 50:
return None, 0.0
centre = diff.median(0).values
share = 1.0
# start at the median change, then keep only the cells near it: a new
# background or new hair is a big change, but not the same one everywhere
for _ in range(5):
near = (diff - centre).norm(dim=1) < NEAR
if int(near.sum()) < 50:
break
centre = diff[near].mean(0)
share = float(near.double().mean())
return [float(v) for v in centre], share
def looks_like_tint(shift: list[float] | None, share: float) -> bool:
"""True for the model's own lighter-pinker-bluer shift; False for an intended recolor."""
if shift is None or share < MIN_SHARE:
return False
light, a, b = shift
return MIN_A <= a <= MAX_A and MIN_B <= b <= MAX_B and MIN_L <= light <= MAX_L
def remove_tint(image: torch.Tensor, shift: list[float]) -> torch.Tensor:
"""Subtract one (L, a, b) shift from every pixel. image: [B, H, W, C] in 0..1."""
lab = rgb_to_lab(image[..., :3]) - torch.tensor(shift, dtype=torch.float64)
out = image.clone()
out[..., :3] = lab_to_rgb(lab).to(image.dtype)
return out
def take_out_tint(images: torch.Tensor, source: torch.Tensor) -> tuple[torch.Tensor, list[float] | None]:
"""Each picture of the batch with its tint taken out when it looks like the model's tint.
Returns the pictures and the largest shift taken out (None when none was)."""
out = images.clone()
taken: list[float] | None = None
for index in range(int(images.shape[0])):
shift, share = measure_tint(images[index], source[0])
if looks_like_tint(shift, share):
out[index:index + 1] = remove_tint(images[index:index + 1], shift)
if taken is None or abs(shift[1]) > abs(taken[1]):
taken = shift
return out, taken
+34 -5
View File
@@ -19,6 +19,7 @@ from ._edit_lock_helpers import (
describe,
restore_pixels,
)
from ._edit_tint_helpers import take_out_tint
class AusBossEditLock:
@@ -141,8 +142,10 @@ class AusBossEditLockRestore:
"Puts your picture's own pixels back in every part the edit did not "
"change. Use it right after the VAE decode, with the edit_lock "
"output of Edit Lock 🆎. The untouched parts then match your "
"original pixel for pixel; only the edit is new. When the lock "
"stepped aside the picture passes through unchanged."
"original pixel for pixel; only the edit is new. It also takes out "
"the faint pink tint some edit models add to the whole picture, "
"which matters most when the lock stepped aside (a new hair color, "
"background or weather)."
)
SEARCH_ALIASES = [
"edit lock restore",
@@ -178,6 +181,19 @@ class AusBossEditLockRestore:
},
),
},
"optional": {
"remove_tint": (
"BOOLEAN",
{
"default": True,
"tooltip": (
"Takes out the faint pink tint the edit model adds "
"to the whole picture. Edits meant to recolor "
"everything, like golden hour, are left alone."
),
},
),
},
}
RETURN_TYPES = ("IMAGE", "MASK", "STRING")
@@ -189,17 +205,30 @@ class AusBossEditLockRestore:
)
FUNCTION = "restore"
def restore(self, image, edit_lock, feather):
def restore(self, image, edit_lock, feather, remove_tint=True):
if image.ndim != 4:
raise ValueError("Edit Lock Restore: image must be an IMAGE batch.")
if not isinstance(edit_lock, EditLock):
raise ValueError("Edit Lock Restore: edit_lock needs the output of Edit Lock.")
height, width = int(image.shape[1]), int(image.shape[2])
report = describe(edit_lock)
if remove_tint and edit_lock.source is not None:
# before the paste, so the edit itself loses the tint too and meets the original evenly
image, taken = take_out_tint(image, edit_lock.source)
if taken is not None:
report += " " + tint_report(taken)
mask = edit_lock.pixel_mask(height, width)
if mask is None or edit_lock.source is None:
return (image, torch.ones((int(image.shape[0]), height, width)), describe(edit_lock))
return (image, torch.ones((int(image.shape[0]), height, width)), report)
result, edit_mask = restore_pixels(image, edit_lock.source, mask, int(feather))
return (result, edit_mask, describe(edit_lock))
return (result, edit_mask, report)
def tint_report(shift):
"""One plain sentence; the numbers are Lab units, for anyone who wants them."""
light, a, b = shift
return (f"It also took out the faint pink tint the model added to the whole picture "
f"(L {light:+.1f}, a {a:+.1f}, b {b:+.1f}).")
NODE_CLASS_MAPPINGS = {
+2 -19
View File
@@ -4,7 +4,6 @@ from __future__ import annotations
import asyncio
from ._audio_cleanup_helpers import clean_up_voice
from ._audio_trim_helpers import decode_audio_window, list_input_audio
from ._execution_helpers import ignore_extra_inputs
from ._media_helpers import resolve_input_path
@@ -18,8 +17,7 @@ class AusBossLoadAudio:
"handles to set where it starts and stops, and press play to hear just that "
"part. Set Max length to the longest sound the next step takes: anything "
"longer shows in red as cut off, so you see exactly what comes out before "
"you run anything. Turn on Clean up for a voice recording: it comes out as one "
"channel, with the quiet ends cut, less hum and hiss, and an even volume."
"you run anything."
)
SEARCH_ALIASES = [
"load audio",
@@ -30,7 +28,6 @@ class AusBossLoadAudio:
"voice",
"music",
"waveform",
"clean up voice",
"ausboss",
]
@@ -80,17 +77,6 @@ class AusBossLoadAudio:
),
},
),
"clean_up": (
"BOOLEAN",
{
"default": False,
"tooltip": (
"For a voice recording. On cuts the quiet bits at the start and end, "
"takes out hum and hiss, and evens out the volume. Off hands on the "
"sound as it is."
),
},
),
},
}
@@ -102,7 +88,7 @@ class AusBossLoadAudio:
)
FUNCTION = "load_audio"
async def load_audio(self, audio, start_seconds, end_seconds, max_seconds=0.0, clean_up=False):
async def load_audio(self, audio, start_seconds, end_seconds, max_seconds=0.0):
path = resolve_input_path(audio)
# Decoding blocks for as long as the pick is; a worker thread keeps
# ComfyUI's event loop answering, and carries the context ComfyUI
@@ -110,9 +96,6 @@ class AusBossLoadAudio:
sound = await asyncio.to_thread(
decode_audio_window, path, start_seconds, end_seconds, max_seconds
)
if clean_up is True:
# After the trim, so seconds is the length of what is handed on.
sound = await asyncio.to_thread(clean_up_voice, sound)
seconds = sound["waveform"].shape[-1] / float(sound["sample_rate"])
return (sound, float(seconds))
+1 -1
View File
@@ -3,7 +3,7 @@ name = "ausboss-nodes"
# Manager/registry search matches against this text, so it names the actual
# nodes instead of describing the collection abstractly.
description = "ComfyUI-AusBoss (AusBoss nodes): image and video nodes with compact controls: Image and Video Crop + Rotate + Pad (frame or clip), Load Image, Load Image + Pad, Image Folder, Image Resize, Align Image, Image Size, Color Match, Image Compare A/B, Realign to Source (experimental), Save Image (PNG, WebP/JXL, captions), Load Video, Load Audio, Save Video, Select Frame, Frame Interpolate, Mask Refine, Mask by Name and Mask by Touch, LaMa Inpaint, Crop For Inpaint, Stitch Inpaint, Tiled Upscale and Tiled Upscale Stitch, FlashVSR Upscale, Edit Lock and Edit Lock Restore, LoRA Loader, Krea 2 Encode and Outpaint Model Patch, Latent Size, Layout Boxes, Seed, Select Every Nth, Split Batch, Merge Batches, Text, Text Lines, Prompt Variations, H3 Prompt Writer, Integer, Float, Math Expression, Show Text, Free Memory, Workflow Note, Workflow Note Tile, Callout, Callout Tile, Workflow Switches, Run Timer, and Step Preview. Example workflows for Qwen Image 2.1, Krea 2, Klein 9B and LTX 2.3."
version = "2.25.0"
version = "2.25.2"
# comfy-cli serializes this to JSON verbatim, so `file` publishes the literal
# string {"file": "LICENSE"} to the registry listing. `text` puts a readable
# name there instead. Both forms are supported; the LICENSE file is unchanged.
+9 -3
View File
@@ -27,12 +27,19 @@ test("the graph gets the wheel unless the panel has a use for it", () => {
const list = { tagName: "DIV", parentElement: root, ...box() };
const field = { tagName: "INPUT", parentElement: root };
const styleOf = (el) => ({ overflowY: el === list ? "auto" : "visible" });
const ctx = (extra = {}) => ({ root, styleOf, activeElement: null, ...extra });
const ctx = (extra = {}) => ({ root, styleOf, ...extra });
assert.equal(panelKeepsWheel(stage, { deltaY: -100 }, ctx()), false, "plain picture stage");
assert.equal(panelKeepsWheel(list, { deltaY: 100 }, ctx()), true, "list with room to scroll");
assert.equal(panelKeepsWheel(list, { deltaY: -100 }, ctx()), false, "list at its top, wheel up zooms");
assert.equal(panelKeepsWheel(field, { deltaY: 100 }, ctx()), false, "idle field");
assert.equal(panelKeepsWheel(field, { deltaY: 100 }, ctx({ activeElement: field })), true, "field being typed in");
// A text box with the cursor in it and nothing to scroll: the graph zooms
// (issue #115, the prompt box in a workflow).
assert.equal(panelKeepsWheel(field, { deltaY: 100 }, ctx({ activeElement: field })), false, "field being typed in, nothing to scroll");
const prompt = { tagName: "TEXTAREA", parentElement: root, ...box() };
const scrolls = (el) => ({ overflowY: el === prompt || el === list ? "auto" : "visible" });
assert.equal(panelKeepsWheel(prompt, { deltaY: 100 }, ctx({ styleOf: scrolls })), true, "long text still scrolls");
assert.equal(panelKeepsWheel(prompt, { deltaY: -100 }, ctx({ styleOf: scrolls })), false, "long text at its top, wheel up zooms");
assert.equal(panelKeepsWheel(list, { deltaY: 100, ctrlKey: true }, ctx()), false, "Ctrl + wheel and pinch always zoom the graph");
});
test("middle button and Ctrl+Shift drag go to the graph; ordinary clicks do not", () => {
@@ -131,7 +138,6 @@ test("only the listed parts of the pack listen for the wheel", () => {
// face of the node.
const js = join(dirname(fileURLToPath(import.meta.url)), "..", "js");
const listed = [
"callout/index.js wrap", // the text box while a note is being edited
"canvas_passthrough/index.js window", // hands the wheel to the graph
"image_folder/index.js root", // the copy question, a window over the page
"image_folder/index.js root", // the folder list, a window over the page
+4 -3
View File
@@ -144,7 +144,9 @@
"edit_lock",
"feather"
],
"optional": [],
"optional": [
"remove_tint"
],
"outputs": [
[
"IMAGE",
@@ -570,8 +572,7 @@
"end_seconds"
],
"optional": [
"max_seconds",
"clean_up"
"max_seconds"
],
"outputs": [
[
-169
View File
@@ -1,169 +0,0 @@
from __future__ import annotations
import asyncio
import math
import sys
import tempfile
import unittest
from pathlib import Path
from unittest import mock
import av
import numpy as np
import torch
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
del sys.modules["nodes"]
from nodes._audio_cleanup_helpers import (
PEAK_CEILING_DB,
TARGET_LUFS,
clean_up_voice,
integrated_loudness,
)
RATE = 16000
def seconds(n: float) -> int:
return int(n * RATE)
def voice_like(length: float, rng: np.random.Generator) -> np.ndarray:
"""A buzzy vowel at 150 Hz with a syllable rhythm: something with the
harmonics and the stops a voice has."""
t = np.arange(seconds(length)) / RATE
tone = sum(np.sin(2 * np.pi * 150 * k * t) / k for k in range(1, 20))
rhythm = np.clip(np.sin(2 * np.pi * 3.0 * t), 0, None) ** 0.5
return (0.2 * tone * rhythm).astype(np.float32)
def sound(*channels: np.ndarray, rate: int = RATE) -> dict:
data = np.stack(channels).astype(np.float32)
return {"waveform": torch.from_numpy(data).unsqueeze(0), "sample_rate": rate}
def band_level(signal: np.ndarray, low: float, high: float) -> float:
spectrum = np.abs(np.fft.rfft(signal)) ** 2
hz = np.fft.rfftfreq(signal.size, 1 / RATE)
return 10 * math.log10(spectrum[(hz >= low) & (hz < high)].sum() + 1e-20)
class CleanUpTests(unittest.TestCase):
def setUp(self):
self.rng = np.random.default_rng(7)
def test_comes_out_as_one_channel_at_the_same_rate(self):
voice = voice_like(2.0, self.rng)
out = clean_up_voice(sound(voice, voice * 0.5, rate=RATE))
self.assertEqual(out["sample_rate"], RATE)
self.assertEqual(out["waveform"].shape[:2], (1, 1))
def test_cuts_quiet_ends_and_keeps_a_short_pad(self):
quiet = np.zeros(seconds(1.5), dtype=np.float32)
voice = voice_like(2.0, self.rng)
noisy = np.concatenate([quiet, voice, quiet, quiet])
noisy = noisy + self.rng.normal(0, 1e-4, noisy.size).astype(np.float32)
out = clean_up_voice(sound(noisy))["waveform"][0, 0]
length = out.shape[-1] / RATE
self.assertLess(length, 2.6)
self.assertGreater(length, 1.9)
def test_level_is_even_and_never_clips(self):
ceiling = 10 ** (PEAK_CEILING_DB / 20)
for scale in (0.05, 0.2, 4.0): # very quiet, quiet, clipping hot
out = clean_up_voice(sound(voice_like(3.0, self.rng) * scale))["waveform"][0, 0]
self.assertLessEqual(float(out.abs().max()), ceiling + 1e-6)
self.assertAlmostEqual(integrated_loudness(out, RATE), TARGET_LUFS, delta=1.5)
def test_takes_out_rumble_and_hum(self):
voice = voice_like(3.0, self.rng)
t = np.arange(voice.size) / RATE
hum = (0.05 * np.sin(2 * np.pi * 30 * t) + 0.05 * np.sin(2 * np.pi * 50 * t)).astype(np.float32)
before = voice + hum
out = clean_up_voice(sound(before))["waveform"][0, 0].numpy()
# Under 60 Hz against the voice's own 150 Hz and up, before and after.
gap_before = band_level(before, 140, 4000) - band_level(before, 20, 60)
gap_after = band_level(out, 140, 4000) - band_level(out, 20, 60)
self.assertGreater(gap_after - gap_before, 12)
def test_turns_down_hiss_between_words(self):
voice = voice_like(4.0, self.rng)
hiss = self.rng.normal(0, 0.01, voice.size).astype(np.float32)
before = voice + hiss
out = clean_up_voice(sound(before))["waveform"][0, 0].numpy()
def floor_gap(signal: np.ndarray) -> float:
frames = signal[: signal.size // 320 * 320].reshape(-1, 320)
levels = 10 * np.log10((frames**2).mean(axis=1) + 1e-20)
return float(levels.max() - np.quantile(levels, 0.1))
self.assertGreater(floor_gap(out) - floor_gap(before), 6)
def test_silence_and_tiny_sounds_pass_without_error(self):
silent = clean_up_voice(sound(np.zeros(seconds(1.0), dtype=np.float32)))["waveform"]
self.assertEqual(float(silent.abs().max()), 0.0)
short = clean_up_voice(sound(voice_like(0.05, self.rng)))["waveform"]
self.assertTrue(torch.isfinite(short).all())
empty = clean_up_voice(sound(np.zeros(0, dtype=np.float32)))["waveform"]
self.assertEqual(tuple(empty.shape), (1, 1, 0))
def test_loudness_meter_reads_a_full_scale_tone_as_broadcast_meters_do(self):
# A 997 Hz sine at 0 dBFS reads -3.01 LUFS on a BS.1770 meter.
t = np.arange(seconds(3.0)) / RATE
tone = torch.from_numpy(np.sin(2 * np.pi * 997 * t).astype(np.float32))
self.assertAlmostEqual(integrated_loudness(tone, RATE), -3.01, delta=0.1)
class NodeTests(unittest.TestCase):
def setUp(self):
self.folder = tempfile.TemporaryDirectory()
self.path = Path(self.folder.name, "voice.wav")
rng = np.random.default_rng(3)
voice = voice_like(2.0, rng)
quiet = np.zeros(seconds(1.0), dtype=np.float32)
data = np.stack([np.concatenate([quiet, voice, quiet])] * 2)
with av.open(str(self.path), "w") as container:
stream = container.add_stream("pcm_f32le", rate=RATE, layout="stereo")
frame = av.AudioFrame.from_ndarray(data, format="fltp", layout="stereo")
frame.sample_rate = RATE
for packet in stream.encode(frame):
container.mux(packet)
for packet in stream.encode(None):
container.mux(packet)
def tearDown(self):
self.folder.cleanup()
def run_node(self, **values):
from nodes import node_load_audio
node = node_load_audio.AusBossLoadAudio()
with mock.patch.object(node_load_audio, "resolve_input_path", return_value=self.path):
return asyncio.run(node.load_audio("voice.wav", 0.0, 0.0, **values))
def test_off_hands_on_the_sound_as_it_is(self):
audio, length = self.run_node()
self.assertEqual(tuple(audio["waveform"].shape), (1, 2, seconds(4.0)))
self.assertAlmostEqual(length, 4.0, places=3)
audio, _length = self.run_node(clean_up=False)
self.assertEqual(audio["waveform"].shape[1], 2)
def test_on_cleans_after_the_trim_and_seconds_matches(self):
audio, length = self.run_node(clean_up=True, max_seconds=3.0)
waveform = audio["waveform"]
self.assertEqual(waveform.shape[1], 1)
self.assertAlmostEqual(length, waveform.shape[-1] / RATE, places=6)
# Max length took the first 3 s (1 s quiet, 2 s voice), then the
# quiet start went: about 2 s of voice and its pads are left.
self.assertLess(length, 2.5)
def test_an_older_saves_empty_value_leaves_it_off(self):
audio, _length = self.run_node(clean_up="")
self.assertEqual(audio["waveform"].shape[1], 2)
if __name__ == "__main__":
unittest.main()
+118
View File
@@ -0,0 +1,118 @@
"""Edit Lock Restore's tint fix: the model's pink tint goes, an intended recolor stays."""
from __future__ import annotations
import sys
import unittest
from pathlib import Path
import torch
import torch.nn.functional as F
ROOT = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(ROOT))
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
del sys.modules["nodes"]
from nodes._edit_lock_helpers import EditLock # noqa: E402
from nodes._edit_tint_helpers import ( # noqa: E402
lab_to_rgb,
looks_like_tint,
measure_tint,
remove_tint,
rgb_to_lab,
take_out_tint,
)
from nodes.node_edit_lock import AusBossEditLockRestore # noqa: E402
TINT = [2.0, 5.0, -3.0] # what MiniMax H3 Image adds: lighter, pinker, bluer
def picture(seed: int = 0, size: int = 384) -> torch.Tensor:
"""A textured photo stand-in, [1, H, W, 3], mid tones so a shift never clips."""
gen = torch.Generator().manual_seed(seed)
coarse = torch.rand(1, 3, size // 16, size // 16, generator=gen)
fine = torch.rand(1, 3, size // 4, size // 4, generator=gen)
img = 0.6 * F.interpolate(coarse, size=(size, size), mode="bicubic", align_corners=False)
img = img + 0.4 * F.interpolate(fine, size=(size, size), mode="bilinear", align_corners=False)
return (0.2 + 0.6 * img).clamp(0.0, 1.0).movedim(1, -1)
def shifted(image: torch.Tensor, shift) -> torch.Tensor:
return lab_to_rgb(rgb_to_lab(image) + torch.tensor(shift, dtype=torch.float64)).float()
def edited(source: torch.Tensor, shift) -> torch.Tensor:
"""A local edit (a new patch) on top of an even shift of the whole picture."""
out = shifted(source, shift)
out[:, 120:260, 140:280] = torch.tensor([0.85, 0.85, 0.88])
return out
class TintTests(unittest.TestCase):
def test_lab_round_trip(self):
img = picture(1)
self.assertLess(float((lab_to_rgb(rgb_to_lab(img)).float() - img).abs().max()), 1e-4)
def test_reads_the_tint_on_the_untouched_part(self):
source = picture(2)
shift, share = measure_tint(edited(source, TINT)[0], source[0])
self.assertIsNotNone(shift)
for got, want in zip(shift, TINT):
self.assertAlmostEqual(got, want, delta=0.4)
self.assertGreater(share, 0.5)
def test_takes_the_tint_out(self):
source = picture(3)
out, taken = take_out_tint(edited(source, TINT), source)
self.assertIsNotNone(taken)
outside = (slice(0, 100), slice(0, 384))
self.assertLess(float((out[0][outside] - source[0][outside]).abs().mean()), 0.004)
def test_leaves_an_intended_recolor(self):
source = picture(4)
golden = [6.0, 20.0, 30.0] # golden hour: warmer and much yellower
image = edited(source, golden)
out, taken = take_out_tint(image, source)
self.assertIsNone(taken)
self.assertTrue(torch.equal(out, image))
def test_the_tint_rule(self):
self.assertTrue(looks_like_tint([2.6, 6.3, -3.4], 0.49)) # silver hair
self.assertTrue(looks_like_tint([1.8, 3.5, -1.8], 0.62)) # rain
self.assertFalse(looks_like_tint([-2.0, 29.4, 2.9], 0.05)) # red light
self.assertFalse(looks_like_tint([19.5, 2.5, -5.5], 0.07)) # anime restyle
self.assertFalse(looks_like_tint([7.8, 23.9, 31.3], 1.0)) # golden hour
self.assertFalse(looks_like_tint([0.0, 0.1, -0.1], 0.9)) # already clean
self.assertFalse(looks_like_tint(None, 0.0))
def test_noise_is_left_alone(self):
gen = torch.Generator().manual_seed(5)
image = torch.rand(1, 64, 64, 3, generator=gen)
out, taken = take_out_tint(image, torch.rand(1, 64, 64, 3, generator=gen))
self.assertIsNone(taken)
self.assertTrue(torch.equal(out, image))
def test_restore_node_takes_it_out_when_the_lock_stepped_aside(self):
source = picture(6)
lock = EditLock()
lock.status = "whole picture"
lock.source = source
image = edited(source, TINT)
out, mask, report = AusBossEditLockRestore().restore(image, lock, 16)
self.assertLess(float((out[0, :100] - source[0, :100]).abs().mean()), 0.004)
self.assertIn("tint", report)
self.assertTrue(torch.all(mask == 1.0))
# switched off, it passes through as before
out, _, report = AusBossEditLockRestore().restore(image, lock, 16, remove_tint=False)
self.assertTrue(torch.equal(out, image))
self.assertNotIn("tint", report)
def test_remove_tint_keeps_alpha(self):
image = torch.cat([picture(7), torch.ones(1, 384, 384, 1)], -1)
out = remove_tint(image, TINT)
self.assertTrue(torch.equal(out[..., 3], image[..., 3]))
if __name__ == "__main__":
unittest.main()
+11 -1
View File
@@ -181,7 +181,7 @@ test("fixed frame duration follows output fps while position follows source fps"
});
// --- One way to set the length -------------------------------------------------
import { clipLengthPlan, framesThrough, lastFrameFor, latestFirstFor, lengthMode, snapStep, snapToValid } from "../js/shared/timeline_math.mjs";
import { clipLengthPlan, framesThrough, lastFrameFor, latestFirstFor, latestInFor, lengthMode, snapStep, snapToValid } from "../js/shared/timeline_math.mjs";
test("one mode sets the length: a wired input, an old fixed count, a Length, or OUT", () => {
assert.equal(lengthMode({}), "free");
@@ -214,6 +214,16 @@ test("OUT lands on a frame the run keeps", () => {
assert.equal(latestFirstFor(info, 20), 77);
assert.equal(latestFirstFor(info, 97), 0);
assert.equal(latestFirstFor(info, 200), 0);
// A Length that fits stops IN where OUT meets the end; one as long as the
// clip or longer only caps it, so IN reaches every frame (a 240-frame
// Length on a 116-frame clip used to pin IN at 0).
assert.equal(latestInFor(info, 20), 77);
assert.equal(latestInFor(info, 97), 96);
assert.equal(latestInFor(info, 200), 96);
const short = clipInfo({ fps: 24, frame_count: 116, duration: 116 / 24 });
assert.equal(latestInFor(short, 240), 115);
const late = clipLengthPlan(short, { start: windowSeconds(short, 40, 40).start_seconds, end: 0, maxFrames: 240 });
assert.deepEqual([late.first, late.last, late.frames, late.truncated], [40, 115, 76, true]);
});
test("free trim: the handles are IN and the last frame the run keeps", () => {