Compare commits

...
36 Commits
Author SHA1 Message Date
ausboss 51993c0a2c Merge pull request #367 from ausboss/release/2.26.3 2026-11-17 14:12:39 -06:00
ausboss b043f6f1a7 Merge pull request #365 from ausboss/fix/stitch-tone-match 2026-11-17 14:12:27 -06:00
ausboss 29c986cfdc Merge main into release/2.26.3 2026-11-17 19:02:33 +00:00
ausboss 171ad422b9 Merge pull request #368 from ausboss/registry-scan-fixes 2026-11-17 13:01:54 -06:00
ausboss f4061f4255 Accept either answer for a thumbnail inside a looping link
Python 3.12 raises on the loop while resolving and answers 400; 3.13
resolves it and answers 404. Neither names the path.
2026-11-17 18:57:30 +00:00
ausboss 8d139cdfc5 Merge branch 'main' of https://github.com/ausboss/ComfyUI-AusBoss into fix/stitch-tone-match
# Conflicts:
#	CHANGELOG.md
2026-11-17 11:58:55 -06:00
ausboss fae68ecb77 Merge pull request #358 from ausboss/codex/fix-crop-image-copy
Copy original pictures by default with optional edits in the gear menu
2026-11-17 11:53:33 -06:00
ausboss 661c33a926 Merge remote-tracking branch 'origin/main' into codex/fix-crop-image-copy
# Conflicts:
#	CHANGELOG.md
2026-11-17 17:52:23 +00:00
ausboss 9b27bc9c4c Merge pull request #363 from ausboss/fix/looks-line-same-line
H3 Prompt Writer: a swap reads the new person's looks however they are typed
2026-11-17 11:51:58 -06:00
ausboss de530e2eac Merge remote-tracking branch 'origin/main' into codex/fix-crop-image-copy
# Conflicts:
#	CHANGELOG.md
2026-11-17 17:24:29 +00:00
ausboss 7d81a43133 H3 Prompt Writer: split a looks sentence off only after a full stop
"Put @1 looks like a friend here." was cut into "Put" and a looks line
for @1, because the split happened after any character. Now it happens
only where a new sentence starts (after ".", "!" or "?"), so the typed
swap line "with @1 .@1 looks like ..." still splits and a sentence that
merely holds the words stays whole. test_h3_swap_appearance passes again,
and test_h3_prompt_writer checks both cases.
2026-11-17 17:24:00 +00:00
ausboss 0067575b0c Merge branch 'main' of https://github.com/ausboss/ComfyUI-AusBoss into fix/stitch-tone-match
# Conflicts:
#	CHANGELOG.md
2026-11-17 11:23:08 -06:00
ausboss 8cc90794d8 Merge main into fix/looks-line-same-line 2026-11-17 11:23:08 -06:00
ausboss d82ed510b9 Merge pull request #362 from ausboss/fix/h3-writer-all-lines
H3 Prompt Writer: every line in a row is spoken, and a looks line keeps only the looks
2026-11-17 11:17:38 -06:00
ausboss 3f06362f9c Merge pull request #359 from ausboss/fix/h3-writer-video-jobs
H3 Prompt Writer: more video jobs work, carry a line across a cut
2026-11-17 11:17:14 -06:00
ausboss d37e0cd026 Merge pull request #366 from ausboss/docs/skill-example-count
Agent skill: Person Swap page now has 4 examples
2026-11-17 11:17:04 -06:00
ausboss 38bdddded8 Keep tool attribution off commits and PRs 2026-11-17 17:03:06 +00:00
ausboss f0f6ca9f0c Check names as text before the disk and keep paths out of errors
LoRA routes match a name against ComfyUI's list before any lookup.
Image Folder refuses drive letters and control characters anywhere in a
name, checks the joined path before resolving it, and sends a hash in
place of each picture's size and time. Library errors keep their kind
and drop their words when they name a server path.
2026-11-17 17:03:06 +00:00
ausboss d47a6743e3 Release 2.26.3 2026-11-17 16:38:19 +00:00
ausboss ab8389d598 Match new content to the picture beside it when Tone match is on
Skin painted where a shirt was has no before of its own to be read
against, so Tone match left it as painted. It is now compared with the
untouched picture right beside it and matched where the two look alike.
Something different from what is beside it is still left as painted.
2026-11-16 23:19:08 -06:00
ausboss 5b10b3a23f Leave new content as painted when Tone match is on
When the painted area held something new, the inside reading stood down
and the soft edge was read instead. A model that repaints the edge too
puts the new colour there, so that reading took the new colour for drift
and shifted the whole patch the other way: a yellow swimsuit over skin
came back with blue skin and a blue wall.

New content now holds at a zero shift. The edge is read only when the
rim is too small to judge.
2026-11-16 21:45:28 -06:00
ausboss f2d966afd1 Read the inpaint's tone drift at the size the model painted
The reading inside the painted area ran at the pasted size. On a large
picture that made the stitch several times slower, and its 32 px rim was a
sliver at the very edge of what the model painted, where the colour drifts
least.

It is now taken at the model's size and the rim is found with one pass
along each axis. A 3072 px crop reads in 2.6 s instead of 16 s on a CPU, and
a picture of any size is read the same way.
2026-11-16 20:20:23 -06:00
ausboss 182abca56f Agent skill: Person Swap page now has 4 examples 2026-11-16 19:50:23 -06:00
ausboss c1c9831378 Read an inpaint's tone drift inside the painted area
Tone match on Stitch Inpaint barely moved the colour of an inpaint. The
drift was read from the soft edge of the patch, where the sampler had
mostly kept the original, so only a fraction of it was measured.

An inpaint is now read just inside the fully painted area: the middle
colour change of the same pixels in a 32 px rim. It is used when at least
half of those pixels agree and the change is 20 LAB or less; new content
keeps the old reading and is left as painted. Outpaints, blend in and
Tone match 0 are unchanged.

Fixes #361
2026-11-16 18:59:55 -06:00
ausboss 6418940761 Merge branch 'main' of https://github.com/ausboss/ComfyUI-AusBoss into fix/h3-writer-video-jobs
# Conflicts:
#	CHANGELOG.md
2026-11-16 14:09:31 -06:00
ausboss 72af858975 Merge main into fix/looks-line-same-line 2026-11-16 14:09:24 -06:00
ausboss 80e5221403 Merge pull request #364 from ausboss/docs/agent-skill
Add an agent skill for the video workflows
2026-11-16 14:08:32 -06:00
ausboss f658ac58b0 Add an agent skill for the video workflows
npx skills add ausboss/ComfyUI-AusBoss installs skills/ausboss-workflows:
how to write the words for the MiniMax H3 Pika Person Swap and Pika Remix
workflows, which model files to get, and how to run a workflow in ComfyUI.
The folder stays out of the Registry archive.
2026-11-16 13:55:12 -06:00
ausboss adb1eaacc4 H3 Prompt Writer: read a swap's looks sentence on the same line, and clothes on their own line
Both were dropped without a message, so the swap ran with no description of
the new person and the clip often came back unchanged.
2026-11-16 12:25:33 -06:00
ausboss b0f8b5ec55 H3 Prompt Writer: a Chinese or Japanese line keeps its own full stop
Spoken words ending in 。 got a second full stop ('。.'), and words ending
in a full-width comma got ',.'. Those count as line ends now, and a line
in Chinese or Japanese that needs a stop gets 。.
2026-11-16 12:25:19 -06:00
ausboss a3984bf656 H3 Prompt Writer: every line in a row is spoken, and a looks line keeps only the looks
A line holding two quotes, like '"Get a job." @1 looks up and says "What?"'
or '@1 says "Hi" and @2 says "Bye"', was one moment, and only its first
quote reached the prompt. A quote followed by an @ (or by 'and @2') now ends
the moment.

A swap's '@1 looks like' line took the whole line as the appearance, so a
later sentence such as 'The trees behind @1 stay the same.' went into the
description with a raw @1. A sentence there that names an @ again is read
as its own script line.

Spoken words that end in a comma ('"Hi," she says') end in a full stop
instead of ',.'.
2026-11-16 11:50:03 -06:00
ausboss 232e1dcdb7 H3 Prompt Writer: more video jobs work, carry a line across a cut
- Copy its moves with no person picture keeps the video's own person and
  takes its length from the video.
- Change something with a place or style picture restyles the video from
  the picture, and an empty clip no longer gets a made-up person.
- A thing picture that the lines make do the action takes the moves in its
  own look; a held or worn thing stays a prop.
- Words ending in "..." carry across the next cut once, or end the clip
  mid-sentence on the last line.
- Clips can be 4 seconds long.
- The job lists on the chips are the node's own menu and take a click.
2026-11-15 20:05:06 -06:00
ausboss ab67a1f77f Copy the original picture by default with optional edits in settings 2026-11-15 19:05:26 -06:00
ausboss 2af3fd9c83 Fix crop image clipboard copies including editor controls 2026-11-15 16:36:14 -06:00
ausboss 9986ded109 Merge pull request #356 from ausboss/fix/cache-guard-version-order
Ask for a reload only when the pack is newer than the page
2026-11-14 18:00:18 -06:00
ausboss 197ac04a8c Ask for a reload only when the pack is newer than the page
The update guard asked for a reload whenever the server's version and the
page's differed. After the Manager switches the pack on disk, the server
keeps reporting the old version until ComfyUI restarts, so a reload in
between loads the new files and the message said the page was older when
it was newer.

Now the guard speaks only when the server's version is the newer one, and
versions compare part by part as numbers. A version that is not three
plain numbers, such as "unknown", stays silent.

A page newer than the server stays silent rather than asking for a
restart: from two version numbers it cannot be told apart from a downgrade
with the tab left open, which needs a reload, and the Manager already asks
for its own restart after a switch.
2026-11-14 17:47:11 -06:00
43 changed files with 1903 additions and 171 deletions
+6
View File
@@ -0,0 +1,6 @@
{
"attribution": {
"commit": "",
"pr": ""
}
}
+4
View File
@@ -24,6 +24,10 @@ SECURITY.md
.claude/
/docs/
# the skill an AI agent installs from the repository (npx skills add);
# nothing in the pack loads it
/skills/
# README media: GitHub shows it on the repository page, nothing in the
# pack loads it
/assets/
+101
View File
@@ -2,6 +2,107 @@
All notable changes to ComfyUI-AusBoss are documented here.
## 2.26.3 - 2026-11-17
- **Tone match now works on inpaints.** On a Crop For Inpaint 🆎 crop,
Stitch Inpaint 🆎 read the colour shift in the soft edge around the
painted area. Workflows such as Krea 2 Inpaint Masked let the model paint
a little wider than that edge, so the edge came back close to your picture
and the slider hardly did anything: in a test on skin it took off a
third of the shift at 1. It is now read just inside the painted area and
takes off more than half, on a picture of any size. A workflow that already had Tone match on for
an inpaint gives a slightly different picture, closer to the original's
colour. When the painted area holds something new, it is matched to your
picture right beside it where the two look alike, such as new skin where
a shirt was. Something different from what is beside it, such as a
swimsuit over skin, is left as painted; before, Tone match could tint the
whole patch with the opposite colour. Outpaints, blend in and Tone match 0
are unchanged.
- **Copy a crop node's picture without its handles.** Right-click the picture
on Image Crop + Rotate + Pad and choose Copy image. By default it copies
the original picture. To include crop, rotation, padding and resize, turn
on Copy with crop, rotation and padding in the gear menu. Editing controls
and guides are never copied. Open image provides a clean
picture to copy with the browser on plain HTTP addresses too.
- **H3 Prompt Writer: a swap reads the new person's looks however you type
them.** `@1 looks like ...` on the same line as the swap sentence, and
clothes on a line of their own (`@1 is wearing a black suit.`), were
left out of the prompt without a word, and the clip often came back
unchanged. Both are now used.
- **H3 Prompt Writer no longer drops lines.** When one line held two
people talking, like `The phone says: "Get a job." @1 looks up and says:
"What?"`, only the first words reached the prompt. Each quote is its own
moment now. A swap's `@1 looks like` line also keeps only the looks: a
sentence after it that names an `@` again, like `The trees behind @1 stay
the same.`, is read as its own line instead of going into the description
with a raw `@1`. `"Hi," she says` is spoken as "Hi." instead of "Hi,.",
and a Chinese or Japanese line keeps its own full stop: "。", not "。.".
- **H3 Prompt Writer can carry a line across a cut.** End the words with
`...` and the person keeps talking into the next shot, without saying the
words twice. On the last line, `...` lets the clip end mid-sentence. A line
too long for its shot carries on the same way instead of a warning.
- **H3 Prompt Writer makes 4-second clips.** The shortest length went from
5 to 4 seconds.
- **H3 Prompt Writer can put the person from your video somewhere new.** With
Copy its moves and no person picture, the person in the video keeps their
own look and moves. Give a picture the job place and type `He dances in @1.`
Before, the node asked for a person picture and stopped. Copy its moves
also takes its length from the video now, like Swap, Change something and
New voice.
- **H3 Prompt Writer can restyle a video from a picture.** With Change
something and a style or place picture, the look comes from the picture
instead of the video. Type `It looks like @1.` A clip with nobody in it,
like an empty hallway, no longer gets a made-up person in its prompt.
- **H3 Prompt Writer can make a thing do the moves.** With Copy its moves or
Swap the person and no person picture, a thing picture your lines make do
the action (a cartoon hand, a toy) takes the moves in its own look. A thing
that is held or worn stays a prop.
- **H3 Prompt Writer's job lists take a click again.** Picking a job from a
chip's list closed the list without changing the job.
- **H3 Prompt Writer's job lists match the node.** The pick list on each
picture and on the video opens under its chip in the node's own colours
and grows with the zoom. Before, it was the browser's small plain list.
- **An AI agent can now learn the video workflows from the repository.**
`npx skills add ausboss/ComfyUI-AusBoss` gives your agent a skill for the
MiniMax H3 Pika Person Swap and Pika Remix workflows: how to write the
words, which model files to get, and how to run a workflow in your
ComfyUI. The same page is at https://ausboss-examples.vercel.app/skill.md.
- **The "AusBoss was updated" message no longer asks for a reload by
mistake.** After the Manager switched the pack to another version, a
page reload before ComfyUI restarted brought up the message, telling you
the page was older than the pack when it was newer. A restart was what
was needed. Now the message only comes up when the pack really is newer
than the page, and versions such as 2.25.10 and 2.25.9 are compared as
numbers.
- **LoRA routes only open files on ComfyUI's LoRA list.** The LoRA
Loader's info, hash and preview routes used to hand the name they were
given to ComfyUI's file lookup and check the answer afterwards. Now the
name is matched against the list first, and a name that is not on it is
refused before anything on disk is looked up.
- **Image Folder 🆎 checks a folder or picture name as text before the
disk.** A drive letter in the middle of a name, or a control character,
is refused like `..`, and the joined path is checked before it is
resolved as well as after. The list no longer sends each picture's size
and time; the thumbnail's cache tag is a short hash instead.
- **Error messages no longer pass on server paths.** When a model,
decoder or encoder fails, its message can name a file on the server's
disk. SAM 3 masks, LoRA loading and video saving now keep the kind of
error and drop its words when they name a path, since node errors reach
the browser and stay in the history.
## 2.26.2 - 2026-11-14
- **Loaders never look up a file name they refused.** Before each run,
+10
View File
@@ -49,6 +49,16 @@ After an update, through Manager or `git pull`, restart ComfyUI and hard-refresh
The pack uses the Pillow, NumPy, Torch and PyAV that come with ComfyUI. Each example lists the models it needs on its Workflow Note. LaMa Inpaint also needs [`big-lama.pt`](https://github.com/Sanster/models/releases/download/add_big_lama/big-lama.pt) in `ComfyUI/models/lama/`. The pack never downloads weights by itself.
### Using an AI agent
Your agent (Claude Code, Codex, Cursor or another one) can learn the video workflows: how to write the words for them, set them up, and run them on your computer.
```bash
npx skills add ausboss/ComfyUI-AusBoss
```
Or tell it: "Read https://ausboss-examples.vercel.app/skill.md and help me use these workflows." The skill is [skills/ausboss-workflows/SKILL.md](skills/ausboss-workflows/SKILL.md). For picking and wiring single nodes, point it at the [agent guide](js/docs/agents/README.md).
### Upgrading from 1.x
2.0 removed **LM Studio Chat** (use **Text** for a fixed prompt) and Align Image's `offset_x` and `offset_y` outputs. Math Expression's `a`, `b` and `c` are sockets now instead of typed boxes, so put constants in the expression itself. 1.2 removed **Drop Shadow**, **Pad Image** (use **Load Image + Pad**) and **Frame Chooser**. Save Image writes only inside ComfyUI's output folder. The [changelog](CHANGELOG.md) has the details.
+3 -3
View File
@@ -1,9 +1,9 @@
<svg xmlns="http://www.w3.org/2000/svg" width="105" height="20" role="img" aria-label="release: v2.26.2">
<title>release: v2.26.2</title>
<svg xmlns="http://www.w3.org/2000/svg" width="105" height="20" role="img" aria-label="release: v2.26.3">
<title>release: v2.26.3</title>
<path fill="#555" d="M0 0h48v20H0z"/>
<path fill="#00b4aa" d="M48 0h57v20H48z"/>
<g fill="#fff" text-anchor="middle" font-family="Verdana,Geneva,DejaVu Sans,sans-serif" font-size="11">
<text x="24" y="14">release</text>
<text x="76.5" y="14">v2.26.2</text>
<text x="76.5" y="14">v2.26.3</text>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 425 B

After

Width:  |  Height:  |  Size: 425 B

+20 -1
View File
@@ -49,6 +49,25 @@ function askForReload(installed) {
}
}
// "2.26.2" -> [2, 26, 2]. Anything else, such as "unknown", is null.
function parseVersion(version) {
const match = /^(\d+)\.(\d+)\.(\d+)$/.exec(typeof version === "string" ? version : "");
return match ? match.slice(1).map(Number) : null;
}
// Only a server newer than the page means this tab runs old files. The page
// is newer when the Manager switched the pack on disk and the page was
// reloaded before ComfyUI restarted, and then a restart is what is needed.
// Two version numbers cannot tell that apart from a downgrade with the tab
// left open, which needs a reload, so in that case the guard says nothing.
function serverIsNewer(installed, running) {
const server = parseVersion(installed);
const page = parseVersion(running);
if (!server || !page) return false;
const step = server.findIndex((part, index) => part !== page[index]);
return step >= 0 && server[step] > page[step];
}
// The installed version this page already asked a reload for. A server that
// drops and comes back several times must not stack the same message.
let asked = null;
@@ -58,7 +77,7 @@ async function check() {
try {
const installed = (await (await api.fetchApi("/ausboss/pack_version")).json())?.version;
const running = await runningVersion();
if (!installed || installed === "unknown" || !running || installed === running || installed === asked) return;
if (!serverIsNewer(installed, running) || installed === asked) return;
asked = installed;
const listing = await (await api.fetchApi("/ausboss/pack_modules")).json();
await refreshModules(Array.isArray(listing?.modules) ? listing.modules : []);
+22 -8
View File
@@ -84,31 +84,42 @@ from.
Connect a video, then pick what it is for:
- **copy its moves**: your person does the moves in the video. It needs a
picture of your person. Say where in your lines:
`@1 dances on a rooftop at sunset.` The clothes can come from words or from an
outfit picture.
- **copy its moves**: someone does the moves in the video. With a picture of
your person, your person does them. Say where in your lines:
`@1 dances on a rooftop at sunset.` With no person picture, the person in
the video keeps their own moves and goes somewhere new: give a picture the
job **place** and type `He dances in @1.` The clothes can come from words or
from an outfit picture. It doesn't have to be a person: give a cartoon hand,
a toy or a mascot the job **thing** and type `@1 waves.` It does the moves in
its own look, and nothing of the video's subject is kept.
- **swap the person**: your person takes over from the one in the video. The
place, the moves and the camera stay. With two person pictures, the first
replaces the person on the left and the second replaces the person on the
right. To choose who goes, type one line per picture, for example
`@1 replaces the man in the hat`. Put spoken words in quotes and name
the speaker. If a voice comes from off screen, say so in the line.
A line like `@1 dances.` is not read as who goes; only a description such
as `the man in the hat` is. With two people, name who speaks.
- **change something**: keep the video and describe the change, for example
`She wears a red raincoat`. Be specific about what should change.
`She wears a red raincoat`. Be specific about what should change. To change
the whole look, connect a picture with the job **style** (or **place**, for
the same room looking different) and type `It looks like @1.` A clip with
nobody in it, such as an empty hallway, stays empty.
- **keep going**: more of the same clip, from where it ends. Say what happens
next, or leave the box empty and it just carries on. Connect the whole clip:
the node sends H3 only the last 2 seconds.
- **new voice**: the same clip with a new voice. Connect the voice, and type
what they say in quotes.
This keeps the person in the clip, so a person picture is not allowed here;
use **swap the person** to change who is in it.
If your swap keeps the video's audio, connect it to **video_sound** and turn on
**More → Clip's own sound**. The writer uses the clip to estimate when each
line is spoken and keeps only the incoming people's words in the swap prompt.
Type a time such as `3.2s:` to set it yourself. This switch starts off.
With an active video, **swap the person**, **change something** and **new voice**
use the incoming clip to set the frame count. **Length** says **From the clip**;
With an active video, **copy its moves**, **swap the person**, **change
something** and **new voice** use the incoming clip to set the frame count. **Length** says **From the clip**;
set the clip's length upstream. The bar does not show a guessed duration or offer
retiming controls for these jobs. H3 rounds the frame count to a supported length.
Other video jobs, or turning the video off, use the editable **Length** again.
@@ -137,6 +148,9 @@ workflows and scripts keep their previous behavior when it is absent.
`ends on her sitting by the window`.
- **Speech**: put the words in quotes: `@1 says "we are late."` Each person
keeps their own voice.
- **Talking over a cut**: end the words with `...` and the line keeps going
into the next shot, said once. On the last line, `...` means the clip ends
mid-sentence.
- **Words on a sign**: name the sign: `a neon sign reads "OPEN"`.
- **Sound and music**: use the **Sound** and **Music** boxes, or type
`sound: rain on a tin roof` and `music: slow sad piano`. With no music you get
@@ -195,7 +209,7 @@ as not plugged in.
- **Sound** (`sound`): what the clip sounds like, for the whole clip. Same as a
`sound:` line.
- **Music** (`music`): the music for the whole clip. Same as a `music:` line.
- **Length** (`seconds`): how long the clip is, 5 to 15.
- **Length** (`seconds`): how long the clip is, 4 to 15.
- **Camera** (`camera`): how the camera moves the whole way through.
- **Look** (`look`): from the picture keeps your photo's look. The others redraw
the clip. Clay and 3D don't work from words, so change your picture first with
@@ -24,6 +24,15 @@ pan. **Save & close** keeps your edits. **Cancel** or Escape puts everything
back, and asks first if anything changed. You do not need the editor: a run
uses what the node shows.
Right-click the picture and choose **Copy image** to copy the original picture,
without added padding, rotation, crop, resize, handles or guides. To include
your edits, open the gear and turn on **Copy with crop, rotation and padding**.
That also includes resize, and is remembered for this browser. **Open image**
follows the same choice. This works on the node and in the editor.
Direct copying needs localhost or HTTPS. On a plain HTTP address, choose
**Open image**, then right-click that
picture and use the browser's **Copy image**.
## On the node
From top to bottom:
+19 -3
View File
@@ -155,9 +155,25 @@ correction turns the corner without a crease.
The shift is subtracted from the pasted region before the blend. In the
padding that is the whole shift; inside the feathered band the paste
alpha scales it, which is exactly the share of each band pixel the
sampler painted. Source pixels stay bit-identical. A crop stitcher, which
has no source rectangle, gets a single global shift measured in its
blend band instead.
sampler painted. Source pixels stay bit-identical.
A Crop For Inpaint crop has no source rectangle, so it gets one shift for
the whole painted area. That shift is read inside the painted area, in the
32 pixels next to its edge: the same pixels before and after, taking the
middle value. Those 32 pixels are counted at the size the model painted, so
a large picture is read the same way and takes no longer. Most inpaint workflows let the model paint a little wider
than the paste, so the soft edge itself comes back close to your picture
and shows only a fraction of the drift. When the painted area holds
something new (less than half of that rim agrees, or the change is larger
than 20 LAB units, such as a black jacket over skin), it is not drift: the
old pixels in that place show something else. The new part is then
compared with your untouched picture right beside it, each side averaged
within 48 pixels of the seam. Where the two sides look alike (within 20
LAB units, on at least a quarter of the seam, such as new skin next to
skin), the gap between them is taken off. Where they do not, such as a
jacket next to skin, Tone match leaves that picture as painted. Before,
the soft edge was read there instead, and a model that repaints the edge
too could tint the whole patch: a yellow swimsuit over skin came back blue.
On a Crop + Rotate + Pad canvas the source rectangle is the crop, so it
can also hold the rotation's empty corners and any transparency. The
+114 -11
View File
@@ -4,6 +4,7 @@
// source of truth: a chip only changes their value.
import { app } from "../../../scripts/app.js";
import { BRAND, chainCallback } from "./index.mjs";
import { THEME as T, FONT_UI } from "./theme.mjs";
import { liftSocket } from "./widget_card.mjs";
import { commitWidgetValue, comboValues } from "./widget_card_math.mjs";
@@ -21,9 +22,15 @@ function ensureCss() {
.ausboss-chip img{flex:none;width:18px;height:18px;margin-left:-5px;border-radius:9px;object-fit:cover}
.ausboss-chip b{flex:none;color:${BRAND};font:700 10px ui-monospace,SFMono-Regular,Menlo,monospace}
.ausboss-chip em{flex:none;color:#6f8886;font-style:normal;font-weight:500}
.ausboss-chip select{field-sizing:content;min-width:0;max-width:120px;height:22px;padding:0 11px 0 0;border:none;background:transparent url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='8' height='5' viewBox='0 0 8 5'%3E%3Cpath d='M1 1l3 3 3-3' fill='none' stroke='%236f8886' stroke-width='1.4'/%3E%3C/svg%3E") no-repeat right 1px center;color:#d8ecea;font:inherit;appearance:none;-webkit-appearance:none;cursor:pointer;text-overflow:ellipsis}
.ausboss-chip select:focus-visible{outline:2px solid ${BRAND};outline-offset:2px}
.ausboss-chip select option{background:#0f1516;color:#d8ecea}
.ausboss-chip-pick{min-width:0;max-width:120px;height:22px;padding:0 12px 0 0;border:none;background:transparent url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='8' height='5' viewBox='0 0 8 5'%3E%3Cpath d='M1 1l3 3 3-3' fill='none' stroke='%236f8886' stroke-width='1.4'/%3E%3C/svg%3E") no-repeat right 1px center;color:#d8ecea;font:inherit;cursor:pointer;overflow:hidden;white-space:nowrap;text-overflow:ellipsis;text-align:left}
.ausboss-chip-pick:focus-visible{outline:2px solid ${BRAND};outline-offset:2px;border-radius:3px}
.ausboss-chip.open{border-color:${BRAND}}
.ausboss-chip-menu{position:fixed;z-index:10000;box-sizing:border-box;padding:.33em;border:1px solid ${T.rule};border-radius:.66em;background:${T.card};box-shadow:0 10px 32px rgba(0,0,0,.55);font:500 12px/1.25 ${FONT_UI};color:${T.inkSoft}}
.ausboss-chip-item{display:flex;align-items:center;gap:.5em;padding:.45em .75em .45em .5em;border-radius:.4em;white-space:nowrap;cursor:pointer;user-select:none}
.ausboss-chip-item::before{content:"";flex:none;width:.9em;text-align:center;color:${T.brand}}
.ausboss-chip-item.current::before{content:"\\2713"}
.ausboss-chip-item.current{color:${T.brandInk}}
.ausboss-chip-item.hot{background:${T.fill};color:${T.ink}}
.ausboss-chip.off{border-style:dashed;background:transparent;color:#56706d}
.ausboss-chip.off:hover{border-color:#2a3437}
.ausboss-chip.wired{color:#8ba3a1}
@@ -56,17 +63,109 @@ function add(parent, tag, text, className) {
return child;
}
// The open job list, one at a time across every chip on the page.
let openMenu = null;
// The job list a chip opens. A native <select> lets the browser draw its own
// list (small system font, light highlight, pinned to the select's left
// edge); this one wears the node's colours, sits under the whole chip and
// grows with the canvas zoom like the chip does.
function openJobMenu(chipElement, button, values, current, choose) {
openMenu?.close();
const menu = add(document.body, "div", undefined, "ausboss-chip-menu");
menu.setAttribute("role", "listbox");
const items = values.map((value) => {
const item = add(menu, "div", value, "ausboss-chip-item");
item.setAttribute("role", "option");
if (value === current) { item.classList.add("current"); item.setAttribute("aria-selected", "true"); }
return item;
});
let hot = -1;
const setHot = (index) => {
items[hot]?.classList.remove("hot");
hot = (index + items.length) % items.length;
items[hot]?.classList.add("hot");
items[hot]?.scrollIntoView({ block: "nearest" });
};
setHot(Math.max(0, values.indexOf(current)));
// The chip is drawn scaled with the canvas: the list matches that size,
// starts at the chip's left edge and is at least as wide as the chip.
function place() {
const box = chipElement.getBoundingClientRect();
const zoom = Math.min(2.5, Math.max(0.85, box.height / (chipElement.offsetHeight || 24)));
menu.style.fontSize = `${12 * zoom}px`;
menu.style.minWidth = `${box.width}px`;
const width = menu.offsetWidth;
const height = menu.offsetHeight;
let top = box.bottom + 4;
if (top + height > window.innerHeight - 8 && box.top - height - 4 >= 8) top = box.top - height - 4;
menu.style.left = `${Math.max(8, Math.min(box.left, window.innerWidth - width - 8))}px`;
menu.style.top = `${Math.max(8, top)}px`;
}
const abort = new AbortController();
const close = () => {
abort.abort();
menu.remove();
chipElement.classList.remove("open");
if (openMenu?.menu === menu) openMenu = null;
};
const pick = (index) => { close(); button.focus(); choose(values[index]); };
items.forEach((item, index) => {
item.addEventListener("pointermove", () => { if (hot !== index) setHot(index); });
item.addEventListener("click", () => pick(index));
});
menu.addEventListener("pointerdown", (event) => event.stopPropagation());
const outside = (event) => !menu.contains(event.target) && !button.contains(event.target);
const on = { capture: true, signal: abort.signal };
window.addEventListener("pointerdown", (event) => { if (outside(event)) close(); }, on);
window.addEventListener("wheel", (event) => { if (outside(event)) close(); }, { ...on, passive: true });
// The page fires resize on its own now and then: the list follows the chip
// instead of vanishing, and goes only when the chip itself is gone.
window.addEventListener("resize", () => (chipElement.isConnected ? place() : close()), on);
// Only the window itself losing focus (another app, another tab). Pressing an item takes
// focus off the chip's button, and a capturing blur listener would hear that and close
// the list before the click lands. The press keeps focus on the button instead.
window.addEventListener("blur", (event) => { if (event.target === window) close(); }, { signal: abort.signal });
menu.addEventListener("mousedown", (event) => event.preventDefault());
window.addEventListener("keydown", (event) => {
const step = { ArrowDown: 1, ArrowUp: -1 }[event.key];
if (step) setHot(hot + step);
else if (event.key === "Home") setHot(0);
else if (event.key === "End") setHot(items.length - 1);
else if (event.key === "Enter" || event.key === " ") pick(hot);
else if (event.key === "Escape" || event.key === "Tab") { close(); if (event.key === "Escape") button.focus(); }
else return;
if (event.key !== "Tab") { event.preventDefault(); event.stopPropagation(); }
}, on);
place();
chipElement.classList.add("open");
openMenu = { owner: button, menu, close };
}
function choice(node, name, into) {
const widget = widgetOf(node, name);
const select = add(into, "select");
for (const value of comboValues(widget)) add(select, "option", value).value = value;
select.value = widget?.value ?? "";
select.title = widget?.options?.tooltip ?? "";
select.addEventListener("change", () => {
if (widget) commitWidgetValue(node, widget, select.value, app.canvas);
const button = add(into, "button", undefined, "ausboss-chip-pick");
button.type = "button";
button.setAttribute("aria-haspopup", "listbox");
button.title = widget?.options?.tooltip ?? "";
const show = (value) => { button.textContent = String(value ?? ""); };
show(widget?.value);
const choose = (value) => {
show(value);
if (widget && value !== widget.value) commitWidgetValue(node, widget, value, app.canvas);
node.setDirtyCanvas?.(true, true);
};
button.addEventListener("pointerdown", (event) => event.stopPropagation());
button.addEventListener("click", (event) => {
event.preventDefault();
if (openMenu?.owner === button) { openMenu.close(); return; }
const values = comboValues(widget).map(String);
openJobMenu(into, button, values, String(widget?.value ?? ""), choose);
});
return (values) => { if (document.activeElement !== select) select.value = values[name] ?? ""; };
return (values) => { if (openMenu?.owner !== button) show(values[name]); };
}
// The jobs have no socket of their own, but a page saved by an earlier
@@ -90,6 +189,7 @@ export function castChips(node) {
let syncs = [];
let latest = {};
const build = (values) => {
if (openMenu && element.contains(openMenu.owner)) openMenu.close();
element.textContent = "";
syncs = [];
for (const number of [1, 2, 3, 4]) {
@@ -136,7 +236,10 @@ export function castChips(node) {
for (const run of syncs) run(values);
};
const watch = setInterval(() => sync(latest), WATCH);
chainCallback(node, "onRemoved", () => clearInterval(watch));
chainCallback(node, "onRemoved", () => {
clearInterval(watch);
if (openMenu && element.contains(openMenu.owner)) openMenu.close();
});
chainCallback(node, "onConfigure", () => queueMicrotask(() => keepOldJobLinks(node)));
return { element, sync };
}
+1 -1
View File
@@ -50,6 +50,6 @@ export function clipLengthWords(timing) {
// Keep in step with Python KEEPS_ITS_LENGTH. Other jobs use seconds normally.
export function clipSetsDuration(node, values) {
return ["swap the person", "new voice", "change something"].includes(values.video_job)
return ["copy its moves", "swap the person", "new voice", "change something"].includes(values.video_job)
&& linkSource(node, "video") !== null;
}
+1 -1
View File
@@ -9,7 +9,7 @@ import { app } from "../../../scripts/app.js";
// Must match `version` in pyproject.toml — scripts/release_preflight.py
// enforces it. The backend serves its copy at /ausboss/pack_version; a
// mismatch means the browser cached JavaScript from an older install.
export const AUSBOSS_JS_VERSION = "2.26.2";
export const AUSBOSS_JS_VERSION = "2.26.3";
export const BRAND = "#00b4aa"; // AusBoss teal — keep node accents consistent.
export const BRAND_DARK = "#007f78";
+9 -2
View File
@@ -38,7 +38,8 @@ function ensureCss() {
.ausboss-strip-handle::before{content:"";position:absolute;left:5px;top:5px;bottom:5px;width:2px;border-radius:1px;background:rgba(207,232,229,.35)}
.ausboss-strip-handle:hover::before,.ausboss-strip-handle.dragging::before{background:#cfe8e5}
.ausboss-strip-handle span{position:absolute;bottom:100%;left:50%;transform:translateX(-50%);padding:1px 4px;border-radius:3px;background:#cfe8e5;color:#0b0f10;font-size:9px;white-space:nowrap;display:none}
.ausboss-strip-handle.dragging span{display:block}
.ausboss-strip-handle:hover span,.ausboss-strip-handle.dragging span{display:block}
.ausboss-strip-handle.cuts::before{background:${BRAND}}
.ausboss-strip-block.movable{cursor:grab;touch-action:none}
.ausboss-strip-block.moving{z-index:3;cursor:grabbing;opacity:.85;box-shadow:0 4px 14px rgba(0,0,0,.5)}
.ausboss-strip-drop{position:absolute;top:-3px;bottom:-3px;z-index:4;width:3px;margin-left:-2px;border-radius:2px;background:#f5b53d;pointer-events:none}
@@ -108,7 +109,12 @@ export function momentStrip(node) {
const label = document.createElement("span");
handle.append(label);
handle.style.left = `${rows[index].from}%`;
handle.title = range ? "Drag to set when this moment starts. Click to add or remove a cut." : "Click to add or remove a cut.";
const isCut = Boolean(rows[index].cut);
handle.classList.toggle("cuts", isCut);
// Shown on hover: what a click does here. Dragging swaps it for the time.
const hint = isCut ? "\u2702 join" : "\u2702 cut";
label.textContent = hint;
handle.title = `${isCut ? "Click to join these two shots." : "Click to cut to a new shot here."}${range ? " Drag to set when this moment starts." : ""}`;
track.append(handle);
handle.addEventListener("pointerdown", (event) => {
if (event.button !== 0) return;
@@ -134,6 +140,7 @@ export function momentStrip(node) {
handle.removeEventListener("pointerup", stop);
handle.removeEventListener("pointercancel", stop);
handle.classList.remove("dragging");
label.textContent = hint;
if (!moved) commitScript((text) => toggleCut(text, rows[index].line));
else if (second !== null && second !== rows[index].start) commitScript((text) => retime(text, rows[index].line, second));
};
+5 -4
View File
@@ -59,14 +59,15 @@ export function dragRange(rows, index, seconds) {
export function stripCaption(report) {
if (report?.warning) return { text: report.warning, warn: true };
const moments = report?.moments ?? [];
if (!moments.length) return { text: "Each line becomes a moment.", warn: false };
if (!moments.length) return { text: "Each line becomes a moment. A blank line between lines is a cut.", warn: false };
const count = moments.length === 1 ? "1 moment" : `${moments.length} moments`;
const cuts = moments.filter((moment) => moment.cut).length;
if (cuts) return { text: `${count}, ${cuts === 1 ? "1 cut" : `${cuts} cuts`}.`, warn: false };
// Say how to cut, since nothing on the bar shows it until you hover an edge.
if (cuts) return { text: `${count}, ${cuts === 1 ? "1 cut" : `${cuts} cuts`}. Click a cut on the bar to join the shots.`, warn: false };
const each = moments.map((moment) => trimmed(moment.end - moment.start));
if (moments.length === 1) return { text: `${count}, ${each[0]} s, one take.`, warn: false };
if (moments.length === 1) return { text: `${count}, ${each[0]} s, one take. Add a line for the next moment, a blank line to cut.`, warn: false };
const even = each.every((length) => length === each[0]);
return { text: even ? `${count}, ${each[0]} s each, one take.` : `${count}, one take.`, warn: false };
return { text: `${even ? `${count}, ${each[0]} s each` : count}, one take. Click between two moments to cut there.`, warn: false };
}
// The seconds the bar under the script marks: every second up to 10 s,
+65 -1
View File
@@ -82,6 +82,7 @@ import {
} from "./transform_geometry.mjs";
import { clampFrame, clipInfo, frameTime } from "./timeline_math.mjs";
import { liftSocket } from "./widget_card.mjs";
import { COPY_EDITS_SETTING, copyPicture, transformPicture } from "./transform_picture.mjs";
const HIDDEN_WIDGETS = [
"image", "upload",
@@ -107,7 +108,7 @@ const RESIZE_METHODS = ["lanczos", "area", "bicubic", "bilinear", "nearest-exact
const TRANSFORM_DEFAULTS = resetTransformValues(false);
const CORE_IMAGE_PREVIEW_WIDGET = "$$canvas-image-preview";
// The gear on the image node's source card. Display choices only, kept for
// The gear on the image node's source card. Browser preferences, kept for
// every image node on this browser (settings_menu.mjs stores them).
const SETTINGS_SCOPE = "image_crop_rotate_pad";
const SETTINGS_SCHEMA = [
@@ -115,6 +116,7 @@ const SETTINGS_SCHEMA = [
key: "show_mask", label: "Show the mask on the picture", type: "toggle", default: true,
hint: "Teal over the parts of your picture the model paints: a mask drawn in the MaskEditor, or see-through parts of a PNG.",
},
COPY_EDITS_SETTING,
];
let transformSettings = null;
function settings() {
@@ -3106,6 +3108,68 @@ function drawRotateGlyph(context, x, y, radius, color) {
// node itself can still drag, and pointercancel plus mouseleave both end a
// gesture in case the capture was refused.
function attachStageHandlers(state, canvas, signal) {
if (state.kind === "image") {
canvas.addEventListener("contextmenu", (event) => {
event.preventDefault(); event.stopPropagation();
if (!state.image) return;
const picture = async () => {
let source = state.image;
const sourceWidth = state.sourceWidth, sourceHeight = state.sourceHeight;
const current = values(state.node), resize = resizeRequest(state.node);
const includeEdits = settings().copy_edits;
if (includeEdits && maskLayer(state)) {
if (value(state.node, "painted_area", "fill") === "keep picture") {
// The browser loses colours under fully transparent pixels. Core's
// RGB view restores the stored picture under MaskEditor strokes.
source = new Image();
source.src = api.apiURL(`/view?${mediaViewQuery(value(state.node, "image", ""))}&channel=rgb`);
await source.decode();
} else {
// The stage's mask layer is capped at 2048px. Copy from the source
// at full size, using the same rule for which pixels stay picture.
const full = document.createElement("canvas");
full.width = sourceWidth; full.height = sourceHeight;
const context = full.getContext("2d");
context.drawImage(source, 0, 0);
const pixels = context.getImageData(0, 0, full.width, full.height);
const map = seeThroughMap(pixels.data);
if (map) for (let i = 0; i < map.length; i++) pixels.data[i * 4 + 3] = map[i] ? 0 : 255;
context.putImageData(pixels, 0, 0);
source = full;
}
}
return transformPicture({
sourceWidth, sourceHeight, values: current, resize, includeEdits,
fill: normalizeColor(current.fill_color),
drawSource: (context) => context.drawImage(source, -sourceWidth / 2, -sourceHeight / 2, sourceWidth, sourceHeight),
});
};
const report = (error) => showToast({ severity: "error", summary: "Copy image", detail: error?.message ?? String(error) });
new globalThis.LiteGraph.ContextMenu([
{ content: "Copy image", callback: () => {
try {
Promise.resolve(copyPicture(picture)).then(
() => showToast({ summary: "Copy image", severity: "success", detail: "The picture is on the clipboard." }), report,
);
} catch (error) { report(error); }
} },
{ content: "Open image", callback: async () => {
try {
// A real image retains the browser's native Copy image on plain
// HTTP too. A data URL stays usable for as long as the tab is open.
const tab = window.open("about:blank", "_blank");
if (!tab) throw new Error("Allow pop-ups to open the picture.");
tab.opener = null;
const image = tab.document.createElement("img");
image.src = (await picture()).toDataURL("image/png");
image.style.maxWidth = "100%";
tab.document.title = "Crop image — right-click to copy";
tab.document.body.append(image);
} catch (error) { report(error); }
} },
], { event });
}, { signal });
}
canvas.addEventListener("pointerdown", (event) => pointerDown(state, canvas, event), { signal });
canvas.addEventListener("pointermove", (event) => pointerMove(state, canvas, event), { signal });
canvas.addEventListener("pointerup", (event) => pointerUp(state, canvas, event), { signal });
+54
View File
@@ -0,0 +1,54 @@
import { keptCrop, resolveCrop, resolvePadding, rotatedSize, scaleToMegapixels } from "./transform_geometry.mjs";
export const COPY_EDITS_SETTING = Object.freeze({
key: "copy_edits", label: "Copy with crop, rotation and padding", type: "toggle", default: false,
hint: "Off: copy the original picture. On: include crop, rotation, padding and resize. Handles and guides are never copied.",
});
// Render picture pixels into a separate canvas. The editor's canvas contains
// handles, hatching and stage margins and must never be a clipboard source.
export function transformPicture({ sourceWidth, sourceHeight, values, resize, fill, drawSource, includeEdits = false, createCanvas = () => document.createElement("canvas") }) {
const canvas = createCanvas();
if (!includeEdits) {
canvas.width = sourceWidth; canvas.height = sourceHeight;
const context = canvas.getContext("2d");
if (!context) throw new Error("The browser could not make the picture.");
context.translate(sourceWidth / 2, sourceHeight / 2);
drawSource(context);
return canvas;
}
const source = rotatedSize(sourceWidth, sourceHeight, values.rotation_degrees);
const whole = resolveCrop(values, source);
const padding = resolvePadding(values, whole);
const crop = keptCrop(whole, padding);
const size = resize
? scaleToMegapixels(padding.outputWidth, padding.outputHeight, resize.megapixels, resize.steps)
: { width: padding.outputWidth, height: padding.outputHeight };
canvas.width = size.width; canvas.height = size.height;
const context = canvas.getContext("2d");
if (!context) throw new Error("The browser could not make the picture.");
context.fillStyle = fill;
context.fillRect(0, 0, canvas.width, canvas.height);
context.save();
context.scale(size.width / padding.outputWidth, size.height / padding.outputHeight);
context.beginPath();
context.rect(padding.left, padding.top, crop.width, crop.height);
context.clip();
context.translate(padding.left - crop.x + source.width / 2, padding.top - crop.y + source.height / 2);
context.rotate((Number(values.rotation_degrees) || 0) * Math.PI / 180);
drawSource(context);
context.restore();
return canvas;
}
// Start the clipboard write in the click's activation, including in browsers
// that require a promised PNG rather than a write after toBlob has finished.
export function copyPicture(picture, { clipboard = globalThis.navigator?.clipboard, ClipboardItem = globalThis.ClipboardItem } = {}) {
if (!clipboard?.write || !ClipboardItem) throw new Error("Open ComfyUI on localhost or HTTPS to copy directly. Choose Open image, then right-click the picture to copy it instead.");
const png = Promise.resolve().then(() => typeof picture === "function" ? picture() : picture).then((canvas) => new Promise((resolve, reject) => {
canvas.toBlob((blob) => blob ? resolve(blob) : reject(new Error("The browser could not make the picture.")), "image/png");
}));
// Also observe encoding failure when write() is refused before it reads png.
png.catch(() => {});
return clipboard.write([new ClipboardItem({ "image/png": png })]);
}
+230 -40
View File
@@ -19,6 +19,10 @@ STYLE = "style"
START = "start on it"
END = "end on it"
JOBS = (PERSON, SHEET, PLACE, OUTFIT, THING, STYLE, START, END)
# Not a pick on the node: a thing picture that does the action of a move copy or a swap (a cartoon hand, a toy,
# a mascot). It takes the place a person would, with its own shape and look.
PERFORMER = "thing that moves"
ACTORS = (PERSON, SHEET, PERFORMER)
MOVES = "copy its moves"
SWAP = "swap the person"
@@ -26,7 +30,7 @@ KEEP = "keep going"
REVOICE = "new voice"
EDIT = "change something"
VIDEO_JOBS = (MOVES, SWAP, KEEP, REVOICE, EDIT)
KEEPS_ITS_LENGTH = (SWAP, REVOICE, EDIT) # These change a clip, so the new one runs as long as the old one.
KEEPS_ITS_LENGTH = (MOVES, SWAP, REVOICE, EDIT) # These follow a clip, so the new one runs as long as the old one.
NEEDS_NO_WORDS = (SWAP, KEEP)
TOKEN = re.compile(r"@(\d)\b")
@@ -35,14 +39,31 @@ MAIN = "@0" # The main person, for lines the node writes itself.
BACKGROUND = "The background of {picture} is not kept."
NOTHING_IN = "Connect a picture or a video. H3 needs one to start from."
NEEDS_A_PERSON = {
MOVES: "To copy the moves, connect a picture of your person and pick person for it.",
SWAP: "To swap the person, connect a picture of the new person and pick person for it.",
}
# A hat or a shirt with no person picture: the user wants the video's own person to wear it.
DRESS_THE_VIDEO_PERSON = 'To put this on the person in your video, pick the video job "change something" and type "He wears @1".'
SWAP_NEEDS_PERSON_THING = (
'A swap needs a picture of the new person, and pick person for it. To show a thing in the clip, '
'pick the video job "change something" and type "He holds @1".'
)
SWAP_NEEDS_PERSON_PLACE = (
'A swap keeps the place of the video. Connect a picture of the new person and pick person for it. '
'To change the place, pick the video job "change something" and type "It looks like @1".'
)
REVOICE_KEEPS_PERSON = (
"New voice keeps the person in the clip, so a person picture can't change who is in it. "
'Take the picture off, or pick the video job "swap the person".'
)
HE_SHE = re.compile(r"\b(?:he|she|they)\b", re.I)
# (what the label is, how it is kept). {picture} is H3's number for the picture.
JOB_WORDS = {
PERFORMER: (
"{subject} is the character in {picture}, with the same shape, colours and drawing style.",
"fully_preserved - the shape, colours and drawing style are retained.",
),
PERSON: (
"{subject} is the person in {picture}, with the same face, hair and clothes.",
"fully_preserved - the face, hair and clothes are retained.",
@@ -91,6 +112,18 @@ MOVES_KEPT = (
"keep the source performer's face, body, clothes, props, background and colours out of the target.\n"
"<Video 1>: attribute_transfer - apply only its camera movement to the target scene."
)
# Copy its moves with no person picture: the person in the video keeps their own moves and goes where the
# script or a place picture says.
OWN_MOVES_ARE = (
"{main} is the person in <Video 1>, with the same face, hair, body{clothes} and the same moves, gestures and "
"timing.\n<Video 1> provides the camera movement."
)
OWN_MOVES_KEPT = (
"{main} (appears in {shots}): {kept}\n"
"<Video 1>: attribute_transfer - apply its person, their moves and its camera movement to the target scene; {place}"
)
OWN_PLACE_FROM = "the background of <Video 1> is not kept, and the place is {place}."
OWN_PLACE_KEPT = "keep its place unless the script names a new one."
SWAP_VIDEO = "<Video 1> supplies the place, the action, the timing, the camera movement and the look."
SWAP_IS = "{replaced} is {who} in <Video 1>, selected for replacement."
SWAP_VIDEO_KEPT = (
@@ -99,7 +132,28 @@ SWAP_VIDEO_KEPT = (
)
SWAP_KEPT = "{replaced}: removed - replace {who} with {main} throughout."
ONE_GOES = "the original person"
THE_PERSON = "the person in the shot"
APPEARANCE_LINE = re.compile(r'^\s*@([1-4])\s+looks like\s+([^"\n]+?)\s*$', re.I)
# Two ways people type a swap that used to be dropped without a word, so the swap ran with no description and H3
# often gave the clip back: the looks sentence on the same line as the swap sentence, and the clothes on a line of
# their own ("@1 is wearing a black suit"). A looks sentence starts after a full stop, ! or ?; "Put @1 looks like a
# friend here." is one sentence and stays whole.
LOOKS_MID_LINE = re.compile(r"(?<=[.!?])[ \t]*(?=@[1-4]\s+looks like\b)", re.I)
WEARS_LINE = re.compile(r'^\s*@([1-4])\s+(?:is\s+wearing|wears|is\s+dressed\s+in|has\s+on)\s+([^"@\n]+?)\s*$', re.I)
QUOTE = re.compile(r'["\u201c\u201d]')
def looks_on_own_lines(text):
"""The script with each @N looks like sentence on a line of its own. A spoken line is left alone."""
text = str(text or "")
lines = []
for line in text.splitlines():
if QUOTE.search(line) or not LOOKS_MID_LINE.search(line):
lines.append(line)
continue
first, *rest = LOOKS_MID_LINE.sub("\n", line).splitlines()
lines += [re.sub(r"\s+([.,;:])$", r"\1", first), *rest]
return "\n".join(lines) if len(lines) != len(text.splitlines()) else text
# Hair and face only, no clothes, expression or background. When the line names clothes that differ from what the
@@ -115,6 +169,7 @@ def add_looks(text, jobs, describe):
"""A swap's script with an @N looks like line for each new person who has none, written by describe(slot), a
small picture-reading model shown that person's picture. Without such a line H3 kept the original person for the
first seconds (2026-11-07), so a connected model writes it when the user didn't."""
text = looks_on_own_lines(text)
typed = {int(match.group(1)) for match in map(APPEARANCE_LINE.match, text.splitlines()) if match}
for slot in sorted(jobs):
if jobs[slot] not in (PERSON, SHEET) or slot in typed:
@@ -137,8 +192,12 @@ def swap_appearances(text, jobs, video):
"""
if video != SWAP:
return text, {}
kept, descriptions = [], {}
for line in str(text or "").splitlines():
kept, descriptions, clothes = [], {}, {}
for line in looks_on_own_lines(text).splitlines():
worn = WEARS_LINE.fullmatch(line)
if worn and (jobs or {}).get(int(worn.group(1))) in (PERSON, SHEET):
clothes.setdefault(int(worn.group(1)), []).append(worn.group(2).strip().rstrip(" ."))
continue
match = APPEARANCE_LINE.fullmatch(line)
if match is None:
kept.append(line)
@@ -150,10 +209,18 @@ def swap_appearances(text, jobs, video):
raise ValueError(f"Picture {slot} must be a person or character sheet for a swap's looks like line.")
if slot in descriptions:
raise ValueError(f"Describe Picture {slot} once, in one looks like line.")
description = match.group(2).strip().rstrip(" .")
# A sentence on the looks line that names an @ again ("The trees behind @1 stay the same.") is not how
# the person looks: it goes back to the script as its own line, or H3 reads a raw @1 in the appearance.
sentences = re.split(r"(?<=[.!?])\s+", match.group(2).strip())
looks = [sentence for sentence in sentences if not TOKEN.search(sentence)]
kept.extend(sentence for sentence in sentences if TOKEN.search(sentence))
description = " ".join(looks).strip().rstrip(" .")
if not description:
raise ValueError(f"Say what Picture {slot}'s person looks like, or remove its looks like line.")
descriptions[slot] = description
for slot, worn in clothes.items():
# Clothes typed on their own line join the description; with none typed, the picture still gives the hair.
descriptions[slot] = f"{descriptions.get(slot, 'the same hair')}, wearing {' and '.join(worn)}"
return ("\n".join(kept), descriptions) if descriptions else (text, {})
@@ -204,23 +271,24 @@ CLOTHES_STAY = (
"attribute_transfer - apply only the face and hair to the original performance; the clothes of {who} in <Video 1> "
"stay as they are; the background of {picture} is not transferred."
)
KEEP_IS = (
"<Video 1> is the video being continued, and the target video starts from its last moment.\n"
"{main} is the main subject in <Video 1>, with the same look."
)
KEEP_KEPT = (
"<Video 1> (continuation point): fully_preserved - the place, the light, the colours and the camera carry on from its last moment.\n"
"{main} (appears in {shots}): fully_preserved - the look is retained."
)
KEEP_IS = "<Video 1> is the video being continued, and the target video starts from its last moment."
KEEP_MAIN = "{main} is the main subject in <Video 1>, with the same look."
KEEP_KEPT = "<Video 1> (continuation point): fully_preserved - the place, the light, the colours and the camera carry on from its last moment."
KEEP_KEPT_PLACE = "<Video 1> (continuation point): fully_preserved - the light, the colours and the camera carry on from its last moment."
KEEP_MAIN_KEPT = "{main} (appears in {shots}): fully_preserved - the look is retained."
SOUND_GOES_ON = "<Audio 1> is the soundtrack of <Video 1>, whose sound carries on in the target video."
SOUND_GOES_ON_KEPT = "<Audio 1>: reference - the sound continues without copying the source signal."
EDIT_IS = (
"<Video 1> supplies the place, the person, the action, the timing, the camera movement, the look and the sound.\n"
"{main} is the person in <Video 1>, kept as they are: the same face, hair, body and movements, with only what the script names changed."
)
EDIT_KEPT = (
"<Video 1>: partially_preserved - keep the place, the person, the action, the timing, the camera movement, the look and the sound; change only what the script names.\n"
"{main} (appears in {shots}): fully_preserved - the face, hair and every movement are retained; only what the script names changes."
EDIT_VIDEO = "<Video 1> supplies {supplies}."
EDIT_IS = "{main} is the person in <Video 1>, kept as they are: the same face, hair, body and movements, with only what the script names changed."
EDIT_VIDEO_KEPT = "<Video 1>: partially_preserved - keep {supplies}; change only what the script names."
EDIT_KEPT = "{main} (appears in {shots}): fully_preserved - the face, hair and every movement are retained; only what the script names changes."
# What a source video gives an edit, less what a connected picture takes over.
EDIT_SUPPLIES = ("the place", "the person", "the action", "the timing", "the camera movement", "the look", "the sound")
# A line that is about someone. An edit or a move copy of a clip with nobody in it (an empty hallway) names nobody.
NAMES_SOMEONE = re.compile(
r"@0|\b(?:he|she|him|her|his|hers|they|them|their|someone|somebody|anyone|everyone|person|people|man|men|woman|women|"
r"guy|girl|boy|kid|child|baby|dancer|singer|actor|actress)\b",
re.IGNORECASE,
)
REVOICE_IS = "<Video 1> is the source video for the target video edit.\n{main} is the person speaking in <Video 1>, with the same face, hair and clothing."
REVOICE_KEPT = (
@@ -229,6 +297,15 @@ REVOICE_KEPT = (
)
# A line only names who goes when it is a noun phrase like "the child in the seat", not an action or a pronoun.
NOUN_PHRASE = re.compile(r"^(?:the|a|an|this|that|these|those|his|her|their|everyone|someone|somebody|both|each|all)\b", re.I)
ACTION_WORD = re.compile(
r"\b(?:dances|walks|runs|talks|says|sits|stands|smiles|waves|jumps|enters|leaves|laughs|sings|eats|drinks|waits|"
r"is\s+\w+ing|are\s+\w+ing)\b",
re.I,
)
def who_goes_in(text):
"""Who a typed swap line says goes, without the rest of the line: "Elon is @1, he replaces the man at the
counter" is the man at the counter, and so is "Replace the man at the counter with @1."."""
@@ -244,7 +321,10 @@ def who_goes_in(text):
text = re.split(r"[.;!?]", text.strip(" .;!?"), maxsplit=1)[0]
text = NAMES_NEXT.sub("", text)
text = SAYS_WHO.sub("", TOKEN.sub("", text).strip(" .,;")).strip(" .,;")
return re.sub(r"\s+and$", "", text).strip(" .,;")
text = re.sub(r"\s+and$", "", text).strip(" .,;")
if not (replaces or plain) and not (NOUN_PHRASE.match(text) and not ACTION_WORD.search(text)):
return "" # "@1 dances" or "He dances" says what happens, not who goes.
return text
def lower_first(text):
@@ -274,7 +354,30 @@ class Picture:
def swapped_in(jobs):
"""How many people a swap puts in: one for each person or sheet picture."""
return max(1, sum(job in (PERSON, SHEET) for job in jobs.values()))
return max(1, sum(job in ACTORS for job in jobs.values()))
# A thing a line holds, wears or uses stays a prop; any other thing named in a line is the one doing the action.
HELD = re.compile(
r"\b(?:holds?|holding|carr(?:y|ies|ying)|wears?|wearing|puts? on|picks? up|uses?|using|grabs?|throws?|"
r"eats?|drinks?|with)\s+(?:\w+\s+){0,2}@(\d)\b",
re.IGNORECASE,
)
def things_that_perform(script, jobs, video):
"""A move copy or a swap with no person picture: a thing picture the lines make do the action (a cartoon hand
waving, a toy dancing) takes the moves, or comes in for the one in the video. With nothing typed, a swap's thing
comes in. A thing that is held or worn stays a thing."""
if video not in (MOVES, SWAP) or any(job in (PERSON, SHEET) for job in (jobs or {}).values()):
return jobs
held = {int(slot) for slot in HELD.findall(script or "")}
named = {int(slot) for slot in TOKEN.findall(script or "")}
jobs = dict(jobs)
for slot, job in jobs.items():
if job == THING and slot not in held and (slot in named or (video == SWAP and not named)):
jobs[slot] = PERFORMER
return jobs
def number_pictures(jobs, video=""):
@@ -295,15 +398,20 @@ class Cast:
self.pictures = pictures
self.video = video
last = max((picture.subject for picture in pictures), default=0)
self.people = [picture for picture in pictures if picture.job in (*ACTORS, START)]
self.moves = subject_label(last + 1) if video == MOVES else ""
# Nobody to give the moves to: the person in the video keeps them and is the main person.
self.own_moves = video == MOVES and not self.people
self.video_person = True # The video's own person is in the result; notice_people can say there is none.
# In a swap each person or sheet picture takes one person's place in the video, in picture order. The script
# can name who goes: "the child in the seat", or "@1 the man on the left. @2 the woman in red".
self.swappers = [picture for picture in pictures if picture.job in (PERSON, SHEET)] if video == SWAP else []
self.swappers = [picture for picture in pictures if picture.job in ACTORS] if video == SWAP else []
goes = len(self.swappers)
self.whos = [f"the person on the {spot}" for spot in SPOTS[goes]] if goes > 1 else [ONE_GOES]
self.replaced = subject_label(1) if video == SWAP else ""
self.in_video = subject_label(last + 1) if video in (KEEP, REVOICE, EDIT) else ""
self.people = [picture for picture in pictures if picture.job in (PERSON, SHEET, START)]
self.in_video = subject_label(last + 1) if video in (KEEP, REVOICE, EDIT) or self.own_moves else ""
if video == KEEP and self.people: # A person picture is the main person; the video adds no second subject.
self.in_video = ""
self.subjects = {picture.slot: subject_label(picture.subject) for picture in pictures if picture.subject}
self.clip_clothes = False # A swap that keeps the clothes of whoever goes.
self.appearances = {} # Explicit looks-like lines, by the visible replacement picture slot.
@@ -311,6 +419,29 @@ class Cast:
self.speaks_as_video = False # With that sound and no words typed: the new person says what the video says.
self.typed_lines = False # The script typed words in quotes for someone to say.
def notice_people(self, text):
"""An edit or a move copy with no person picture keeps the video's person only when the lines are about
someone, or something is put on them. A clip of an empty hallway gets no made-up person."""
if self.video == KEEP:
if not self.people and (text or "").strip():
self.video_person = bool(NAMES_SOMEONE.search(text))
elif self.video == EDIT or self.own_moves:
self.video_person = bool(self.of(OUTFIT) or NAMES_SOMEONE.search(text or ""))
@property
def supplies(self):
"""What the source video of an edit still gives: a place picture brings the place and its look, a style
picture the look."""
gone = set()
if self.of(PLACE):
gone |= {"the place", "the look"}
if self.of(STYLE):
gone.add("the look")
if not self.video_person:
gone.add("the person")
items = [item for item in EDIT_SUPPLIES if item not in gone]
return ", ".join(items[:-1]) + " and " + items[-1]
def of(self, *jobs):
return [picture for picture in self.pictures if picture.job in jobs]
@@ -356,12 +487,32 @@ class Cast:
@property
def main(self):
"""Who a line is about when it names nobody."""
if self.in_video:
if self.in_video and self.video_person:
return self.in_video
if self.people:
return subject_label(self.people[0].subject)
return next(iter(self.subjects.values()), subject_label(1))
@property
def main_in_shot(self):
"""Whether main is a person or thing the clip defines, so "<Subject N> is in the shot" can name it."""
if self.in_video and self.video_person and self.main == self.in_video:
return True
defined = {subject_label(picture.subject) for picture in self.of(*ACTORS, THING, START)}
return self.main in defined
def unclear_speaker(self, moments):
"""A refusal when two or more people are connected and a spoken line does not say which one speaks."""
if self.video not in ("", MOVES, KEEP) or len(self.people) < 2:
return ""
people = {picture.slot for picture in self.people}
for moment in moments:
if moment.speech and not moment.voice and not moment.voiceover:
if not any(int(slot) in people for slot in TOKEN.findall(moment.text)):
count = "two people" if len(self.people) == 2 else "more than one person"
return f'With {count}, name who speaks: @{self.people[1].slot} says "hello".'
return ""
@property
def opens(self):
return next(iter(self.of(START)), None)
@@ -378,9 +529,15 @@ class Cast:
if not self.pictures and not self.video:
return NOTHING_IN
if self.video in NEEDS_A_PERSON and not self.people:
if self.of(OUTFIT, THING):
if self.of(OUTFIT):
return DRESS_THE_VIDEO_PERSON
if self.of(THING):
return SWAP_NEEDS_PERSON_THING
if self.of(PLACE):
return SWAP_NEEDS_PERSON_PLACE
return NEEDS_A_PERSON[self.video]
if self.video == REVOICE and self.of(PERSON, SHEET):
return REVOICE_KEEPS_PERSON
typed = sorted({int(number) for number in TOKEN.findall(text)} - {0})
slots = {picture.slot: picture for picture in self.pictures}
for slot in typed:
@@ -389,16 +546,23 @@ class Cast:
if slots[slot].job == END:
return f"Picture {slot} is where the clip ends, so @{slot} can't be in a line. Say what happens on the way there."
if typed:
for picture in self.of(PERSON, SHEET, THING, OUTFIT):
if picture.slot not in typed:
return f"Picture {picture.slot} is on, but no line names @{picture.slot}."
for picture in self.of(*ACTORS, THING, OUTFIT):
if picture.slot in typed:
continue
# The only person picture is who "He" or "She" means.
if picture.job in (PERSON, SHEET) and len(self.of(PERSON, SHEET)) == 1 and HE_SHE.search(text):
continue
hint = f' Type who wears it, like "He wears @{picture.slot}."' if picture.job == OUTFIT else ""
return f"Picture {picture.slot} is on, but no line names @{picture.slot}.{hint}"
return ""
def wearers(self, moments):
"""Who wears each outfit: the person named just before it, or else the main person."""
found = {}
outfits = {subject_label(picture.subject) for picture in self.of(OUTFIT)}
default = THE_PERSON if self.main in outfits else self.main # An outfit is never worn by itself.
for picture in self.of(OUTFIT):
found[picture.slot] = self.main
found[picture.slot] = default
token = f"@{picture.slot}"
for moment in moments:
before = moment.text.partition(token)[0]
@@ -414,7 +578,9 @@ class Cast:
ends = self.ends
lines = []
if self.video == EDIT:
lines.append(EDIT_IS.format(main=self.in_video))
lines.append(EDIT_VIDEO.format(supplies=self.supplies))
if self.video_person:
lines.append(EDIT_IS.format(main=self.in_video))
if self.video == SWAP:
lines.append(SWAP_VIDEO)
lines += [SWAP_IS.format(replaced=gone, who=who) for gone, who, _ in self.swaps]
@@ -445,10 +611,17 @@ class Cast:
lines.append(FIRST_FRAME.format(picture=picture_label(self.opens.picture)))
if ends:
lines.append(LAST_FRAME.format(picture=picture_label(ends.picture), last=last))
if self.moves:
if self.own_moves and not self.video_person:
lines.append("<Video 1> provides the camera movement.")
elif self.own_moves:
clothes = "" if self.of(OUTFIT) else " and clothes,"
lines.append(OWN_MOVES_ARE.format(main=self.in_video, clothes=clothes or ","))
elif self.moves:
lines.append(MOVES_ARE.format(moves=self.moves))
if self.video == KEEP:
lines.append(KEEP_IS.format(main=self.in_video))
lines.append(KEEP_IS)
if self.in_video and self.video_person:
lines.append(KEEP_MAIN.format(main=self.in_video))
if self.video == REVOICE:
lines.append(REVOICE_IS.format(main=self.in_video))
return "\n".join(lines)
@@ -458,7 +631,7 @@ class Cast:
subject = subject_label(picture.subject)
label = picture_label(picture.picture)
wearing = [slot for slot, wearer in wearers.items() if wearer == subject]
if self.video == SWAP and picture.job in (PERSON, SHEET):
if self.video == SWAP and picture.job in ACTORS:
if self.clip_clothes:
return CLOTHES_STAY.format(who=self.goes_for(picture), picture=label)
return SWAPPED.format(picture=label)
@@ -467,9 +640,9 @@ class Cast:
elif picture.job == PERSON and wearing:
words = DRESSED[1].format(outfit=self.subjects[wearing[0]])
else:
wearer = wearers.get(picture.slot, self.main)
wearer = wearers.get(picture.slot, THE_PERSON if picture.job == OUTFIT else self.main)
words = JOB_WORDS[picture.job][1].format(wearer=wearer)
if picture.job in (PERSON, SHEET, THING):
if picture.job in (*ACTORS, THING):
words += " " + BACKGROUND.format(picture=label)
return words
@@ -482,7 +655,9 @@ class Cast:
lines = []
if self.video == EDIT:
lines.append(EDIT_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
lines.append(EDIT_VIDEO_KEPT.format(supplies=self.supplies))
if self.video_person:
lines.append(EDIT_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
if self.video == SWAP:
lines.append(SWAP_VIDEO_KEPT.format(people="people" if len(self.swaps) > 1 else "person"))
lines += [SWAP_KEPT.format(replaced=gone, who=who, main=new) for gone, who, new in self.swaps]
@@ -496,10 +671,25 @@ class Cast:
lines.append(FIRST_KEPT.format(picture=picture_label(self.opens.picture)))
if self.ends:
lines.append(LAST_KEPT.format(picture=picture_label(self.ends.picture), last=len(shots)))
if self.moves:
if self.own_moves and not self.video_person:
places = [self.subjects[picture.slot] for picture in self.of(PLACE)]
place = OWN_PLACE_FROM.format(place=places[0]) if places else OWN_PLACE_KEPT
lines.append(f"<Video 1>: attribute_transfer - apply only its camera movement to the target scene; {place}")
elif self.own_moves:
outfits = [self.subjects[picture.slot] for picture in self.of(OUTFIT)]
kept = (
f"partially_preserved - the face, hair, body, moves and timing are retained, and the clothes change to {outfits[0]}."
if outfits else "fully_preserved - the face, hair, body, clothes, moves and timing are retained."
)
places = [self.subjects[picture.slot] for picture in self.of(PLACE)]
place = OWN_PLACE_FROM.format(place=places[0]) if places else OWN_PLACE_KEPT
lines.append(OWN_MOVES_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video)), kept=kept, place=place))
elif self.moves:
lines.append(MOVES_KEPT.format(moves=self.moves, shots=shot_list(seen(self.main)), main=self.main))
if self.video == KEEP:
lines.append(KEEP_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
lines.append(KEEP_KEPT_PLACE if self.of(PLACE) else KEEP_KEPT)
if self.in_video and self.video_person:
lines.append(KEEP_MAIN_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
if self.video == REVOICE:
lines.append(REVOICE_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
return "\n".join(lines)
+11 -4
View File
@@ -13,7 +13,7 @@ import re
import numpy as np
from ._h3_cast_helpers import EDIT, KEEP, NEEDS_NO_WORDS, ONE_GOES, REVOICE, SWAP, Cast, number_pictures, swap_appearances
from ._h3_cast_helpers import TOKEN
from ._h3_cast_helpers import TOKEN, things_that_perform
from ._h3_script_helpers import (
ACTION_GOES_ON,
PRONOUN,
@@ -78,9 +78,10 @@ def one_line(text):
def clean_words(text):
"""Spoken words: one line, no tags, ending the way H3 wants a line to end."""
text = one_line(text).replace("<", "").replace(">", "")
if text and text[-1] not in ".?!":
text += "."
# '"Hi," she says' is "Hi.", not "Hi,.". Chinese and Japanese lines end in their own stops: "。", not "。.".
text = one_line(text).replace("<", "").replace(">", "").rstrip(" ,;:,、;:")
if text and text[-1] not in ".?!。?!…":
text += "。" if re.search(r"[\u3040-\u30ff\u4e00-\u9fff]", text) else "."
return text
@@ -312,6 +313,7 @@ def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="E
longest) puts a question to a small model, if one is connected: it fills in
a one-line idea and the sound of the place."""
parts = split_sections(script)
written = parts is None # Words someone typed in six parts go through as they are.
audio = next_label(script, "Audio")
speaker = None
@@ -319,10 +321,13 @@ def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="E
jobs = jobs if jobs is not None else {1: "person"}
if video == EDIT:
script, jobs = put_on_from_pictures(script, jobs)
jobs = things_that_perform(script, jobs, video)
cast = Cast(number_pictures(jobs, video), video)
script, cast.appearances = swap_appearances(script, jobs, video)
scene = read_script(script)
cast.notice_people(" ".join(moment.text for moment in scene.moments))
problem = cast.problem(" ".join(moment.text for moment in scene.moments))
problem = problem or cast.unclear_speaker(scene.moments)
if problem:
raise ValueError(problem)
if not scene.moments and video == EDIT:
@@ -370,6 +375,8 @@ def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="E
scene, seconds, camera=camera, look=look, sound=sound, say=say, cast=cast, sounds=not keeps_song, soundtrack=soundtrack
)
if written and (video == EDIT or (video == SWAP and clip_sound)): # The clip's own sound stays in the result.
parts["summary"] = add_task(parts["summary"], "audio reuse")
if voice:
add_voice(parts, voice_label, speaker)
if song_beats is not None and voice:
+69 -21
View File
@@ -17,15 +17,19 @@ from ._h3_cast_helpers import (
MAIN,
OUTFIT,
PERSON,
PERFORMER,
PLACE,
REVOICE,
SHEET,
START as START_JOB,
END as END_JOB,
STYLE,
SWAP,
SOUND_GOES_ON,
SOUND_GOES_ON_KEPT,
THING,
TOKEN,
things_that_perform,
ONE_GOES,
SPOTS,
Cast,
@@ -43,7 +47,9 @@ THEN = r"(?:,?\s*(?:and\s+)?then\b|,?\s*and\s+after\s+that\b|,?\s*after\s+that\b
# A word that starts a new moment and stays in it.
TURN = r"(?=,?\s*\b(?:suddenly|all\s+of\s+a\s+sudden|out\s+of\s+nowhere|finally|at\s+last|(?:hard\s+)?cut\s+to|(?:it\s+|and\s+|(?<!\bit\s))ends?\s+(?:with|on|in|as)|ending\s+(?:with|on|in))\b)"
# Where a line breaks into sentences: a full stop that is not a title, the end of a quote, or a semicolon.
SENTENCE = r"(?<!\bMr\.)(?<!\bMs\.)(?<!\bDr\.)(?<!\bSt\.)(?<!\bMrs\.)(?<=[.!?])\s+(?=\S)|(?<=\x00)\s+(?=[A-Z])|\s*;\s*"
# After a quote, a new sentence starts with a capital or an @: in '"Get a job." @1 looks up' the @1 is a new moment,
# or its own words are lost behind the first quote's. So is '"Hi" and @2 says "Bye"': one moment holds one quote.
SENTENCE = r"(?<!\bMr\.)(?<!\bMs\.)(?<!\bDr\.)(?<!\bSt\.)(?<!\bMrs\.)(?<=[.!?])\s+(?=\S)|(?<=\x00)\s+(?=[A-Z@])|(?<=\x00),?\s+(?:and|but|while)\s+(?=@\d)|\s*;\s*"
CUT_WORDS = re.compile(r"^(?:hard\s+)?(?:cut|cuts|smash\s+cut|jump\s+cut)\s+to\b[:,]?\s*", re.I)
NEW_SHOT = re.compile(r"^(?:new\s+shot|next\s+shot|shot\s+\d+)\b[:,.\-]?\s*", re.I)
START = re.compile(r"^(?:it\s+|the\s+(?:clip|video|scene|shot)\s+)?(?:starts?|begins?|opens?)(?:\s+(?:with|on|in|at|as)\b[:,]?|\s*:)\s*", re.I)
@@ -110,6 +116,30 @@ SOUND_WORDS = 22
NAMED = {PLACE: "the place", OUTFIT: "the outfit", THING: "the thing"} # What @2 reads as to the small model.
WORDS_A_SECOND = 3 # How fast H3 speaks.
# Words that trail off ("I never thought...") keep going: across the next cut, or past the end of the clip.
TRAILS_OFF = re.compile(r"(?:\.\.\.|\u2026|\u2014|--)\s*$")
LINE_CARRIES = "<scenetrans> The line from the previous shot continues seamlessly across the cut, and the words are not repeated."
CUT_OFF = "The line is cut off by the end of the video. <cutoff>"
def speech_runs_on(script, seconds):
"""(carried, cut_off): the moments whose words go on across the next cut, and the last moment's words if the
clip ends mid-line. Words carry on when they trail off with "..." or need more time than their shot has."""
starts = timeline(script, seconds)
ends = [*starts[1:], seconds]
carried, cut_off = set(), None
for index, moment in enumerate(script.moments):
if not moment.speech:
continue
trails = bool(TRAILS_OFF.search(moment.speech))
long = len(moment.speech.split()) / WORDS_A_SECOND > ends[index] - starts[index] + 0.5
following = script.moments[index + 1] if index + 1 < len(script.moments) else None
if following is not None and following.cut and not script.one_take and (trails or long):
carried.add(index)
elif following is None and trails:
cut_off = index
return carried, cut_off
HOLD = f"{MAIN} holds still" # What fills the time before a first moment set later.
KEEP_LOOK = "The target video keeps the look, the light and the colours of {picture}."
KEEP_COLOURS = "The target video keeps the colours and the film grain of {picture}."
@@ -130,6 +160,7 @@ OPENS_ON = "The shot begins from {picture}."
ENDS_ON = "The shot ends on {picture}."
ON_SHEET = "{subject} appears once, without the panels, labels or other views of {picture}."
THE_MOVES = "{main} performs the same moves as {moves}, at the same timing."
OWN_MOVES = "{main} moves exactly as in <Video 1>, at the same timing."
IN_THE_PLACE = "The shot takes place in {place}."
WEARS = "{wearer} wears {outfit}."
IN_THE_SHOT = "{main} is in the shot."
@@ -142,10 +173,10 @@ SAYS_NOTHING = "{main} keeps the lips closed and says nothing the whole time."
SPEAKS_AS_VIDEO = "{main} speaks as in <Video 1>, each word landing exactly on the lip movement of <Video 1>."
CLIP_SOUND = "The original sound of <Video 1> continues throughout."
FOLLOWS_CAMERA = "The camera follows the movement of <Video 1>."
REST_AS_VIDEO = "Everything else stays as it is in <Video 1>."
SAME_AS_VIDEO = "{main} has the same words, mouth movements and moves as in <Video 1>. Everything else stays as it is in <Video 1>."
CARRIES_ON = "The shot carries straight on from the end of <Video 1>."
ACTION_GOES_ON = "The action keeps going the way it was heading, and the camera keeps moving the same way."
FACES_US = "{main} faces the camera as in <Video 1>."
VIDEO_LOOKS = {
SWAP: "Retain the look of <Video 1>.",
KEEP: "The target video keeps the look, colours and camera of <Video 1>.",
@@ -557,12 +588,13 @@ def script_warning(script, seconds, cast=None):
if any((moment.at or 0.0) >= seconds for moment in script.moments):
return f"A moment starts after the clip ends, at {seconds:g} seconds."
starts = timeline(script, seconds)
for moment, start, end in zip(script.moments, starts, [*starts[1:], seconds]):
carried, cut_off = speech_runs_on(script, seconds)
for index, (moment, start, end) in enumerate(zip(script.moments, starts, [*starts[1:], seconds])):
unseen = UNSEEN.search(without_speech(moment))
if unseen:
return f'A camera can\'t see "{unseen.group(0).lower()}". Say what we would see, like "she smiles".'
takes = len(moment.speech.split()) / WORDS_A_SECOND
if takes > end - start + 0.5:
if takes > end - start + 0.5 and index not in carried and index != cut_off:
return f"A spoken line takes about {takes:.0f} seconds to say, and its moment has {end - start:.0f}. Give it more room."
return ""
@@ -585,6 +617,7 @@ def script_report(text, seconds, jobs=None, video=""):
}
for moment, start, end in zip(script.moments, starts, ends)
]
jobs = things_that_perform(text, jobs, video) if jobs is not None else None
cast = Cast(number_pictures(jobs, video), video) if jobs is not None else None
return {"moments": moments, "seconds": seconds, "warning": script_warning(script, seconds, cast)}
@@ -732,6 +765,8 @@ def before_the_action(number, text, cast, wearers):
# or for the whole clip at Fast preview; with it the new
# person is there from the first frame. An @1 looks like line only adds its own words.
looks = (f"has the defined appearance from {picture_label(picture.picture)}: {appearance}" if appearance
else f"has the shape, colours and drawing style of the character in {picture_label(picture.picture)}"
if picture.job == PERFORMER
else f"has the face, hair and clothes of the person in {picture_label(picture.picture)}")
notes.append(
f"{new} occupies the starting position of {who} and follows the same route, movements, timing and pace. "
@@ -749,8 +784,6 @@ def before_the_action(number, text, cast, wearers):
notes.append((OWN_LINES_ONLY if f"{new} (S" in text else SAYS_NOTHING).format(main=new))
if number == 1 and cast.video == KEEP:
notes += [CARRIES_ON]
if number == 1 and cast.video == REVOICE:
notes += [FACES_US.format(main=cast.main)]
places = [subject_label(picture.subject) for picture in cast.of(PLACE)]
if places and not any(place in text for place in places):
notes.append(IN_THE_PLACE.format(place=places[0]))
@@ -760,7 +793,8 @@ def before_the_action(number, text, cast, wearers):
notes.append(sentence(", ".join(spots[:-1]) + f" and {spots[-1]} of the frame"))
for slot, wearer in wearers.items():
outfit = cast.subjects[slot]
if outfit not in text and (wearer in people or wearer in text):
# An edit has no person picture: the person in the video wears an outfit that no line names.
if outfit not in text and (wearer in people or wearer in text or (cast.video == EDIT and wearer == cast.in_video)):
notes.append(WEARS.format(wearer=wearer, outfit=outfit))
return notes
@@ -768,10 +802,10 @@ def before_the_action(number, text, cast, wearers):
def after_the_action(text, cast):
"""What H3 needs to hear about a shot once its action has been said."""
notes = []
if cast.moves and cast.main in text:
notes.append(THE_MOVES.format(main=cast.main, moves=cast.moves))
if cast.moves and cast.video_person and cast.main in text:
notes.append(OWN_MOVES.format(main=cast.main) if cast.own_moves else THE_MOVES.format(main=cast.main, moves=cast.moves))
if cast.video == EDIT:
notes.append(SAME_AS_VIDEO.format(main=cast.main))
notes.append(SAME_AS_VIDEO.format(main=cast.main) if cast.video_person else REST_AS_VIDEO)
if cast.video in (SWAP, EDIT):
notes.append(FOLLOWS_CAMERA)
for picture in cast.of(SHEET):
@@ -799,6 +833,7 @@ def shot_lines(script, seconds, say, cast, wearers, move="", hold="", sounds=Tru
talking = speakers(script.moments, cast)
quiet = " ".join(QUIET.format(speaker=speaker) for speaker in talking if speaker.startswith("<"))
kept = still_true(script.moments)
carried, cut_off = speech_runs_on(script, seconds)
lines = []
for number, shot in enumerate(shots_of(script), 1):
shot_camera = script.moments[shot[0]].camera
@@ -841,8 +876,17 @@ def shot_lines(script, seconds, say, cast, wearers, move="", hold="", sounds=Tru
*([hold] if number > 1 else []),
capital(script.moments[first].sound) if script.moments[first].sound and not script.silent else "",
]
lines.append(f"[Shot {number}] " + " ".join(part for part in body if part))
if not any(cast.main in line for line in lines):
line = " ".join(part for part in body if part)
if shot[0] - 1 in carried:
# The speaker is mid-line as the shot opens, so the mouth is not closed here.
line = f"{LINE_CARRIES} " + re.sub(r" ?<Subject \d+> stays silent with the lips closed\.", "", line, count=1)
if shot[-1] in carried:
# The speaker is still talking at the cut, so the mouth does not close here.
line = re.sub(r" <Subject \d+> closes the lips\.", "", line) + " <scenetrans>"
if shot[-1] == cut_off:
line = re.sub(r" <Subject \d+> closes the lips\.", "", line) + f" {CUT_OFF}"
lines.append(f"[Shot {number}] " + line)
if cast.video_person and cast.main_in_shot and not any(cast.main in line for line in lines):
lines[0] = lines[0].replace("[Shot 1] ", f"[Shot 1] {IN_THE_SHOT.format(main=cast.main)} ", 1)
return lines
@@ -881,16 +925,19 @@ def summary_of(script, cast, new_places, soundtrack):
return f"[{task}] {changes} Everything else in <Video 1> is kept."
if cast.video == KEEP:
tag = f"{task} + audio reference" if soundtrack else task
return f"[{tag}] The target video continues directly from the end of <Video 1> in the same place."
opening = with_labels(without_speech(script.moments[0]), cast) or f"{cast.main} speaks"
where = "" if cast.of(PLACE) else " in the same place"
return f"[{tag}] The target video continues directly from the end of <Video 1>{where}."
talking = [who for who in speakers(script.moments, cast) if who.startswith("<")]
who_speaks = f"{' and '.join(talking)} speak" if len(talking) > 1 else f"{cast.main} speaks"
opening = with_labels(without_speech(script.moments[0]), cast) or who_speaks
later = [with_labels(without_speech(script.moments[shot[0]]), cast) for shot in shots_of(script)[1:]]
if cast.opens and cast.ends:
moving = f"moving from {picture_label(cast.opens.picture)} to {picture_label(cast.ends.picture)}"
return f"[{task}] In the target video, {lower_first(opening.rstrip('.!? '))}, {moving}."
summary = f"[{task}] In the target video, {lower_first(sentence(opening))}"
summary += "".join(f" {AND_THEN.format(line=lower_first(sentence(line)))}" for line in later if line)
if cast.moves:
summary += " " + THE_MOVES.format(main=cast.main, moves=cast.moves)
if cast.moves and cast.video_person:
summary += " " + (OWN_MOVES.format(main=cast.main) if cast.own_moves else THE_MOVES.format(main=cast.main, moves=cast.moves))
return f"{summary} {EVERY_PLACE}" if new_places else summary
@@ -916,14 +963,15 @@ def script_parts(script, seconds, camera="", look="", sound="", say=None, cast=N
cuts = [script.moments[shot[0]] for shot in shots_of(script)[1:]]
moved = any(not FRAME_WORDS.match(without_speech(moment)) for moment in cuts)
new_places = last > 1 and not cast.of(PLACE) and moved and all(not FRAME_WORDS.match(without_speech(moment)) for moment in cuts)
first = next(iter(cast.of(PLACE) or cast.pictures), None)
if not style and cast.video == EDIT:
# Only a place, a start frame or a person sets the look; an outfit, a thing or a style never does.
first = next(iter(cast.of(PLACE) or cast.of(START_JOB) or cast.of(PERSON, SHEET, PERFORMER, END_JOB)), None)
if not style and cast.video == EDIT and not cast.of(PLACE):
style = VIDEO_LOOKS[EDIT]
if not style and len(cast.of(PLACE)) > 1:
style = OWN_COLOURS
elif not style and not (first and first.job == THING):
elif not style and (first or cast.video in VIDEO_LOOKS):
words = KEEP_LOOK if cast.opens else KEEP_COLOURS
style = VIDEO_LOOKS.get(cast.video) or words.format(picture=picture_label(first.picture if first else 1))
style = (cast.video != EDIT and VIDEO_LOOKS.get(cast.video)) or words.format(picture=picture_label(first.picture if first else 1))
takes = [EVERY_PLACE] if new_places else []
if last == 1 and (len(script.moments) > 1 or cast.moves or script.one_take):
takes = [ONE_SHOT]
@@ -947,7 +995,7 @@ def script_parts(script, seconds, camera="", look="", sound="", say=None, cast=N
"subject_definitions": subjects,
"summary": summary_of(script, cast, new_places, soundtrack),
"retention_analysis": kept,
"detailed_description": "\n".join([" ".join(part for part in (style, move, *takes) if part), *lines]),
"detailed_description": "\n".join([*[" ".join(part for part in (style, move, *takes) if part)] * bool(style or move or takes), *lines]),
"overall_soundscape": ambience,
"non_diegetic_music": tune,
}
+22 -4
View File
@@ -40,6 +40,9 @@ OUTSIDE = (
"Image Folder reads only folders inside ComfyUI's input or output folder. "
"Drop your folder on the node, or press Add, and its pictures are copied in."
)
# A drive ("D:") in any part of a folder name, and control characters.
_DRIVE = re.compile(r"^[A-Za-z]:")
_CONTROL = re.compile(r"[\x00-\x1f\x7f]")
NO_FOLDER = 'Image Folder: there is no folder "{folder}" in ComfyUI\'s {source} folder.'
NO_PICTURES = 'Image Folder: no pictures in "{folder}". Add some, or choose another folder.'
NONE_LEFT = "Image Folder: none of the picked pictures are in the folder anymore. Pick again on the node."
@@ -64,7 +67,9 @@ def clean_folder(folder) -> str:
if text.startswith("/") or text.startswith("~") or re.match(r"^[A-Za-z]:", text):
raise ValueError(OUTSIDE)
parts = [part for part in text.split("/") if part not in ("", ".")]
if any(part == ".." for part in parts):
# ".." climbs out; a drive in any part ("a/D:/x") jumps to another drive
# on Windows; a control character is never part of a folder name.
if any(part == ".." or _DRIVE.match(part) for part in parts) or _CONTROL.search(text):
raise ValueError(OUTSIDE)
return "/".join(parts)
@@ -95,14 +100,27 @@ def place_inside(folder, roots: dict) -> tuple[str, str] | None:
return None
def inside_root(top: Path, relative: str) -> Path:
"""`relative` below the resolved root `top`, as a resolved path. It is
checked as text before the disk is asked to resolve it, and again after,
so a link that leads out of the root is refused too."""
if not relative:
return top
lexical = top.joinpath(*relative.split("/"))
if top not in lexical.parents:
raise ValueError(OUTSIDE)
target = lexical.resolve()
if target != top and top not in target.parents:
raise ValueError(OUTSIDE)
return target
def safe_folder(root, folder, source: str = "input") -> Path:
"""The folder as a real path below `root`. A link that leads out of the
root is refused like any other way out."""
relative = clean_folder(folder)
top = Path(root).resolve()
target = top.joinpath(*relative.split("/")).resolve() if relative else top
if target != top and top not in target.parents:
raise ValueError(OUTSIDE)
target = inside_root(top, relative)
if not target.is_dir():
raise ValueError(NO_FOLDER.format(folder=relative, source=source))
return target
+183 -5
View File
@@ -653,10 +653,13 @@ def apply_stitch(
exact with the toggle off, since the spread deliberately rewrites the
feathered band.
``color_match`` (0..1) shifts the patch's tone by the offset measured
in the feathered band (:func:`estimate_tone_offset`) before blending,
so an outpaint whose new bands came out a touch lighter or warmer than
the picture lands on the picture's own tone. 0 leaves the patch alone.
``color_match`` (0..1) shifts the patch's tone by the measured drift
before blending, so a new area that came out a touch lighter or warmer
than the picture lands on the picture's own tone. An outpaint is read
across its seams (:func:`tone_offset_field`); an inpaint, which has no
source rectangle, inside its painted area (:func:`inner_tone_offset`),
or in the feathered band (:func:`estimate_tone_offset`) when the
painted area holds new content. 0 leaves the patch alone.
``seam`` "classic" is everything above. "blend in" replaces it on a
stitcher that knows where the picture's edge is (Load Image + Pad and
@@ -788,7 +791,13 @@ def apply_stitch(
# Trimmed like the blend when fewer frames came back.
generated = generated[:frames] if generated.shape[0] > frames else generated
generated = generated[:, cy : cy + ch, cx : cx + cw].to(out.device)
field = tone_offset_field(patch, out[:, cy : cy + ch, cx : cx + cw, :], alpha, bbox, generated)
region = out[:, cy : cy + ch, cx : cx + cw, :]
field = tone_offset_field(patch, region, alpha, bbox, generated)
if bbox is None:
# An inpaint has no source rectangle to read a seam across. Its
# drift is read just inside the painted area, where it holds.
inner, held = inner_tone_offset(patch, region, alpha, came_back)
field = torch.where(held.view(-1, 1, 1, 1), inner.view(-1, 1, 1, 3).expand_as(field), field)
# Weighted by the blend: a fully generated pixel drifted by the whole
# field, a band pixel the sampler mixed at alpha drifted by alpha of
# it, and the blend below scales the correction by alpha once more.
@@ -841,6 +850,175 @@ def estimate_tone_offset(
return torch.where(measured, offset, torch.zeros_like(offset))
# An inpaint's drift is read in the painted area's own rim: this many
# pixels in from where the paste becomes whole. The rim is where the new
# part most often still shows what was there before (skin beside skin).
INNER_RIM_PX = 32
# Rim pixels within this many LAB units of the rim's median change count as
# the same content, drifted. The reading is only used when at least
# INNER_MIN_SHARE of the rim agrees and there are INNER_MIN_PIXELS of it.
INNER_SAME_CONTENT = 8.0
INNER_MIN_SHARE = 0.5
INNER_MIN_PIXELS = 64
# A larger change than this is new content (a jacket over skin), not a
# drift: matching it would pull the new thing toward the old colour.
INNER_MAX_SHIFT = 20.0
# New content has no "before" in its own place (skin where a shirt was), so
# it is read against the untouched picture beside it instead: the painted
# area and the picture, each averaged within this many pixels of a point.
NEIGHBOUR_REACH_PX = 48
# A point counts as on the seam when each side fills this much of its reach.
NEIGHBOUR_MIN_SIDE = 0.1
# Seam points within INNER_MAX_SHIFT of each other show the same thing on
# both sides (skin beside skin). Fewer than this share of the seam and the
# new part is something else all round, so nothing is matched.
NEIGHBOUR_MIN_SHARE = 0.25
def _neighbour_offset(
new: torch.Tensor, old: torch.Tensor, whole: torch.Tensor, outside: torch.Tensor
) -> torch.Tensor | None:
"""LAB offset of new content against the untouched picture beside it.
``new`` and ``old`` are one frame in LAB [H, W, 3]; ``whole`` marks the
fully pasted pixels and ``outside`` the untouched ones. Along the seam
the painted side's local mean is compared with the picture's. Where the
two are alike the model drew more of what is already there, and the gap
between them is its drift. None when too little of the seam is alike or
the alike part does not agree on one gap.
"""
size = 2 * NEIGHBOUR_REACH_PX + 1
def local(values: torch.Tensor, mask: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
weight = mask.to(values.dtype).unsqueeze(-1)
stack = torch.cat([values * weight, weight], dim=-1).movedim(-1, 0).unsqueeze(0)
stack = functional.avg_pool2d(stack, (1, size), stride=1, padding=(0, NEIGHBOUR_REACH_PX))
stack = functional.avg_pool2d(stack, (size, 1), stride=1, padding=(NEIGHBOUR_REACH_PX, 0))[0]
return (stack[:3] / stack[3:].clamp_min(1e-6)).movedim(0, -1), stack[3]
painted, painted_share = local(new, whole)
picture, picture_share = local(old, outside)
seam = (painted_share >= NEIGHBOUR_MIN_SIDE) & (picture_share >= NEIGHBOUR_MIN_SIDE)
gaps = (painted - picture)[seam]
if gaps.shape[0] < INNER_MIN_PIXELS:
return None
alike = gaps.norm(dim=-1) <= INNER_MAX_SHIFT
if int(alike.sum()) < INNER_MIN_PIXELS or float(alike.float().mean()) < NEIGHBOUR_MIN_SHARE:
return None
gaps = gaps[alike]
agree = (gaps - gaps.median(dim=0).values).norm(dim=-1) < INNER_SAME_CONTENT
if float(agree.float().mean()) < INNER_MIN_SHARE:
return None
return gaps[agree].median(dim=0).values
def _area_resize(image: torch.Tensor, width: int, height: int) -> torch.Tensor:
"""Shrink [B, H, W, C] by averaging the pixels each new one covers."""
return functional.interpolate(image.movedim(-1, 1), size=(height, width), mode="area").movedim(1, -1)
def inner_tone_offset(
inpainted: torch.Tensor,
canvas: torch.Tensor,
blend: torch.Tensor,
sampled: tuple[int, int] | None = None,
) -> tuple[torch.Tensor, torch.Tensor]:
"""Per-frame LAB drift of an inpaint, read inside the painted area.
:func:`estimate_tone_offset` reads the feathered band and assumes the
sampler mixed original and generated there by ``blend``. A workflow that
samples with its own, wider or softer noise mask breaks that: the model
leaves the band close to the original and drifts further in, so the band
reads a fraction of the drift (issue #361: 0.8 of 2.5 LAB units).
This reads the same pixels before and after in the rim of the fully
pasted area (``blend`` > 0.999, INNER_RIM_PX deep): the median change,
then the median again over the pixels that agree with it. Returns the
offsets [B, 3] and which frames hold [B]. A frame whose rim mostly
disagrees, or changed by more than INNER_MAX_SHIFT, is new content
rather than drift. That frame is read against the untouched picture
beside it (:func:`_neighbour_offset`), and holds at zero, left as
painted, when the picture beside it shows something else. Only a rim too
small to judge does not hold, and the caller keeps the band reading for it.
``sampled`` is the (width, height) the model worked at. A crop from a
large picture is pasted back several times that size, and a rim measured
in pasted pixels would be a sliver at the very edge of what the model
painted, where it drifts least. The reading is taken at the model's
size instead, which also keeps its cost the same for any picture.
"""
from ._color_helpers import rgb_to_lab
if sampled is not None:
width, height = int(sampled[0]), int(sampled[1])
if 0 < width < inpainted.shape[2] and 0 < height < inpainted.shape[1]:
inpainted = _area_resize(inpainted, width, height)
canvas = _area_resize(canvas, width, height)
blend = _area_resize(blend.unsqueeze(-1), width, height).squeeze(-1)
new = rgb_to_lab(inpainted[..., :3])
old = rgb_to_lab(canvas[..., :3])
diff = new - old
frames = diff.shape[0]
whole = (blend > 0.999).to(diff.dtype)
size = 2 * INNER_RIM_PX + 1
# A pixel is deeper than the rim when nothing within INNER_RIM_PX of it
# is outside the whole paste. The crop's own border does not count as
# outside: the painted area simply runs off the crop there.
# One pass along each axis: the same square reach at a fraction of the cost.
outside = (1.0 - whole).unsqueeze(1)
outside = functional.max_pool2d(outside, (1, size), stride=1, padding=(0, INNER_RIM_PX))
outside = functional.max_pool2d(outside, (size, 1), stride=1, padding=(INNER_RIM_PX, 0))
deep = 1.0 - outside.squeeze(1)
rim = (whole > 0) & (deep < 0.5)
rim = torch.where(rim.flatten(1).any(dim=1).view(-1, 1, 1), rim, whole > 0)
pasted = whole > 0
untouched = blend < 0.001
if rim.shape[0] != frames:
rim = rim.expand(frames, -1, -1)
pasted = pasted.expand(frames, -1, -1)
untouched = untouched.expand(frames, -1, -1)
# See-through pixels hold whatever colour sits under the clear part.
if inpainted.shape[-1] > 3:
rim = rim & (inpainted[..., 3] >= 0.999)
pasted = pasted & (inpainted[..., 3] >= 0.999)
if canvas.shape[-1] > 3:
rim = rim & (canvas[..., 3] >= 0.999)
untouched = untouched & (canvas[..., 3] >= 0.999)
def beside(index: int) -> torch.Tensor:
gap = _neighbour_offset(new[index], old[index], pasted[index], untouched[index])
return offsets[index] if gap is None else gap
offsets = torch.zeros((frames, 3), dtype=diff.dtype, device=diff.device)
held = torch.zeros((frames,), dtype=torch.bool, device=diff.device)
for index in range(frames):
changes = diff[index][rim[index]]
if changes.shape[0] < INNER_MIN_PIXELS:
continue
# From here the rim is large enough to judge, so this reading stands
# and the band is not asked. New content keeps a zero shift: the band
# of a repainted edge reads the new colour itself as drift (a yellow
# swimsuit over skin turned the whole patch blue).
held[index] = True
middle = changes.median(dim=0).values
agree = (changes - middle).norm(dim=-1) < INNER_SAME_CONTENT
if float(agree.float().mean()) < INNER_MIN_SHARE:
offsets[index] = beside(index)
continue
offset = changes[agree].median(dim=0).values
if float(offset.norm()) > INNER_MAX_SHIFT:
offsets[index] = beside(index)
continue
offsets[index] = offset
return offsets, held
def _band_weights(blend: torch.Tensor) -> torch.Tensor:
return torch.where((blend > 0.001) & (blend < 0.999), blend, torch.zeros_like(blend))
+23 -19
View File
@@ -13,6 +13,8 @@ from collections import OrderedDict
from pathlib import Path
from typing import Any
from ._route_errors import error_detail
try:
import folder_paths
except ImportError: # pragma: no cover - only outside ComfyUI
@@ -136,27 +138,29 @@ _remap_noted: set[str] = set()
def resolve_lora_path(name: str) -> Path:
if folder_paths is None:
raise ValueError("folder_paths is unavailable outside ComfyUI.")
full = folder_paths.get_full_path("loras", name)
if not full:
# Only a name on ComfyUI's own LoRA list reaches the disk. A route's
# name or a widget value is never joined to a folder as given: it is
# matched against the list first, and anything else is refused here.
candidate, status = match_lora_name(str(name), list_loras())
if status == "remapped":
# The saved path is stale; a moved file should just keep working.
candidate, status = match_lora_name(str(name), list_loras())
if status == "remapped":
if name not in _remap_noted:
_remap_noted.add(name)
print(
f"[AusBoss] LoRA Loader: '{name}' is not at its saved "
f"path; using '{candidate}'."
)
full = folder_paths.get_full_path("loras", candidate)
elif status == "ambiguous":
raise ValueError(
f"LoRA name matches several files in models/loras: {name}"
if name not in _remap_noted:
_remap_noted.add(name)
print(
f"[AusBoss] LoRA Loader: '{name}' is not at its saved "
f"path; using '{candidate}'."
)
elif status == "ambiguous":
raise ValueError(
f"LoRA name matches several files in models/loras: {name}"
)
elif status != "exact":
raise ValueError(f"LoRA file not found in models/loras: {name}")
full = folder_paths.get_full_path("loras", candidate)
if not full:
raise ValueError(f"LoRA file not found in models/loras: {name}")
# Containment is checked on the path as written, with ".." folded but
# links not followed: a LoRA kept on another drive and linked into
# models/loras is the user's own choice, and ComfyUI loads it too.
# The listed file must also sit below one of the loras folders, read
# from the path as ComfyUI built it, with ".." folded.
path = Path(os.path.abspath(full))
roots = [Path(os.path.abspath(root)) for root in folder_paths.get_folder_paths("loras")]
if not any(root in path.parents for root in roots):
@@ -388,8 +392,8 @@ def apply_lora_stack(
except MemoryError:
raise
except Exception as exc:
# ComfyUI's own message here is PyTorch's and never names the file.
detail = f"{type(exc).__name__}: {' '.join(str(exc).split())}"
# ComfyUI can add the file's path; error_detail drops words that name one.
detail = error_detail(exc)
raise ValueError(
unreadable_lora_message(row["name"], unreadable_lora_reason(path), detail)
) from exc
+8 -1
View File
@@ -242,13 +242,20 @@ def _locate_comfy_file(text: str, rule: str) -> tuple[str, str, Path]:
if (
parts[0] == "~"
or any(part in ("", ".", "..") for part in parts)
or any(re.match(r"[A-Za-z]:", part) for part in parts)
or (os.name == "nt" and ":" in inside)
or re.search(r"[\x00-\x1f\x7f]", inside)
):
raise ValueError(rule)
root = _server_folders().get(folder)
if root is None:
raise ValueError(rule)
path = root.joinpath(*parts).resolve()
# Checked as text before the disk resolves it, and again after, so a
# link that leads out of the folder is refused too.
lexical = root.joinpath(*parts)
if root not in lexical.parents:
raise ValueError(rule)
path = lexical.resolve()
if root not in path.parents:
raise ValueError(rule)
return folder, inside, path
+15 -3
View File
@@ -1,11 +1,12 @@
"""What a route says back when a request fails."""
"""What a route or a node says back when something fails."""
from __future__ import annotations
import re
# A whole path: a word that opens with "/", "\" or a drive letter.
_SERVER_PATH = re.compile(r"""(?:^|[\s'"(\[=:,;])(?:[A-Za-z]:[\\/]|[\\/])[^\s'"]""")
# A whole path: a word that opens with "/", "\", "~/" or a drive letter,
# after anything but a letter, digit, "_" or "." ("and/or" and "3/4" stay).
_SERVER_PATH = re.compile(r"""(?:^|[^\w.])(?:[A-Za-z]:[\\/]|~[\\/]|[\\/])[^\s'"]""")
def safe_route_error(exc: Exception, fallback: str = "The request failed.") -> str:
@@ -16,3 +17,14 @@ def safe_route_error(exc: Exception, fallback: str = "The request failed.") -> s
if not text or _SERVER_PATH.search(text):
return fallback
return text
def error_detail(exc: BaseException) -> str:
"""A library's error for a node's message: its kind and its words, or
its kind alone when the words name a place on the server's disk. Node
errors reach the browser that queued the run and stay in the history."""
kind = type(exc).__name__
text = " ".join(str(exc).split())
if not text or _SERVER_PATH.search(text):
return kind
return f"{kind}: {text}"
+3 -2
View File
@@ -13,6 +13,7 @@ import numpy as np
import torch
from av.video.reformatter import ColorRange, Colorspace
from ._route_errors import error_detail
from ._execution_helpers import advance_progress, frame_progress, raise_if_interrupted
AAC_FRAME_SIZE = 1024
@@ -363,9 +364,9 @@ def _encoder_start_error(codec: str, exc: Exception) -> RuntimeError:
return RuntimeError(
"Save Video: the nvenc formats need an NVIDIA GPU with a working "
"driver. Pick a software format such as 'mp4 h264' instead. "
f"(The '{codec}' encoder said: {exc})"
f"(The '{codec}' encoder said: {error_detail(exc)})"
)
return RuntimeError(f"Save Video could not start the '{codec}' encoder: {exc}.")
return RuntimeError(f"Save Video could not start the '{codec}' encoder: {error_detail(exc)}.")
def _add_video_stream(container, spec: VideoFormat, rate, width: int, height: int, crf: int):
+4 -4
View File
@@ -164,7 +164,7 @@ class AusBossH3PromptWriter:
"INT",
{
"default": 8,
"min": 5,
"min": 4,
"max": 15,
"step": 1,
"tooltip": "How long the clip is. Plan on about one moment a second at most.",
@@ -254,8 +254,8 @@ class AusBossH3PromptWriter:
"default": VIDEO_JOBS[0],
"tooltip": (
"What the video is for: copy its moves, swap the person in it, keep it going, give it "
"a new voice, or change something in it and keep the person (a hat, a shirt). Swap, "
"new voice and change something ignore Seconds and keep the video's length."
"a new voice, or change something in it and keep the person (a hat, a shirt). Copy its "
"moves, swap, new voice and change something ignore Seconds and keep the video's length."
),
},
),
@@ -393,7 +393,7 @@ def _register_routes():
async def read(request):
try:
body = await request.json()
seconds = min(max(int(body.get("seconds", 8)), 5), LONGEST_SECONDS)
seconds = min(max(int(body.get("seconds", 8)), 4), LONGEST_SECONDS)
jobs = {int(slot): job for slot, job in dict(body.get("jobs") or {}).items() if job in JOBS and int(slot) in PICTURES}
video = body.get("video") if body.get("video") in VIDEO_JOBS else ""
return web.json_response(script_report(str(body.get("script", ""))[:LONGEST_SCRIPT], seconds, jobs, video))
+15 -3
View File
@@ -2,6 +2,8 @@
from __future__ import annotations
import hashlib
from pathlib import Path
from ._execution_helpers import ignore_extra_inputs
@@ -18,6 +20,7 @@ from ._image_folder_helpers import (
choose,
clean_folder,
fingerprint,
inside_root,
list_folders,
list_pictures,
load_picture,
@@ -69,6 +72,13 @@ def _picked(source, folder, subfolders, sort, pictures) -> list[dict]:
return choose(listed, parse_picked(pictures))
def _version_tag(item: dict) -> str:
"""A short tag that changes when the picture does, for the browser's
thumbnail cache. It carries no file size or time."""
seen = f"{item['mtime_ns']}-{item['size']}".encode()
return hashlib.sha1(seen, usedforsecurity=False).hexdigest()[:12]
def _register_routes() -> None:
"""The gallery's three questions: which folders, which pictures, and a
small copy of one. Every path is attacker-controlled; each is resolved
@@ -117,7 +127,7 @@ def _register_routes() -> None:
"source": source,
"folder": shown,
"full": len(listed) >= MAX_PICTURES,
"pictures": [{"name": item["name"], "v": f"{item['mtime_ns']}-{item['size']}"} for item in listed],
"pictures": [{"name": item["name"], "v": _version_tag(item)} for item in listed],
})
@routes.get("/ausboss/image_folder/thumb")
@@ -129,8 +139,10 @@ def _register_routes() -> None:
base = clean_folder(query(request, "folder"))
# The name is a path below the folder; it goes through the same door.
relative = clean_folder(f"{base}/{name}" if base else name)
path = root.joinpath(*relative.split("/")).resolve()
if root not in path.parents or not path.is_file() or not path.name.lower().endswith(IMAGE_EXTENSIONS):
if not relative or not relative.lower().endswith(IMAGE_EXTENSIONS):
return web.json_response({"error": "no such picture"}, status=404)
path = inside_root(root, relative)
if not path.is_file() or not path.name.lower().endswith(IMAGE_EXTENSIONS):
return web.json_response({"error": "no such picture"}, status=404)
try:
edge = int(query(request, "size") or 160)
+2 -1
View File
@@ -6,6 +6,7 @@ import sys
import torch
from ._route_errors import error_detail
from ._mask_by_name_helpers import (
IF_NOTHING,
NEEDS_SAM3,
@@ -238,7 +239,7 @@ class AusBossMaskByName:
raise
# A model or text reader that is not SAM 3's fails somewhere
# inside; say what to wire instead of showing that.
raise RuntimeError(f"{NOT_SAM3} ({type(error).__name__}: {error})") from error
raise RuntimeError(f"{NOT_SAM3} ({error_detail(error)})") from error
masks, boxes = output.result
counts = [len(frame) for frame in boxes]
# How sure SAM 3 was of each match in the first picture, best first.
+6 -5
View File
@@ -15,6 +15,7 @@ import torch
import torch.nn.functional as F
from PIL import Image
from ._route_errors import error_detail, safe_route_error
from ._execution_helpers import advance_progress, progress_bar, raise_if_interrupted, warn_once
from ._fine_edge_helpers import OFF as FINE_OFF
from ._fine_edge_helpers import (
@@ -485,7 +486,7 @@ class _Session:
except Exception as error: # noqa: BLE001 - told apart just below
if _stopped_by_user(error) or _out_of_memory(error):
raise
raise RuntimeError(f"{NOT_SAM3} ({type(error).__name__}: {error})") from error
raise RuntimeError(f"{NOT_SAM3} ({error_detail(error)})") from error
_WORDS[key] = held
while len(_WORDS) > WORDS_KEPT:
_WORDS.popitem(last=False)
@@ -561,14 +562,14 @@ class _Session:
if self._open is None:
# A model or a text reader that is not SAM 3's fails
# somewhere inside; say what to wire instead of showing that.
raise RuntimeError(f"{NOT_SAM3} ({type(error).__name__}: {error})") from error
raise RuntimeError(f"{NOT_SAM3} ({error_detail(error)})") from error
self.close()
try:
answer = self._call(kind, frame, what)
except Exception as error: # noqa: BLE001
if _stopped_by_user(error) or _out_of_memory(error):
raise
raise RuntimeError(f"{NOT_SAM3} ({type(error).__name__}: {error})") from error
raise RuntimeError(f"{NOT_SAM3} ({error_detail(error)})") from error
# The plain way answers where the quick way failed.
self._distrust("a question failed with it")
return answer
@@ -1288,7 +1289,7 @@ class AusBossMaskByTouch:
if item.kind not in ("subject", "background") or not asked or _stopped_by_user(error):
raise
area, used, which = None, "none", ()
subject_trouble = str(error)
subject_trouble = safe_route_error(error, type(error).__name__)
if index == 0:
share = 0.0 if area is None else round(float(area.sum()) / (height * width), 5)
item_notes.append({"ok": used != "none", "share": share, "as": used})
@@ -1348,7 +1349,7 @@ class AusBossMaskByTouch:
# and the picture says why.
if not asked or _stopped_by_user(error):
raise
fine, fine_trouble = False, str(error)
fine, fine_trouble = False, safe_route_error(error, type(error).__name__)
if fine and soft:
# Where a soft brush had the last word its fade stays as painted.
fade = torch.stack(painted) & (raw > 0) & (raw < 1)
+1 -1
View File
@@ -3,7 +3,7 @@ name = "ausboss-nodes"
# Manager/registry search matches against this text, so it names the actual
# nodes instead of describing the collection abstractly.
description = "ComfyUI-AusBoss (AusBoss nodes): image and video nodes with compact controls: Image and Video Crop + Rotate + Pad (frame or clip), Load Image, Load Image + Pad, Image Folder, Image Resize, Align Image, Image Size, Color Match, Image Compare A/B, Realign to Source (experimental), Save Image (PNG, WebP/JXL, captions), Load Video, Load Audio, Save Video, Select Frame, Frame Interpolate, Mask Refine, Mask by Name and Mask by Touch, LaMa Inpaint, Crop For Inpaint, Stitch Inpaint, Tiled Upscale and Tiled Upscale Stitch, FlashVSR Upscale, Edit Lock and Edit Lock Restore, LoRA Loader, Krea 2 Encode and Outpaint Model Patch, Latent Size, Layout Boxes, Seed, Select Every Nth, Split Batch, Merge Batches, Text, Text Lines, Prompt Variations, H3 Prompt Writer, Integer, Float, Math Expression, Show Text, Free Memory, Workflow Note, Workflow Note Tile, Callout, Callout Tile, Workflow Switches, Run Timer, and Step Preview. Example workflows for Qwen Image 2.1, Krea 2, Klein 9B and LTX 2.3."
version = "2.26.2"
version = "2.26.3"
# comfy-cli serializes this to JSON verbatim, so `file` publishes the literal
# string {"file": "LICENSE"} to the registry listing. `text` puts a readable
# name there instead. Both forms are supported; the LICENSE file is unchanged.
+305
View File
@@ -0,0 +1,305 @@
---
name: ausboss-workflows
description: Write the words for AusBoss ComfyUI video workflows (MiniMax H3 Pika Person Swap and Pika Remix, through the H3 Prompt Writer node), and set up and run those workflows on the person's own computer. Use when someone shares ausboss-examples.vercel.app, asks for a person swap, a video remix or an H3 prompt, or wants an AusBoss workflow installed or run for them.
license: MIT
compatibility: Writing the words needs nothing. Running needs ComfyUI 0.38 or newer on the person's computer, an 8 GB or larger graphics card, and a browser the agent can drive.
metadata:
author: AusBoss
homepage: https://ausboss-examples.vercel.app/
updated: "2026-11-16"
---
# AusBoss workflows
AusBoss makes a pack of ComfyUI nodes and the workflows built on them. This
page is for an AI agent helping someone use those workflows. It covers three
jobs, from light to heavy:
1. **Write the words.** The person says what they want; you write what goes in
the workflow's text box and say which settings to pick.
2. **Set it up.** Install the node pack and the model files.
3. **Run it for them.** Load the workflow in their ComfyUI, put in their files
and your words, run it, and hand back the clip.
Offer the third job when you can run commands on the person's computer. Ask
before you start it: it downloads large files (see "Model files") and uses
their graphics card for several minutes a clip.
## Where everything is
| What | Where |
|---|---|
| Node pack (source, help pages, issues) | https://github.com/ausboss/ComfyUI-AusBoss |
| The pack's own guide for agents (which node does which job) | https://github.com/ausboss/ComfyUI-AusBoss/blob/main/js/docs/agents/README.md |
| Workflows | https://civitai.com/user/AusBoss |
| Models and LoRAs | https://huggingface.co/ausboss |
| Real example runs (this site) | https://ausboss-examples.vercel.app/ |
| Help from people, and every workflow file | https://discord.gg/RwVxY9Fuaj |
| Video guides | https://www.youtube.com/@ausbossdev |
The examples on this site are real runs. Each page has a plain-text copy with
every script, setting, prompt and file link:
- Person Swap Examples (4 examples): https://ausboss-examples.vercel.app/person-swap/index.md
- Pika Remix Examples (8 examples): https://ausboss-examples.vercel.app/pika-remix/index.md
- Fix a Pika Remix or Person Swap Run (a guide): https://ausboss-examples.vercel.app/fix/index.md
- Faster Video in ComfyUI (a guide): https://ausboss-examples.vercel.app/speed/index.md
Read the matching page before you write for a workflow. A real example close
to the person's request is the best starting point.
## The workflows
| Workflow | What it does | Get it | Real runs |
|---|---|---|---|
| MiniMax H3 Pika Person Swap | Swaps the person in a clip for the person in one photo. The place, camera, moves and sound stay. | https://civitai.com/models/3040833 | https://ausboss-examples.vercel.app/person-swap/index.md |
| MiniMax H3 Pika Remix | Changes, swaps or copies a video using up to four pictures. Also starts a clip from a picture, or goes from one picture to another. | https://civitai.com/models/3050155 | https://ausboss-examples.vercel.app/pika-remix/index.md |
Both make video with sound, up to 15 seconds, with the MiniMax H3 model and
the Pika H3 Flex LoRA. Both have a High VRAM setting (24 GB or more) and a Low
VRAM setting (8 to 24 GB; on 8 GB, clips up to 8 seconds and 32 GB of RAM).
## Write the words
Both workflows have one text box, on the node titled
**H3 Prompt Writer - What happens**. The person types plain lines there and
the node writes the model's long prompt itself. So you write short plain
lines, never the long prompt. The full rules are on the node's help page:
https://github.com/ausboss/ComfyUI-AusBoss/blob/main/js/docs/AUSBOSS_NODES_H3PromptWriter.md
If that page and this one disagree, that page is right.
### The basics
- One line is one moment. Lines play in order. About one moment a second is
the most the model can play.
- A blank line cuts to a new place.
- `@1` to `@4` name the pictures plugged in. Each picture has a job, picked
under the text box.
- Say what a camera would see. `She smiles` works; `She feels happy` does not.
- Nothing moves or exists unless a line says so.
- Say which way people face, and where they end up.
- Spoken words go in quotes, with the speaker named: `@1 says "we are late."`
- Put the spoken line last. With time left after it, the model keeps talking.
### Picture jobs
| Job | Use it for |
|---|---|
| person | This person goes where the lines say. A clear photo. |
| character sheet | One character drawn from several sides. |
| place | Where it happens. A picture with nobody in it. |
| outfit | Clothes for the person. Best with no head in the picture. |
| thing | An object, product, toy or mascot. |
| style | Only the look is used, for the whole clip. |
| start on it | The clip starts from this picture. |
| end on it | The clip lands on this picture. It can't be named with `@`. |
### Video jobs
| Job | What happens | What to type |
|---|---|---|
| swap the person | The pictured person takes over from the one in the video. | Who goes, by what they look like: `Replace the man in the hat with @1.` |
| copy its moves | The pictured person does the video's moves somewhere new. | `@1 dances on a rooftop at sunset.` |
| change something | The video stays, with one change. | The change, exactly: `She wears a red raincoat.` |
| keep going | More of the same clip, from where it ends. | What happens next, or nothing. |
| new voice | The same clip with a new voice. | The words, in quotes. Needs a voice clip. |
For **swap the person**, **change something** and **new voice**, the clip's
length comes from the video. To make it shorter, trim the video on its loader.
### Person swap: the lines that work
```
Replace the man in the suit holding an ice cream with @1.
@1 looks like a heavy-set man with round glasses, a white shirt and green trousers.
```
- Line 1 says who goes. Describe them by what is seen in the clip (clothes,
hair, position), so the others in the shot are left alone.
- Line 2 is optional and describes the new person. Write it from their photo.
It helps the look; it does not promise an exact likeness. The Person Swap
workflow can write this line itself: the **Describe @1 for me** switch.
- If the photo shows only a face, write the clothes too. What the photo does
not show, the model makes up.
- The face in the clip has to be clear. A small or turned-away face keeps the
clip's own person.
- **Keep the clip's sound** on: the clip's own voice and music stay and the
new mouth follows them. Off, with words in quotes: the model makes a new
voice.
### Other things a line can do
- A time: start a line with `3s:` and it happens at 3 seconds.
- `sound: rain on a tin roof` and `music: slow sad piano`. With no music line
there is no music.
- `camera: slow push in` and `look: a 1970s film`.
- `no cuts` makes blank lines stop being cuts.
- `ends on her sitting by the window` sets the last moment.
- A sign: `a neon sign reads "OPEN"`.
### What to hand the person
When you only write the words, give them:
1. The lines, in a code block they can paste into the text box.
2. Which file goes in which loader (`@1` is Picture 1, and so on).
3. The job for each picture and for the video.
4. Any switch to change, such as **Keep the clip's sound**.
A voice clip should be the person's own voice, or one they have permission to
use. Only swap a real person into a clip when the person asking has the right
to use that photo.
## Set it up
Check before you install anything:
```bash
curl -s http://127.0.0.1:8188/system_stats
```
That answers when ComfyUI is running on its usual port, with its version and
the graphics card's memory. If nothing answers, ask the person where their
ComfyUI is and how they start it. If they have none, the official install
guide is https://docs.comfy.org/installation and that choice is theirs.
### The node pack
Either of these, then restart ComfyUI:
```bash
comfy node install ausboss-nodes
```
```bash
git clone https://github.com/ausboss/ComfyUI-AusBoss.git
```
The first needs comfy-cli. The second runs inside ComfyUI's `custom_nodes`
folder. In the ComfyUI window, the person can also open the Manager, search
for **AusBoss** and install it there. After the restart this returns the
node, which shows the pack loaded:
```bash
curl -s http://127.0.0.1:8188/object_info/AUSBOSS_NODES_H3PromptWriter
```
### Model files
The files go in ComfyUI's `models` folder, in the subfolder named below. They
are large. Tell the person the total and get a yes before you download. Skip
any file they already have.
| File | Folder in `models` | Size | Needed |
|---|---|---|---|
| [qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors) | `text_encoders` | 15.7 GB | always |
| [minimax_h3_audio_vae_fp32.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors) | `vae` | 0.6 GB | always |
| [pika_h3_flex_ref2va_v1_bf16.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/loras/pika_h3_flex_ref2va_v1_bf16.safetensors) | `loras` | 2.0 GB | always |
| [minimax_h3_ref2va_pruned_int8_convrot.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/diffusion_models/minimax_h3_ref2va_pruned_int8_convrot.safetensors) | `diffusion_models` | 21.0 GB | High VRAM |
| [minimax_h3_video_vae_fp16.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_fp16.safetensors) | `vae` | 5.2 GB | High VRAM |
| [minimax_h3_ref2va_pruned_w6a8.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/diffusion_models/minimax_h3_ref2va_pruned_w6a8.safetensors) | `diffusion_models` | 16.0 GB | Low VRAM |
| [minimax_h3_video_vae_int8_convrot.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_int8_convrot.safetensors) | `vae` | 2.8 GB | Low VRAM |
| [flashvsr_v1.1_fp16.safetensors](https://huggingface.co/ausboss/FlashVSR-v1.1-ComfyUI/resolve/main/flashvsr_v1.1_fp16.safetensors) | `diffusion_models` | 3.5 GB | only for Finish: FlashVSR |
High VRAM needs about 44 GB of files, Low VRAM about 37 GB. The optional switches that write text for you (such as **Describe @1 for me**) use a small Qwen3-VL 8B text model; the workflow's Guide note names that file and where to get it.
Each workflow opens with a Guide note that lists its files. If the note and
this table disagree, the note in the newest workflow is right.
### Faster runs
One node or one launch setting makes these workflows about twice as fast on
most recent cards: https://ausboss-examples.vercel.app/speed/index.md
## Run it for them
### Get the workflow file
- The newest file is on the workflow's Civitai page (links above). Civitai may
ask for a login. If it does, ask the person to download the file and tell
you where it is. The same files are in the Discord.
- Each Person Swap example on this site also has a zip with the workflow set
up exactly as it ran, its input files and its result. Those can be older
than the Civitai file. They are good for a first test run: the result is
there to compare with.
### Put the person's files in place
Copy their clip and pictures into ComfyUI's `input` folder, or upload them:
```bash
curl -s -F "image=@/path/to/photo.jpg" http://127.0.0.1:8188/upload/image
```
The same address takes video and audio files.
### Load it in ComfyUI's own page and run
These workflows have switches and grouped parts that only ComfyUI's page
turns into a run. So do not rewrite the file into an API prompt by hand:
open the page, load the file there, and let the page queue it. Any browser
you can drive works, headless or not (Chrome started with
`--remote-debugging-port`, Playwright, or a browser tool).
On the page at `http://127.0.0.1:8188/`, once `window.app.graph` exists:
```js
await app.loadGraphData(workflow); // the parsed workflow file
const node = (title) => app.graph.nodes.find((n) => n.title === title);
const set = (title, name, value) => {
const w = node(title).widgets.find((w) => w.name === name);
w.value = value;
w.callback?.(value);
};
set("H3 Prompt Writer - What happens", "script", "Replace the man in the hat with @1.");
set("Video - Your clip", "video", "their_clip.mp4");
set("Picture 1 - @1", "image", "their_photo.jpg");
set("Seed", "seed", 6202);
await app.queuePrompt(0);
```
Find a node by its title and a setting by its name. The titles above are the
ones in Person Swap; read `app.graph.nodes.map((n) => n.title)` for any other
workflow. `await app.graphToPrompt()` shows what would be sent without running
anything, which is a cheap check before you queue.
**Switches** (VRAM, Keep the clip's sound, Describe @1 for me, Finish) are
cards on the canvas. To flip one without clicking, change the workflow file
before you load it, as the pack's guide describes:
https://github.com/ausboss/ComfyUI-AusBoss/blob/main/js/docs/agents/workflow_switches.md
With a browser tool that can click, clicking the switch on the card is
simpler. Pick **Low VRAM** when the card has less than 24 GB.
### Wait, then hand back the clip
`queuePrompt` returns right away. Watch the run with:
```bash
curl -s http://127.0.0.1:8188/queue
```
```bash
curl -s http://127.0.0.1:8188/history
```
A clip takes a few minutes; the first run is longer because the models load.
The finished clip is in ComfyUI's `output` folder, and the history entry
names the file. Tell the person where it is, and show it if you can.
### If it fails
- A loader turns red or the run stops on a file name: that model file is
missing or has another name. Pick the person's copy in that loader.
- Out of memory: pick **Low VRAM**, use a shorter clip, or turn on
**Fast preview**.
- The swap kept the clip's own person: the face in the clip is too small or
turned away, or line 1 did not describe who goes.
- A clip came out wrong but ran: run again with a new seed.
- Anything else: the pack's issues
(https://github.com/ausboss/ComfyUI-AusBoss/issues) and the Discord.
## About this page
Everything here is public information from AusBoss about his own nodes and
workflows. It gives no instruction to act without the person's say: installs,
downloads and runs are theirs to approve.
+36
View File
@@ -102,3 +102,39 @@ test("a server that does not answer keeps the page quiet", async (t) => {
await page.serverComesBack("2.7.0");
assert.equal(page.toasts.length, 1);
});
// The Manager switches the pack's files on disk, and the server keeps running
// the old version until ComfyUI restarts. A reload in between gives the page
// the new files: the page is newer than the server, and "press F5" is wrong.
test("a page newer than the server says nothing", async (t) => {
const page = await openPage(t, { running: "2.26.2", installed: "2.25.2" });
assert.deepEqual(page.toasts, []);
assert.deepEqual(page.refreshed, []);
await page.serverComesBack("2.26.2");
assert.deepEqual(page.toasts, [], "after the restart the two match");
});
test("versions compare as numbers, not as text", async (t) => {
const older = await openPage(t, { running: "2.25.10", installed: "2.25.9" });
assert.deepEqual(older.toasts, [], "2.25.9 is older than 2.25.10");
const newer = await openPage(t, { running: "2.25.9", installed: "2.25.10" });
assert.equal(newer.toasts.length, 1, "2.25.10 is newer than 2.25.9");
assert.match(newer.toasts[0].detail, /v2\.25\.10 is installed/);
});
test("an unknown or odd version says nothing", async (t) => {
for (const [running, installed] of [
["2.26.2", "unknown"],
["2.26.2", "2.27"],
["2.26.2", "2.27.0rc1"],
["2.26.2", "v2.27.0"],
["2.26.2", 3],
["unknown", "2.27.0"],
["dev", "2.27.0"],
[null, "2.27.0"],
]) {
const page = await openPage(t, { running, installed });
assert.deepEqual(page.toasts, [], `page ${running}, server ${installed}`);
assert.deepEqual(page.refreshed, [], `page ${running}, server ${installed}`);
}
});
+1
View File
@@ -142,6 +142,7 @@ test("only the listed parts of the pack listen for the wheel", () => {
"image_folder/index.js root", // the copy question, a window over the page
"image_folder/index.js root", // the folder list, a window over the page
"shared/discard_prompt.mjs backdrop", // the keep or discard question
"shared/h3_cast_chips.mjs window", // closes a chip's job list when the graph moves
"shared/media_picker.mjs window", // closes its list when the graph moves
"shared/transform_editor.mjs canvas", // the full-screen editor zooms its picture
"shared/widget_card.mjs area", // a text box scrolls its own text first
+14
View File
@@ -0,0 +1,14 @@
import test from "node:test";
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
const source = readFileSync(new URL("../js/shared/h3_cast_chips.mjs", import.meta.url), "utf-8");
test("a chip's job list stays open while an item is pressed", () => {
// Pressing an item takes focus off the chip's button. A capturing blur
// listener heard that and closed the list before the click could pick.
const blur = source.match(/addEventListener\("blur",[^\n]*/)?.[0] ?? "";
assert.match(blur, /event\.target === window/, "only the window losing focus closes the list");
assert.doesNotMatch(blur, /\bon\)/, "the blur listener is not the capturing set");
assert.match(source, /menu\.addEventListener\("mousedown", \(event\) => event\.preventDefault\(\)\)/, "a press keeps focus on the button");
});
+4 -4
View File
@@ -11,10 +11,10 @@ const graphWith = (mode = 0, useMap = false) => {
getNodeById:id=>id===150?source:null,
} } };
};
test('only the three backend duration overrides use the clip', () => {
test('only the four backend duration overrides use the clip', () => {
const {node}=graphWith();
for(const job of ['swap the person','new voice','change something']) assert.equal(clipSetsDuration(node,{video_job:job}),true);
for(const job of ['copy its moves','keep going','',undefined]) assert.equal(clipSetsDuration(node,{video_job:job}),false);
for(const job of ['copy its moves','swap the person','new voice','change something']) assert.equal(clipSetsDuration(node,{video_job:job}),true);
for(const job of ['keep going','',undefined]) assert.equal(clipSetsDuration(node,{video_job:job}),false);
});
test('Full Video OFF/bypassed/missing/disconnected restore seconds; both link stores work', () => {
for(const map of [false,true]) {
@@ -73,7 +73,7 @@ test('timeline omits guessed duration/retiming and rejects an old async report;
const {momentStrip}=await import('data:text/javascript;base64,'+Buffer.from(source).toString('base64'));
const {node,source:video}=graphWith();let refreshes=0;node.__ausbossCard={refresh(){refreshes++;}};
const panel=momentStrip(node), [ruler,track,note]=panel.element.children;
const values={seconds:12,script:'She walks.',video_job:'copy its moves'};
const values={seconds:12,script:'She walks.',video_job:'keep going'};
panel.sync(values);assert.equal(timers.length,1);timers.shift()();await Promise.resolve();
assert.equal(requests[0].seconds,12);
panel.sync({...values,video_job:'swap the person'});
+6 -6
View File
@@ -24,15 +24,15 @@ test("cuts and speech are marked", () => {
test("no report and an empty script draw nothing and say what to do", () => {
assert.deepEqual(stripRows(null), []);
assert.deepEqual(stripCaption(null), { text: "Each line becomes a moment.", warn: false });
assert.deepEqual(stripCaption({ moments: [] }), { text: "Each line becomes a moment.", warn: false });
assert.deepEqual(stripCaption(null), { text: "Each line becomes a moment. A blank line between lines is a cut.", warn: false });
assert.deepEqual(stripCaption({ moments: [] }), { text: "Each line becomes a moment. A blank line between lines is a cut.", warn: false });
});
test("the caption adds the script up", () => {
assert.equal(stripCaption({ moments: [row(0, 2.5), row(2.5, 5)] }).text, "2 moments, 2.5 s each, one take.");
assert.equal(stripCaption({ moments: [row(0, 8)] }).text, "1 moment, 8 s, one take.");
assert.equal(stripCaption({ moments: [row(0, 3), row(3, 8)] }).text, "2 moments, one take.");
assert.equal(stripCaption({ moments: [row(0, 3), row(3, 8, { cut: true })] }).text, "2 moments, 1 cut.");
assert.equal(stripCaption({ moments: [row(0, 2.5), row(2.5, 5)] }).text, "2 moments, 2.5 s each, one take. Click between two moments to cut there.");
assert.equal(stripCaption({ moments: [row(0, 8)] }).text, "1 moment, 8 s, one take. Add a line for the next moment, a blank line to cut.");
assert.equal(stripCaption({ moments: [row(0, 3), row(3, 8)] }).text, "2 moments, one take. Click between two moments to cut there.");
assert.equal(stripCaption({ moments: [row(0, 3), row(3, 8, { cut: true })] }).text, "2 moments, 1 cut. Click a cut on the bar to join the shots.");
});
test("only connected pictures send their job", () => {
+225 -8
View File
@@ -182,6 +182,20 @@ class SplitTests(unittest.TestCase):
def test_a_quote_can_end_a_sentence(self):
self.assertEqual(moments('A voiceover says "this is how it ended." She walks away.'), ['A voiceover says "this is how it ended."', "She walks away"])
def test_an_at_after_a_quote_starts_a_new_moment(self):
self.assertEqual(moments('@1 says "Hi." @1 waves and says "Bye."'), ['@1 says "Hi."', '@1 waves and says "Bye."'])
self.assertEqual(moments('@1 says "Hi" and @2 says "Bye."'), ['@1 says "Hi"', '@2 says "Bye."'])
self.assertEqual(moments('"Hi," she says. @1 laughs.'), ['"Hi," she says', "@1 laughs"])
def test_every_line_in_a_row_is_spoken(self):
def said(script, jobs):
return re.findall(r"<d>\[English\] (.*?)</d>", write_prompt(script, seconds=8.0, jobs=jobs))
self.assertEqual(said('@1 says "Hi!" @2 answers "Hello." @1 says "Bye."', {1: "person", 2: "person"}), ["Hi!", "Hello.", "Bye."])
self.assertEqual(said('@1 says "Hi" and @2 says "Bye."', {1: "person", 2: "person"}), ["Hi.", "Bye."])
self.assertEqual(said('"Hi," @1 says.', {1: "person"}), ["Hi."])
self.assertEqual(said('@1 says "我有冰淇淋。" @1 says "好吗?" @1 says "但是,"', {1: "person"}), ["我有冰淇淋。", "好吗?", "但是。"])
def test_the_timeline_shows_plain_words_for_a_hold(self):
self.assertEqual(script_report("3s: she jumps", 8)["moments"][0]["text"], "Holds still")
@@ -643,7 +657,7 @@ class VideoJobTests(unittest.TestCase):
for job in ("outfit", "thing"):
text = write_prompt("He wears @1 in place of his blue cap.", seconds=8.0, jobs={1: job}, video="change something")
self.assertIn("<Subject 2> is the person in <Video 1>, kept as they are", text)
self.assertIn("[video editing] <Subject 2> wears <Subject 1> in place of his blue cap. Everything else in <Video 1> is kept.", text)
self.assertIn("[video editing + audio reuse] <Subject 2> wears <Subject 1> in place of his blue cap. Everything else in <Video 1> is kept.", text)
self.assertIn("Retain the look of <Video 1>.", text)
self.assertIn("<Subject 2> has the same words, mouth movements and moves as in <Video 1>.", text)
self.assertIn("The original sound of <Video 1> continues throughout.", text)
@@ -659,7 +673,7 @@ class VideoJobTests(unittest.TestCase):
self.assertIn("put only this outfit on <Subject 2>", text)
self.assertNotIn("<Subject 1> is the person in <Picture 1>", text)
text = write_prompt("He wears the propeller hat instead of his cap.", seconds=8.0, jobs={1: "person"}, video="change something")
self.assertIn("[video editing] <Subject 2> wears <Subject 1> instead of his cap.", text)
self.assertIn("[video editing + audio reuse] <Subject 2> wears <Subject 1> instead of his cap.", text)
def test_change_something_keeps_a_person_picture_that_is_not_put_on(self):
self.assertEqual(put_on_from_pictures("@1 stands next to him.", {1: "person"}), ("@1 stands next to him.", {1: "person"}))
@@ -696,9 +710,8 @@ class VideoJobTests(unittest.TestCase):
def test_a_hat_with_no_person_picture_points_to_change_something(self):
for job in ("outfit", "thing"):
for video in ("swap the person", "copy its moves"):
cast = Cast(number_pictures({1: job}, video), video)
self.assertIn('pick the video job "change something"', cast.problem("He wears @1."))
cast = Cast(number_pictures({1: job}, "swap the person"), "swap the person")
self.assertIn('pick the video job "change something"', cast.problem("He wears @1."))
def test_copying_moves_keeps_the_performer_out(self):
parts = parts_of("@1 dances on a rooftop.", jobs={1: "person"}, video="copy its moves")
@@ -710,11 +723,62 @@ class VideoJobTests(unittest.TestCase):
def test_the_timeline_says_so_when_nothing_is_connected(self):
self.assertIn("Connect a picture or a video", script_report("She runs.", 6, {})["warning"])
self.assertIn("To copy the moves", script_report("She runs.", 6, {}, "copy its moves")["warning"])
self.assertEqual(script_report("She runs.", 6, {}, "copy its moves")["warning"], "")
def test_copying_moves_with_nobody_to_give_them_to_says_so(self):
def test_copying_moves_with_no_person_picture_keeps_the_person_in_the_video(self):
cast = Cast(number_pictures({1: "place"}, "copy its moves"), "copy its moves")
self.assertIn("To copy the moves", cast.problem("She dances."))
self.assertEqual(cast.problem("She dances in @1."), "")
parts = parts_of("She dances in @1.", jobs={1: "place"}, video="copy its moves")
self.assertIn("<Subject 2> is the person in <Video 1>, with the same face, hair, body and clothes", parts["subject_definitions"])
self.assertIn("the background of <Video 1> is not kept, and the place is <Subject 1>", parts["retention_analysis"])
self.assertIn("<Subject 2> dances in <Subject 1>. <Subject 2> moves exactly as in <Video 1>", parts["detailed_description"])
self.assertNotIn("performs the same moves as", parts["detailed_description"])
def test_restyling_an_empty_clip_names_no_person_and_drops_the_look(self):
parts = parts_of("It looks real, like @1.", jobs={1: "style"}, video="change something")
self.assertIn("<Video 1> supplies the place, the action, the timing, the camera movement and the sound.", parts["subject_definitions"])
self.assertNotIn("the person in <Video 1>", parts["subject_definitions"])
self.assertNotIn("the look", parts["retention_analysis"])
self.assertNotIn("Retain the look of <Video 1>", parts["detailed_description"])
def test_an_edit_with_a_place_picture_takes_the_place_and_look_from_it(self):
parts = parts_of("The hallway looks like @1.", jobs={1: "place"}, video="change something")
self.assertIn("<Video 1> supplies the action, the timing, the camera movement and the sound.", parts["subject_definitions"])
self.assertIn("the colours and the film grain of <Picture 1>", parts["detailed_description"])
self.assertNotIn("is in the shot", parts["detailed_description"])
def test_an_edit_about_someone_still_keeps_them(self):
parts = parts_of("She looks real, like @1.", jobs={1: "style"}, video="change something")
self.assertIn("is the person in <Video 1>, kept as they are", parts["subject_definitions"])
self.assertIn("the person", parts["subject_definitions"].splitlines()[0])
def test_copying_only_a_camera_move_names_no_person(self):
parts = parts_of("The camera glides down @1.", jobs={1: "place"}, video="copy its moves")
self.assertNotIn("the person in <Video 1>", parts["subject_definitions"])
self.assertIn("apply only its camera movement", parts["retention_analysis"])
self.assertNotIn("moves exactly as in", parts["detailed_description"])
def test_a_thing_that_does_the_action_takes_the_moves(self):
parts = parts_of("@1 waves its finger.", jobs={1: "thing"}, video="copy its moves")
self.assertIn("<Subject 1> is the character in <Picture 1>, with the same shape, colours and drawing style.", parts["subject_definitions"])
self.assertIn("apply only the body motion, choreography and timing to <Subject 1>", parts["retention_analysis"])
self.assertIn("<Subject 1> performs the same moves as <Subject 2>", parts["detailed_description"])
def test_a_thing_can_come_in_for_the_one_in_a_swap(self):
parts = parts_of("@1 replaces the yellow hand.", jobs={1: "thing"}, video="swap the person")
self.assertIn("<Subject 1> is the yellow hand in <Video 1>, selected for replacement.", parts["subject_definitions"])
self.assertIn("has the shape, colours and drawing style of the character in <Picture 1>", parts["detailed_description"])
self.assertEqual(script_report("", 6, {1: "thing"}, "swap the person")["warning"], "")
def test_a_held_thing_stays_a_prop(self):
parts = parts_of("He holds @1 and dances.", jobs={1: "thing"}, video="copy its moves")
self.assertIn("<Subject 1> is the object in <Picture 1>", parts["subject_definitions"])
self.assertIn("<Subject 2> moves exactly as in <Video 1>", parts["detailed_description"])
def test_copying_moves_with_only_an_outfit_dresses_the_person_in_the_video(self):
parts = parts_of("He wears @1.", jobs={1: "outfit"}, video="copy its moves")
self.assertIn("the clothes change to <Subject 1>", parts["retention_analysis"])
self.assertIn("keep its place unless the script names a new one", parts["retention_analysis"])
def test_a_swap_with_nobody_to_put_in_says_so(self):
with self.assertRaisesRegex(ValueError, "new person"):
@@ -755,6 +819,30 @@ class VideoJobTests(unittest.TestCase):
self.assertEqual(add_looks("Swap him for @1.", {1: "person"}, lambda slot: said),
"Swap him for @1.\n@1 looks like a man with curly brown hair, wearing dark-framed glasses")
def test_a_swap_reads_the_looks_sentence_on_the_same_line(self):
from nodes._h3_cast_helpers import add_looks
typed = "replace the girl on the car with @1 .@1 looks like a young woman with long dark hair"
text = write_prompt(typed, seconds=8.0, jobs={1: "person"}, video="swap the person")
self.assertIn("<Subject 1> is the girl on the car in <Video 1>", text)
self.assertIn("this appearance: a young woman with long dark hair.", text)
self.assertNotIn("looks like", text)
# A typed sentence still wins over the model, wherever it sits on the line.
self.assertNotIn("bald", add_looks(typed, {1: "person"}, lambda slot: "a bald man"))
# A spoken line that happens to hold the words is left alone.
said = 'Replace the guy with @1.\n@1 says: "He said @1 looks like trouble."'
self.assertNotIn("this appearance", write_prompt(said, seconds=8.0, jobs={1: "person"}, video="swap the person"))
# Only a new sentence is split off: inside a sentence the words are left where they are.
from nodes._h3_cast_helpers import looks_on_own_lines
self.assertEqual(looks_on_own_lines("Put @1 looks like a friend here."), "Put @1 looks like a friend here.")
self.assertEqual(looks_on_own_lines("Swap him for @1! @1 looks like a pilot"), "Swap him for @1!\n@1 looks like a pilot")
def test_a_swap_reads_clothes_typed_on_their_own_line(self):
swap = dict(seconds=8.0, jobs={1: "person"}, video="swap the person")
text = write_prompt("Replace the man in the checked shirt with @1.\n@1 is wearing a black suit.", **swap)
self.assertIn("this appearance: the same hair, wearing a black suit.", text)
both = "Replace the guy with @1.\n@1 looks like a man with short dark hair.\n@1 wears a black suit."
self.assertIn("this appearance: a man with short dark hair, wearing a black suit.", write_prompt(both, **swap))
def test_a_swap_gets_the_firm_wording_without_a_looks_like_line(self):
text = write_prompt("Replace the guy with @1.", seconds=8.0, jobs={1: "person"}, video="swap the person")
self.assertIn("<Subject 2> has the face, hair and clothes of the person in <Picture 1>", text)
@@ -948,6 +1036,30 @@ class VideoJobTests(unittest.TestCase):
self.assertIn("each word landing exactly on the lip movement of <Video 1>", parts["detailed_description"])
self.assertIn("Quiet room tone", parts["overall_soundscape"])
def test_a_swap_keeps_every_line_the_new_person_says(self):
script = (
"Replace the young man in the black t-shirt with @1.\n"
'@1 holds a phone up and says: "How do I get rich?" The phone answers: "Get a job." '
'@1 looks up, confused, and says: "Bro, what are they talking about?"'
)
text = write_prompt(script, seconds=8.0, jobs={1: "person"}, video="swap the person")
self.assertEqual(re.findall(r"<d>\[English\] (.*?)</d>", text),
["How do I get rich?", "Get a job.", "Bro, what are they talking about?"])
# With the clip's sound kept, the phone's words come from the clip and only the new person's own lines are written.
text = write_prompt(script, seconds=8.0, jobs={1: "person"}, video="swap the person", clip_sound=True)
self.assertEqual(re.findall(r"<d>\[English\] (.*?)</d>", text), ["How do I get rich?", "Bro, what are they talking about?"])
def test_a_looks_line_keeps_only_the_looks(self):
script = (
"Replace the young man with @1.\n"
"@1 looks like a cartoon wooden log, no clothes. The trees behind @1 stay the same."
)
parts = parts_of(script, jobs={1: "person"}, video="swap the person")
self.assertIn("this appearance: a cartoon wooden log, no clothes. The background", parts["subject_definitions"])
self.assertNotIn("trees", parts["subject_definitions"])
for name, part in parts.items():
self.assertNotRegex(part, r"@\d", name)
def test_a_video_needs_words_unless_it_can_go_without(self):
with self.assertRaises(ValueError):
write_prompt("", jobs={1: "person"}, video="copy its moves")
@@ -1521,5 +1633,110 @@ class ClipSoundTests(unittest.TestCase):
self.assertEqual(moves[0], node.write('@1 dances.', 6, picture_1=PICTURE, video=frames, video_sound=sound, video_job="copy its moves")[0])
class PromptBugBatchTests(unittest.TestCase):
def test_a_swap_action_line_does_not_name_who_goes(self):
for script in ("@1 dances.", "He dances."):
text = write_prompt(script, seconds=6, jobs={1: "person"}, video="swap the person")
self.assertIn("<Subject 1> is the original person in <Video 1>, selected for replacement.", text)
self.assertNotIn("dances", text)
def test_two_people_need_a_named_speaker(self):
jobs = {1: "person", 2: "person"}
with self.assertRaisesRegex(ValueError, r"name who speaks: @2 says"):
write_prompt('@1 and @2 talk. She says "hi". He says "hello".', seconds=6, jobs=jobs)
parts = parts_of('@1 says "hi". @2 says "hello".', seconds=6, jobs=jobs)
self.assertIn("<Subject 1> and <Subject 2> speak", parts["summary"])
def test_keep_going_uses_the_person_picture_as_the_main_person(self):
parts = parts_of("He walks.", seconds=6, jobs={1: "person"}, video="keep going")
text = join_sections(parts)
self.assertNotIn("<Subject 2>", text)
self.assertNotIn("main subject in <Video 1>", text)
self.assertIn("<Subject 1> walks", parts["detailed_description"])
placed = parts_of("He walks in @2.", seconds=6, jobs={1: "person", 2: "place"}, video="keep going")
self.assertNotIn("same place", join_sections(placed))
empty = parts_of("The room is empty.", seconds=6, jobs={}, video="keep going")
self.assertNotIn("is in the shot", join_sections(empty))
self.assertNotIn("main subject", join_sections(empty))
def test_an_outfit_with_no_person_goes_on_the_person_in_the_shot(self):
parts = parts_of("He wears @1.", seconds=6, jobs={1: "outfit"})
self.assertIn("put only this outfit on the person in the shot", parts["retention_analysis"])
self.assertNotIn("on <Subject 1>", parts["retention_analysis"])
def test_in_the_shot_is_only_for_defined_people_and_things(self):
for jobs in ({1: "style"}, {1: "end on it"}, {1: "outfit"}):
self.assertNotIn("is in the shot", write_prompt("A calm lake.", seconds=6, jobs=jobs), jobs)
self.assertIn("<Subject 1> is in the shot", write_prompt("A calm lake.", seconds=6, jobs={1: "thing"}))
def test_new_voice_keeps_the_clips_person(self):
with self.assertRaisesRegex(ValueError, "swap the person"):
write_prompt('She says "hi".', seconds=6, jobs={1: "person"}, video="new voice")
text = write_prompt('She says "hi".', seconds=6, jobs={}, video="new voice")
self.assertNotIn("faces the camera", text)
def test_he_or_she_binds_the_only_person_picture(self):
parts = parts_of("She walks in @2.", seconds=6, jobs={1: "person", 2: "place"})
self.assertIn("<Subject 1> walks in <Subject 2>", parts["detailed_description"])
with self.assertRaisesRegex(ValueError, "no line names @1"):
write_prompt("A dog sits in @2.", seconds=6, jobs={1: "person", 2: "place"})
def test_a_swap_with_a_thing_or_place_gets_its_own_advice(self):
with self.assertRaisesRegex(ValueError, "He holds @1"):
write_prompt("He holds @1.", seconds=6, jobs={1: "thing"}, video="swap the person")
with self.assertRaisesRegex(ValueError, "keeps the place"):
write_prompt("He dances.", seconds=6, jobs={1: "place"}, video="swap the person")
def test_the_task_tag_adds_audio_reuse_when_the_clips_sound_stays(self):
edit = parts_of("She wears a red coat.", seconds=6, jobs={}, video="change something")["summary"]
self.assertTrue(edit.startswith("[video editing + audio reuse]"))
kept = parts_of("", seconds=6, jobs={1: "person"}, video="swap the person", clip_sound=True)["summary"]
self.assertTrue(kept.startswith("[video editing + audio reuse]"))
plain = parts_of("", seconds=6, jobs={1: "person"}, video="swap the person")["summary"]
self.assertNotIn("audio reuse", plain)
def test_only_a_place_or_start_picture_sets_the_look(self):
for job in ("outfit", "thing"):
self.assertNotIn("film grain", write_prompt("He wears @1.", seconds=6, jobs={1: job}), job)
self.assertIn("film grain of <Picture 1>", write_prompt("He sits in @1.", seconds=6, jobs={1: "place"}))
def test_change_something_puts_an_unnamed_outfit_on_the_video_person(self):
parts = parts_of("Add a hat.", seconds=6, jobs={1: "outfit"}, video="change something")
self.assertIn("<Subject 2> wears <Subject 1>.", parts["detailed_description"])
class SpeechAcrossCutTests(unittest.TestCase):
CARRIED = 'She looks out the window. She says "I never thought this would happen..."\n\nA close-up of the road ahead.'
def test_a_trailing_line_carries_across_the_cut(self):
first, second = shots(self.CARRIED, seconds=8)
self.assertTrue(first.endswith("<scenetrans>"))
self.assertNotIn("closes the lips", first)
self.assertIn("<scenetrans> The line from the previous shot continues", second)
self.assertNotIn("stays silent", second)
self.assertNotIn("<Video 1>", second)
self.assertEqual(write_prompt(self.CARRIED, seconds=8).count("I never thought"), 1)
def test_a_trailing_last_line_is_cut_off(self):
(only,) = shots('She says "Wait, I was going to tell you that..."', seconds=4)
self.assertTrue(only.endswith("<cutoff>"))
self.assertNotIn("closes the lips", only)
def test_a_carried_line_is_not_warned_about(self):
long = 'She says "this is a very long line that would never fit inside two short seconds of video"\n\nA close-up of the road.'
self.assertEqual(script_warning(read_script(long), 4), "")
self.assertIn("<scenetrans>", write_prompt(long, seconds=4))
def test_plain_lines_carry_nothing(self):
text = write_prompt('She says "hello."\n\nHe waves back.', seconds=6)
self.assertNotIn("<scenetrans>", text)
self.assertNotIn("<cutoff>", text)
def test_four_seconds_is_allowed(self):
seconds = AusBossH3PromptWriter.INPUT_TYPES()["optional"].get("seconds") or AusBossH3PromptWriter.INPUT_TYPES()["required"]["seconds"]
self.assertEqual(seconds[1]["min"], 4)
if __name__ == "__main__":
unittest.main()
+8 -2
View File
@@ -127,7 +127,7 @@ class ListingTests(FolderCase):
def test_no_folder_name_leads_out_of_the_root(self):
for bad in ("..", "../private", "people/../../private", "/etc", "~/Pictures", "C:/Windows", "//server/share",
"people\\..\\..\\private"):
"people\\..\\..\\private", "people/C:/Windows", "people/\x00/x"):
with self.assertRaises(ValueError, msg=bad) as refused:
safe_folder(self.root, bad)
self.assertEqual(str(refused.exception), OUTSIDE, bad)
@@ -381,7 +381,13 @@ class RouteTests(FolderCase):
("thumb", {"name": "loop/picture.png"})]:
with self.subTest(route=route):
response = self.request(route, **query)
self.assertEqual(response.status, 400)
# A looped link is a bad folder to the lists. The thumbnail
# answers 400 or 404 depending on whether this Python's
# resolve() raises on the loop; neither answer names the path.
if route == "thumb":
self.assertIn(response.status, (400, 404))
else:
self.assertEqual(response.status, 400)
self.assertNotIn(str(self.root), json.loads(response.text)["error"])
def test_thumbnail_versions_change_with_subsecond_rewrites_and_size(self):
+151
View File
@@ -1148,6 +1148,157 @@ class CanvasStitcherBboxTests(unittest.TestCase):
class ToneMatchInpaintTests(unittest.TestCase):
"""color_match on a Crop For Inpaint crop: the drift is read inside the
painted area, so it holds when the sampler used its own, wider noise
mask and left the soft edge close to the original (issue #361)."""
def _case(self, repaint):
generator = torch.Generator().manual_seed(4)
rows = torch.linspace(0.0, 1.0, 320).view(1, 320, 1, 1)
image = torch.tensor([0.72, 0.55, 0.45]).view(1, 1, 1, 3) - 0.15 * rows
image = (image + 0.01 * torch.randn((1, 320, 320, 3), generator=generator)).clamp(0.0, 1.0)
mask = torch.zeros((1, 320, 320))
mask[:, 110:210, 110:210] = 1.0
crop, sampling, stitcher = build_crop(image, mask, 2.0, 16, 16)
# The model drifts where it paints, but the workflow's noise mask and
# the model itself keep the soft edge on the original: the drift
# fades out before the paste's feathered band begins.
painted = inpaint_helpers.blur_mask(sampling, 3.0).unsqueeze(-1)
patch = crop + painted * (repaint(crop) - crop)
cx, cy, cw, ch = stitcher["crop_to_canvas"]
ox, oy, ow, oh = stitcher["canvas_to_original"]
inside = stitcher["blend"][0, oy : oy + oh, ox : ox + ow] > 0.999
return image, patch, stitcher, inside
def _left_over(self, image, stitched, inside):
from nodes._color_helpers import rgb_to_lab
return float((rgb_to_lab(stitched) - rgb_to_lab(image))[0][inside].median(dim=0).values.norm())
def test_a_washed_out_repaint_comes_back_to_the_picture(self):
image, patch, stitcher, inside = self._case(lambda crop: crop * 0.85 + 0.11)
off = self._left_over(image, apply_stitch(stitcher, patch, color_match=0.0), inside)
half = self._left_over(image, apply_stitch(stitcher, patch, color_match=0.5), inside)
full = self._left_over(image, apply_stitch(stitcher, patch, color_match=1.0), inside)
self.assertGreater(off, 3.0)
self.assertLess(half, 0.65 * off)
self.assertLess(full, 0.25 * off)
def test_the_band_alone_read_a_fraction_of_that_drift(self):
# What the slider had to work with before: the regression in numbers.
image, patch, stitcher, inside = self._case(lambda crop: crop * 0.85 + 0.11)
cx, cy, cw, ch = stitcher["crop_to_canvas"]
region = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
alpha = stitcher["blend"][:, cy : cy + ch, cx : cx + cw]
band = inpaint_helpers.estimate_tone_offset(patch, region, alpha)
inner, held = inpaint_helpers.inner_tone_offset(patch, region, alpha)
self.assertTrue(bool(held[0]))
self.assertLess(float(band.norm()), 0.5 * float(inner.norm()))
def test_new_content_is_not_pulled_toward_the_old_colour(self):
# A black jacket over skin is a change, not a drift: the reading
# holds at zero, so the band is not asked and nothing is shifted.
image, patch, stitcher, inside = self._case(lambda crop: torch.full_like(crop, 0.05))
cx, cy, cw, ch = stitcher["crop_to_canvas"]
region = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
alpha = stitcher["blend"][:, cy : cy + ch, cx : cx + cw]
inner, held = inpaint_helpers.inner_tone_offset(patch, region, alpha)
self.assertTrue(bool(held[0]))
self.assertEqual(float(inner.abs().max()), 0.0)
stitched = apply_stitch(stitcher, patch, color_match=1.0)
self.assertLess(float(stitched[0][inside].mean()), 0.12)
def test_new_content_painted_to_the_edge_is_left_as_painted(self):
# A model with no noise mask repaints the soft edge too. The band
# then reads the new colour as drift; Tone match must not act on it.
def repaint(crop):
return torch.tensor([0.85, 0.7, 0.1]).view(1, 1, 1, 3).expand_as(crop).clone()
image, _patch, stitcher, _inside = self._case(repaint)
cx, cy, cw, ch = stitcher["crop_to_canvas"]
crop = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
patch = repaint(crop)
plain = apply_stitch(stitcher, patch, color_match=0.0)
matched = apply_stitch(stitcher, patch, color_match=1.0)
self.assertLess(float((matched - plain).abs().max()), 1.0 / 255)
def _shirt_case(self, paint):
"""Skin with a dark shirt on it. The mask covers the shirt with a
thin margin, and the model paints skin there, so the painted area
has almost no "before" of its own to be read against."""
generator = torch.Generator().manual_seed(7)
skin = torch.tensor([0.72, 0.55, 0.45]).view(1, 1, 1, 3)
rows = torch.linspace(0.0, 1.0, 320).view(1, 320, 1, 1)
clean = (skin - 0.1 * rows + 0.01 * torch.randn((1, 320, 320, 3), generator=generator)).clamp(0.0, 1.0)
image = clean.clone()
image[:, 104:216, 104:216] = 0.06
mask = torch.zeros((1, 320, 320))
mask[:, 100:220, 100:220] = 1.0
crop, sampling, stitcher = build_crop(image, mask, 2.0, 16, 16)
cx, cy, cw, ch = stitcher["crop_to_canvas"]
ox, oy, ow, oh = stitcher["canvas_to_original"]
wanted = clean[:, oy : oy + oh, ox : ox + ow][:, cy : cy + ch, cx : cx + cw]
painted = inpaint_helpers.blur_mask(sampling, 3.0).unsqueeze(-1)
patch = crop + painted * (paint(wanted) - crop)
inside = stitcher["blend"][0, oy : oy + oh, ox : ox + ow] > 0.999
return clean, patch, stitcher, inside
def test_new_skin_is_matched_to_the_skin_beside_it(self):
# Skin where a shirt was, painted paler than the skin around it:
# the area's own "before" is the shirt, so it is read against the
# untouched skin just outside the paste.
clean, patch, stitcher, inside = self._shirt_case(lambda skin: skin * 0.85 + 0.13)
off = self._left_over(clean, apply_stitch(stitcher, patch, color_match=0.0), inside)
full = self._left_over(clean, apply_stitch(stitcher, patch, color_match=1.0), inside)
self.assertGreater(off, 3.0)
self.assertLess(full, 0.35 * off)
def test_new_skin_that_already_matches_is_left_alone(self):
clean, patch, stitcher, inside = self._shirt_case(lambda skin: skin)
plain = apply_stitch(stitcher, patch, color_match=0.0)
matched = apply_stitch(stitcher, patch, color_match=1.0)
self.assertLess(float((matched - plain).abs().max()), 2.0 / 255)
def test_a_mixed_rim_is_not_read_as_drift(self):
# Half the painted area turned into something else: too little of
# the rim agrees, so nothing inside is trusted.
def repaint(crop):
out = crop.clone()
out[:, :, : crop.shape[2] // 2] = 0.05
return out
image, patch, stitcher, inside = self._case(repaint)
cx, cy, cw, ch = stitcher["crop_to_canvas"]
region = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
alpha = stitcher["blend"][:, cy : cy + ch, cx : cx + cw]
offset, held = inpaint_helpers.inner_tone_offset(patch, region, alpha)
self.assertLess(float(offset.norm()), 1.0)
def test_an_unchanged_crop_reads_no_drift(self):
image, _patch, stitcher, _inside = self._case(lambda crop: crop)
cx, cy, cw, ch = stitcher["crop_to_canvas"]
crop = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
stitched = apply_stitch(stitcher, crop, color_match=1.0)
# Nothing to undo; what is left is the colour-space round trip.
self.assertLess(float((stitched - image).abs().max()), 0.5 / 255)
def test_a_large_picture_is_read_at_the_size_the_model_painted(self):
_image, patch, stitcher, _inside = self._case(lambda crop: crop * 0.85 + 0.11)
cx, cy, cw, ch = stitcher["crop_to_canvas"]
canvas = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
blend = stitcher["blend"][:, cy : cy + ch, cx : cx + cw]
small, small_held = inpaint_helpers.inner_tone_offset(patch, canvas, blend)
def grow(image):
return torch.nn.functional.interpolate(image.movedim(-1, 1), scale_factor=4, mode="bilinear").movedim(1, -1)
big, big_held = inpaint_helpers.inner_tone_offset(
grow(patch), grow(canvas), grow(blend.unsqueeze(-1)).squeeze(-1), sampled=(cw, ch)
)
self.assertTrue(bool(small_held[0]) and bool(big_held[0]))
# Pasted back four times larger, the reading is the same one.
self.assertLess(float((big - small).norm()), 0.3)
class ToneMatchTests(unittest.TestCase):
"""color_match: the drift is read off the feathered band and undone."""
+18 -1
View File
@@ -108,6 +108,15 @@ class _FolderPaths:
def get_folder_paths(self, kind: str) -> list[str]:
return [str(self.lora_root)] if kind == "loras" else []
def get_filename_list(self, kind: str) -> list[str]:
if kind != "loras":
return []
return [
path.relative_to(self.lora_root).as_posix()
for path in self.lora_root.rglob("*")
if path.is_file() and path.suffix == ".safetensors"
]
def get_user_directory(self) -> str:
return str(self.user_root)
@@ -605,7 +614,15 @@ class ResolveLoraPathTests(unittest.TestCase):
def test_path_outside_the_folders_is_refused(self):
self.full_path = lambda kind, name: str(self.root / ".." / "other_drive" / "real.safetensors")
with self.assertRaisesRegex(ValueError, "escapes the loras folders"):
_lora_helpers.resolve_lora_path("../other_drive/real.safetensors")
_lora_helpers.resolve_lora_path("plain.safetensors")
def test_a_name_off_the_list_never_reaches_the_disk(self):
asked = []
self.full_path = lambda kind, name: asked.append(name)
for name in ("../other_drive/real.safetensors", str(self.outside / "real.safetensors")):
with self.assertRaisesRegex(ValueError, "not found"):
_lora_helpers.resolve_lora_path(name)
self.assertEqual(asked, [])
if __name__ == "__main__":
+7 -1
View File
@@ -61,7 +61,7 @@ from nodes._media_helpers import ( # noqa: E402
resolve_server_file,
resolve_video_path,
)
from nodes._route_errors import safe_route_error # noqa: E402
from nodes._route_errors import error_detail, safe_route_error # noqa: E402
NETWORK_REFUSAL = "Network source paths are not allowed."
WHOLE_PATH_REFUSAL = "A whole path is not read"
@@ -425,6 +425,12 @@ class RouteErrorTests(unittest.TestCase):
with self.subTest(text=text):
self.assertEqual(safe_route_error(ValueError(text)), text)
def test_a_node_error_keeps_its_words_unless_they_name_a_path(self):
self.assertEqual(error_detail(ValueError("bad shape (3, 4)")), "ValueError: bad shape (3, 4)")
for text in ("cannot open /home/me/models/x.pt", "Error in `C:\\Comfy\\models\\sam3.pt`", "~/loras/a.safetensors"):
with self.subTest(text=text):
self.assertEqual(error_detail(OSError(text)), "OSError")
def test_no_route_sends_a_file_by_its_path(self):
for path in sorted((ROOT / "nodes").glob("*.py")):
with self.subTest(module=path.name):
+14 -1
View File
@@ -9,6 +9,13 @@ image and blend_mask with the output the 2.3.0 code gave.
The legacy inpaint crop keeps its original geometry here; its expected
mask is limited to the paste window. Crop sizing is tested separately.
A Crop For Inpaint crop with Tone match on is the one case no longer held to
2.3.0: that release read the drift in the soft edge, which read a fraction of
it when the workflow samples with a wider noise mask, so Tone match hardly
did anything (issue #361). It is read inside the painted area now, and
ToneMatchInpaintTests in test_inpaint_crop_helpers.py holds that. Its Tone
match 0 stitch and its blend mask stay pinned.
Blend in with Tone match at 0 is pinned the same way, to the 2.4.0 code:
the published Qwen Image 2.1 Rotate + Outpaint workflow stitches that way.
@@ -78,6 +85,11 @@ PINS = {
}
# Tone match on an inpaint crop is read inside the painted area since issue
# #361, so these two no longer give the 2.3.0 picture. See the docstring.
REREAD = {("crop for inpaint", 0.5), ("crop for inpaint", 1.0)}
def picture(width: int, height: int, seed: int) -> Image.Image:
"""Smooth colour structure plus fine grain, as 8-bit RGB."""
y, x = np.mgrid[0:height, 0:width].astype(np.float32)
@@ -207,7 +219,8 @@ class ClassicSeamPinTests(unittest.TestCase):
with self.subTest(case=name, color_match=strength):
stitcher, patch_image = build()
image, blend_mask = stitch(stitcher, patch_image, strength)
assert_released_image(image, reference_image(name, strength))
if (name, strength) not in REREAD:
assert_released_image(image, reference_image(name, strength))
expected_mask = reference_mask(name)
if name == "crop for inpaint":
cx, cy, cw, ch = stitcher["crop_to_canvas"]
+79
View File
@@ -0,0 +1,79 @@
import assert from "node:assert/strict";
import test from "node:test";
import { readFileSync } from "node:fs";
import { COPY_EDITS_SETTING, copyPicture, transformPicture } from "../js/shared/transform_picture.mjs";
import { mergeSettings } from "../js/shared/node_settings.mjs";
function render(values = {}, resize = null, includeEdits = false) {
const calls = [];
const context = new Proxy({}, { get: (target, name) => name in target ? target[name] : (...args) => calls.push([name, ...args]) });
const canvas = { getContext: () => context };
const result = transformPicture({
sourceWidth: 100, sourceHeight: 60,
values: { canvas_multiple: 1, ...values }, resize, includeEdits, fill: "#123456",
drawSource: (ctx) => ctx.drawImage("source", -50, -30, 100, 60),
createCanvas: () => canvas,
});
return { result, calls, context };
}
test("default copy uses the original picture, ignoring crop, rotation, padding and resize", () => {
const { result, calls } = render({ crop_x: 10, crop_width: 40, rotation_degrees: 90, pad_left: 80, pad_top: 20, canvas_multiple: 64 }, { megapixels: 1, steps: 8 });
assert.deepEqual([result.width, result.height], [100, 60]);
assert.deepEqual(calls, [
["translate", 50, 30], ["drawImage", "source", -50, -30, 100, 60],
]);
});
test("the gear preference defaults off for existing browsers and accepts an explicit opt-in", () => {
assert.equal(mergeSettings([COPY_EDITS_SETTING], {}).copy_edits, false);
assert.equal(mergeSettings([COPY_EDITS_SETTING], { show_mask: true }).copy_edits, false);
assert.equal(mergeSettings([COPY_EDITS_SETTING], { copy_edits: "true" }).copy_edits, false);
assert.equal(mergeSettings([COPY_EDITS_SETTING], { copy_edits: true }).copy_edits, true);
});
test("opting in includes crop, padding and alignment in the copied bounds and source offset", () => {
const { result, calls } = render({ crop_x: 10, crop_y: 5, crop_width: 40, crop_height: 30, pad_left: 8, pad_top: 4, pad_right: 8, pad_bottom: 6, canvas_multiple: 8 }, null, true);
assert.deepEqual([result.width, result.height], [56, 40]);
assert.ok(calls.some((c) => JSON.stringify(c) === JSON.stringify(["rect", 8, 4, 40, 30])));
assert.ok(calls.some((c) => JSON.stringify(c) === JSON.stringify(["translate", 48, 29])));
});
test("opting in applies rotation and resize without copying a screen-sized preview", () => {
const { result, calls } = render({ rotation_degrees: 90 }, { megapixels: 0.25, steps: 8 }, true);
assert.deepEqual([result.width, result.height], [400, 664]);
assert.ok(calls.some((c) => c[0] === "rotate" && c[1] === Math.PI / 2));
assert.ok(calls.some((c) => c[0] === "scale" && c[1] === 400 / 60 && c[2] === 664 / 100));
});
test("clipboard write starts before PNG encoding finishes and contains only the clean PNG", async () => {
let finish, item, written = false;
const canvas = { toBlob: (callback, type) => { assert.equal(type, "image/png"); finish = callback; } };
class ClipboardItem { constructor(data) { this.data = data; } }
const clipboard = { write: async (items) => { written = true; [item] = items; await item.data["image/png"]; } };
const pending = copyPicture(canvas, { clipboard, ClipboardItem });
assert.equal(written, true);
assert.deepEqual(Object.keys(item.data), ["image/png"]);
await Promise.resolve(); await Promise.resolve();
const png = new Blob(["picture"], { type: "image/png" });
finish(png);
await pending;
assert.equal(await item.data["image/png"], png);
});
test("clipboard and encoding failures are reported instead of false success", async () => {
assert.throws(() => copyPicture({}, { clipboard: null, ClipboardItem: null }), /Open image/);
class ClipboardItem { constructor(data) { this.data = data; } }
const canvas = { toBlob: (callback) => callback(null) };
const clipboard = { write: async ([item]) => await item.data["image/png"] };
await assert.rejects(copyPicture(canvas, { clipboard, ClipboardItem }), /could not make/);
await assert.rejects(copyPicture({ toBlob: () => {} }, { clipboard: { write: async () => { throw new Error("Denied"); } }, ClipboardItem }), /Denied/);
});
test("both image editing surfaces replace native canvas copying, video is unchanged", () => {
const source = readFileSync(new URL("../js/shared/transform_editor.mjs", import.meta.url), "utf8");
assert.match(source, /attachStageHandlers\(state, preview, state.panelAbort.signal\)/);
assert.match(source, /attachStageHandlers\(state, canvas, abort.signal\)/);
assert.match(source, /if \(state.kind === "image"\) \{\s*canvas.addEventListener\("contextmenu"/);
assert.match(source, /copyPicture\(picture\)/);
});