Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
51993c0a2c | ||
|
|
b043f6f1a7 | ||
|
|
29c986cfdc | ||
|
|
171ad422b9 | ||
|
|
f4061f4255 | ||
|
|
8d139cdfc5 | ||
|
|
fae68ecb77 | ||
|
|
661c33a926 | ||
|
|
9b27bc9c4c | ||
|
|
de530e2eac | ||
|
|
7d81a43133 | ||
|
|
0067575b0c | ||
|
|
8cc90794d8 | ||
|
|
d82ed510b9 | ||
|
|
3f06362f9c | ||
|
|
d37e0cd026 | ||
|
|
38bdddded8 | ||
|
|
f0f6ca9f0c | ||
|
|
d47a6743e3 | ||
|
|
ab8389d598 | ||
|
|
5b10b3a23f | ||
|
|
f2d966afd1 | ||
|
|
182abca56f | ||
|
|
c1c9831378 | ||
|
|
6418940761 | ||
|
|
72af858975 | ||
|
|
80e5221403 | ||
|
|
f658ac58b0 | ||
|
|
adb1eaacc4 | ||
|
|
b0f8b5ec55 | ||
|
|
a3984bf656 | ||
|
|
232e1dcdb7 | ||
|
|
ab67a1f77f | ||
|
|
2af3fd9c83 | ||
|
|
9986ded109 | ||
|
|
197ac04a8c | ||
|
|
be2a2a1444 | ||
|
|
76eb4fc7e7 | ||
|
|
057f2bd05a | ||
|
|
172252ab02 | ||
|
|
ebc81ea57e | ||
|
|
bc235e96e5 | ||
|
|
6572b49386 |
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"attribution": {
|
||||
"commit": "",
|
||||
"pr": ""
|
||||
}
|
||||
}
|
||||
@@ -24,6 +24,10 @@ SECURITY.md
|
||||
.claude/
|
||||
/docs/
|
||||
|
||||
# the skill an AI agent installs from the repository (npx skills add);
|
||||
# nothing in the pack loads it
|
||||
/skills/
|
||||
|
||||
# README media: GitHub shows it on the repository page, nothing in the
|
||||
# pack loads it
|
||||
/assets/
|
||||
|
||||
+134
@@ -2,6 +2,140 @@
|
||||
|
||||
All notable changes to ComfyUI-AusBoss are documented here.
|
||||
|
||||
## 2.26.3 - 2026-11-17
|
||||
|
||||
- **Tone match now works on inpaints.** On a Crop For Inpaint 🆎 crop,
|
||||
Stitch Inpaint 🆎 read the colour shift in the soft edge around the
|
||||
painted area. Workflows such as Krea 2 Inpaint Masked let the model paint
|
||||
a little wider than that edge, so the edge came back close to your picture
|
||||
and the slider hardly did anything: in a test on skin it took off a
|
||||
third of the shift at 1. It is now read just inside the painted area and
|
||||
takes off more than half, on a picture of any size. A workflow that already had Tone match on for
|
||||
an inpaint gives a slightly different picture, closer to the original's
|
||||
colour. When the painted area holds something new, it is matched to your
|
||||
picture right beside it where the two look alike, such as new skin where
|
||||
a shirt was. Something different from what is beside it, such as a
|
||||
swimsuit over skin, is left as painted; before, Tone match could tint the
|
||||
whole patch with the opposite colour. Outpaints, blend in and Tone match 0
|
||||
are unchanged.
|
||||
|
||||
- **Copy a crop node's picture without its handles.** Right-click the picture
|
||||
on Image Crop + Rotate + Pad and choose Copy image. By default it copies
|
||||
the original picture. To include crop, rotation, padding and resize, turn
|
||||
on Copy with crop, rotation and padding in the gear menu. Editing controls
|
||||
and guides are never copied. Open image provides a clean
|
||||
picture to copy with the browser on plain HTTP addresses too.
|
||||
|
||||
- **H3 Prompt Writer: a swap reads the new person's looks however you type
|
||||
them.** `@1 looks like ...` on the same line as the swap sentence, and
|
||||
clothes on a line of their own (`@1 is wearing a black suit.`), were
|
||||
left out of the prompt without a word, and the clip often came back
|
||||
unchanged. Both are now used.
|
||||
|
||||
- **H3 Prompt Writer no longer drops lines.** When one line held two
|
||||
people talking, like `The phone says: "Get a job." @1 looks up and says:
|
||||
"What?"`, only the first words reached the prompt. Each quote is its own
|
||||
moment now. A swap's `@1 looks like` line also keeps only the looks: a
|
||||
sentence after it that names an `@` again, like `The trees behind @1 stay
|
||||
the same.`, is read as its own line instead of going into the description
|
||||
with a raw `@1`. `"Hi," she says` is spoken as "Hi." instead of "Hi,.",
|
||||
and a Chinese or Japanese line keeps its own full stop: "。", not "。.".
|
||||
|
||||
- **H3 Prompt Writer can carry a line across a cut.** End the words with
|
||||
`...` and the person keeps talking into the next shot, without saying the
|
||||
words twice. On the last line, `...` lets the clip end mid-sentence. A line
|
||||
too long for its shot carries on the same way instead of a warning.
|
||||
|
||||
- **H3 Prompt Writer makes 4-second clips.** The shortest length went from
|
||||
5 to 4 seconds.
|
||||
|
||||
- **H3 Prompt Writer can put the person from your video somewhere new.** With
|
||||
Copy its moves and no person picture, the person in the video keeps their
|
||||
own look and moves. Give a picture the job place and type `He dances in @1.`
|
||||
Before, the node asked for a person picture and stopped. Copy its moves
|
||||
also takes its length from the video now, like Swap, Change something and
|
||||
New voice.
|
||||
|
||||
- **H3 Prompt Writer can restyle a video from a picture.** With Change
|
||||
something and a style or place picture, the look comes from the picture
|
||||
instead of the video. Type `It looks like @1.` A clip with nobody in it,
|
||||
like an empty hallway, no longer gets a made-up person in its prompt.
|
||||
|
||||
- **H3 Prompt Writer can make a thing do the moves.** With Copy its moves or
|
||||
Swap the person and no person picture, a thing picture your lines make do
|
||||
the action (a cartoon hand, a toy) takes the moves in its own look. A thing
|
||||
that is held or worn stays a prop.
|
||||
|
||||
- **H3 Prompt Writer's job lists take a click again.** Picking a job from a
|
||||
chip's list closed the list without changing the job.
|
||||
|
||||
- **H3 Prompt Writer's job lists match the node.** The pick list on each
|
||||
picture and on the video opens under its chip in the node's own colours
|
||||
and grows with the zoom. Before, it was the browser's small plain list.
|
||||
|
||||
- **An AI agent can now learn the video workflows from the repository.**
|
||||
`npx skills add ausboss/ComfyUI-AusBoss` gives your agent a skill for the
|
||||
MiniMax H3 Pika Person Swap and Pika Remix workflows: how to write the
|
||||
words, which model files to get, and how to run a workflow in your
|
||||
ComfyUI. The same page is at https://ausboss-examples.vercel.app/skill.md.
|
||||
|
||||
- **The "AusBoss was updated" message no longer asks for a reload by
|
||||
mistake.** After the Manager switched the pack to another version, a
|
||||
page reload before ComfyUI restarted brought up the message, telling you
|
||||
the page was older than the pack when it was newer. A restart was what
|
||||
was needed. Now the message only comes up when the pack really is newer
|
||||
than the page, and versions such as 2.25.10 and 2.25.9 are compared as
|
||||
numbers.
|
||||
|
||||
- **LoRA routes only open files on ComfyUI's LoRA list.** The LoRA
|
||||
Loader's info, hash and preview routes used to hand the name they were
|
||||
given to ComfyUI's file lookup and check the answer afterwards. Now the
|
||||
name is matched against the list first, and a name that is not on it is
|
||||
refused before anything on disk is looked up.
|
||||
- **Image Folder 🆎 checks a folder or picture name as text before the
|
||||
disk.** A drive letter in the middle of a name, or a control character,
|
||||
is refused like `..`, and the joined path is checked before it is
|
||||
resolved as well as after. The list no longer sends each picture's size
|
||||
and time; the thumbnail's cache tag is a short hash instead.
|
||||
- **Error messages no longer pass on server paths.** When a model,
|
||||
decoder or encoder fails, its message can name a file on the server's
|
||||
disk. SAM 3 masks, LoRA loading and video saving now keep the kind of
|
||||
error and drop its words when they name a path, since node errors reach
|
||||
the browser and stay in the history.
|
||||
|
||||
## 2.26.2 - 2026-11-14
|
||||
|
||||
- **Loaders never look up a file name they refused.** Before each run,
|
||||
Load Image 🆎, Load Image + Pad 🆎, Image Crop + Rotate + Pad 🆎 and both
|
||||
Video Crop + Rotate + Pad nodes check whether their file changed. When the
|
||||
name was one they do not accept, such as a whole path or a network share,
|
||||
that check still looked it up on disk. On Windows, looking up a share
|
||||
contacts its server and hands it the computer's login. Now a refused name
|
||||
is only compared as text.
|
||||
- **The pack's scripts are found without looking up the address.** The pack
|
||||
tags its scripts with a version as the browser asks for them. It found
|
||||
each one by joining the address the browser sent to its own folder and
|
||||
looking the result up on disk, and only then checked that the file was
|
||||
its own. Now it looks the address up in a list of its scripts made when
|
||||
ComfyUI starts, so no address reaches the disk.
|
||||
- **Save Image's Save button copies only pictures it holds.** With Auto
|
||||
save off, the list of a run's held pictures was a file in ComfyUI's temp
|
||||
folder. That folder takes uploads, so a planted list could copy another
|
||||
file on the server into the output folder. The list now stays in
|
||||
ComfyUI's memory, and a held picture is found only by its run, its number
|
||||
and one of Save Image's formats. The Save button works for the last 256
|
||||
runs, up to 10,000 pictures each; held pictures still go when ComfyUI
|
||||
restarts, as before.
|
||||
|
||||
## 2.26.1 - 2026-11-14
|
||||
|
||||
- **A path that starts with `~` is no longer expanded.** Image Folder used
|
||||
to turn a pasted `~/...` path into a full one before checking that it sits
|
||||
inside ComfyUI's folders; now only a path written out in full is checked.
|
||||
LaMa Inpaint only opens a model from ComfyUI's `models/lama` list. Nothing
|
||||
outside those folders could be read before either; this removes the code
|
||||
that looked as if it could.
|
||||
|
||||
## 2.26.0 - 2026-11-14
|
||||
|
||||
- **Missing LoRAs show in ComfyUI's missing-models list.** When a workflow
|
||||
|
||||
@@ -49,6 +49,16 @@ After an update, through Manager or `git pull`, restart ComfyUI and hard-refresh
|
||||
|
||||
The pack uses the Pillow, NumPy, Torch and PyAV that come with ComfyUI. Each example lists the models it needs on its Workflow Note. LaMa Inpaint also needs [`big-lama.pt`](https://github.com/Sanster/models/releases/download/add_big_lama/big-lama.pt) in `ComfyUI/models/lama/`. The pack never downloads weights by itself.
|
||||
|
||||
### Using an AI agent
|
||||
|
||||
Your agent (Claude Code, Codex, Cursor or another one) can learn the video workflows: how to write the words for them, set them up, and run them on your computer.
|
||||
|
||||
```bash
|
||||
npx skills add ausboss/ComfyUI-AusBoss
|
||||
```
|
||||
|
||||
Or tell it: "Read https://ausboss-examples.vercel.app/skill.md and help me use these workflows." The skill is [skills/ausboss-workflows/SKILL.md](skills/ausboss-workflows/SKILL.md). For picking and wiring single nodes, point it at the [agent guide](js/docs/agents/README.md).
|
||||
|
||||
### Upgrading from 1.x
|
||||
|
||||
2.0 removed **LM Studio Chat** (use **Text** for a fixed prompt) and Align Image's `offset_x` and `offset_y` outputs. Math Expression's `a`, `b` and `c` are sockets now instead of typed boxes, so put constants in the expression itself. 1.2 removed **Drop Shadow**, **Pad Image** (use **Load Image + Pad**) and **Frame Chooser**. Save Image writes only inside ComfyUI's output folder. The [changelog](CHANGELOG.md) has the details.
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="105" height="20" role="img" aria-label="release: v2.26.0">
|
||||
<title>release: v2.26.0</title>
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="105" height="20" role="img" aria-label="release: v2.26.3">
|
||||
<title>release: v2.26.3</title>
|
||||
<path fill="#555" d="M0 0h48v20H0z"/>
|
||||
<path fill="#00b4aa" d="M48 0h57v20H48z"/>
|
||||
<g fill="#fff" text-anchor="middle" font-family="Verdana,Geneva,DejaVu Sans,sans-serif" font-size="11">
|
||||
<text x="24" y="14">release</text>
|
||||
<text x="76.5" y="14">v2.26.0</text>
|
||||
<text x="76.5" y="14">v2.26.3</text>
|
||||
</g>
|
||||
</svg>
|
||||
|
||||
|
Before Width: | Height: | Size: 425 B After Width: | Height: | Size: 425 B |
+20
-1
@@ -49,6 +49,25 @@ function askForReload(installed) {
|
||||
}
|
||||
}
|
||||
|
||||
// "2.26.2" -> [2, 26, 2]. Anything else, such as "unknown", is null.
|
||||
function parseVersion(version) {
|
||||
const match = /^(\d+)\.(\d+)\.(\d+)$/.exec(typeof version === "string" ? version : "");
|
||||
return match ? match.slice(1).map(Number) : null;
|
||||
}
|
||||
|
||||
// Only a server newer than the page means this tab runs old files. The page
|
||||
// is newer when the Manager switched the pack on disk and the page was
|
||||
// reloaded before ComfyUI restarted, and then a restart is what is needed.
|
||||
// Two version numbers cannot tell that apart from a downgrade with the tab
|
||||
// left open, which needs a reload, so in that case the guard says nothing.
|
||||
function serverIsNewer(installed, running) {
|
||||
const server = parseVersion(installed);
|
||||
const page = parseVersion(running);
|
||||
if (!server || !page) return false;
|
||||
const step = server.findIndex((part, index) => part !== page[index]);
|
||||
return step >= 0 && server[step] > page[step];
|
||||
}
|
||||
|
||||
// The installed version this page already asked a reload for. A server that
|
||||
// drops and comes back several times must not stack the same message.
|
||||
let asked = null;
|
||||
@@ -58,7 +77,7 @@ async function check() {
|
||||
try {
|
||||
const installed = (await (await api.fetchApi("/ausboss/pack_version")).json())?.version;
|
||||
const running = await runningVersion();
|
||||
if (!installed || installed === "unknown" || !running || installed === running || installed === asked) return;
|
||||
if (!serverIsNewer(installed, running) || installed === asked) return;
|
||||
asked = installed;
|
||||
const listing = await (await api.fetchApi("/ausboss/pack_modules")).json();
|
||||
await refreshModules(Array.isArray(listing?.modules) ? listing.modules : []);
|
||||
|
||||
@@ -84,31 +84,42 @@ from.
|
||||
|
||||
Connect a video, then pick what it is for:
|
||||
|
||||
- **copy its moves**: your person does the moves in the video. It needs a
|
||||
picture of your person. Say where in your lines:
|
||||
`@1 dances on a rooftop at sunset.` The clothes can come from words or from an
|
||||
outfit picture.
|
||||
- **copy its moves**: someone does the moves in the video. With a picture of
|
||||
your person, your person does them. Say where in your lines:
|
||||
`@1 dances on a rooftop at sunset.` With no person picture, the person in
|
||||
the video keeps their own moves and goes somewhere new: give a picture the
|
||||
job **place** and type `He dances in @1.` The clothes can come from words or
|
||||
from an outfit picture. It doesn't have to be a person: give a cartoon hand,
|
||||
a toy or a mascot the job **thing** and type `@1 waves.` It does the moves in
|
||||
its own look, and nothing of the video's subject is kept.
|
||||
- **swap the person**: your person takes over from the one in the video. The
|
||||
place, the moves and the camera stay. With two person pictures, the first
|
||||
replaces the person on the left and the second replaces the person on the
|
||||
right. To choose who goes, type one line per picture, for example
|
||||
`@1 replaces the man in the hat`. Put spoken words in quotes and name
|
||||
the speaker. If a voice comes from off screen, say so in the line.
|
||||
A line like `@1 dances.` is not read as who goes; only a description such
|
||||
as `the man in the hat` is. With two people, name who speaks.
|
||||
- **change something**: keep the video and describe the change, for example
|
||||
`She wears a red raincoat`. Be specific about what should change.
|
||||
`She wears a red raincoat`. Be specific about what should change. To change
|
||||
the whole look, connect a picture with the job **style** (or **place**, for
|
||||
the same room looking different) and type `It looks like @1.` A clip with
|
||||
nobody in it, such as an empty hallway, stays empty.
|
||||
- **keep going**: more of the same clip, from where it ends. Say what happens
|
||||
next, or leave the box empty and it just carries on. Connect the whole clip:
|
||||
the node sends H3 only the last 2 seconds.
|
||||
- **new voice**: the same clip with a new voice. Connect the voice, and type
|
||||
what they say in quotes.
|
||||
This keeps the person in the clip, so a person picture is not allowed here;
|
||||
use **swap the person** to change who is in it.
|
||||
|
||||
If your swap keeps the video's audio, connect it to **video_sound** and turn on
|
||||
**More → Clip's own sound**. The writer uses the clip to estimate when each
|
||||
line is spoken and keeps only the incoming people's words in the swap prompt.
|
||||
Type a time such as `3.2s:` to set it yourself. This switch starts off.
|
||||
|
||||
With an active video, **swap the person**, **change something** and **new voice**
|
||||
use the incoming clip to set the frame count. **Length** says **From the clip**;
|
||||
With an active video, **copy its moves**, **swap the person**, **change
|
||||
something** and **new voice** use the incoming clip to set the frame count. **Length** says **From the clip**;
|
||||
set the clip's length upstream. The bar does not show a guessed duration or offer
|
||||
retiming controls for these jobs. H3 rounds the frame count to a supported length.
|
||||
Other video jobs, or turning the video off, use the editable **Length** again.
|
||||
@@ -137,6 +148,9 @@ workflows and scripts keep their previous behavior when it is absent.
|
||||
`ends on her sitting by the window`.
|
||||
- **Speech**: put the words in quotes: `@1 says "we are late."` Each person
|
||||
keeps their own voice.
|
||||
- **Talking over a cut**: end the words with `...` and the line keeps going
|
||||
into the next shot, said once. On the last line, `...` means the clip ends
|
||||
mid-sentence.
|
||||
- **Words on a sign**: name the sign: `a neon sign reads "OPEN"`.
|
||||
- **Sound and music**: use the **Sound** and **Music** boxes, or type
|
||||
`sound: rain on a tin roof` and `music: slow sad piano`. With no music you get
|
||||
@@ -195,7 +209,7 @@ as not plugged in.
|
||||
- **Sound** (`sound`): what the clip sounds like, for the whole clip. Same as a
|
||||
`sound:` line.
|
||||
- **Music** (`music`): the music for the whole clip. Same as a `music:` line.
|
||||
- **Length** (`seconds`): how long the clip is, 5 to 15.
|
||||
- **Length** (`seconds`): how long the clip is, 4 to 15.
|
||||
- **Camera** (`camera`): how the camera moves the whole way through.
|
||||
- **Look** (`look`): from the picture keeps your photo's look. The others redraw
|
||||
the clip. Clay and 3D don't work from words, so change your picture first with
|
||||
|
||||
@@ -24,6 +24,15 @@ pan. **Save & close** keeps your edits. **Cancel** or Escape puts everything
|
||||
back, and asks first if anything changed. You do not need the editor: a run
|
||||
uses what the node shows.
|
||||
|
||||
Right-click the picture and choose **Copy image** to copy the original picture,
|
||||
without added padding, rotation, crop, resize, handles or guides. To include
|
||||
your edits, open the gear and turn on **Copy with crop, rotation and padding**.
|
||||
That also includes resize, and is remembered for this browser. **Open image**
|
||||
follows the same choice. This works on the node and in the editor.
|
||||
Direct copying needs localhost or HTTPS. On a plain HTTP address, choose
|
||||
**Open image**, then right-click that
|
||||
picture and use the browser's **Copy image**.
|
||||
|
||||
## On the node
|
||||
|
||||
From top to bottom:
|
||||
|
||||
@@ -65,7 +65,7 @@ file's name. A caption on the `caption_text` input writes the paired `.txt`.
|
||||
are saved under the name and folder set above, in the format picked,
|
||||
with the workflow inside if **Embed workflow** is on. They wait in
|
||||
ComfyUI's temp folder, which is emptied when ComfyUI restarts, so save
|
||||
what you want before that.
|
||||
what you want before that. The Save button works for the last 256 runs.
|
||||
- **Preview steps** (`step_preview`, off by default): the switch left of
|
||||
**preview** shows the picture while it is being made, one step at a time,
|
||||
where the saved image goes, with a **STOP** button on the bar. Stop a run
|
||||
|
||||
@@ -155,9 +155,25 @@ correction turns the corner without a crease.
|
||||
The shift is subtracted from the pasted region before the blend. In the
|
||||
padding that is the whole shift; inside the feathered band the paste
|
||||
alpha scales it, which is exactly the share of each band pixel the
|
||||
sampler painted. Source pixels stay bit-identical. A crop stitcher, which
|
||||
has no source rectangle, gets a single global shift measured in its
|
||||
blend band instead.
|
||||
sampler painted. Source pixels stay bit-identical.
|
||||
|
||||
A Crop For Inpaint crop has no source rectangle, so it gets one shift for
|
||||
the whole painted area. That shift is read inside the painted area, in the
|
||||
32 pixels next to its edge: the same pixels before and after, taking the
|
||||
middle value. Those 32 pixels are counted at the size the model painted, so
|
||||
a large picture is read the same way and takes no longer. Most inpaint workflows let the model paint a little wider
|
||||
than the paste, so the soft edge itself comes back close to your picture
|
||||
and shows only a fraction of the drift. When the painted area holds
|
||||
something new (less than half of that rim agrees, or the change is larger
|
||||
than 20 LAB units, such as a black jacket over skin), it is not drift: the
|
||||
old pixels in that place show something else. The new part is then
|
||||
compared with your untouched picture right beside it, each side averaged
|
||||
within 48 pixels of the seam. Where the two sides look alike (within 20
|
||||
LAB units, on at least a quarter of the seam, such as new skin next to
|
||||
skin), the gap between them is taken off. Where they do not, such as a
|
||||
jacket next to skin, Tone match leaves that picture as painted. Before,
|
||||
the soft edge was read there instead, and a model that repaints the edge
|
||||
too could tint the whole patch: a yellow swimsuit over skin came back blue.
|
||||
|
||||
On a Crop + Rotate + Pad canvas the source rectangle is the crop, so it
|
||||
can also hold the rotation's empty corners and any transparency. The
|
||||
|
||||
+114
-11
@@ -4,6 +4,7 @@
|
||||
// source of truth: a chip only changes their value.
|
||||
import { app } from "../../../scripts/app.js";
|
||||
import { BRAND, chainCallback } from "./index.mjs";
|
||||
import { THEME as T, FONT_UI } from "./theme.mjs";
|
||||
import { liftSocket } from "./widget_card.mjs";
|
||||
import { commitWidgetValue, comboValues } from "./widget_card_math.mjs";
|
||||
|
||||
@@ -21,9 +22,15 @@ function ensureCss() {
|
||||
.ausboss-chip img{flex:none;width:18px;height:18px;margin-left:-5px;border-radius:9px;object-fit:cover}
|
||||
.ausboss-chip b{flex:none;color:${BRAND};font:700 10px ui-monospace,SFMono-Regular,Menlo,monospace}
|
||||
.ausboss-chip em{flex:none;color:#6f8886;font-style:normal;font-weight:500}
|
||||
.ausboss-chip select{field-sizing:content;min-width:0;max-width:120px;height:22px;padding:0 11px 0 0;border:none;background:transparent url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='8' height='5' viewBox='0 0 8 5'%3E%3Cpath d='M1 1l3 3 3-3' fill='none' stroke='%236f8886' stroke-width='1.4'/%3E%3C/svg%3E") no-repeat right 1px center;color:#d8ecea;font:inherit;appearance:none;-webkit-appearance:none;cursor:pointer;text-overflow:ellipsis}
|
||||
.ausboss-chip select:focus-visible{outline:2px solid ${BRAND};outline-offset:2px}
|
||||
.ausboss-chip select option{background:#0f1516;color:#d8ecea}
|
||||
.ausboss-chip-pick{min-width:0;max-width:120px;height:22px;padding:0 12px 0 0;border:none;background:transparent url("data:image/svg+xml,%3Csvg xmlns='http://www.w3.org/2000/svg' width='8' height='5' viewBox='0 0 8 5'%3E%3Cpath d='M1 1l3 3 3-3' fill='none' stroke='%236f8886' stroke-width='1.4'/%3E%3C/svg%3E") no-repeat right 1px center;color:#d8ecea;font:inherit;cursor:pointer;overflow:hidden;white-space:nowrap;text-overflow:ellipsis;text-align:left}
|
||||
.ausboss-chip-pick:focus-visible{outline:2px solid ${BRAND};outline-offset:2px;border-radius:3px}
|
||||
.ausboss-chip.open{border-color:${BRAND}}
|
||||
.ausboss-chip-menu{position:fixed;z-index:10000;box-sizing:border-box;padding:.33em;border:1px solid ${T.rule};border-radius:.66em;background:${T.card};box-shadow:0 10px 32px rgba(0,0,0,.55);font:500 12px/1.25 ${FONT_UI};color:${T.inkSoft}}
|
||||
.ausboss-chip-item{display:flex;align-items:center;gap:.5em;padding:.45em .75em .45em .5em;border-radius:.4em;white-space:nowrap;cursor:pointer;user-select:none}
|
||||
.ausboss-chip-item::before{content:"";flex:none;width:.9em;text-align:center;color:${T.brand}}
|
||||
.ausboss-chip-item.current::before{content:"\\2713"}
|
||||
.ausboss-chip-item.current{color:${T.brandInk}}
|
||||
.ausboss-chip-item.hot{background:${T.fill};color:${T.ink}}
|
||||
.ausboss-chip.off{border-style:dashed;background:transparent;color:#56706d}
|
||||
.ausboss-chip.off:hover{border-color:#2a3437}
|
||||
.ausboss-chip.wired{color:#8ba3a1}
|
||||
@@ -56,17 +63,109 @@ function add(parent, tag, text, className) {
|
||||
return child;
|
||||
}
|
||||
|
||||
// The open job list, one at a time across every chip on the page.
|
||||
let openMenu = null;
|
||||
|
||||
// The job list a chip opens. A native <select> lets the browser draw its own
|
||||
// list (small system font, light highlight, pinned to the select's left
|
||||
// edge); this one wears the node's colours, sits under the whole chip and
|
||||
// grows with the canvas zoom like the chip does.
|
||||
function openJobMenu(chipElement, button, values, current, choose) {
|
||||
openMenu?.close();
|
||||
const menu = add(document.body, "div", undefined, "ausboss-chip-menu");
|
||||
menu.setAttribute("role", "listbox");
|
||||
const items = values.map((value) => {
|
||||
const item = add(menu, "div", value, "ausboss-chip-item");
|
||||
item.setAttribute("role", "option");
|
||||
if (value === current) { item.classList.add("current"); item.setAttribute("aria-selected", "true"); }
|
||||
return item;
|
||||
});
|
||||
let hot = -1;
|
||||
const setHot = (index) => {
|
||||
items[hot]?.classList.remove("hot");
|
||||
hot = (index + items.length) % items.length;
|
||||
items[hot]?.classList.add("hot");
|
||||
items[hot]?.scrollIntoView({ block: "nearest" });
|
||||
};
|
||||
setHot(Math.max(0, values.indexOf(current)));
|
||||
|
||||
// The chip is drawn scaled with the canvas: the list matches that size,
|
||||
// starts at the chip's left edge and is at least as wide as the chip.
|
||||
function place() {
|
||||
const box = chipElement.getBoundingClientRect();
|
||||
const zoom = Math.min(2.5, Math.max(0.85, box.height / (chipElement.offsetHeight || 24)));
|
||||
menu.style.fontSize = `${12 * zoom}px`;
|
||||
menu.style.minWidth = `${box.width}px`;
|
||||
const width = menu.offsetWidth;
|
||||
const height = menu.offsetHeight;
|
||||
let top = box.bottom + 4;
|
||||
if (top + height > window.innerHeight - 8 && box.top - height - 4 >= 8) top = box.top - height - 4;
|
||||
menu.style.left = `${Math.max(8, Math.min(box.left, window.innerWidth - width - 8))}px`;
|
||||
menu.style.top = `${Math.max(8, top)}px`;
|
||||
}
|
||||
|
||||
const abort = new AbortController();
|
||||
const close = () => {
|
||||
abort.abort();
|
||||
menu.remove();
|
||||
chipElement.classList.remove("open");
|
||||
if (openMenu?.menu === menu) openMenu = null;
|
||||
};
|
||||
const pick = (index) => { close(); button.focus(); choose(values[index]); };
|
||||
items.forEach((item, index) => {
|
||||
item.addEventListener("pointermove", () => { if (hot !== index) setHot(index); });
|
||||
item.addEventListener("click", () => pick(index));
|
||||
});
|
||||
menu.addEventListener("pointerdown", (event) => event.stopPropagation());
|
||||
const outside = (event) => !menu.contains(event.target) && !button.contains(event.target);
|
||||
const on = { capture: true, signal: abort.signal };
|
||||
window.addEventListener("pointerdown", (event) => { if (outside(event)) close(); }, on);
|
||||
window.addEventListener("wheel", (event) => { if (outside(event)) close(); }, { ...on, passive: true });
|
||||
// The page fires resize on its own now and then: the list follows the chip
|
||||
// instead of vanishing, and goes only when the chip itself is gone.
|
||||
window.addEventListener("resize", () => (chipElement.isConnected ? place() : close()), on);
|
||||
// Only the window itself losing focus (another app, another tab). Pressing an item takes
|
||||
// focus off the chip's button, and a capturing blur listener would hear that and close
|
||||
// the list before the click lands. The press keeps focus on the button instead.
|
||||
window.addEventListener("blur", (event) => { if (event.target === window) close(); }, { signal: abort.signal });
|
||||
menu.addEventListener("mousedown", (event) => event.preventDefault());
|
||||
window.addEventListener("keydown", (event) => {
|
||||
const step = { ArrowDown: 1, ArrowUp: -1 }[event.key];
|
||||
if (step) setHot(hot + step);
|
||||
else if (event.key === "Home") setHot(0);
|
||||
else if (event.key === "End") setHot(items.length - 1);
|
||||
else if (event.key === "Enter" || event.key === " ") pick(hot);
|
||||
else if (event.key === "Escape" || event.key === "Tab") { close(); if (event.key === "Escape") button.focus(); }
|
||||
else return;
|
||||
if (event.key !== "Tab") { event.preventDefault(); event.stopPropagation(); }
|
||||
}, on);
|
||||
|
||||
place();
|
||||
chipElement.classList.add("open");
|
||||
openMenu = { owner: button, menu, close };
|
||||
}
|
||||
|
||||
function choice(node, name, into) {
|
||||
const widget = widgetOf(node, name);
|
||||
const select = add(into, "select");
|
||||
for (const value of comboValues(widget)) add(select, "option", value).value = value;
|
||||
select.value = widget?.value ?? "";
|
||||
select.title = widget?.options?.tooltip ?? "";
|
||||
select.addEventListener("change", () => {
|
||||
if (widget) commitWidgetValue(node, widget, select.value, app.canvas);
|
||||
const button = add(into, "button", undefined, "ausboss-chip-pick");
|
||||
button.type = "button";
|
||||
button.setAttribute("aria-haspopup", "listbox");
|
||||
button.title = widget?.options?.tooltip ?? "";
|
||||
const show = (value) => { button.textContent = String(value ?? ""); };
|
||||
show(widget?.value);
|
||||
const choose = (value) => {
|
||||
show(value);
|
||||
if (widget && value !== widget.value) commitWidgetValue(node, widget, value, app.canvas);
|
||||
node.setDirtyCanvas?.(true, true);
|
||||
};
|
||||
button.addEventListener("pointerdown", (event) => event.stopPropagation());
|
||||
button.addEventListener("click", (event) => {
|
||||
event.preventDefault();
|
||||
if (openMenu?.owner === button) { openMenu.close(); return; }
|
||||
const values = comboValues(widget).map(String);
|
||||
openJobMenu(into, button, values, String(widget?.value ?? ""), choose);
|
||||
});
|
||||
return (values) => { if (document.activeElement !== select) select.value = values[name] ?? ""; };
|
||||
return (values) => { if (openMenu?.owner !== button) show(values[name]); };
|
||||
}
|
||||
|
||||
// The jobs have no socket of their own, but a page saved by an earlier
|
||||
@@ -90,6 +189,7 @@ export function castChips(node) {
|
||||
let syncs = [];
|
||||
let latest = {};
|
||||
const build = (values) => {
|
||||
if (openMenu && element.contains(openMenu.owner)) openMenu.close();
|
||||
element.textContent = "";
|
||||
syncs = [];
|
||||
for (const number of [1, 2, 3, 4]) {
|
||||
@@ -136,7 +236,10 @@ export function castChips(node) {
|
||||
for (const run of syncs) run(values);
|
||||
};
|
||||
const watch = setInterval(() => sync(latest), WATCH);
|
||||
chainCallback(node, "onRemoved", () => clearInterval(watch));
|
||||
chainCallback(node, "onRemoved", () => {
|
||||
clearInterval(watch);
|
||||
if (openMenu && element.contains(openMenu.owner)) openMenu.close();
|
||||
});
|
||||
chainCallback(node, "onConfigure", () => queueMicrotask(() => keepOldJobLinks(node)));
|
||||
return { element, sync };
|
||||
}
|
||||
|
||||
@@ -50,6 +50,6 @@ export function clipLengthWords(timing) {
|
||||
|
||||
// Keep in step with Python KEEPS_ITS_LENGTH. Other jobs use seconds normally.
|
||||
export function clipSetsDuration(node, values) {
|
||||
return ["swap the person", "new voice", "change something"].includes(values.video_job)
|
||||
return ["copy its moves", "swap the person", "new voice", "change something"].includes(values.video_job)
|
||||
&& linkSource(node, "video") !== null;
|
||||
}
|
||||
|
||||
+1
-1
@@ -9,7 +9,7 @@ import { app } from "../../../scripts/app.js";
|
||||
// Must match `version` in pyproject.toml — scripts/release_preflight.py
|
||||
// enforces it. The backend serves its copy at /ausboss/pack_version; a
|
||||
// mismatch means the browser cached JavaScript from an older install.
|
||||
export const AUSBOSS_JS_VERSION = "2.26.0";
|
||||
export const AUSBOSS_JS_VERSION = "2.26.3";
|
||||
|
||||
export const BRAND = "#00b4aa"; // AusBoss teal — keep node accents consistent.
|
||||
export const BRAND_DARK = "#007f78";
|
||||
|
||||
@@ -38,7 +38,8 @@ function ensureCss() {
|
||||
.ausboss-strip-handle::before{content:"";position:absolute;left:5px;top:5px;bottom:5px;width:2px;border-radius:1px;background:rgba(207,232,229,.35)}
|
||||
.ausboss-strip-handle:hover::before,.ausboss-strip-handle.dragging::before{background:#cfe8e5}
|
||||
.ausboss-strip-handle span{position:absolute;bottom:100%;left:50%;transform:translateX(-50%);padding:1px 4px;border-radius:3px;background:#cfe8e5;color:#0b0f10;font-size:9px;white-space:nowrap;display:none}
|
||||
.ausboss-strip-handle.dragging span{display:block}
|
||||
.ausboss-strip-handle:hover span,.ausboss-strip-handle.dragging span{display:block}
|
||||
.ausboss-strip-handle.cuts::before{background:${BRAND}}
|
||||
.ausboss-strip-block.movable{cursor:grab;touch-action:none}
|
||||
.ausboss-strip-block.moving{z-index:3;cursor:grabbing;opacity:.85;box-shadow:0 4px 14px rgba(0,0,0,.5)}
|
||||
.ausboss-strip-drop{position:absolute;top:-3px;bottom:-3px;z-index:4;width:3px;margin-left:-2px;border-radius:2px;background:#f5b53d;pointer-events:none}
|
||||
@@ -108,7 +109,12 @@ export function momentStrip(node) {
|
||||
const label = document.createElement("span");
|
||||
handle.append(label);
|
||||
handle.style.left = `${rows[index].from}%`;
|
||||
handle.title = range ? "Drag to set when this moment starts. Click to add or remove a cut." : "Click to add or remove a cut.";
|
||||
const isCut = Boolean(rows[index].cut);
|
||||
handle.classList.toggle("cuts", isCut);
|
||||
// Shown on hover: what a click does here. Dragging swaps it for the time.
|
||||
const hint = isCut ? "\u2702 join" : "\u2702 cut";
|
||||
label.textContent = hint;
|
||||
handle.title = `${isCut ? "Click to join these two shots." : "Click to cut to a new shot here."}${range ? " Drag to set when this moment starts." : ""}`;
|
||||
track.append(handle);
|
||||
handle.addEventListener("pointerdown", (event) => {
|
||||
if (event.button !== 0) return;
|
||||
@@ -134,6 +140,7 @@ export function momentStrip(node) {
|
||||
handle.removeEventListener("pointerup", stop);
|
||||
handle.removeEventListener("pointercancel", stop);
|
||||
handle.classList.remove("dragging");
|
||||
label.textContent = hint;
|
||||
if (!moved) commitScript((text) => toggleCut(text, rows[index].line));
|
||||
else if (second !== null && second !== rows[index].start) commitScript((text) => retime(text, rows[index].line, second));
|
||||
};
|
||||
|
||||
@@ -59,14 +59,15 @@ export function dragRange(rows, index, seconds) {
|
||||
export function stripCaption(report) {
|
||||
if (report?.warning) return { text: report.warning, warn: true };
|
||||
const moments = report?.moments ?? [];
|
||||
if (!moments.length) return { text: "Each line becomes a moment.", warn: false };
|
||||
if (!moments.length) return { text: "Each line becomes a moment. A blank line between lines is a cut.", warn: false };
|
||||
const count = moments.length === 1 ? "1 moment" : `${moments.length} moments`;
|
||||
const cuts = moments.filter((moment) => moment.cut).length;
|
||||
if (cuts) return { text: `${count}, ${cuts === 1 ? "1 cut" : `${cuts} cuts`}.`, warn: false };
|
||||
// Say how to cut, since nothing on the bar shows it until you hover an edge.
|
||||
if (cuts) return { text: `${count}, ${cuts === 1 ? "1 cut" : `${cuts} cuts`}. Click a cut on the bar to join the shots.`, warn: false };
|
||||
const each = moments.map((moment) => trimmed(moment.end - moment.start));
|
||||
if (moments.length === 1) return { text: `${count}, ${each[0]} s, one take.`, warn: false };
|
||||
if (moments.length === 1) return { text: `${count}, ${each[0]} s, one take. Add a line for the next moment, a blank line to cut.`, warn: false };
|
||||
const even = each.every((length) => length === each[0]);
|
||||
return { text: even ? `${count}, ${each[0]} s each, one take.` : `${count}, one take.`, warn: false };
|
||||
return { text: `${even ? `${count}, ${each[0]} s each` : count}, one take. Click between two moments to cut there.`, warn: false };
|
||||
}
|
||||
|
||||
// The seconds the bar under the script marks: every second up to 10 s,
|
||||
|
||||
@@ -82,6 +82,7 @@ import {
|
||||
} from "./transform_geometry.mjs";
|
||||
import { clampFrame, clipInfo, frameTime } from "./timeline_math.mjs";
|
||||
import { liftSocket } from "./widget_card.mjs";
|
||||
import { COPY_EDITS_SETTING, copyPicture, transformPicture } from "./transform_picture.mjs";
|
||||
|
||||
const HIDDEN_WIDGETS = [
|
||||
"image", "upload",
|
||||
@@ -107,7 +108,7 @@ const RESIZE_METHODS = ["lanczos", "area", "bicubic", "bilinear", "nearest-exact
|
||||
const TRANSFORM_DEFAULTS = resetTransformValues(false);
|
||||
const CORE_IMAGE_PREVIEW_WIDGET = "$$canvas-image-preview";
|
||||
|
||||
// The gear on the image node's source card. Display choices only, kept for
|
||||
// The gear on the image node's source card. Browser preferences, kept for
|
||||
// every image node on this browser (settings_menu.mjs stores them).
|
||||
const SETTINGS_SCOPE = "image_crop_rotate_pad";
|
||||
const SETTINGS_SCHEMA = [
|
||||
@@ -115,6 +116,7 @@ const SETTINGS_SCHEMA = [
|
||||
key: "show_mask", label: "Show the mask on the picture", type: "toggle", default: true,
|
||||
hint: "Teal over the parts of your picture the model paints: a mask drawn in the MaskEditor, or see-through parts of a PNG.",
|
||||
},
|
||||
COPY_EDITS_SETTING,
|
||||
];
|
||||
let transformSettings = null;
|
||||
function settings() {
|
||||
@@ -3106,6 +3108,68 @@ function drawRotateGlyph(context, x, y, radius, color) {
|
||||
// node itself can still drag, and pointercancel plus mouseleave both end a
|
||||
// gesture in case the capture was refused.
|
||||
function attachStageHandlers(state, canvas, signal) {
|
||||
if (state.kind === "image") {
|
||||
canvas.addEventListener("contextmenu", (event) => {
|
||||
event.preventDefault(); event.stopPropagation();
|
||||
if (!state.image) return;
|
||||
const picture = async () => {
|
||||
let source = state.image;
|
||||
const sourceWidth = state.sourceWidth, sourceHeight = state.sourceHeight;
|
||||
const current = values(state.node), resize = resizeRequest(state.node);
|
||||
const includeEdits = settings().copy_edits;
|
||||
if (includeEdits && maskLayer(state)) {
|
||||
if (value(state.node, "painted_area", "fill") === "keep picture") {
|
||||
// The browser loses colours under fully transparent pixels. Core's
|
||||
// RGB view restores the stored picture under MaskEditor strokes.
|
||||
source = new Image();
|
||||
source.src = api.apiURL(`/view?${mediaViewQuery(value(state.node, "image", ""))}&channel=rgb`);
|
||||
await source.decode();
|
||||
} else {
|
||||
// The stage's mask layer is capped at 2048px. Copy from the source
|
||||
// at full size, using the same rule for which pixels stay picture.
|
||||
const full = document.createElement("canvas");
|
||||
full.width = sourceWidth; full.height = sourceHeight;
|
||||
const context = full.getContext("2d");
|
||||
context.drawImage(source, 0, 0);
|
||||
const pixels = context.getImageData(0, 0, full.width, full.height);
|
||||
const map = seeThroughMap(pixels.data);
|
||||
if (map) for (let i = 0; i < map.length; i++) pixels.data[i * 4 + 3] = map[i] ? 0 : 255;
|
||||
context.putImageData(pixels, 0, 0);
|
||||
source = full;
|
||||
}
|
||||
}
|
||||
return transformPicture({
|
||||
sourceWidth, sourceHeight, values: current, resize, includeEdits,
|
||||
fill: normalizeColor(current.fill_color),
|
||||
drawSource: (context) => context.drawImage(source, -sourceWidth / 2, -sourceHeight / 2, sourceWidth, sourceHeight),
|
||||
});
|
||||
};
|
||||
const report = (error) => showToast({ severity: "error", summary: "Copy image", detail: error?.message ?? String(error) });
|
||||
new globalThis.LiteGraph.ContextMenu([
|
||||
{ content: "Copy image", callback: () => {
|
||||
try {
|
||||
Promise.resolve(copyPicture(picture)).then(
|
||||
() => showToast({ summary: "Copy image", severity: "success", detail: "The picture is on the clipboard." }), report,
|
||||
);
|
||||
} catch (error) { report(error); }
|
||||
} },
|
||||
{ content: "Open image", callback: async () => {
|
||||
try {
|
||||
// A real image retains the browser's native Copy image on plain
|
||||
// HTTP too. A data URL stays usable for as long as the tab is open.
|
||||
const tab = window.open("about:blank", "_blank");
|
||||
if (!tab) throw new Error("Allow pop-ups to open the picture.");
|
||||
tab.opener = null;
|
||||
const image = tab.document.createElement("img");
|
||||
image.src = (await picture()).toDataURL("image/png");
|
||||
image.style.maxWidth = "100%";
|
||||
tab.document.title = "Crop image — right-click to copy";
|
||||
tab.document.body.append(image);
|
||||
} catch (error) { report(error); }
|
||||
} },
|
||||
], { event });
|
||||
}, { signal });
|
||||
}
|
||||
canvas.addEventListener("pointerdown", (event) => pointerDown(state, canvas, event), { signal });
|
||||
canvas.addEventListener("pointermove", (event) => pointerMove(state, canvas, event), { signal });
|
||||
canvas.addEventListener("pointerup", (event) => pointerUp(state, canvas, event), { signal });
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
import { keptCrop, resolveCrop, resolvePadding, rotatedSize, scaleToMegapixels } from "./transform_geometry.mjs";
|
||||
|
||||
export const COPY_EDITS_SETTING = Object.freeze({
|
||||
key: "copy_edits", label: "Copy with crop, rotation and padding", type: "toggle", default: false,
|
||||
hint: "Off: copy the original picture. On: include crop, rotation, padding and resize. Handles and guides are never copied.",
|
||||
});
|
||||
|
||||
// Render picture pixels into a separate canvas. The editor's canvas contains
|
||||
// handles, hatching and stage margins and must never be a clipboard source.
|
||||
export function transformPicture({ sourceWidth, sourceHeight, values, resize, fill, drawSource, includeEdits = false, createCanvas = () => document.createElement("canvas") }) {
|
||||
const canvas = createCanvas();
|
||||
if (!includeEdits) {
|
||||
canvas.width = sourceWidth; canvas.height = sourceHeight;
|
||||
const context = canvas.getContext("2d");
|
||||
if (!context) throw new Error("The browser could not make the picture.");
|
||||
context.translate(sourceWidth / 2, sourceHeight / 2);
|
||||
drawSource(context);
|
||||
return canvas;
|
||||
}
|
||||
const source = rotatedSize(sourceWidth, sourceHeight, values.rotation_degrees);
|
||||
const whole = resolveCrop(values, source);
|
||||
const padding = resolvePadding(values, whole);
|
||||
const crop = keptCrop(whole, padding);
|
||||
const size = resize
|
||||
? scaleToMegapixels(padding.outputWidth, padding.outputHeight, resize.megapixels, resize.steps)
|
||||
: { width: padding.outputWidth, height: padding.outputHeight };
|
||||
canvas.width = size.width; canvas.height = size.height;
|
||||
const context = canvas.getContext("2d");
|
||||
if (!context) throw new Error("The browser could not make the picture.");
|
||||
context.fillStyle = fill;
|
||||
context.fillRect(0, 0, canvas.width, canvas.height);
|
||||
context.save();
|
||||
context.scale(size.width / padding.outputWidth, size.height / padding.outputHeight);
|
||||
context.beginPath();
|
||||
context.rect(padding.left, padding.top, crop.width, crop.height);
|
||||
context.clip();
|
||||
context.translate(padding.left - crop.x + source.width / 2, padding.top - crop.y + source.height / 2);
|
||||
context.rotate((Number(values.rotation_degrees) || 0) * Math.PI / 180);
|
||||
drawSource(context);
|
||||
context.restore();
|
||||
return canvas;
|
||||
}
|
||||
|
||||
// Start the clipboard write in the click's activation, including in browsers
|
||||
// that require a promised PNG rather than a write after toBlob has finished.
|
||||
export function copyPicture(picture, { clipboard = globalThis.navigator?.clipboard, ClipboardItem = globalThis.ClipboardItem } = {}) {
|
||||
if (!clipboard?.write || !ClipboardItem) throw new Error("Open ComfyUI on localhost or HTTPS to copy directly. Choose Open image, then right-click the picture to copy it instead.");
|
||||
const png = Promise.resolve().then(() => typeof picture === "function" ? picture() : picture).then((canvas) => new Promise((resolve, reject) => {
|
||||
canvas.toBlob((blob) => blob ? resolve(blob) : reject(new Error("The browser could not make the picture.")), "image/png");
|
||||
}));
|
||||
// Also observe encoding failure when write() is refused before it reads png.
|
||||
png.catch(() => {});
|
||||
return clipboard.write([new ClipboardItem({ "image/png": png })]);
|
||||
}
|
||||
@@ -60,6 +60,26 @@ def add_module_version(source, token):
|
||||
return _MODULE_IMPORT.sub(lambda m: f"{m.group(1)}{m.group(2)}{m.group(3)}?v={token}{m.group(2)}", source)
|
||||
|
||||
|
||||
def list_served_scripts(js_root):
|
||||
"""The pack's own scripts by their address below the js folder:
|
||||
{"shared/index.mjs": <path>}. Made once at startup, so a request is
|
||||
only looked up in it and the address a browser sends never reaches the
|
||||
disk: no whole path, network share or ".." is ever opened. Links are
|
||||
left out, and anything that resolves outside the js folder."""
|
||||
root = Path(js_root).resolve()
|
||||
scripts = {}
|
||||
for path in sorted(root.rglob("*")):
|
||||
try:
|
||||
if path.suffix not in (".js", ".mjs") or path.is_symlink() or not path.is_file():
|
||||
continue
|
||||
if not path.resolve().is_relative_to(root):
|
||||
continue
|
||||
except OSError:
|
||||
continue
|
||||
scripts[path.relative_to(root).as_posix()] = path
|
||||
return scripts
|
||||
|
||||
|
||||
def make_revalidate_middleware(pack_name, js_root=None, token=None):
|
||||
"""aiohttp middleware for this pack's scripts, nothing else touched.
|
||||
|
||||
@@ -69,7 +89,7 @@ def make_revalidate_middleware(pack_name, js_root=None, token=None):
|
||||
from aiohttp import web
|
||||
|
||||
marker = f"/extensions/{pack_name}/"
|
||||
root = Path(js_root).resolve() if js_root and token else None
|
||||
scripts = list_served_scripts(js_root) if js_root and token else None
|
||||
served = {}
|
||||
|
||||
def versioned(path):
|
||||
@@ -90,15 +110,15 @@ def make_revalidate_middleware(pack_name, js_root=None, token=None):
|
||||
|
||||
def tagged_response(request):
|
||||
"""The tagged script for this request, or None to let ComfyUI serve it."""
|
||||
if root is None or request.method not in ("GET", "HEAD"):
|
||||
if not scripts or request.method not in ("GET", "HEAD"):
|
||||
return None
|
||||
start = request.path.find(marker)
|
||||
if start < 0 or not request.path.endswith((".js", ".mjs")):
|
||||
return None
|
||||
target = scripts.get(request.path[start + len(marker):])
|
||||
if target is None:
|
||||
return None
|
||||
try:
|
||||
target = (root / request.path[start + len(marker):]).resolve()
|
||||
if not target.is_relative_to(root) or not target.is_file():
|
||||
return None
|
||||
found = versioned(target)
|
||||
except (OSError, ValueError, UnicodeDecodeError):
|
||||
return None
|
||||
|
||||
+230
-40
@@ -19,6 +19,10 @@ STYLE = "style"
|
||||
START = "start on it"
|
||||
END = "end on it"
|
||||
JOBS = (PERSON, SHEET, PLACE, OUTFIT, THING, STYLE, START, END)
|
||||
# Not a pick on the node: a thing picture that does the action of a move copy or a swap (a cartoon hand, a toy,
|
||||
# a mascot). It takes the place a person would, with its own shape and look.
|
||||
PERFORMER = "thing that moves"
|
||||
ACTORS = (PERSON, SHEET, PERFORMER)
|
||||
|
||||
MOVES = "copy its moves"
|
||||
SWAP = "swap the person"
|
||||
@@ -26,7 +30,7 @@ KEEP = "keep going"
|
||||
REVOICE = "new voice"
|
||||
EDIT = "change something"
|
||||
VIDEO_JOBS = (MOVES, SWAP, KEEP, REVOICE, EDIT)
|
||||
KEEPS_ITS_LENGTH = (SWAP, REVOICE, EDIT) # These change a clip, so the new one runs as long as the old one.
|
||||
KEEPS_ITS_LENGTH = (MOVES, SWAP, REVOICE, EDIT) # These follow a clip, so the new one runs as long as the old one.
|
||||
NEEDS_NO_WORDS = (SWAP, KEEP)
|
||||
|
||||
TOKEN = re.compile(r"@(\d)\b")
|
||||
@@ -35,14 +39,31 @@ MAIN = "@0" # The main person, for lines the node writes itself.
|
||||
BACKGROUND = "The background of {picture} is not kept."
|
||||
NOTHING_IN = "Connect a picture or a video. H3 needs one to start from."
|
||||
NEEDS_A_PERSON = {
|
||||
MOVES: "To copy the moves, connect a picture of your person and pick person for it.",
|
||||
SWAP: "To swap the person, connect a picture of the new person and pick person for it.",
|
||||
}
|
||||
# A hat or a shirt with no person picture: the user wants the video's own person to wear it.
|
||||
DRESS_THE_VIDEO_PERSON = 'To put this on the person in your video, pick the video job "change something" and type "He wears @1".'
|
||||
|
||||
SWAP_NEEDS_PERSON_THING = (
|
||||
'A swap needs a picture of the new person, and pick person for it. To show a thing in the clip, '
|
||||
'pick the video job "change something" and type "He holds @1".'
|
||||
)
|
||||
SWAP_NEEDS_PERSON_PLACE = (
|
||||
'A swap keeps the place of the video. Connect a picture of the new person and pick person for it. '
|
||||
'To change the place, pick the video job "change something" and type "It looks like @1".'
|
||||
)
|
||||
REVOICE_KEEPS_PERSON = (
|
||||
"New voice keeps the person in the clip, so a person picture can't change who is in it. "
|
||||
'Take the picture off, or pick the video job "swap the person".'
|
||||
)
|
||||
HE_SHE = re.compile(r"\b(?:he|she|they)\b", re.I)
|
||||
|
||||
# (what the label is, how it is kept). {picture} is H3's number for the picture.
|
||||
JOB_WORDS = {
|
||||
PERFORMER: (
|
||||
"{subject} is the character in {picture}, with the same shape, colours and drawing style.",
|
||||
"fully_preserved - the shape, colours and drawing style are retained.",
|
||||
),
|
||||
PERSON: (
|
||||
"{subject} is the person in {picture}, with the same face, hair and clothes.",
|
||||
"fully_preserved - the face, hair and clothes are retained.",
|
||||
@@ -91,6 +112,18 @@ MOVES_KEPT = (
|
||||
"keep the source performer's face, body, clothes, props, background and colours out of the target.\n"
|
||||
"<Video 1>: attribute_transfer - apply only its camera movement to the target scene."
|
||||
)
|
||||
# Copy its moves with no person picture: the person in the video keeps their own moves and goes where the
|
||||
# script or a place picture says.
|
||||
OWN_MOVES_ARE = (
|
||||
"{main} is the person in <Video 1>, with the same face, hair, body{clothes} and the same moves, gestures and "
|
||||
"timing.\n<Video 1> provides the camera movement."
|
||||
)
|
||||
OWN_MOVES_KEPT = (
|
||||
"{main} (appears in {shots}): {kept}\n"
|
||||
"<Video 1>: attribute_transfer - apply its person, their moves and its camera movement to the target scene; {place}"
|
||||
)
|
||||
OWN_PLACE_FROM = "the background of <Video 1> is not kept, and the place is {place}."
|
||||
OWN_PLACE_KEPT = "keep its place unless the script names a new one."
|
||||
SWAP_VIDEO = "<Video 1> supplies the place, the action, the timing, the camera movement and the look."
|
||||
SWAP_IS = "{replaced} is {who} in <Video 1>, selected for replacement."
|
||||
SWAP_VIDEO_KEPT = (
|
||||
@@ -99,7 +132,28 @@ SWAP_VIDEO_KEPT = (
|
||||
)
|
||||
SWAP_KEPT = "{replaced}: removed - replace {who} with {main} throughout."
|
||||
ONE_GOES = "the original person"
|
||||
THE_PERSON = "the person in the shot"
|
||||
APPEARANCE_LINE = re.compile(r'^\s*@([1-4])\s+looks like\s+([^"\n]+?)\s*$', re.I)
|
||||
# Two ways people type a swap that used to be dropped without a word, so the swap ran with no description and H3
|
||||
# often gave the clip back: the looks sentence on the same line as the swap sentence, and the clothes on a line of
|
||||
# their own ("@1 is wearing a black suit"). A looks sentence starts after a full stop, ! or ?; "Put @1 looks like a
|
||||
# friend here." is one sentence and stays whole.
|
||||
LOOKS_MID_LINE = re.compile(r"(?<=[.!?])[ \t]*(?=@[1-4]\s+looks like\b)", re.I)
|
||||
WEARS_LINE = re.compile(r'^\s*@([1-4])\s+(?:is\s+wearing|wears|is\s+dressed\s+in|has\s+on)\s+([^"@\n]+?)\s*$', re.I)
|
||||
QUOTE = re.compile(r'["\u201c\u201d]')
|
||||
|
||||
|
||||
def looks_on_own_lines(text):
|
||||
"""The script with each @N looks like sentence on a line of its own. A spoken line is left alone."""
|
||||
text = str(text or "")
|
||||
lines = []
|
||||
for line in text.splitlines():
|
||||
if QUOTE.search(line) or not LOOKS_MID_LINE.search(line):
|
||||
lines.append(line)
|
||||
continue
|
||||
first, *rest = LOOKS_MID_LINE.sub("\n", line).splitlines()
|
||||
lines += [re.sub(r"\s+([.,;:])$", r"\1", first), *rest]
|
||||
return "\n".join(lines) if len(lines) != len(text.splitlines()) else text
|
||||
|
||||
|
||||
# Hair and face only, no clothes, expression or background. When the line names clothes that differ from what the
|
||||
@@ -115,6 +169,7 @@ def add_looks(text, jobs, describe):
|
||||
"""A swap's script with an @N looks like line for each new person who has none, written by describe(slot), a
|
||||
small picture-reading model shown that person's picture. Without such a line H3 kept the original person for the
|
||||
first seconds (2026-11-07), so a connected model writes it when the user didn't."""
|
||||
text = looks_on_own_lines(text)
|
||||
typed = {int(match.group(1)) for match in map(APPEARANCE_LINE.match, text.splitlines()) if match}
|
||||
for slot in sorted(jobs):
|
||||
if jobs[slot] not in (PERSON, SHEET) or slot in typed:
|
||||
@@ -137,8 +192,12 @@ def swap_appearances(text, jobs, video):
|
||||
"""
|
||||
if video != SWAP:
|
||||
return text, {}
|
||||
kept, descriptions = [], {}
|
||||
for line in str(text or "").splitlines():
|
||||
kept, descriptions, clothes = [], {}, {}
|
||||
for line in looks_on_own_lines(text).splitlines():
|
||||
worn = WEARS_LINE.fullmatch(line)
|
||||
if worn and (jobs or {}).get(int(worn.group(1))) in (PERSON, SHEET):
|
||||
clothes.setdefault(int(worn.group(1)), []).append(worn.group(2).strip().rstrip(" ."))
|
||||
continue
|
||||
match = APPEARANCE_LINE.fullmatch(line)
|
||||
if match is None:
|
||||
kept.append(line)
|
||||
@@ -150,10 +209,18 @@ def swap_appearances(text, jobs, video):
|
||||
raise ValueError(f"Picture {slot} must be a person or character sheet for a swap's looks like line.")
|
||||
if slot in descriptions:
|
||||
raise ValueError(f"Describe Picture {slot} once, in one looks like line.")
|
||||
description = match.group(2).strip().rstrip(" .")
|
||||
# A sentence on the looks line that names an @ again ("The trees behind @1 stay the same.") is not how
|
||||
# the person looks: it goes back to the script as its own line, or H3 reads a raw @1 in the appearance.
|
||||
sentences = re.split(r"(?<=[.!?])\s+", match.group(2).strip())
|
||||
looks = [sentence for sentence in sentences if not TOKEN.search(sentence)]
|
||||
kept.extend(sentence for sentence in sentences if TOKEN.search(sentence))
|
||||
description = " ".join(looks).strip().rstrip(" .")
|
||||
if not description:
|
||||
raise ValueError(f"Say what Picture {slot}'s person looks like, or remove its looks like line.")
|
||||
descriptions[slot] = description
|
||||
for slot, worn in clothes.items():
|
||||
# Clothes typed on their own line join the description; with none typed, the picture still gives the hair.
|
||||
descriptions[slot] = f"{descriptions.get(slot, 'the same hair')}, wearing {' and '.join(worn)}"
|
||||
return ("\n".join(kept), descriptions) if descriptions else (text, {})
|
||||
|
||||
|
||||
@@ -204,23 +271,24 @@ CLOTHES_STAY = (
|
||||
"attribute_transfer - apply only the face and hair to the original performance; the clothes of {who} in <Video 1> "
|
||||
"stay as they are; the background of {picture} is not transferred."
|
||||
)
|
||||
KEEP_IS = (
|
||||
"<Video 1> is the video being continued, and the target video starts from its last moment.\n"
|
||||
"{main} is the main subject in <Video 1>, with the same look."
|
||||
)
|
||||
KEEP_KEPT = (
|
||||
"<Video 1> (continuation point): fully_preserved - the place, the light, the colours and the camera carry on from its last moment.\n"
|
||||
"{main} (appears in {shots}): fully_preserved - the look is retained."
|
||||
)
|
||||
KEEP_IS = "<Video 1> is the video being continued, and the target video starts from its last moment."
|
||||
KEEP_MAIN = "{main} is the main subject in <Video 1>, with the same look."
|
||||
KEEP_KEPT = "<Video 1> (continuation point): fully_preserved - the place, the light, the colours and the camera carry on from its last moment."
|
||||
KEEP_KEPT_PLACE = "<Video 1> (continuation point): fully_preserved - the light, the colours and the camera carry on from its last moment."
|
||||
KEEP_MAIN_KEPT = "{main} (appears in {shots}): fully_preserved - the look is retained."
|
||||
SOUND_GOES_ON = "<Audio 1> is the soundtrack of <Video 1>, whose sound carries on in the target video."
|
||||
SOUND_GOES_ON_KEPT = "<Audio 1>: reference - the sound continues without copying the source signal."
|
||||
EDIT_IS = (
|
||||
"<Video 1> supplies the place, the person, the action, the timing, the camera movement, the look and the sound.\n"
|
||||
"{main} is the person in <Video 1>, kept as they are: the same face, hair, body and movements, with only what the script names changed."
|
||||
)
|
||||
EDIT_KEPT = (
|
||||
"<Video 1>: partially_preserved - keep the place, the person, the action, the timing, the camera movement, the look and the sound; change only what the script names.\n"
|
||||
"{main} (appears in {shots}): fully_preserved - the face, hair and every movement are retained; only what the script names changes."
|
||||
EDIT_VIDEO = "<Video 1> supplies {supplies}."
|
||||
EDIT_IS = "{main} is the person in <Video 1>, kept as they are: the same face, hair, body and movements, with only what the script names changed."
|
||||
EDIT_VIDEO_KEPT = "<Video 1>: partially_preserved - keep {supplies}; change only what the script names."
|
||||
EDIT_KEPT = "{main} (appears in {shots}): fully_preserved - the face, hair and every movement are retained; only what the script names changes."
|
||||
# What a source video gives an edit, less what a connected picture takes over.
|
||||
EDIT_SUPPLIES = ("the place", "the person", "the action", "the timing", "the camera movement", "the look", "the sound")
|
||||
# A line that is about someone. An edit or a move copy of a clip with nobody in it (an empty hallway) names nobody.
|
||||
NAMES_SOMEONE = re.compile(
|
||||
r"@0|\b(?:he|she|him|her|his|hers|they|them|their|someone|somebody|anyone|everyone|person|people|man|men|woman|women|"
|
||||
r"guy|girl|boy|kid|child|baby|dancer|singer|actor|actress)\b",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
REVOICE_IS = "<Video 1> is the source video for the target video edit.\n{main} is the person speaking in <Video 1>, with the same face, hair and clothing."
|
||||
REVOICE_KEPT = (
|
||||
@@ -229,6 +297,15 @@ REVOICE_KEPT = (
|
||||
)
|
||||
|
||||
|
||||
# A line only names who goes when it is a noun phrase like "the child in the seat", not an action or a pronoun.
|
||||
NOUN_PHRASE = re.compile(r"^(?:the|a|an|this|that|these|those|his|her|their|everyone|someone|somebody|both|each|all)\b", re.I)
|
||||
ACTION_WORD = re.compile(
|
||||
r"\b(?:dances|walks|runs|talks|says|sits|stands|smiles|waves|jumps|enters|leaves|laughs|sings|eats|drinks|waits|"
|
||||
r"is\s+\w+ing|are\s+\w+ing)\b",
|
||||
re.I,
|
||||
)
|
||||
|
||||
|
||||
def who_goes_in(text):
|
||||
"""Who a typed swap line says goes, without the rest of the line: "Elon is @1, he replaces the man at the
|
||||
counter" is the man at the counter, and so is "Replace the man at the counter with @1."."""
|
||||
@@ -244,7 +321,10 @@ def who_goes_in(text):
|
||||
text = re.split(r"[.;!?]", text.strip(" .;!?"), maxsplit=1)[0]
|
||||
text = NAMES_NEXT.sub("", text)
|
||||
text = SAYS_WHO.sub("", TOKEN.sub("", text).strip(" .,;")).strip(" .,;")
|
||||
return re.sub(r"\s+and$", "", text).strip(" .,;")
|
||||
text = re.sub(r"\s+and$", "", text).strip(" .,;")
|
||||
if not (replaces or plain) and not (NOUN_PHRASE.match(text) and not ACTION_WORD.search(text)):
|
||||
return "" # "@1 dances" or "He dances" says what happens, not who goes.
|
||||
return text
|
||||
|
||||
|
||||
def lower_first(text):
|
||||
@@ -274,7 +354,30 @@ class Picture:
|
||||
|
||||
def swapped_in(jobs):
|
||||
"""How many people a swap puts in: one for each person or sheet picture."""
|
||||
return max(1, sum(job in (PERSON, SHEET) for job in jobs.values()))
|
||||
return max(1, sum(job in ACTORS for job in jobs.values()))
|
||||
|
||||
|
||||
# A thing a line holds, wears or uses stays a prop; any other thing named in a line is the one doing the action.
|
||||
HELD = re.compile(
|
||||
r"\b(?:holds?|holding|carr(?:y|ies|ying)|wears?|wearing|puts? on|picks? up|uses?|using|grabs?|throws?|"
|
||||
r"eats?|drinks?|with)\s+(?:\w+\s+){0,2}@(\d)\b",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
|
||||
|
||||
def things_that_perform(script, jobs, video):
|
||||
"""A move copy or a swap with no person picture: a thing picture the lines make do the action (a cartoon hand
|
||||
waving, a toy dancing) takes the moves, or comes in for the one in the video. With nothing typed, a swap's thing
|
||||
comes in. A thing that is held or worn stays a thing."""
|
||||
if video not in (MOVES, SWAP) or any(job in (PERSON, SHEET) for job in (jobs or {}).values()):
|
||||
return jobs
|
||||
held = {int(slot) for slot in HELD.findall(script or "")}
|
||||
named = {int(slot) for slot in TOKEN.findall(script or "")}
|
||||
jobs = dict(jobs)
|
||||
for slot, job in jobs.items():
|
||||
if job == THING and slot not in held and (slot in named or (video == SWAP and not named)):
|
||||
jobs[slot] = PERFORMER
|
||||
return jobs
|
||||
|
||||
|
||||
def number_pictures(jobs, video=""):
|
||||
@@ -295,15 +398,20 @@ class Cast:
|
||||
self.pictures = pictures
|
||||
self.video = video
|
||||
last = max((picture.subject for picture in pictures), default=0)
|
||||
self.people = [picture for picture in pictures if picture.job in (*ACTORS, START)]
|
||||
self.moves = subject_label(last + 1) if video == MOVES else ""
|
||||
# Nobody to give the moves to: the person in the video keeps them and is the main person.
|
||||
self.own_moves = video == MOVES and not self.people
|
||||
self.video_person = True # The video's own person is in the result; notice_people can say there is none.
|
||||
# In a swap each person or sheet picture takes one person's place in the video, in picture order. The script
|
||||
# can name who goes: "the child in the seat", or "@1 the man on the left. @2 the woman in red".
|
||||
self.swappers = [picture for picture in pictures if picture.job in (PERSON, SHEET)] if video == SWAP else []
|
||||
self.swappers = [picture for picture in pictures if picture.job in ACTORS] if video == SWAP else []
|
||||
goes = len(self.swappers)
|
||||
self.whos = [f"the person on the {spot}" for spot in SPOTS[goes]] if goes > 1 else [ONE_GOES]
|
||||
self.replaced = subject_label(1) if video == SWAP else ""
|
||||
self.in_video = subject_label(last + 1) if video in (KEEP, REVOICE, EDIT) else ""
|
||||
self.people = [picture for picture in pictures if picture.job in (PERSON, SHEET, START)]
|
||||
self.in_video = subject_label(last + 1) if video in (KEEP, REVOICE, EDIT) or self.own_moves else ""
|
||||
if video == KEEP and self.people: # A person picture is the main person; the video adds no second subject.
|
||||
self.in_video = ""
|
||||
self.subjects = {picture.slot: subject_label(picture.subject) for picture in pictures if picture.subject}
|
||||
self.clip_clothes = False # A swap that keeps the clothes of whoever goes.
|
||||
self.appearances = {} # Explicit looks-like lines, by the visible replacement picture slot.
|
||||
@@ -311,6 +419,29 @@ class Cast:
|
||||
self.speaks_as_video = False # With that sound and no words typed: the new person says what the video says.
|
||||
self.typed_lines = False # The script typed words in quotes for someone to say.
|
||||
|
||||
def notice_people(self, text):
|
||||
"""An edit or a move copy with no person picture keeps the video's person only when the lines are about
|
||||
someone, or something is put on them. A clip of an empty hallway gets no made-up person."""
|
||||
if self.video == KEEP:
|
||||
if not self.people and (text or "").strip():
|
||||
self.video_person = bool(NAMES_SOMEONE.search(text))
|
||||
elif self.video == EDIT or self.own_moves:
|
||||
self.video_person = bool(self.of(OUTFIT) or NAMES_SOMEONE.search(text or ""))
|
||||
|
||||
@property
|
||||
def supplies(self):
|
||||
"""What the source video of an edit still gives: a place picture brings the place and its look, a style
|
||||
picture the look."""
|
||||
gone = set()
|
||||
if self.of(PLACE):
|
||||
gone |= {"the place", "the look"}
|
||||
if self.of(STYLE):
|
||||
gone.add("the look")
|
||||
if not self.video_person:
|
||||
gone.add("the person")
|
||||
items = [item for item in EDIT_SUPPLIES if item not in gone]
|
||||
return ", ".join(items[:-1]) + " and " + items[-1]
|
||||
|
||||
def of(self, *jobs):
|
||||
return [picture for picture in self.pictures if picture.job in jobs]
|
||||
|
||||
@@ -356,12 +487,32 @@ class Cast:
|
||||
@property
|
||||
def main(self):
|
||||
"""Who a line is about when it names nobody."""
|
||||
if self.in_video:
|
||||
if self.in_video and self.video_person:
|
||||
return self.in_video
|
||||
if self.people:
|
||||
return subject_label(self.people[0].subject)
|
||||
return next(iter(self.subjects.values()), subject_label(1))
|
||||
|
||||
@property
|
||||
def main_in_shot(self):
|
||||
"""Whether main is a person or thing the clip defines, so "<Subject N> is in the shot" can name it."""
|
||||
if self.in_video and self.video_person and self.main == self.in_video:
|
||||
return True
|
||||
defined = {subject_label(picture.subject) for picture in self.of(*ACTORS, THING, START)}
|
||||
return self.main in defined
|
||||
|
||||
def unclear_speaker(self, moments):
|
||||
"""A refusal when two or more people are connected and a spoken line does not say which one speaks."""
|
||||
if self.video not in ("", MOVES, KEEP) or len(self.people) < 2:
|
||||
return ""
|
||||
people = {picture.slot for picture in self.people}
|
||||
for moment in moments:
|
||||
if moment.speech and not moment.voice and not moment.voiceover:
|
||||
if not any(int(slot) in people for slot in TOKEN.findall(moment.text)):
|
||||
count = "two people" if len(self.people) == 2 else "more than one person"
|
||||
return f'With {count}, name who speaks: @{self.people[1].slot} says "hello".'
|
||||
return ""
|
||||
|
||||
@property
|
||||
def opens(self):
|
||||
return next(iter(self.of(START)), None)
|
||||
@@ -378,9 +529,15 @@ class Cast:
|
||||
if not self.pictures and not self.video:
|
||||
return NOTHING_IN
|
||||
if self.video in NEEDS_A_PERSON and not self.people:
|
||||
if self.of(OUTFIT, THING):
|
||||
if self.of(OUTFIT):
|
||||
return DRESS_THE_VIDEO_PERSON
|
||||
if self.of(THING):
|
||||
return SWAP_NEEDS_PERSON_THING
|
||||
if self.of(PLACE):
|
||||
return SWAP_NEEDS_PERSON_PLACE
|
||||
return NEEDS_A_PERSON[self.video]
|
||||
if self.video == REVOICE and self.of(PERSON, SHEET):
|
||||
return REVOICE_KEEPS_PERSON
|
||||
typed = sorted({int(number) for number in TOKEN.findall(text)} - {0})
|
||||
slots = {picture.slot: picture for picture in self.pictures}
|
||||
for slot in typed:
|
||||
@@ -389,16 +546,23 @@ class Cast:
|
||||
if slots[slot].job == END:
|
||||
return f"Picture {slot} is where the clip ends, so @{slot} can't be in a line. Say what happens on the way there."
|
||||
if typed:
|
||||
for picture in self.of(PERSON, SHEET, THING, OUTFIT):
|
||||
if picture.slot not in typed:
|
||||
return f"Picture {picture.slot} is on, but no line names @{picture.slot}."
|
||||
for picture in self.of(*ACTORS, THING, OUTFIT):
|
||||
if picture.slot in typed:
|
||||
continue
|
||||
# The only person picture is who "He" or "She" means.
|
||||
if picture.job in (PERSON, SHEET) and len(self.of(PERSON, SHEET)) == 1 and HE_SHE.search(text):
|
||||
continue
|
||||
hint = f' Type who wears it, like "He wears @{picture.slot}."' if picture.job == OUTFIT else ""
|
||||
return f"Picture {picture.slot} is on, but no line names @{picture.slot}.{hint}"
|
||||
return ""
|
||||
|
||||
def wearers(self, moments):
|
||||
"""Who wears each outfit: the person named just before it, or else the main person."""
|
||||
found = {}
|
||||
outfits = {subject_label(picture.subject) for picture in self.of(OUTFIT)}
|
||||
default = THE_PERSON if self.main in outfits else self.main # An outfit is never worn by itself.
|
||||
for picture in self.of(OUTFIT):
|
||||
found[picture.slot] = self.main
|
||||
found[picture.slot] = default
|
||||
token = f"@{picture.slot}"
|
||||
for moment in moments:
|
||||
before = moment.text.partition(token)[0]
|
||||
@@ -414,7 +578,9 @@ class Cast:
|
||||
ends = self.ends
|
||||
lines = []
|
||||
if self.video == EDIT:
|
||||
lines.append(EDIT_IS.format(main=self.in_video))
|
||||
lines.append(EDIT_VIDEO.format(supplies=self.supplies))
|
||||
if self.video_person:
|
||||
lines.append(EDIT_IS.format(main=self.in_video))
|
||||
if self.video == SWAP:
|
||||
lines.append(SWAP_VIDEO)
|
||||
lines += [SWAP_IS.format(replaced=gone, who=who) for gone, who, _ in self.swaps]
|
||||
@@ -445,10 +611,17 @@ class Cast:
|
||||
lines.append(FIRST_FRAME.format(picture=picture_label(self.opens.picture)))
|
||||
if ends:
|
||||
lines.append(LAST_FRAME.format(picture=picture_label(ends.picture), last=last))
|
||||
if self.moves:
|
||||
if self.own_moves and not self.video_person:
|
||||
lines.append("<Video 1> provides the camera movement.")
|
||||
elif self.own_moves:
|
||||
clothes = "" if self.of(OUTFIT) else " and clothes,"
|
||||
lines.append(OWN_MOVES_ARE.format(main=self.in_video, clothes=clothes or ","))
|
||||
elif self.moves:
|
||||
lines.append(MOVES_ARE.format(moves=self.moves))
|
||||
if self.video == KEEP:
|
||||
lines.append(KEEP_IS.format(main=self.in_video))
|
||||
lines.append(KEEP_IS)
|
||||
if self.in_video and self.video_person:
|
||||
lines.append(KEEP_MAIN.format(main=self.in_video))
|
||||
if self.video == REVOICE:
|
||||
lines.append(REVOICE_IS.format(main=self.in_video))
|
||||
return "\n".join(lines)
|
||||
@@ -458,7 +631,7 @@ class Cast:
|
||||
subject = subject_label(picture.subject)
|
||||
label = picture_label(picture.picture)
|
||||
wearing = [slot for slot, wearer in wearers.items() if wearer == subject]
|
||||
if self.video == SWAP and picture.job in (PERSON, SHEET):
|
||||
if self.video == SWAP and picture.job in ACTORS:
|
||||
if self.clip_clothes:
|
||||
return CLOTHES_STAY.format(who=self.goes_for(picture), picture=label)
|
||||
return SWAPPED.format(picture=label)
|
||||
@@ -467,9 +640,9 @@ class Cast:
|
||||
elif picture.job == PERSON and wearing:
|
||||
words = DRESSED[1].format(outfit=self.subjects[wearing[0]])
|
||||
else:
|
||||
wearer = wearers.get(picture.slot, self.main)
|
||||
wearer = wearers.get(picture.slot, THE_PERSON if picture.job == OUTFIT else self.main)
|
||||
words = JOB_WORDS[picture.job][1].format(wearer=wearer)
|
||||
if picture.job in (PERSON, SHEET, THING):
|
||||
if picture.job in (*ACTORS, THING):
|
||||
words += " " + BACKGROUND.format(picture=label)
|
||||
return words
|
||||
|
||||
@@ -482,7 +655,9 @@ class Cast:
|
||||
|
||||
lines = []
|
||||
if self.video == EDIT:
|
||||
lines.append(EDIT_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
|
||||
lines.append(EDIT_VIDEO_KEPT.format(supplies=self.supplies))
|
||||
if self.video_person:
|
||||
lines.append(EDIT_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
|
||||
if self.video == SWAP:
|
||||
lines.append(SWAP_VIDEO_KEPT.format(people="people" if len(self.swaps) > 1 else "person"))
|
||||
lines += [SWAP_KEPT.format(replaced=gone, who=who, main=new) for gone, who, new in self.swaps]
|
||||
@@ -496,10 +671,25 @@ class Cast:
|
||||
lines.append(FIRST_KEPT.format(picture=picture_label(self.opens.picture)))
|
||||
if self.ends:
|
||||
lines.append(LAST_KEPT.format(picture=picture_label(self.ends.picture), last=len(shots)))
|
||||
if self.moves:
|
||||
if self.own_moves and not self.video_person:
|
||||
places = [self.subjects[picture.slot] for picture in self.of(PLACE)]
|
||||
place = OWN_PLACE_FROM.format(place=places[0]) if places else OWN_PLACE_KEPT
|
||||
lines.append(f"<Video 1>: attribute_transfer - apply only its camera movement to the target scene; {place}")
|
||||
elif self.own_moves:
|
||||
outfits = [self.subjects[picture.slot] for picture in self.of(OUTFIT)]
|
||||
kept = (
|
||||
f"partially_preserved - the face, hair, body, moves and timing are retained, and the clothes change to {outfits[0]}."
|
||||
if outfits else "fully_preserved - the face, hair, body, clothes, moves and timing are retained."
|
||||
)
|
||||
places = [self.subjects[picture.slot] for picture in self.of(PLACE)]
|
||||
place = OWN_PLACE_FROM.format(place=places[0]) if places else OWN_PLACE_KEPT
|
||||
lines.append(OWN_MOVES_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video)), kept=kept, place=place))
|
||||
elif self.moves:
|
||||
lines.append(MOVES_KEPT.format(moves=self.moves, shots=shot_list(seen(self.main)), main=self.main))
|
||||
if self.video == KEEP:
|
||||
lines.append(KEEP_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
|
||||
lines.append(KEEP_KEPT_PLACE if self.of(PLACE) else KEEP_KEPT)
|
||||
if self.in_video and self.video_person:
|
||||
lines.append(KEEP_MAIN_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
|
||||
if self.video == REVOICE:
|
||||
lines.append(REVOICE_KEPT.format(main=self.in_video, shots=shot_list(seen(self.in_video))))
|
||||
return "\n".join(lines)
|
||||
|
||||
@@ -13,7 +13,7 @@ import re
|
||||
import numpy as np
|
||||
|
||||
from ._h3_cast_helpers import EDIT, KEEP, NEEDS_NO_WORDS, ONE_GOES, REVOICE, SWAP, Cast, number_pictures, swap_appearances
|
||||
from ._h3_cast_helpers import TOKEN
|
||||
from ._h3_cast_helpers import TOKEN, things_that_perform
|
||||
from ._h3_script_helpers import (
|
||||
ACTION_GOES_ON,
|
||||
PRONOUN,
|
||||
@@ -78,9 +78,10 @@ def one_line(text):
|
||||
|
||||
def clean_words(text):
|
||||
"""Spoken words: one line, no tags, ending the way H3 wants a line to end."""
|
||||
text = one_line(text).replace("<", "").replace(">", "")
|
||||
if text and text[-1] not in ".?!":
|
||||
text += "."
|
||||
# '"Hi," she says' is "Hi.", not "Hi,.". Chinese and Japanese lines end in their own stops: "。", not "。.".
|
||||
text = one_line(text).replace("<", "").replace(">", "").rstrip(" ,;:,、;:")
|
||||
if text and text[-1] not in ".?!。?!…":
|
||||
text += "。" if re.search(r"[\u3040-\u30ff\u4e00-\u9fff]", text) else "."
|
||||
return text
|
||||
|
||||
|
||||
@@ -312,6 +313,7 @@ def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="E
|
||||
longest) puts a question to a small model, if one is connected: it fills in
|
||||
a one-line idea and the sound of the place."""
|
||||
parts = split_sections(script)
|
||||
written = parts is None # Words someone typed in six parts go through as they are.
|
||||
audio = next_label(script, "Audio")
|
||||
speaker = None
|
||||
|
||||
@@ -319,10 +321,13 @@ def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="E
|
||||
jobs = jobs if jobs is not None else {1: "person"}
|
||||
if video == EDIT:
|
||||
script, jobs = put_on_from_pictures(script, jobs)
|
||||
jobs = things_that_perform(script, jobs, video)
|
||||
cast = Cast(number_pictures(jobs, video), video)
|
||||
script, cast.appearances = swap_appearances(script, jobs, video)
|
||||
scene = read_script(script)
|
||||
cast.notice_people(" ".join(moment.text for moment in scene.moments))
|
||||
problem = cast.problem(" ".join(moment.text for moment in scene.moments))
|
||||
problem = problem or cast.unclear_speaker(scene.moments)
|
||||
if problem:
|
||||
raise ValueError(problem)
|
||||
if not scene.moments and video == EDIT:
|
||||
@@ -370,6 +375,8 @@ def write_prompt(script, seconds=15.0, camera="", look="", sound="", language="E
|
||||
scene, seconds, camera=camera, look=look, sound=sound, say=say, cast=cast, sounds=not keeps_song, soundtrack=soundtrack
|
||||
)
|
||||
|
||||
if written and (video == EDIT or (video == SWAP and clip_sound)): # The clip's own sound stays in the result.
|
||||
parts["summary"] = add_task(parts["summary"], "audio reuse")
|
||||
if voice:
|
||||
add_voice(parts, voice_label, speaker)
|
||||
if song_beats is not None and voice:
|
||||
|
||||
+69
-21
@@ -17,15 +17,19 @@ from ._h3_cast_helpers import (
|
||||
MAIN,
|
||||
OUTFIT,
|
||||
PERSON,
|
||||
PERFORMER,
|
||||
PLACE,
|
||||
REVOICE,
|
||||
SHEET,
|
||||
START as START_JOB,
|
||||
END as END_JOB,
|
||||
STYLE,
|
||||
SWAP,
|
||||
SOUND_GOES_ON,
|
||||
SOUND_GOES_ON_KEPT,
|
||||
THING,
|
||||
TOKEN,
|
||||
things_that_perform,
|
||||
ONE_GOES,
|
||||
SPOTS,
|
||||
Cast,
|
||||
@@ -43,7 +47,9 @@ THEN = r"(?:,?\s*(?:and\s+)?then\b|,?\s*and\s+after\s+that\b|,?\s*after\s+that\b
|
||||
# A word that starts a new moment and stays in it.
|
||||
TURN = r"(?=,?\s*\b(?:suddenly|all\s+of\s+a\s+sudden|out\s+of\s+nowhere|finally|at\s+last|(?:hard\s+)?cut\s+to|(?:it\s+|and\s+|(?<!\bit\s))ends?\s+(?:with|on|in|as)|ending\s+(?:with|on|in))\b)"
|
||||
# Where a line breaks into sentences: a full stop that is not a title, the end of a quote, or a semicolon.
|
||||
SENTENCE = r"(?<!\bMr\.)(?<!\bMs\.)(?<!\bDr\.)(?<!\bSt\.)(?<!\bMrs\.)(?<=[.!?])\s+(?=\S)|(?<=\x00)\s+(?=[A-Z])|\s*;\s*"
|
||||
# After a quote, a new sentence starts with a capital or an @: in '"Get a job." @1 looks up' the @1 is a new moment,
|
||||
# or its own words are lost behind the first quote's. So is '"Hi" and @2 says "Bye"': one moment holds one quote.
|
||||
SENTENCE = r"(?<!\bMr\.)(?<!\bMs\.)(?<!\bDr\.)(?<!\bSt\.)(?<!\bMrs\.)(?<=[.!?])\s+(?=\S)|(?<=\x00)\s+(?=[A-Z@])|(?<=\x00),?\s+(?:and|but|while)\s+(?=@\d)|\s*;\s*"
|
||||
CUT_WORDS = re.compile(r"^(?:hard\s+)?(?:cut|cuts|smash\s+cut|jump\s+cut)\s+to\b[:,]?\s*", re.I)
|
||||
NEW_SHOT = re.compile(r"^(?:new\s+shot|next\s+shot|shot\s+\d+)\b[:,.\-]?\s*", re.I)
|
||||
START = re.compile(r"^(?:it\s+|the\s+(?:clip|video|scene|shot)\s+)?(?:starts?|begins?|opens?)(?:\s+(?:with|on|in|at|as)\b[:,]?|\s*:)\s*", re.I)
|
||||
@@ -110,6 +116,30 @@ SOUND_WORDS = 22
|
||||
NAMED = {PLACE: "the place", OUTFIT: "the outfit", THING: "the thing"} # What @2 reads as to the small model.
|
||||
|
||||
WORDS_A_SECOND = 3 # How fast H3 speaks.
|
||||
# Words that trail off ("I never thought...") keep going: across the next cut, or past the end of the clip.
|
||||
TRAILS_OFF = re.compile(r"(?:\.\.\.|\u2026|\u2014|--)\s*$")
|
||||
LINE_CARRIES = "<scenetrans> The line from the previous shot continues seamlessly across the cut, and the words are not repeated."
|
||||
CUT_OFF = "The line is cut off by the end of the video. <cutoff>"
|
||||
|
||||
|
||||
def speech_runs_on(script, seconds):
|
||||
"""(carried, cut_off): the moments whose words go on across the next cut, and the last moment's words if the
|
||||
clip ends mid-line. Words carry on when they trail off with "..." or need more time than their shot has."""
|
||||
starts = timeline(script, seconds)
|
||||
ends = [*starts[1:], seconds]
|
||||
carried, cut_off = set(), None
|
||||
for index, moment in enumerate(script.moments):
|
||||
if not moment.speech:
|
||||
continue
|
||||
trails = bool(TRAILS_OFF.search(moment.speech))
|
||||
long = len(moment.speech.split()) / WORDS_A_SECOND > ends[index] - starts[index] + 0.5
|
||||
following = script.moments[index + 1] if index + 1 < len(script.moments) else None
|
||||
if following is not None and following.cut and not script.one_take and (trails or long):
|
||||
carried.add(index)
|
||||
elif following is None and trails:
|
||||
cut_off = index
|
||||
return carried, cut_off
|
||||
|
||||
HOLD = f"{MAIN} holds still" # What fills the time before a first moment set later.
|
||||
KEEP_LOOK = "The target video keeps the look, the light and the colours of {picture}."
|
||||
KEEP_COLOURS = "The target video keeps the colours and the film grain of {picture}."
|
||||
@@ -130,6 +160,7 @@ OPENS_ON = "The shot begins from {picture}."
|
||||
ENDS_ON = "The shot ends on {picture}."
|
||||
ON_SHEET = "{subject} appears once, without the panels, labels or other views of {picture}."
|
||||
THE_MOVES = "{main} performs the same moves as {moves}, at the same timing."
|
||||
OWN_MOVES = "{main} moves exactly as in <Video 1>, at the same timing."
|
||||
IN_THE_PLACE = "The shot takes place in {place}."
|
||||
WEARS = "{wearer} wears {outfit}."
|
||||
IN_THE_SHOT = "{main} is in the shot."
|
||||
@@ -142,10 +173,10 @@ SAYS_NOTHING = "{main} keeps the lips closed and says nothing the whole time."
|
||||
SPEAKS_AS_VIDEO = "{main} speaks as in <Video 1>, each word landing exactly on the lip movement of <Video 1>."
|
||||
CLIP_SOUND = "The original sound of <Video 1> continues throughout."
|
||||
FOLLOWS_CAMERA = "The camera follows the movement of <Video 1>."
|
||||
REST_AS_VIDEO = "Everything else stays as it is in <Video 1>."
|
||||
SAME_AS_VIDEO = "{main} has the same words, mouth movements and moves as in <Video 1>. Everything else stays as it is in <Video 1>."
|
||||
CARRIES_ON = "The shot carries straight on from the end of <Video 1>."
|
||||
ACTION_GOES_ON = "The action keeps going the way it was heading, and the camera keeps moving the same way."
|
||||
FACES_US = "{main} faces the camera as in <Video 1>."
|
||||
VIDEO_LOOKS = {
|
||||
SWAP: "Retain the look of <Video 1>.",
|
||||
KEEP: "The target video keeps the look, colours and camera of <Video 1>.",
|
||||
@@ -557,12 +588,13 @@ def script_warning(script, seconds, cast=None):
|
||||
if any((moment.at or 0.0) >= seconds for moment in script.moments):
|
||||
return f"A moment starts after the clip ends, at {seconds:g} seconds."
|
||||
starts = timeline(script, seconds)
|
||||
for moment, start, end in zip(script.moments, starts, [*starts[1:], seconds]):
|
||||
carried, cut_off = speech_runs_on(script, seconds)
|
||||
for index, (moment, start, end) in enumerate(zip(script.moments, starts, [*starts[1:], seconds])):
|
||||
unseen = UNSEEN.search(without_speech(moment))
|
||||
if unseen:
|
||||
return f'A camera can\'t see "{unseen.group(0).lower()}". Say what we would see, like "she smiles".'
|
||||
takes = len(moment.speech.split()) / WORDS_A_SECOND
|
||||
if takes > end - start + 0.5:
|
||||
if takes > end - start + 0.5 and index not in carried and index != cut_off:
|
||||
return f"A spoken line takes about {takes:.0f} seconds to say, and its moment has {end - start:.0f}. Give it more room."
|
||||
return ""
|
||||
|
||||
@@ -585,6 +617,7 @@ def script_report(text, seconds, jobs=None, video=""):
|
||||
}
|
||||
for moment, start, end in zip(script.moments, starts, ends)
|
||||
]
|
||||
jobs = things_that_perform(text, jobs, video) if jobs is not None else None
|
||||
cast = Cast(number_pictures(jobs, video), video) if jobs is not None else None
|
||||
return {"moments": moments, "seconds": seconds, "warning": script_warning(script, seconds, cast)}
|
||||
|
||||
@@ -732,6 +765,8 @@ def before_the_action(number, text, cast, wearers):
|
||||
# or for the whole clip at Fast preview; with it the new
|
||||
# person is there from the first frame. An @1 looks like line only adds its own words.
|
||||
looks = (f"has the defined appearance from {picture_label(picture.picture)}: {appearance}" if appearance
|
||||
else f"has the shape, colours and drawing style of the character in {picture_label(picture.picture)}"
|
||||
if picture.job == PERFORMER
|
||||
else f"has the face, hair and clothes of the person in {picture_label(picture.picture)}")
|
||||
notes.append(
|
||||
f"{new} occupies the starting position of {who} and follows the same route, movements, timing and pace. "
|
||||
@@ -749,8 +784,6 @@ def before_the_action(number, text, cast, wearers):
|
||||
notes.append((OWN_LINES_ONLY if f"{new} (S" in text else SAYS_NOTHING).format(main=new))
|
||||
if number == 1 and cast.video == KEEP:
|
||||
notes += [CARRIES_ON]
|
||||
if number == 1 and cast.video == REVOICE:
|
||||
notes += [FACES_US.format(main=cast.main)]
|
||||
places = [subject_label(picture.subject) for picture in cast.of(PLACE)]
|
||||
if places and not any(place in text for place in places):
|
||||
notes.append(IN_THE_PLACE.format(place=places[0]))
|
||||
@@ -760,7 +793,8 @@ def before_the_action(number, text, cast, wearers):
|
||||
notes.append(sentence(", ".join(spots[:-1]) + f" and {spots[-1]} of the frame"))
|
||||
for slot, wearer in wearers.items():
|
||||
outfit = cast.subjects[slot]
|
||||
if outfit not in text and (wearer in people or wearer in text):
|
||||
# An edit has no person picture: the person in the video wears an outfit that no line names.
|
||||
if outfit not in text and (wearer in people or wearer in text or (cast.video == EDIT and wearer == cast.in_video)):
|
||||
notes.append(WEARS.format(wearer=wearer, outfit=outfit))
|
||||
return notes
|
||||
|
||||
@@ -768,10 +802,10 @@ def before_the_action(number, text, cast, wearers):
|
||||
def after_the_action(text, cast):
|
||||
"""What H3 needs to hear about a shot once its action has been said."""
|
||||
notes = []
|
||||
if cast.moves and cast.main in text:
|
||||
notes.append(THE_MOVES.format(main=cast.main, moves=cast.moves))
|
||||
if cast.moves and cast.video_person and cast.main in text:
|
||||
notes.append(OWN_MOVES.format(main=cast.main) if cast.own_moves else THE_MOVES.format(main=cast.main, moves=cast.moves))
|
||||
if cast.video == EDIT:
|
||||
notes.append(SAME_AS_VIDEO.format(main=cast.main))
|
||||
notes.append(SAME_AS_VIDEO.format(main=cast.main) if cast.video_person else REST_AS_VIDEO)
|
||||
if cast.video in (SWAP, EDIT):
|
||||
notes.append(FOLLOWS_CAMERA)
|
||||
for picture in cast.of(SHEET):
|
||||
@@ -799,6 +833,7 @@ def shot_lines(script, seconds, say, cast, wearers, move="", hold="", sounds=Tru
|
||||
talking = speakers(script.moments, cast)
|
||||
quiet = " ".join(QUIET.format(speaker=speaker) for speaker in talking if speaker.startswith("<"))
|
||||
kept = still_true(script.moments)
|
||||
carried, cut_off = speech_runs_on(script, seconds)
|
||||
lines = []
|
||||
for number, shot in enumerate(shots_of(script), 1):
|
||||
shot_camera = script.moments[shot[0]].camera
|
||||
@@ -841,8 +876,17 @@ def shot_lines(script, seconds, say, cast, wearers, move="", hold="", sounds=Tru
|
||||
*([hold] if number > 1 else []),
|
||||
capital(script.moments[first].sound) if script.moments[first].sound and not script.silent else "",
|
||||
]
|
||||
lines.append(f"[Shot {number}] " + " ".join(part for part in body if part))
|
||||
if not any(cast.main in line for line in lines):
|
||||
line = " ".join(part for part in body if part)
|
||||
if shot[0] - 1 in carried:
|
||||
# The speaker is mid-line as the shot opens, so the mouth is not closed here.
|
||||
line = f"{LINE_CARRIES} " + re.sub(r" ?<Subject \d+> stays silent with the lips closed\.", "", line, count=1)
|
||||
if shot[-1] in carried:
|
||||
# The speaker is still talking at the cut, so the mouth does not close here.
|
||||
line = re.sub(r" <Subject \d+> closes the lips\.", "", line) + " <scenetrans>"
|
||||
if shot[-1] == cut_off:
|
||||
line = re.sub(r" <Subject \d+> closes the lips\.", "", line) + f" {CUT_OFF}"
|
||||
lines.append(f"[Shot {number}] " + line)
|
||||
if cast.video_person and cast.main_in_shot and not any(cast.main in line for line in lines):
|
||||
lines[0] = lines[0].replace("[Shot 1] ", f"[Shot 1] {IN_THE_SHOT.format(main=cast.main)} ", 1)
|
||||
return lines
|
||||
|
||||
@@ -881,16 +925,19 @@ def summary_of(script, cast, new_places, soundtrack):
|
||||
return f"[{task}] {changes} Everything else in <Video 1> is kept."
|
||||
if cast.video == KEEP:
|
||||
tag = f"{task} + audio reference" if soundtrack else task
|
||||
return f"[{tag}] The target video continues directly from the end of <Video 1> in the same place."
|
||||
opening = with_labels(without_speech(script.moments[0]), cast) or f"{cast.main} speaks"
|
||||
where = "" if cast.of(PLACE) else " in the same place"
|
||||
return f"[{tag}] The target video continues directly from the end of <Video 1>{where}."
|
||||
talking = [who for who in speakers(script.moments, cast) if who.startswith("<")]
|
||||
who_speaks = f"{' and '.join(talking)} speak" if len(talking) > 1 else f"{cast.main} speaks"
|
||||
opening = with_labels(without_speech(script.moments[0]), cast) or who_speaks
|
||||
later = [with_labels(without_speech(script.moments[shot[0]]), cast) for shot in shots_of(script)[1:]]
|
||||
if cast.opens and cast.ends:
|
||||
moving = f"moving from {picture_label(cast.opens.picture)} to {picture_label(cast.ends.picture)}"
|
||||
return f"[{task}] In the target video, {lower_first(opening.rstrip('.!? '))}, {moving}."
|
||||
summary = f"[{task}] In the target video, {lower_first(sentence(opening))}"
|
||||
summary += "".join(f" {AND_THEN.format(line=lower_first(sentence(line)))}" for line in later if line)
|
||||
if cast.moves:
|
||||
summary += " " + THE_MOVES.format(main=cast.main, moves=cast.moves)
|
||||
if cast.moves and cast.video_person:
|
||||
summary += " " + (OWN_MOVES.format(main=cast.main) if cast.own_moves else THE_MOVES.format(main=cast.main, moves=cast.moves))
|
||||
return f"{summary} {EVERY_PLACE}" if new_places else summary
|
||||
|
||||
|
||||
@@ -916,14 +963,15 @@ def script_parts(script, seconds, camera="", look="", sound="", say=None, cast=N
|
||||
cuts = [script.moments[shot[0]] for shot in shots_of(script)[1:]]
|
||||
moved = any(not FRAME_WORDS.match(without_speech(moment)) for moment in cuts)
|
||||
new_places = last > 1 and not cast.of(PLACE) and moved and all(not FRAME_WORDS.match(without_speech(moment)) for moment in cuts)
|
||||
first = next(iter(cast.of(PLACE) or cast.pictures), None)
|
||||
if not style and cast.video == EDIT:
|
||||
# Only a place, a start frame or a person sets the look; an outfit, a thing or a style never does.
|
||||
first = next(iter(cast.of(PLACE) or cast.of(START_JOB) or cast.of(PERSON, SHEET, PERFORMER, END_JOB)), None)
|
||||
if not style and cast.video == EDIT and not cast.of(PLACE):
|
||||
style = VIDEO_LOOKS[EDIT]
|
||||
if not style and len(cast.of(PLACE)) > 1:
|
||||
style = OWN_COLOURS
|
||||
elif not style and not (first and first.job == THING):
|
||||
elif not style and (first or cast.video in VIDEO_LOOKS):
|
||||
words = KEEP_LOOK if cast.opens else KEEP_COLOURS
|
||||
style = VIDEO_LOOKS.get(cast.video) or words.format(picture=picture_label(first.picture if first else 1))
|
||||
style = (cast.video != EDIT and VIDEO_LOOKS.get(cast.video)) or words.format(picture=picture_label(first.picture if first else 1))
|
||||
takes = [EVERY_PLACE] if new_places else []
|
||||
if last == 1 and (len(script.moments) > 1 or cast.moves or script.one_take):
|
||||
takes = [ONE_SHOT]
|
||||
@@ -947,7 +995,7 @@ def script_parts(script, seconds, camera="", look="", sound="", say=None, cast=N
|
||||
"subject_definitions": subjects,
|
||||
"summary": summary_of(script, cast, new_places, soundtrack),
|
||||
"retention_analysis": kept,
|
||||
"detailed_description": "\n".join([" ".join(part for part in (style, move, *takes) if part), *lines]),
|
||||
"detailed_description": "\n".join([*[" ".join(part for part in (style, move, *takes) if part)] * bool(style or move or takes), *lines]),
|
||||
"overall_soundscape": ambience,
|
||||
"non_diegetic_music": tune,
|
||||
}
|
||||
|
||||
@@ -40,6 +40,9 @@ OUTSIDE = (
|
||||
"Image Folder reads only folders inside ComfyUI's input or output folder. "
|
||||
"Drop your folder on the node, or press Add, and its pictures are copied in."
|
||||
)
|
||||
# A drive ("D:") in any part of a folder name, and control characters.
|
||||
_DRIVE = re.compile(r"^[A-Za-z]:")
|
||||
_CONTROL = re.compile(r"[\x00-\x1f\x7f]")
|
||||
NO_FOLDER = 'Image Folder: there is no folder "{folder}" in ComfyUI\'s {source} folder.'
|
||||
NO_PICTURES = 'Image Folder: no pictures in "{folder}". Add some, or choose another folder.'
|
||||
NONE_LEFT = "Image Folder: none of the picked pictures are in the folder anymore. Pick again on the node."
|
||||
@@ -64,7 +67,9 @@ def clean_folder(folder) -> str:
|
||||
if text.startswith("/") or text.startswith("~") or re.match(r"^[A-Za-z]:", text):
|
||||
raise ValueError(OUTSIDE)
|
||||
parts = [part for part in text.split("/") if part not in ("", ".")]
|
||||
if any(part == ".." for part in parts):
|
||||
# ".." climbs out; a drive in any part ("a/D:/x") jumps to another drive
|
||||
# on Windows; a control character is never part of a folder name.
|
||||
if any(part == ".." or _DRIVE.match(part) for part in parts) or _CONTROL.search(text):
|
||||
raise ValueError(OUTSIDE)
|
||||
return "/".join(parts)
|
||||
|
||||
@@ -82,9 +87,10 @@ def place_inside(folder, roots: dict) -> tuple[str, str] | None:
|
||||
This compares text only. A path that is outside those folders is never
|
||||
looked up on disk, and one on another machine is refused from its text."""
|
||||
text = str(folder or "").strip().strip('"').replace("\\", "/")
|
||||
if not looks_full(text) or text.startswith("//"):
|
||||
# A "~" path is not expanded: only a path written out in full can match.
|
||||
if not looks_full(text) or text.startswith("//") or text.startswith("~"):
|
||||
return None
|
||||
full = os.path.normpath(os.path.expanduser(text))
|
||||
full = os.path.normpath(text)
|
||||
for source, root in roots.items():
|
||||
for base in {os.path.normpath(str(root)), os.path.realpath(str(root))}:
|
||||
if full == base:
|
||||
@@ -94,14 +100,27 @@ def place_inside(folder, roots: dict) -> tuple[str, str] | None:
|
||||
return None
|
||||
|
||||
|
||||
def inside_root(top: Path, relative: str) -> Path:
|
||||
"""`relative` below the resolved root `top`, as a resolved path. It is
|
||||
checked as text before the disk is asked to resolve it, and again after,
|
||||
so a link that leads out of the root is refused too."""
|
||||
if not relative:
|
||||
return top
|
||||
lexical = top.joinpath(*relative.split("/"))
|
||||
if top not in lexical.parents:
|
||||
raise ValueError(OUTSIDE)
|
||||
target = lexical.resolve()
|
||||
if target != top and top not in target.parents:
|
||||
raise ValueError(OUTSIDE)
|
||||
return target
|
||||
|
||||
|
||||
def safe_folder(root, folder, source: str = "input") -> Path:
|
||||
"""The folder as a real path below `root`. A link that leads out of the
|
||||
root is refused like any other way out."""
|
||||
relative = clean_folder(folder)
|
||||
top = Path(root).resolve()
|
||||
target = top.joinpath(*relative.split("/")).resolve() if relative else top
|
||||
if target != top and top not in target.parents:
|
||||
raise ValueError(OUTSIDE)
|
||||
target = inside_root(top, relative)
|
||||
if not target.is_dir():
|
||||
raise ValueError(NO_FOLDER.format(folder=relative, source=source))
|
||||
return target
|
||||
|
||||
@@ -6,18 +6,26 @@ that are wanted: they are copied into the output folder under the names the
|
||||
node would have given them. Everything here works from names this module
|
||||
made itself; the only thing a caller sends is the run's token and which
|
||||
pictures to keep.
|
||||
|
||||
What a run holds (its settings, its count, what was kept) stays in this
|
||||
process's memory, never in a file. ComfyUI's temp folder takes uploads, so a
|
||||
list read back from there could be anyone's. The held pictures themselves
|
||||
are named from the token, their number and one of the node's own formats,
|
||||
and must still be in the held folder when they are copied.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import OrderedDict
|
||||
from datetime import datetime
|
||||
import json
|
||||
from pathlib import Path
|
||||
import re
|
||||
import secrets
|
||||
import shutil
|
||||
import threading
|
||||
|
||||
from ._image_save_helpers import (
|
||||
FORMAT_EXTENSIONS,
|
||||
IMAGE_SAVE_LOCK,
|
||||
existing_action,
|
||||
name_stem,
|
||||
@@ -34,6 +42,14 @@ from ._image_save_helpers import (
|
||||
HELD_FOLDER = "ausboss_held"
|
||||
_TOKEN = re.compile(r"^[0-9a-f]{16}$")
|
||||
GONE = "These pictures are no longer held. Run the workflow again."
|
||||
# The held pictures' formats: the extensions Save Image writes, nothing else.
|
||||
HELD_EXTENSIONS = frozenset(FORMAT_EXTENSIONS.values())
|
||||
# A run holds at most this many pictures, and the server remembers this many
|
||||
# runs; the oldest is forgotten first and then says GONE.
|
||||
MAX_HELD_PICTURES = 10000
|
||||
MAX_HELD_RUNS = 256
|
||||
_RUNS: "OrderedDict[str, dict]" = OrderedDict()
|
||||
_RUNS_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def plan_destinations(settings: dict, count: int, now: datetime, root: Path, output_root: Path, picks=None):
|
||||
@@ -88,15 +104,31 @@ def settle_destinations(planned, has_caption: bool, output_root: Path) -> list[s
|
||||
|
||||
|
||||
def _held_name(token: str, index: int, extension: str) -> str:
|
||||
if not _TOKEN.match(token) or extension not in HELD_EXTENSIONS or not 0 <= index < MAX_HELD_PICTURES:
|
||||
raise ValueError("Save Image: that is not a picture it is holding.")
|
||||
return f"{token}_{index:03d}.{extension}"
|
||||
|
||||
|
||||
def _held_source(folder: Path, token: str, index: int, extension: str) -> Path:
|
||||
"""A held picture's file, which must still sit directly in the held folder."""
|
||||
source = folder / _held_name(token, index, extension)
|
||||
if not source.is_file() or source.is_symlink() or source.resolve().parent != folder.resolve():
|
||||
raise LookupError(GONE)
|
||||
return source
|
||||
|
||||
|
||||
def hold_pictures(temp_root, frames, settings: dict, encode) -> tuple[str, list[dict]]:
|
||||
"""Write a run's pictures to the temp folder and remember how to name them.
|
||||
|
||||
`encode(path, frame)` writes one picture in the run's format. Returns
|
||||
the run's token and the entries the node shows them from.
|
||||
"""
|
||||
frames = list(frames)
|
||||
if len(frames) > MAX_HELD_PICTURES:
|
||||
raise ValueError(
|
||||
f"Save Image: {len(frames)} pictures are too many to hold (at most {MAX_HELD_PICTURES}). "
|
||||
"Turn Auto save on to save them straight away."
|
||||
)
|
||||
folder = Path(temp_root) / HELD_FOLDER
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
token = secrets.token_hex(8)
|
||||
@@ -105,8 +137,10 @@ def hold_pictures(temp_root, frames, settings: dict, encode) -> tuple[str, list[
|
||||
name = _held_name(token, index, settings["extension"])
|
||||
encode(folder / name, frame)
|
||||
previews.append({"filename": name, "subfolder": HELD_FOLDER, "type": "temp"})
|
||||
manifest = dict(settings, count=len(previews), kept={})
|
||||
(folder / f"{token}.json").write_text(json.dumps(manifest), encoding="utf-8")
|
||||
with _RUNS_LOCK:
|
||||
_RUNS[token] = dict(settings, count=len(previews), kept={})
|
||||
while len(_RUNS) > MAX_HELD_RUNS:
|
||||
_RUNS.popitem(last=False)
|
||||
return token, previews
|
||||
|
||||
|
||||
@@ -119,16 +153,18 @@ def keep_pictures(temp_root, output_root, token, picks=None, now: datetime | Non
|
||||
if not isinstance(token, str) or not _TOKEN.match(token):
|
||||
raise ValueError("Save Image: that is not a run it is holding.")
|
||||
folder = Path(temp_root) / HELD_FOLDER
|
||||
manifest_path = folder / f"{token}.json"
|
||||
if not manifest_path.is_file():
|
||||
with _RUNS_LOCK:
|
||||
manifest = _RUNS.get(token)
|
||||
if manifest is None:
|
||||
raise LookupError(GONE)
|
||||
manifest = json.loads(manifest_path.read_text(encoding="utf-8"))
|
||||
count = int(manifest["count"])
|
||||
kept = {int(index): str(path) for index, path in dict(manifest.get("kept") or {}).items()}
|
||||
kept = dict(manifest["kept"])
|
||||
|
||||
if picks is None:
|
||||
wanted = list(range(count))
|
||||
else:
|
||||
if len(picks) > MAX_HELD_PICTURES:
|
||||
raise ValueError("Save Image: that picture is not in this run.")
|
||||
wanted = []
|
||||
for pick in picks:
|
||||
if isinstance(pick, bool) or not isinstance(pick, int) or not 0 <= pick < count:
|
||||
@@ -151,17 +187,14 @@ def keep_pictures(temp_root, output_root, token, picks=None, now: datetime | Non
|
||||
if action == "skip":
|
||||
skipped.append(index)
|
||||
continue
|
||||
source = folder / _held_name(token, index, manifest["extension"])
|
||||
if not source.is_file():
|
||||
raise LookupError(GONE)
|
||||
source = _held_source(folder, token, index, manifest["extension"])
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
shutil.copyfile(source, path)
|
||||
if has_caption:
|
||||
sidecar_path(path).write_text(caption, encoding="utf-8")
|
||||
kept[index] = path.resolve().relative_to(resolved_root).as_posix()
|
||||
saved.append(index)
|
||||
manifest["kept"] = {str(index): path for index, path in kept.items()}
|
||||
manifest_path.write_text(json.dumps(manifest), encoding="utf-8")
|
||||
manifest["kept"] = kept
|
||||
return {
|
||||
"count": count,
|
||||
"saved": [{"index": index, "path": kept[index]} for index in saved],
|
||||
|
||||
@@ -653,10 +653,13 @@ def apply_stitch(
|
||||
exact with the toggle off, since the spread deliberately rewrites the
|
||||
feathered band.
|
||||
|
||||
``color_match`` (0..1) shifts the patch's tone by the offset measured
|
||||
in the feathered band (:func:`estimate_tone_offset`) before blending,
|
||||
so an outpaint whose new bands came out a touch lighter or warmer than
|
||||
the picture lands on the picture's own tone. 0 leaves the patch alone.
|
||||
``color_match`` (0..1) shifts the patch's tone by the measured drift
|
||||
before blending, so a new area that came out a touch lighter or warmer
|
||||
than the picture lands on the picture's own tone. An outpaint is read
|
||||
across its seams (:func:`tone_offset_field`); an inpaint, which has no
|
||||
source rectangle, inside its painted area (:func:`inner_tone_offset`),
|
||||
or in the feathered band (:func:`estimate_tone_offset`) when the
|
||||
painted area holds new content. 0 leaves the patch alone.
|
||||
|
||||
``seam`` "classic" is everything above. "blend in" replaces it on a
|
||||
stitcher that knows where the picture's edge is (Load Image + Pad and
|
||||
@@ -788,7 +791,13 @@ def apply_stitch(
|
||||
# Trimmed like the blend when fewer frames came back.
|
||||
generated = generated[:frames] if generated.shape[0] > frames else generated
|
||||
generated = generated[:, cy : cy + ch, cx : cx + cw].to(out.device)
|
||||
field = tone_offset_field(patch, out[:, cy : cy + ch, cx : cx + cw, :], alpha, bbox, generated)
|
||||
region = out[:, cy : cy + ch, cx : cx + cw, :]
|
||||
field = tone_offset_field(patch, region, alpha, bbox, generated)
|
||||
if bbox is None:
|
||||
# An inpaint has no source rectangle to read a seam across. Its
|
||||
# drift is read just inside the painted area, where it holds.
|
||||
inner, held = inner_tone_offset(patch, region, alpha, came_back)
|
||||
field = torch.where(held.view(-1, 1, 1, 1), inner.view(-1, 1, 1, 3).expand_as(field), field)
|
||||
# Weighted by the blend: a fully generated pixel drifted by the whole
|
||||
# field, a band pixel the sampler mixed at alpha drifted by alpha of
|
||||
# it, and the blend below scales the correction by alpha once more.
|
||||
@@ -841,6 +850,175 @@ def estimate_tone_offset(
|
||||
return torch.where(measured, offset, torch.zeros_like(offset))
|
||||
|
||||
|
||||
# An inpaint's drift is read in the painted area's own rim: this many
|
||||
# pixels in from where the paste becomes whole. The rim is where the new
|
||||
# part most often still shows what was there before (skin beside skin).
|
||||
INNER_RIM_PX = 32
|
||||
|
||||
# Rim pixels within this many LAB units of the rim's median change count as
|
||||
# the same content, drifted. The reading is only used when at least
|
||||
# INNER_MIN_SHARE of the rim agrees and there are INNER_MIN_PIXELS of it.
|
||||
INNER_SAME_CONTENT = 8.0
|
||||
INNER_MIN_SHARE = 0.5
|
||||
INNER_MIN_PIXELS = 64
|
||||
|
||||
# A larger change than this is new content (a jacket over skin), not a
|
||||
# drift: matching it would pull the new thing toward the old colour.
|
||||
INNER_MAX_SHIFT = 20.0
|
||||
|
||||
|
||||
# New content has no "before" in its own place (skin where a shirt was), so
|
||||
# it is read against the untouched picture beside it instead: the painted
|
||||
# area and the picture, each averaged within this many pixels of a point.
|
||||
NEIGHBOUR_REACH_PX = 48
|
||||
|
||||
# A point counts as on the seam when each side fills this much of its reach.
|
||||
NEIGHBOUR_MIN_SIDE = 0.1
|
||||
|
||||
# Seam points within INNER_MAX_SHIFT of each other show the same thing on
|
||||
# both sides (skin beside skin). Fewer than this share of the seam and the
|
||||
# new part is something else all round, so nothing is matched.
|
||||
NEIGHBOUR_MIN_SHARE = 0.25
|
||||
|
||||
|
||||
def _neighbour_offset(
|
||||
new: torch.Tensor, old: torch.Tensor, whole: torch.Tensor, outside: torch.Tensor
|
||||
) -> torch.Tensor | None:
|
||||
"""LAB offset of new content against the untouched picture beside it.
|
||||
|
||||
``new`` and ``old`` are one frame in LAB [H, W, 3]; ``whole`` marks the
|
||||
fully pasted pixels and ``outside`` the untouched ones. Along the seam
|
||||
the painted side's local mean is compared with the picture's. Where the
|
||||
two are alike the model drew more of what is already there, and the gap
|
||||
between them is its drift. None when too little of the seam is alike or
|
||||
the alike part does not agree on one gap.
|
||||
"""
|
||||
size = 2 * NEIGHBOUR_REACH_PX + 1
|
||||
|
||||
def local(values: torch.Tensor, mask: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
weight = mask.to(values.dtype).unsqueeze(-1)
|
||||
stack = torch.cat([values * weight, weight], dim=-1).movedim(-1, 0).unsqueeze(0)
|
||||
stack = functional.avg_pool2d(stack, (1, size), stride=1, padding=(0, NEIGHBOUR_REACH_PX))
|
||||
stack = functional.avg_pool2d(stack, (size, 1), stride=1, padding=(NEIGHBOUR_REACH_PX, 0))[0]
|
||||
return (stack[:3] / stack[3:].clamp_min(1e-6)).movedim(0, -1), stack[3]
|
||||
|
||||
painted, painted_share = local(new, whole)
|
||||
picture, picture_share = local(old, outside)
|
||||
seam = (painted_share >= NEIGHBOUR_MIN_SIDE) & (picture_share >= NEIGHBOUR_MIN_SIDE)
|
||||
gaps = (painted - picture)[seam]
|
||||
if gaps.shape[0] < INNER_MIN_PIXELS:
|
||||
return None
|
||||
alike = gaps.norm(dim=-1) <= INNER_MAX_SHIFT
|
||||
if int(alike.sum()) < INNER_MIN_PIXELS or float(alike.float().mean()) < NEIGHBOUR_MIN_SHARE:
|
||||
return None
|
||||
gaps = gaps[alike]
|
||||
agree = (gaps - gaps.median(dim=0).values).norm(dim=-1) < INNER_SAME_CONTENT
|
||||
if float(agree.float().mean()) < INNER_MIN_SHARE:
|
||||
return None
|
||||
return gaps[agree].median(dim=0).values
|
||||
|
||||
|
||||
def _area_resize(image: torch.Tensor, width: int, height: int) -> torch.Tensor:
|
||||
"""Shrink [B, H, W, C] by averaging the pixels each new one covers."""
|
||||
return functional.interpolate(image.movedim(-1, 1), size=(height, width), mode="area").movedim(1, -1)
|
||||
|
||||
|
||||
def inner_tone_offset(
|
||||
inpainted: torch.Tensor,
|
||||
canvas: torch.Tensor,
|
||||
blend: torch.Tensor,
|
||||
sampled: tuple[int, int] | None = None,
|
||||
) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
"""Per-frame LAB drift of an inpaint, read inside the painted area.
|
||||
|
||||
:func:`estimate_tone_offset` reads the feathered band and assumes the
|
||||
sampler mixed original and generated there by ``blend``. A workflow that
|
||||
samples with its own, wider or softer noise mask breaks that: the model
|
||||
leaves the band close to the original and drifts further in, so the band
|
||||
reads a fraction of the drift (issue #361: 0.8 of 2.5 LAB units).
|
||||
|
||||
This reads the same pixels before and after in the rim of the fully
|
||||
pasted area (``blend`` > 0.999, INNER_RIM_PX deep): the median change,
|
||||
then the median again over the pixels that agree with it. Returns the
|
||||
offsets [B, 3] and which frames hold [B]. A frame whose rim mostly
|
||||
disagrees, or changed by more than INNER_MAX_SHIFT, is new content
|
||||
rather than drift. That frame is read against the untouched picture
|
||||
beside it (:func:`_neighbour_offset`), and holds at zero, left as
|
||||
painted, when the picture beside it shows something else. Only a rim too
|
||||
small to judge does not hold, and the caller keeps the band reading for it.
|
||||
|
||||
``sampled`` is the (width, height) the model worked at. A crop from a
|
||||
large picture is pasted back several times that size, and a rim measured
|
||||
in pasted pixels would be a sliver at the very edge of what the model
|
||||
painted, where it drifts least. The reading is taken at the model's
|
||||
size instead, which also keeps its cost the same for any picture.
|
||||
"""
|
||||
from ._color_helpers import rgb_to_lab
|
||||
|
||||
if sampled is not None:
|
||||
width, height = int(sampled[0]), int(sampled[1])
|
||||
if 0 < width < inpainted.shape[2] and 0 < height < inpainted.shape[1]:
|
||||
inpainted = _area_resize(inpainted, width, height)
|
||||
canvas = _area_resize(canvas, width, height)
|
||||
blend = _area_resize(blend.unsqueeze(-1), width, height).squeeze(-1)
|
||||
new = rgb_to_lab(inpainted[..., :3])
|
||||
old = rgb_to_lab(canvas[..., :3])
|
||||
diff = new - old
|
||||
frames = diff.shape[0]
|
||||
whole = (blend > 0.999).to(diff.dtype)
|
||||
size = 2 * INNER_RIM_PX + 1
|
||||
# A pixel is deeper than the rim when nothing within INNER_RIM_PX of it
|
||||
# is outside the whole paste. The crop's own border does not count as
|
||||
# outside: the painted area simply runs off the crop there.
|
||||
# One pass along each axis: the same square reach at a fraction of the cost.
|
||||
outside = (1.0 - whole).unsqueeze(1)
|
||||
outside = functional.max_pool2d(outside, (1, size), stride=1, padding=(0, INNER_RIM_PX))
|
||||
outside = functional.max_pool2d(outside, (size, 1), stride=1, padding=(INNER_RIM_PX, 0))
|
||||
deep = 1.0 - outside.squeeze(1)
|
||||
rim = (whole > 0) & (deep < 0.5)
|
||||
rim = torch.where(rim.flatten(1).any(dim=1).view(-1, 1, 1), rim, whole > 0)
|
||||
pasted = whole > 0
|
||||
untouched = blend < 0.001
|
||||
if rim.shape[0] != frames:
|
||||
rim = rim.expand(frames, -1, -1)
|
||||
pasted = pasted.expand(frames, -1, -1)
|
||||
untouched = untouched.expand(frames, -1, -1)
|
||||
# See-through pixels hold whatever colour sits under the clear part.
|
||||
if inpainted.shape[-1] > 3:
|
||||
rim = rim & (inpainted[..., 3] >= 0.999)
|
||||
pasted = pasted & (inpainted[..., 3] >= 0.999)
|
||||
if canvas.shape[-1] > 3:
|
||||
rim = rim & (canvas[..., 3] >= 0.999)
|
||||
untouched = untouched & (canvas[..., 3] >= 0.999)
|
||||
|
||||
def beside(index: int) -> torch.Tensor:
|
||||
gap = _neighbour_offset(new[index], old[index], pasted[index], untouched[index])
|
||||
return offsets[index] if gap is None else gap
|
||||
|
||||
offsets = torch.zeros((frames, 3), dtype=diff.dtype, device=diff.device)
|
||||
held = torch.zeros((frames,), dtype=torch.bool, device=diff.device)
|
||||
for index in range(frames):
|
||||
changes = diff[index][rim[index]]
|
||||
if changes.shape[0] < INNER_MIN_PIXELS:
|
||||
continue
|
||||
# From here the rim is large enough to judge, so this reading stands
|
||||
# and the band is not asked. New content keeps a zero shift: the band
|
||||
# of a repainted edge reads the new colour itself as drift (a yellow
|
||||
# swimsuit over skin turned the whole patch blue).
|
||||
held[index] = True
|
||||
middle = changes.median(dim=0).values
|
||||
agree = (changes - middle).norm(dim=-1) < INNER_SAME_CONTENT
|
||||
if float(agree.float().mean()) < INNER_MIN_SHARE:
|
||||
offsets[index] = beside(index)
|
||||
continue
|
||||
offset = changes[agree].median(dim=0).values
|
||||
if float(offset.norm()) > INNER_MAX_SHIFT:
|
||||
offsets[index] = beside(index)
|
||||
continue
|
||||
offsets[index] = offset
|
||||
return offsets, held
|
||||
|
||||
|
||||
def _band_weights(blend: torch.Tensor) -> torch.Tensor:
|
||||
return torch.where((blend > 0.001) & (blend < 0.999), blend, torch.zeros_like(blend))
|
||||
|
||||
|
||||
@@ -61,11 +61,8 @@ def list_lama_models() -> list[str]:
|
||||
|
||||
def resolve_lama_model(model_name: str) -> Path:
|
||||
register_lama_model_folder()
|
||||
if folder_paths is None:
|
||||
path = Path(model_name).expanduser().resolve()
|
||||
if path.is_file():
|
||||
return path
|
||||
else:
|
||||
# Only a name from ComfyUI's lama model folder: a typed path is never opened.
|
||||
if folder_paths is not None:
|
||||
resolved = folder_paths.get_full_path(MODEL_FOLDER, model_name)
|
||||
if resolved:
|
||||
return Path(resolved).resolve()
|
||||
|
||||
+23
-19
@@ -13,6 +13,8 @@ from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ._route_errors import error_detail
|
||||
|
||||
try:
|
||||
import folder_paths
|
||||
except ImportError: # pragma: no cover - only outside ComfyUI
|
||||
@@ -136,27 +138,29 @@ _remap_noted: set[str] = set()
|
||||
def resolve_lora_path(name: str) -> Path:
|
||||
if folder_paths is None:
|
||||
raise ValueError("folder_paths is unavailable outside ComfyUI.")
|
||||
full = folder_paths.get_full_path("loras", name)
|
||||
if not full:
|
||||
# Only a name on ComfyUI's own LoRA list reaches the disk. A route's
|
||||
# name or a widget value is never joined to a folder as given: it is
|
||||
# matched against the list first, and anything else is refused here.
|
||||
candidate, status = match_lora_name(str(name), list_loras())
|
||||
if status == "remapped":
|
||||
# The saved path is stale; a moved file should just keep working.
|
||||
candidate, status = match_lora_name(str(name), list_loras())
|
||||
if status == "remapped":
|
||||
if name not in _remap_noted:
|
||||
_remap_noted.add(name)
|
||||
print(
|
||||
f"[AusBoss] LoRA Loader: '{name}' is not at its saved "
|
||||
f"path; using '{candidate}'."
|
||||
)
|
||||
full = folder_paths.get_full_path("loras", candidate)
|
||||
elif status == "ambiguous":
|
||||
raise ValueError(
|
||||
f"LoRA name matches several files in models/loras: {name}"
|
||||
if name not in _remap_noted:
|
||||
_remap_noted.add(name)
|
||||
print(
|
||||
f"[AusBoss] LoRA Loader: '{name}' is not at its saved "
|
||||
f"path; using '{candidate}'."
|
||||
)
|
||||
elif status == "ambiguous":
|
||||
raise ValueError(
|
||||
f"LoRA name matches several files in models/loras: {name}"
|
||||
)
|
||||
elif status != "exact":
|
||||
raise ValueError(f"LoRA file not found in models/loras: {name}")
|
||||
full = folder_paths.get_full_path("loras", candidate)
|
||||
if not full:
|
||||
raise ValueError(f"LoRA file not found in models/loras: {name}")
|
||||
# Containment is checked on the path as written, with ".." folded but
|
||||
# links not followed: a LoRA kept on another drive and linked into
|
||||
# models/loras is the user's own choice, and ComfyUI loads it too.
|
||||
# The listed file must also sit below one of the loras folders, read
|
||||
# from the path as ComfyUI built it, with ".." folded.
|
||||
path = Path(os.path.abspath(full))
|
||||
roots = [Path(os.path.abspath(root)) for root in folder_paths.get_folder_paths("loras")]
|
||||
if not any(root in path.parents for root in roots):
|
||||
@@ -388,8 +392,8 @@ def apply_lora_stack(
|
||||
except MemoryError:
|
||||
raise
|
||||
except Exception as exc:
|
||||
# ComfyUI's own message here is PyTorch's and never names the file.
|
||||
detail = f"{type(exc).__name__}: {' '.join(str(exc).split())}"
|
||||
# ComfyUI can add the file's path; error_detail drops words that name one.
|
||||
detail = error_detail(exc)
|
||||
raise ValueError(
|
||||
unreadable_lora_message(row["name"], unreadable_lora_reason(path), detail)
|
||||
) from exc
|
||||
|
||||
@@ -242,13 +242,20 @@ def _locate_comfy_file(text: str, rule: str) -> tuple[str, str, Path]:
|
||||
if (
|
||||
parts[0] == "~"
|
||||
or any(part in ("", ".", "..") for part in parts)
|
||||
or any(re.match(r"[A-Za-z]:", part) for part in parts)
|
||||
or (os.name == "nt" and ":" in inside)
|
||||
or re.search(r"[\x00-\x1f\x7f]", inside)
|
||||
):
|
||||
raise ValueError(rule)
|
||||
root = _server_folders().get(folder)
|
||||
if root is None:
|
||||
raise ValueError(rule)
|
||||
path = root.joinpath(*parts).resolve()
|
||||
# Checked as text before the disk resolves it, and again after, so a
|
||||
# link that leads out of the folder is refused too.
|
||||
lexical = root.joinpath(*parts)
|
||||
if root not in lexical.parents:
|
||||
raise ValueError(rule)
|
||||
path = lexical.resolve()
|
||||
if root not in path.parents:
|
||||
raise ValueError(rule)
|
||||
return folder, inside, path
|
||||
|
||||
+15
-3
@@ -1,11 +1,12 @@
|
||||
"""What a route says back when a request fails."""
|
||||
"""What a route or a node says back when something fails."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
# A whole path: a word that opens with "/", "\" or a drive letter.
|
||||
_SERVER_PATH = re.compile(r"""(?:^|[\s'"(\[=:,;])(?:[A-Za-z]:[\\/]|[\\/])[^\s'"]""")
|
||||
# A whole path: a word that opens with "/", "\", "~/" or a drive letter,
|
||||
# after anything but a letter, digit, "_" or "." ("and/or" and "3/4" stay).
|
||||
_SERVER_PATH = re.compile(r"""(?:^|[^\w.])(?:[A-Za-z]:[\\/]|~[\\/]|[\\/])[^\s'"]""")
|
||||
|
||||
|
||||
def safe_route_error(exc: Exception, fallback: str = "The request failed.") -> str:
|
||||
@@ -16,3 +17,14 @@ def safe_route_error(exc: Exception, fallback: str = "The request failed.") -> s
|
||||
if not text or _SERVER_PATH.search(text):
|
||||
return fallback
|
||||
return text
|
||||
|
||||
|
||||
def error_detail(exc: BaseException) -> str:
|
||||
"""A library's error for a node's message: its kind and its words, or
|
||||
its kind alone when the words name a place on the server's disk. Node
|
||||
errors reach the browser that queued the run and stay in the history."""
|
||||
kind = type(exc).__name__
|
||||
text = " ".join(str(exc).split())
|
||||
if not text or _SERVER_PATH.search(text):
|
||||
return kind
|
||||
return f"{kind}: {text}"
|
||||
|
||||
+19
-11
@@ -480,17 +480,25 @@ def transform_tensor_batch_chunked(
|
||||
return output, mask, geometry
|
||||
|
||||
|
||||
def stable_file_fingerprint(path: str | os.PathLike[str], inputs: dict[str, object]) -> str:
|
||||
normalized_path = str(Path(path).resolve()).casefold()
|
||||
try:
|
||||
stat = os.stat(path)
|
||||
file_state: dict[str, object] = {
|
||||
"path": normalized_path,
|
||||
"mtime_ns": stat.st_mtime_ns,
|
||||
"size": stat.st_size,
|
||||
}
|
||||
except OSError:
|
||||
file_state = {"path": normalized_path, "missing": True}
|
||||
def stable_file_fingerprint(path: Path | None, inputs: dict[str, object]) -> str:
|
||||
"""A key that changes when the inputs or the file do.
|
||||
|
||||
`path` is what resolve_input_path or resolve_video_path returned, or
|
||||
None when they refused the value. Only such a path is ever looked at on
|
||||
disk: a refused value is a string, and a string is only hashed, so a
|
||||
whole path or a network share typed into a widget is never opened."""
|
||||
if isinstance(path, Path):
|
||||
try:
|
||||
stat = os.stat(path)
|
||||
file_state: dict[str, object] = {
|
||||
"path": str(path).casefold(),
|
||||
"mtime_ns": stat.st_mtime_ns,
|
||||
"size": stat.st_size,
|
||||
}
|
||||
except OSError:
|
||||
file_state = {"path": str(path).casefold(), "missing": True}
|
||||
else:
|
||||
file_state = {"unresolved": True}
|
||||
payload = {"file": file_state, "inputs": inputs}
|
||||
encoded = json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True)
|
||||
return hashlib.sha256(encoded.encode("utf-8")).hexdigest()
|
||||
|
||||
@@ -13,6 +13,7 @@ import numpy as np
|
||||
import torch
|
||||
from av.video.reformatter import ColorRange, Colorspace
|
||||
|
||||
from ._route_errors import error_detail
|
||||
from ._execution_helpers import advance_progress, frame_progress, raise_if_interrupted
|
||||
|
||||
AAC_FRAME_SIZE = 1024
|
||||
@@ -363,9 +364,9 @@ def _encoder_start_error(codec: str, exc: Exception) -> RuntimeError:
|
||||
return RuntimeError(
|
||||
"Save Video: the nvenc formats need an NVIDIA GPU with a working "
|
||||
"driver. Pick a software format such as 'mp4 h264' instead. "
|
||||
f"(The '{codec}' encoder said: {exc})"
|
||||
f"(The '{codec}' encoder said: {error_detail(exc)})"
|
||||
)
|
||||
return RuntimeError(f"Save Video could not start the '{codec}' encoder: {exc}.")
|
||||
return RuntimeError(f"Save Video could not start the '{codec}' encoder: {error_detail(exc)}.")
|
||||
|
||||
|
||||
def _add_video_stream(container, spec: VideoFormat, rate, width: int, height: int, crf: int):
|
||||
|
||||
@@ -164,7 +164,7 @@ class AusBossH3PromptWriter:
|
||||
"INT",
|
||||
{
|
||||
"default": 8,
|
||||
"min": 5,
|
||||
"min": 4,
|
||||
"max": 15,
|
||||
"step": 1,
|
||||
"tooltip": "How long the clip is. Plan on about one moment a second at most.",
|
||||
@@ -254,8 +254,8 @@ class AusBossH3PromptWriter:
|
||||
"default": VIDEO_JOBS[0],
|
||||
"tooltip": (
|
||||
"What the video is for: copy its moves, swap the person in it, keep it going, give it "
|
||||
"a new voice, or change something in it and keep the person (a hat, a shirt). Swap, "
|
||||
"new voice and change something ignore Seconds and keep the video's length."
|
||||
"a new voice, or change something in it and keep the person (a hat, a shirt). Copy its "
|
||||
"moves, swap, new voice and change something ignore Seconds and keep the video's length."
|
||||
),
|
||||
},
|
||||
),
|
||||
@@ -393,7 +393,7 @@ def _register_routes():
|
||||
async def read(request):
|
||||
try:
|
||||
body = await request.json()
|
||||
seconds = min(max(int(body.get("seconds", 8)), 5), LONGEST_SECONDS)
|
||||
seconds = min(max(int(body.get("seconds", 8)), 4), LONGEST_SECONDS)
|
||||
jobs = {int(slot): job for slot, job in dict(body.get("jobs") or {}).items() if job in JOBS and int(slot) in PICTURES}
|
||||
video = body.get("video") if body.get("video") in VIDEO_JOBS else ""
|
||||
return web.json_response(script_report(str(body.get("script", ""))[:LONGEST_SCRIPT], seconds, jobs, video))
|
||||
|
||||
@@ -168,7 +168,7 @@ class AusBossImageCropRotatePad:
|
||||
try:
|
||||
path = resolve_input_path(image)
|
||||
except Exception:
|
||||
path = image or ""
|
||||
path = None # refused: hashed by its text only, never opened
|
||||
spec = spec_from_values(**values)
|
||||
# Every widget value is already part of ComfyUI's cache key, so a
|
||||
# changed transform or budget re-runs the node without this; the
|
||||
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from ._execution_helpers import ignore_extra_inputs
|
||||
@@ -18,6 +20,7 @@ from ._image_folder_helpers import (
|
||||
choose,
|
||||
clean_folder,
|
||||
fingerprint,
|
||||
inside_root,
|
||||
list_folders,
|
||||
list_pictures,
|
||||
load_picture,
|
||||
@@ -69,6 +72,13 @@ def _picked(source, folder, subfolders, sort, pictures) -> list[dict]:
|
||||
return choose(listed, parse_picked(pictures))
|
||||
|
||||
|
||||
def _version_tag(item: dict) -> str:
|
||||
"""A short tag that changes when the picture does, for the browser's
|
||||
thumbnail cache. It carries no file size or time."""
|
||||
seen = f"{item['mtime_ns']}-{item['size']}".encode()
|
||||
return hashlib.sha1(seen, usedforsecurity=False).hexdigest()[:12]
|
||||
|
||||
|
||||
def _register_routes() -> None:
|
||||
"""The gallery's three questions: which folders, which pictures, and a
|
||||
small copy of one. Every path is attacker-controlled; each is resolved
|
||||
@@ -117,7 +127,7 @@ def _register_routes() -> None:
|
||||
"source": source,
|
||||
"folder": shown,
|
||||
"full": len(listed) >= MAX_PICTURES,
|
||||
"pictures": [{"name": item["name"], "v": f"{item['mtime_ns']}-{item['size']}"} for item in listed],
|
||||
"pictures": [{"name": item["name"], "v": _version_tag(item)} for item in listed],
|
||||
})
|
||||
|
||||
@routes.get("/ausboss/image_folder/thumb")
|
||||
@@ -129,8 +139,10 @@ def _register_routes() -> None:
|
||||
base = clean_folder(query(request, "folder"))
|
||||
# The name is a path below the folder; it goes through the same door.
|
||||
relative = clean_folder(f"{base}/{name}" if base else name)
|
||||
path = root.joinpath(*relative.split("/")).resolve()
|
||||
if root not in path.parents or not path.is_file() or not path.name.lower().endswith(IMAGE_EXTENSIONS):
|
||||
if not relative or not relative.lower().endswith(IMAGE_EXTENSIONS):
|
||||
return web.json_response({"error": "no such picture"}, status=404)
|
||||
path = inside_root(root, relative)
|
||||
if not path.is_file() or not path.name.lower().endswith(IMAGE_EXTENSIONS):
|
||||
return web.json_response({"error": "no such picture"}, status=404)
|
||||
try:
|
||||
edge = int(query(request, "size") or 160)
|
||||
|
||||
@@ -253,7 +253,7 @@ class AusBossLoadImage:
|
||||
try:
|
||||
path = resolve_input_path(image)
|
||||
except Exception:
|
||||
path = image or ""
|
||||
path = None # refused: hashed by its text only, never opened
|
||||
return stable_file_fingerprint(path, {"image": image, **values})
|
||||
|
||||
|
||||
|
||||
@@ -403,7 +403,7 @@ class AusBossLoadImagePad:
|
||||
try:
|
||||
path = resolve_input_path(image)
|
||||
except Exception:
|
||||
path = image or ""
|
||||
path = None # refused: hashed by its text only, never opened
|
||||
return stable_file_fingerprint(path, {"image": image, **values})
|
||||
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ import sys
|
||||
|
||||
import torch
|
||||
|
||||
from ._route_errors import error_detail
|
||||
from ._mask_by_name_helpers import (
|
||||
IF_NOTHING,
|
||||
NEEDS_SAM3,
|
||||
@@ -238,7 +239,7 @@ class AusBossMaskByName:
|
||||
raise
|
||||
# A model or text reader that is not SAM 3's fails somewhere
|
||||
# inside; say what to wire instead of showing that.
|
||||
raise RuntimeError(f"{NOT_SAM3} ({type(error).__name__}: {error})") from error
|
||||
raise RuntimeError(f"{NOT_SAM3} ({error_detail(error)})") from error
|
||||
masks, boxes = output.result
|
||||
counts = [len(frame) for frame in boxes]
|
||||
# How sure SAM 3 was of each match in the first picture, best first.
|
||||
|
||||
@@ -15,6 +15,7 @@ import torch
|
||||
import torch.nn.functional as F
|
||||
from PIL import Image
|
||||
|
||||
from ._route_errors import error_detail, safe_route_error
|
||||
from ._execution_helpers import advance_progress, progress_bar, raise_if_interrupted, warn_once
|
||||
from ._fine_edge_helpers import OFF as FINE_OFF
|
||||
from ._fine_edge_helpers import (
|
||||
@@ -485,7 +486,7 @@ class _Session:
|
||||
except Exception as error: # noqa: BLE001 - told apart just below
|
||||
if _stopped_by_user(error) or _out_of_memory(error):
|
||||
raise
|
||||
raise RuntimeError(f"{NOT_SAM3} ({type(error).__name__}: {error})") from error
|
||||
raise RuntimeError(f"{NOT_SAM3} ({error_detail(error)})") from error
|
||||
_WORDS[key] = held
|
||||
while len(_WORDS) > WORDS_KEPT:
|
||||
_WORDS.popitem(last=False)
|
||||
@@ -561,14 +562,14 @@ class _Session:
|
||||
if self._open is None:
|
||||
# A model or a text reader that is not SAM 3's fails
|
||||
# somewhere inside; say what to wire instead of showing that.
|
||||
raise RuntimeError(f"{NOT_SAM3} ({type(error).__name__}: {error})") from error
|
||||
raise RuntimeError(f"{NOT_SAM3} ({error_detail(error)})") from error
|
||||
self.close()
|
||||
try:
|
||||
answer = self._call(kind, frame, what)
|
||||
except Exception as error: # noqa: BLE001
|
||||
if _stopped_by_user(error) or _out_of_memory(error):
|
||||
raise
|
||||
raise RuntimeError(f"{NOT_SAM3} ({type(error).__name__}: {error})") from error
|
||||
raise RuntimeError(f"{NOT_SAM3} ({error_detail(error)})") from error
|
||||
# The plain way answers where the quick way failed.
|
||||
self._distrust("a question failed with it")
|
||||
return answer
|
||||
@@ -1288,7 +1289,7 @@ class AusBossMaskByTouch:
|
||||
if item.kind not in ("subject", "background") or not asked or _stopped_by_user(error):
|
||||
raise
|
||||
area, used, which = None, "none", ()
|
||||
subject_trouble = str(error)
|
||||
subject_trouble = safe_route_error(error, type(error).__name__)
|
||||
if index == 0:
|
||||
share = 0.0 if area is None else round(float(area.sum()) / (height * width), 5)
|
||||
item_notes.append({"ok": used != "none", "share": share, "as": used})
|
||||
@@ -1348,7 +1349,7 @@ class AusBossMaskByTouch:
|
||||
# and the picture says why.
|
||||
if not asked or _stopped_by_user(error):
|
||||
raise
|
||||
fine, fine_trouble = False, str(error)
|
||||
fine, fine_trouble = False, safe_route_error(error, type(error).__name__)
|
||||
if fine and soft:
|
||||
# Where a soft brush had the last word its fade stays as painted.
|
||||
fade = torch.stack(painted) & (raw > 0) & (raw < 1)
|
||||
|
||||
@@ -152,7 +152,7 @@ class AusBossVideoCropRotatePad:
|
||||
try:
|
||||
path = resolve_video_path(source_mode, video, local_path)
|
||||
except Exception:
|
||||
path = local_path if source_mode == "local path" else video
|
||||
path = None # refused: hashed by its text only, never opened
|
||||
spec = spec_from_values(**values)
|
||||
resize = {name: values.get(name) for name in resize_inputs()}
|
||||
return stable_file_fingerprint(
|
||||
@@ -160,6 +160,7 @@ class AusBossVideoCropRotatePad:
|
||||
{
|
||||
"video": video,
|
||||
"source_mode": source_mode,
|
||||
"local_path": local_path,
|
||||
"seek_mode": seek_mode,
|
||||
"frame_index": int(frame_index),
|
||||
"frame_time": round(float(frame_time), 6),
|
||||
|
||||
@@ -395,7 +395,7 @@ class AusBossVideoCropRotatePadClip:
|
||||
try:
|
||||
path = resolve_video_path(source_mode, video, local_path)
|
||||
except Exception:
|
||||
path = local_path if source_mode == "local path" else video
|
||||
path = None # refused: hashed by its text only, never opened
|
||||
spec = spec_from_values(**values)
|
||||
resize = {name: values.get(name) for name in resize_inputs()}
|
||||
return stable_file_fingerprint(
|
||||
@@ -403,6 +403,7 @@ class AusBossVideoCropRotatePadClip:
|
||||
{
|
||||
"video": video,
|
||||
"source_mode": source_mode,
|
||||
"local_path": local_path,
|
||||
"start_seconds": round(float(start_seconds), 6),
|
||||
"end_seconds": round(float(end_seconds), 6),
|
||||
"every_nth": max(1, int(every_nth)),
|
||||
|
||||
+1
-1
@@ -3,7 +3,7 @@ name = "ausboss-nodes"
|
||||
# Manager/registry search matches against this text, so it names the actual
|
||||
# nodes instead of describing the collection abstractly.
|
||||
description = "ComfyUI-AusBoss (AusBoss nodes): image and video nodes with compact controls: Image and Video Crop + Rotate + Pad (frame or clip), Load Image, Load Image + Pad, Image Folder, Image Resize, Align Image, Image Size, Color Match, Image Compare A/B, Realign to Source (experimental), Save Image (PNG, WebP/JXL, captions), Load Video, Load Audio, Save Video, Select Frame, Frame Interpolate, Mask Refine, Mask by Name and Mask by Touch, LaMa Inpaint, Crop For Inpaint, Stitch Inpaint, Tiled Upscale and Tiled Upscale Stitch, FlashVSR Upscale, Edit Lock and Edit Lock Restore, LoRA Loader, Krea 2 Encode and Outpaint Model Patch, Latent Size, Layout Boxes, Seed, Select Every Nth, Split Batch, Merge Batches, Text, Text Lines, Prompt Variations, H3 Prompt Writer, Integer, Float, Math Expression, Show Text, Free Memory, Workflow Note, Workflow Note Tile, Callout, Callout Tile, Workflow Switches, Run Timer, and Step Preview. Example workflows for Qwen Image 2.1, Krea 2, Klein 9B and LTX 2.3."
|
||||
version = "2.26.0"
|
||||
version = "2.26.3"
|
||||
# comfy-cli serializes this to JSON verbatim, so `file` publishes the literal
|
||||
# string {"file": "LICENSE"} to the registry listing. `text` puts a readable
|
||||
# name there instead. Both forms are supported; the LICENSE file is unchanged.
|
||||
|
||||
@@ -0,0 +1,305 @@
|
||||
---
|
||||
name: ausboss-workflows
|
||||
description: Write the words for AusBoss ComfyUI video workflows (MiniMax H3 Pika Person Swap and Pika Remix, through the H3 Prompt Writer node), and set up and run those workflows on the person's own computer. Use when someone shares ausboss-examples.vercel.app, asks for a person swap, a video remix or an H3 prompt, or wants an AusBoss workflow installed or run for them.
|
||||
license: MIT
|
||||
compatibility: Writing the words needs nothing. Running needs ComfyUI 0.38 or newer on the person's computer, an 8 GB or larger graphics card, and a browser the agent can drive.
|
||||
metadata:
|
||||
author: AusBoss
|
||||
homepage: https://ausboss-examples.vercel.app/
|
||||
updated: "2026-11-16"
|
||||
---
|
||||
|
||||
# AusBoss workflows
|
||||
|
||||
AusBoss makes a pack of ComfyUI nodes and the workflows built on them. This
|
||||
page is for an AI agent helping someone use those workflows. It covers three
|
||||
jobs, from light to heavy:
|
||||
|
||||
1. **Write the words.** The person says what they want; you write what goes in
|
||||
the workflow's text box and say which settings to pick.
|
||||
2. **Set it up.** Install the node pack and the model files.
|
||||
3. **Run it for them.** Load the workflow in their ComfyUI, put in their files
|
||||
and your words, run it, and hand back the clip.
|
||||
|
||||
Offer the third job when you can run commands on the person's computer. Ask
|
||||
before you start it: it downloads large files (see "Model files") and uses
|
||||
their graphics card for several minutes a clip.
|
||||
|
||||
## Where everything is
|
||||
|
||||
| What | Where |
|
||||
|---|---|
|
||||
| Node pack (source, help pages, issues) | https://github.com/ausboss/ComfyUI-AusBoss |
|
||||
| The pack's own guide for agents (which node does which job) | https://github.com/ausboss/ComfyUI-AusBoss/blob/main/js/docs/agents/README.md |
|
||||
| Workflows | https://civitai.com/user/AusBoss |
|
||||
| Models and LoRAs | https://huggingface.co/ausboss |
|
||||
| Real example runs (this site) | https://ausboss-examples.vercel.app/ |
|
||||
| Help from people, and every workflow file | https://discord.gg/RwVxY9Fuaj |
|
||||
| Video guides | https://www.youtube.com/@ausbossdev |
|
||||
|
||||
The examples on this site are real runs. Each page has a plain-text copy with
|
||||
every script, setting, prompt and file link:
|
||||
|
||||
- Person Swap Examples (4 examples): https://ausboss-examples.vercel.app/person-swap/index.md
|
||||
- Pika Remix Examples (8 examples): https://ausboss-examples.vercel.app/pika-remix/index.md
|
||||
- Fix a Pika Remix or Person Swap Run (a guide): https://ausboss-examples.vercel.app/fix/index.md
|
||||
- Faster Video in ComfyUI (a guide): https://ausboss-examples.vercel.app/speed/index.md
|
||||
|
||||
Read the matching page before you write for a workflow. A real example close
|
||||
to the person's request is the best starting point.
|
||||
|
||||
## The workflows
|
||||
|
||||
| Workflow | What it does | Get it | Real runs |
|
||||
|---|---|---|---|
|
||||
| MiniMax H3 Pika Person Swap | Swaps the person in a clip for the person in one photo. The place, camera, moves and sound stay. | https://civitai.com/models/3040833 | https://ausboss-examples.vercel.app/person-swap/index.md |
|
||||
| MiniMax H3 Pika Remix | Changes, swaps or copies a video using up to four pictures. Also starts a clip from a picture, or goes from one picture to another. | https://civitai.com/models/3050155 | https://ausboss-examples.vercel.app/pika-remix/index.md |
|
||||
|
||||
Both make video with sound, up to 15 seconds, with the MiniMax H3 model and
|
||||
the Pika H3 Flex LoRA. Both have a High VRAM setting (24 GB or more) and a Low
|
||||
VRAM setting (8 to 24 GB; on 8 GB, clips up to 8 seconds and 32 GB of RAM).
|
||||
|
||||
## Write the words
|
||||
|
||||
Both workflows have one text box, on the node titled
|
||||
**H3 Prompt Writer - What happens**. The person types plain lines there and
|
||||
the node writes the model's long prompt itself. So you write short plain
|
||||
lines, never the long prompt. The full rules are on the node's help page:
|
||||
https://github.com/ausboss/ComfyUI-AusBoss/blob/main/js/docs/AUSBOSS_NODES_H3PromptWriter.md
|
||||
If that page and this one disagree, that page is right.
|
||||
|
||||
### The basics
|
||||
|
||||
- One line is one moment. Lines play in order. About one moment a second is
|
||||
the most the model can play.
|
||||
- A blank line cuts to a new place.
|
||||
- `@1` to `@4` name the pictures plugged in. Each picture has a job, picked
|
||||
under the text box.
|
||||
- Say what a camera would see. `She smiles` works; `She feels happy` does not.
|
||||
- Nothing moves or exists unless a line says so.
|
||||
- Say which way people face, and where they end up.
|
||||
- Spoken words go in quotes, with the speaker named: `@1 says "we are late."`
|
||||
- Put the spoken line last. With time left after it, the model keeps talking.
|
||||
|
||||
### Picture jobs
|
||||
|
||||
| Job | Use it for |
|
||||
|---|---|
|
||||
| person | This person goes where the lines say. A clear photo. |
|
||||
| character sheet | One character drawn from several sides. |
|
||||
| place | Where it happens. A picture with nobody in it. |
|
||||
| outfit | Clothes for the person. Best with no head in the picture. |
|
||||
| thing | An object, product, toy or mascot. |
|
||||
| style | Only the look is used, for the whole clip. |
|
||||
| start on it | The clip starts from this picture. |
|
||||
| end on it | The clip lands on this picture. It can't be named with `@`. |
|
||||
|
||||
### Video jobs
|
||||
|
||||
| Job | What happens | What to type |
|
||||
|---|---|---|
|
||||
| swap the person | The pictured person takes over from the one in the video. | Who goes, by what they look like: `Replace the man in the hat with @1.` |
|
||||
| copy its moves | The pictured person does the video's moves somewhere new. | `@1 dances on a rooftop at sunset.` |
|
||||
| change something | The video stays, with one change. | The change, exactly: `She wears a red raincoat.` |
|
||||
| keep going | More of the same clip, from where it ends. | What happens next, or nothing. |
|
||||
| new voice | The same clip with a new voice. | The words, in quotes. Needs a voice clip. |
|
||||
|
||||
For **swap the person**, **change something** and **new voice**, the clip's
|
||||
length comes from the video. To make it shorter, trim the video on its loader.
|
||||
|
||||
### Person swap: the lines that work
|
||||
|
||||
```
|
||||
Replace the man in the suit holding an ice cream with @1.
|
||||
@1 looks like a heavy-set man with round glasses, a white shirt and green trousers.
|
||||
```
|
||||
|
||||
- Line 1 says who goes. Describe them by what is seen in the clip (clothes,
|
||||
hair, position), so the others in the shot are left alone.
|
||||
- Line 2 is optional and describes the new person. Write it from their photo.
|
||||
It helps the look; it does not promise an exact likeness. The Person Swap
|
||||
workflow can write this line itself: the **Describe @1 for me** switch.
|
||||
- If the photo shows only a face, write the clothes too. What the photo does
|
||||
not show, the model makes up.
|
||||
- The face in the clip has to be clear. A small or turned-away face keeps the
|
||||
clip's own person.
|
||||
- **Keep the clip's sound** on: the clip's own voice and music stay and the
|
||||
new mouth follows them. Off, with words in quotes: the model makes a new
|
||||
voice.
|
||||
|
||||
### Other things a line can do
|
||||
|
||||
- A time: start a line with `3s:` and it happens at 3 seconds.
|
||||
- `sound: rain on a tin roof` and `music: slow sad piano`. With no music line
|
||||
there is no music.
|
||||
- `camera: slow push in` and `look: a 1970s film`.
|
||||
- `no cuts` makes blank lines stop being cuts.
|
||||
- `ends on her sitting by the window` sets the last moment.
|
||||
- A sign: `a neon sign reads "OPEN"`.
|
||||
|
||||
### What to hand the person
|
||||
|
||||
When you only write the words, give them:
|
||||
|
||||
1. The lines, in a code block they can paste into the text box.
|
||||
2. Which file goes in which loader (`@1` is Picture 1, and so on).
|
||||
3. The job for each picture and for the video.
|
||||
4. Any switch to change, such as **Keep the clip's sound**.
|
||||
|
||||
A voice clip should be the person's own voice, or one they have permission to
|
||||
use. Only swap a real person into a clip when the person asking has the right
|
||||
to use that photo.
|
||||
|
||||
## Set it up
|
||||
|
||||
Check before you install anything:
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:8188/system_stats
|
||||
```
|
||||
|
||||
That answers when ComfyUI is running on its usual port, with its version and
|
||||
the graphics card's memory. If nothing answers, ask the person where their
|
||||
ComfyUI is and how they start it. If they have none, the official install
|
||||
guide is https://docs.comfy.org/installation and that choice is theirs.
|
||||
|
||||
### The node pack
|
||||
|
||||
Either of these, then restart ComfyUI:
|
||||
|
||||
```bash
|
||||
comfy node install ausboss-nodes
|
||||
```
|
||||
|
||||
```bash
|
||||
git clone https://github.com/ausboss/ComfyUI-AusBoss.git
|
||||
```
|
||||
|
||||
The first needs comfy-cli. The second runs inside ComfyUI's `custom_nodes`
|
||||
folder. In the ComfyUI window, the person can also open the Manager, search
|
||||
for **AusBoss** and install it there. After the restart this returns the
|
||||
node, which shows the pack loaded:
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:8188/object_info/AUSBOSS_NODES_H3PromptWriter
|
||||
```
|
||||
|
||||
### Model files
|
||||
|
||||
The files go in ComfyUI's `models` folder, in the subfolder named below. They
|
||||
are large. Tell the person the total and get a yes before you download. Skip
|
||||
any file they already have.
|
||||
|
||||
| File | Folder in `models` | Size | Needed |
|
||||
|---|---|---|---|
|
||||
| [qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors) | `text_encoders` | 15.7 GB | always |
|
||||
| [minimax_h3_audio_vae_fp32.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors) | `vae` | 0.6 GB | always |
|
||||
| [pika_h3_flex_ref2va_v1_bf16.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/loras/pika_h3_flex_ref2va_v1_bf16.safetensors) | `loras` | 2.0 GB | always |
|
||||
| [minimax_h3_ref2va_pruned_int8_convrot.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/diffusion_models/minimax_h3_ref2va_pruned_int8_convrot.safetensors) | `diffusion_models` | 21.0 GB | High VRAM |
|
||||
| [minimax_h3_video_vae_fp16.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_fp16.safetensors) | `vae` | 5.2 GB | High VRAM |
|
||||
| [minimax_h3_ref2va_pruned_w6a8.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/diffusion_models/minimax_h3_ref2va_pruned_w6a8.safetensors) | `diffusion_models` | 16.0 GB | Low VRAM |
|
||||
| [minimax_h3_video_vae_int8_convrot.safetensors](https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_int8_convrot.safetensors) | `vae` | 2.8 GB | Low VRAM |
|
||||
| [flashvsr_v1.1_fp16.safetensors](https://huggingface.co/ausboss/FlashVSR-v1.1-ComfyUI/resolve/main/flashvsr_v1.1_fp16.safetensors) | `diffusion_models` | 3.5 GB | only for Finish: FlashVSR |
|
||||
|
||||
High VRAM needs about 44 GB of files, Low VRAM about 37 GB. The optional switches that write text for you (such as **Describe @1 for me**) use a small Qwen3-VL 8B text model; the workflow's Guide note names that file and where to get it.
|
||||
|
||||
Each workflow opens with a Guide note that lists its files. If the note and
|
||||
this table disagree, the note in the newest workflow is right.
|
||||
|
||||
### Faster runs
|
||||
|
||||
One node or one launch setting makes these workflows about twice as fast on
|
||||
most recent cards: https://ausboss-examples.vercel.app/speed/index.md
|
||||
|
||||
## Run it for them
|
||||
|
||||
### Get the workflow file
|
||||
|
||||
- The newest file is on the workflow's Civitai page (links above). Civitai may
|
||||
ask for a login. If it does, ask the person to download the file and tell
|
||||
you where it is. The same files are in the Discord.
|
||||
- Each Person Swap example on this site also has a zip with the workflow set
|
||||
up exactly as it ran, its input files and its result. Those can be older
|
||||
than the Civitai file. They are good for a first test run: the result is
|
||||
there to compare with.
|
||||
|
||||
### Put the person's files in place
|
||||
|
||||
Copy their clip and pictures into ComfyUI's `input` folder, or upload them:
|
||||
|
||||
```bash
|
||||
curl -s -F "image=@/path/to/photo.jpg" http://127.0.0.1:8188/upload/image
|
||||
```
|
||||
|
||||
The same address takes video and audio files.
|
||||
|
||||
### Load it in ComfyUI's own page and run
|
||||
|
||||
These workflows have switches and grouped parts that only ComfyUI's page
|
||||
turns into a run. So do not rewrite the file into an API prompt by hand:
|
||||
open the page, load the file there, and let the page queue it. Any browser
|
||||
you can drive works, headless or not (Chrome started with
|
||||
`--remote-debugging-port`, Playwright, or a browser tool).
|
||||
|
||||
On the page at `http://127.0.0.1:8188/`, once `window.app.graph` exists:
|
||||
|
||||
```js
|
||||
await app.loadGraphData(workflow); // the parsed workflow file
|
||||
const node = (title) => app.graph.nodes.find((n) => n.title === title);
|
||||
const set = (title, name, value) => {
|
||||
const w = node(title).widgets.find((w) => w.name === name);
|
||||
w.value = value;
|
||||
w.callback?.(value);
|
||||
};
|
||||
set("H3 Prompt Writer - What happens", "script", "Replace the man in the hat with @1.");
|
||||
set("Video - Your clip", "video", "their_clip.mp4");
|
||||
set("Picture 1 - @1", "image", "their_photo.jpg");
|
||||
set("Seed", "seed", 6202);
|
||||
await app.queuePrompt(0);
|
||||
```
|
||||
|
||||
Find a node by its title and a setting by its name. The titles above are the
|
||||
ones in Person Swap; read `app.graph.nodes.map((n) => n.title)` for any other
|
||||
workflow. `await app.graphToPrompt()` shows what would be sent without running
|
||||
anything, which is a cheap check before you queue.
|
||||
|
||||
**Switches** (VRAM, Keep the clip's sound, Describe @1 for me, Finish) are
|
||||
cards on the canvas. To flip one without clicking, change the workflow file
|
||||
before you load it, as the pack's guide describes:
|
||||
https://github.com/ausboss/ComfyUI-AusBoss/blob/main/js/docs/agents/workflow_switches.md
|
||||
With a browser tool that can click, clicking the switch on the card is
|
||||
simpler. Pick **Low VRAM** when the card has less than 24 GB.
|
||||
|
||||
### Wait, then hand back the clip
|
||||
|
||||
`queuePrompt` returns right away. Watch the run with:
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:8188/queue
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -s http://127.0.0.1:8188/history
|
||||
```
|
||||
|
||||
A clip takes a few minutes; the first run is longer because the models load.
|
||||
The finished clip is in ComfyUI's `output` folder, and the history entry
|
||||
names the file. Tell the person where it is, and show it if you can.
|
||||
|
||||
### If it fails
|
||||
|
||||
- A loader turns red or the run stops on a file name: that model file is
|
||||
missing or has another name. Pick the person's copy in that loader.
|
||||
- Out of memory: pick **Low VRAM**, use a shorter clip, or turn on
|
||||
**Fast preview**.
|
||||
- The swap kept the clip's own person: the face in the clip is too small or
|
||||
turned away, or line 1 did not describe who goes.
|
||||
- A clip came out wrong but ran: run again with a new seed.
|
||||
- Anything else: the pack's issues
|
||||
(https://github.com/ausboss/ComfyUI-AusBoss/issues) and the Discord.
|
||||
|
||||
## About this page
|
||||
|
||||
Everything here is public information from AusBoss about his own nodes and
|
||||
workflows. It gives no instruction to act without the person's say: installs,
|
||||
downloads and runs are theirs to approve.
|
||||
@@ -102,3 +102,39 @@ test("a server that does not answer keeps the page quiet", async (t) => {
|
||||
await page.serverComesBack("2.7.0");
|
||||
assert.equal(page.toasts.length, 1);
|
||||
});
|
||||
|
||||
// The Manager switches the pack's files on disk, and the server keeps running
|
||||
// the old version until ComfyUI restarts. A reload in between gives the page
|
||||
// the new files: the page is newer than the server, and "press F5" is wrong.
|
||||
test("a page newer than the server says nothing", async (t) => {
|
||||
const page = await openPage(t, { running: "2.26.2", installed: "2.25.2" });
|
||||
assert.deepEqual(page.toasts, []);
|
||||
assert.deepEqual(page.refreshed, []);
|
||||
await page.serverComesBack("2.26.2");
|
||||
assert.deepEqual(page.toasts, [], "after the restart the two match");
|
||||
});
|
||||
|
||||
test("versions compare as numbers, not as text", async (t) => {
|
||||
const older = await openPage(t, { running: "2.25.10", installed: "2.25.9" });
|
||||
assert.deepEqual(older.toasts, [], "2.25.9 is older than 2.25.10");
|
||||
const newer = await openPage(t, { running: "2.25.9", installed: "2.25.10" });
|
||||
assert.equal(newer.toasts.length, 1, "2.25.10 is newer than 2.25.9");
|
||||
assert.match(newer.toasts[0].detail, /v2\.25\.10 is installed/);
|
||||
});
|
||||
|
||||
test("an unknown or odd version says nothing", async (t) => {
|
||||
for (const [running, installed] of [
|
||||
["2.26.2", "unknown"],
|
||||
["2.26.2", "2.27"],
|
||||
["2.26.2", "2.27.0rc1"],
|
||||
["2.26.2", "v2.27.0"],
|
||||
["2.26.2", 3],
|
||||
["unknown", "2.27.0"],
|
||||
["dev", "2.27.0"],
|
||||
[null, "2.27.0"],
|
||||
]) {
|
||||
const page = await openPage(t, { running, installed });
|
||||
assert.deepEqual(page.toasts, [], `page ${running}, server ${installed}`);
|
||||
assert.deepEqual(page.refreshed, [], `page ${running}, server ${installed}`);
|
||||
}
|
||||
});
|
||||
|
||||
@@ -142,6 +142,7 @@ test("only the listed parts of the pack listen for the wheel", () => {
|
||||
"image_folder/index.js root", // the copy question, a window over the page
|
||||
"image_folder/index.js root", // the folder list, a window over the page
|
||||
"shared/discard_prompt.mjs backdrop", // the keep or discard question
|
||||
"shared/h3_cast_chips.mjs window", // closes a chip's job list when the graph moves
|
||||
"shared/media_picker.mjs window", // closes its list when the graph moves
|
||||
"shared/transform_editor.mjs canvas", // the full-screen editor zooms its picture
|
||||
"shared/widget_card.mjs area", // a text box scrolls its own text first
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
import test from "node:test";
|
||||
import assert from "node:assert/strict";
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
const source = readFileSync(new URL("../js/shared/h3_cast_chips.mjs", import.meta.url), "utf-8");
|
||||
|
||||
test("a chip's job list stays open while an item is pressed", () => {
|
||||
// Pressing an item takes focus off the chip's button. A capturing blur
|
||||
// listener heard that and closed the list before the click could pick.
|
||||
const blur = source.match(/addEventListener\("blur",[^\n]*/)?.[0] ?? "";
|
||||
assert.match(blur, /event\.target === window/, "only the window losing focus closes the list");
|
||||
assert.doesNotMatch(blur, /\bon\)/, "the blur listener is not the capturing set");
|
||||
assert.match(source, /menu\.addEventListener\("mousedown", \(event\) => event\.preventDefault\(\)\)/, "a press keeps focus on the button");
|
||||
});
|
||||
@@ -11,10 +11,10 @@ const graphWith = (mode = 0, useMap = false) => {
|
||||
getNodeById:id=>id===150?source:null,
|
||||
} } };
|
||||
};
|
||||
test('only the three backend duration overrides use the clip', () => {
|
||||
test('only the four backend duration overrides use the clip', () => {
|
||||
const {node}=graphWith();
|
||||
for(const job of ['swap the person','new voice','change something']) assert.equal(clipSetsDuration(node,{video_job:job}),true);
|
||||
for(const job of ['copy its moves','keep going','',undefined]) assert.equal(clipSetsDuration(node,{video_job:job}),false);
|
||||
for(const job of ['copy its moves','swap the person','new voice','change something']) assert.equal(clipSetsDuration(node,{video_job:job}),true);
|
||||
for(const job of ['keep going','',undefined]) assert.equal(clipSetsDuration(node,{video_job:job}),false);
|
||||
});
|
||||
test('Full Video OFF/bypassed/missing/disconnected restore seconds; both link stores work', () => {
|
||||
for(const map of [false,true]) {
|
||||
@@ -73,7 +73,7 @@ test('timeline omits guessed duration/retiming and rejects an old async report;
|
||||
const {momentStrip}=await import('data:text/javascript;base64,'+Buffer.from(source).toString('base64'));
|
||||
const {node,source:video}=graphWith();let refreshes=0;node.__ausbossCard={refresh(){refreshes++;}};
|
||||
const panel=momentStrip(node), [ruler,track,note]=panel.element.children;
|
||||
const values={seconds:12,script:'She walks.',video_job:'copy its moves'};
|
||||
const values={seconds:12,script:'She walks.',video_job:'keep going'};
|
||||
panel.sync(values);assert.equal(timers.length,1);timers.shift()();await Promise.resolve();
|
||||
assert.equal(requests[0].seconds,12);
|
||||
panel.sync({...values,video_job:'swap the person'});
|
||||
|
||||
@@ -24,15 +24,15 @@ test("cuts and speech are marked", () => {
|
||||
|
||||
test("no report and an empty script draw nothing and say what to do", () => {
|
||||
assert.deepEqual(stripRows(null), []);
|
||||
assert.deepEqual(stripCaption(null), { text: "Each line becomes a moment.", warn: false });
|
||||
assert.deepEqual(stripCaption({ moments: [] }), { text: "Each line becomes a moment.", warn: false });
|
||||
assert.deepEqual(stripCaption(null), { text: "Each line becomes a moment. A blank line between lines is a cut.", warn: false });
|
||||
assert.deepEqual(stripCaption({ moments: [] }), { text: "Each line becomes a moment. A blank line between lines is a cut.", warn: false });
|
||||
});
|
||||
|
||||
test("the caption adds the script up", () => {
|
||||
assert.equal(stripCaption({ moments: [row(0, 2.5), row(2.5, 5)] }).text, "2 moments, 2.5 s each, one take.");
|
||||
assert.equal(stripCaption({ moments: [row(0, 8)] }).text, "1 moment, 8 s, one take.");
|
||||
assert.equal(stripCaption({ moments: [row(0, 3), row(3, 8)] }).text, "2 moments, one take.");
|
||||
assert.equal(stripCaption({ moments: [row(0, 3), row(3, 8, { cut: true })] }).text, "2 moments, 1 cut.");
|
||||
assert.equal(stripCaption({ moments: [row(0, 2.5), row(2.5, 5)] }).text, "2 moments, 2.5 s each, one take. Click between two moments to cut there.");
|
||||
assert.equal(stripCaption({ moments: [row(0, 8)] }).text, "1 moment, 8 s, one take. Add a line for the next moment, a blank line to cut.");
|
||||
assert.equal(stripCaption({ moments: [row(0, 3), row(3, 8)] }).text, "2 moments, one take. Click between two moments to cut there.");
|
||||
assert.equal(stripCaption({ moments: [row(0, 3), row(3, 8, { cut: true })] }).text, "2 moments, 1 cut. Click a cut on the bar to join the shots.");
|
||||
});
|
||||
|
||||
test("only connected pictures send their job", () => {
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
"""A loader's change check never opens a value it refused.
|
||||
|
||||
ComfyUI calls a node's IS_CHANGED with the prompt's widget values before it
|
||||
runs anything, and /prompt needs no login. The picture and video loaders
|
||||
first hand the value to their resolver, which takes only a file named below
|
||||
ComfyUI's input, output or temp folder. When the resolver refuses it, the
|
||||
value is hashed as text and nothing else: on Windows, merely looking up a
|
||||
network path ("//host/share/x.png") contacts that host and hands it the
|
||||
server's login.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
|
||||
del sys.modules["nodes"]
|
||||
|
||||
from nodes._transform_engine import stable_file_fingerprint # noqa: E402
|
||||
from nodes.node_image_crop_rotate_pad import AusBossImageCropRotatePad # noqa: E402
|
||||
from nodes.node_load_image import AusBossLoadImage # noqa: E402
|
||||
from nodes.node_load_image_pad import AusBossLoadImagePad # noqa: E402
|
||||
from nodes.node_video_crop_rotate_pad import AusBossVideoCropRotatePad # noqa: E402
|
||||
from nodes.node_video_crop_rotate_pad_clip import AusBossVideoCropRotatePadClip # noqa: E402
|
||||
|
||||
REFUSED = [
|
||||
"//evil-host/share/photo.png",
|
||||
"\\\\evil-host\\share\\photo.png",
|
||||
"/evil-host/etc/photo.png",
|
||||
"../../evil-host/photo.png",
|
||||
]
|
||||
|
||||
|
||||
class DiskWatch:
|
||||
"""Records every stat and resolve of a path that names the evil host."""
|
||||
|
||||
def __init__(self):
|
||||
self.touched = []
|
||||
|
||||
def __enter__(self):
|
||||
real_stat, real_resolve = os.stat, Path.resolve
|
||||
watch = self
|
||||
|
||||
def stat(path, *args, **kwargs):
|
||||
if "evil-host" in str(path):
|
||||
watch.touched.append(("stat", str(path)))
|
||||
return real_stat(path, *args, **kwargs)
|
||||
|
||||
def resolve(self, *args, **kwargs):
|
||||
if "evil-host" in str(self):
|
||||
watch.touched.append(("resolve", str(self)))
|
||||
return real_resolve(self, *args, **kwargs)
|
||||
|
||||
self._patches = [patch("os.stat", stat), patch.object(Path, "resolve", resolve)]
|
||||
for item in self._patches:
|
||||
item.start()
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc):
|
||||
for item in reversed(self._patches):
|
||||
item.stop()
|
||||
|
||||
|
||||
class ChangeCheckTests(unittest.TestCase):
|
||||
def assert_off_disk(self, call):
|
||||
for value in REFUSED:
|
||||
with self.subTest(value=value), DiskWatch() as watch:
|
||||
first = call(value)
|
||||
self.assertEqual(watch.touched, [])
|
||||
# The text still counts: another refused value is another key.
|
||||
self.assertNotEqual(first, call(value + "x"))
|
||||
|
||||
def test_picture_loaders(self):
|
||||
for node in (AusBossLoadImage, AusBossLoadImagePad, AusBossImageCropRotatePad):
|
||||
with self.subTest(node=node.__name__):
|
||||
self.assert_off_disk(lambda value, node=node: node.IS_CHANGED(image=value))
|
||||
|
||||
def test_video_crop_nodes(self):
|
||||
for node in (AusBossVideoCropRotatePad, AusBossVideoCropRotatePadClip):
|
||||
for mode in ("input folder", "local path"):
|
||||
with self.subTest(node=node.__name__, mode=mode):
|
||||
self.assert_off_disk(
|
||||
lambda value, node=node, mode=mode: node.IS_CHANGED(
|
||||
video=value if mode == "input folder" else "",
|
||||
source_mode=mode,
|
||||
local_path=value if mode == "local path" else "",
|
||||
seek_mode="frame index",
|
||||
frame_index=0,
|
||||
frame_time=0.0,
|
||||
**({"start_seconds": 0.0, "end_seconds": 0.0} if node is AusBossVideoCropRotatePadClip else {}),
|
||||
)
|
||||
)
|
||||
|
||||
def test_the_fingerprint_opens_only_a_resolved_path(self):
|
||||
with DiskWatch() as watch:
|
||||
stable_file_fingerprint("//evil-host/share/photo.png", {})
|
||||
stable_file_fingerprint(None, {})
|
||||
self.assertEqual(watch.touched, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -7,6 +7,7 @@ import re
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
@@ -244,6 +245,45 @@ class VersionedServingTests(unittest.TestCase):
|
||||
for path, (_status, body, _headers) in self.fetch(paths).items():
|
||||
self.assertNotIn("?v=tok", body, path)
|
||||
|
||||
def test_an_address_never_reaches_the_disk(self):
|
||||
# The request is unauthenticated: its address is only looked up in
|
||||
# the list of scripts made at startup, never joined to a folder and
|
||||
# resolved. On Windows, resolving "//host/share" contacts that host.
|
||||
web = self.web
|
||||
touched = []
|
||||
|
||||
async def handled(request):
|
||||
return web.Response(text="from the handler", content_type="text/javascript")
|
||||
|
||||
async def run():
|
||||
middleware = make_revalidate_middleware("ausboss-nodes", self.js, "tok")
|
||||
real_resolve, real_stat = Path.resolve, os.stat
|
||||
|
||||
def resolve(path, *args, **kwargs):
|
||||
if "evil-host" in str(path):
|
||||
touched.append(str(path))
|
||||
return real_resolve(path, *args, **kwargs)
|
||||
|
||||
def stat(path, *args, **kwargs):
|
||||
if "evil-host" in str(path):
|
||||
touched.append(str(path))
|
||||
return real_stat(path, *args, **kwargs)
|
||||
|
||||
with patch.object(Path, "resolve", resolve), patch("os.stat", stat):
|
||||
for path in [
|
||||
"/extensions/ausboss-nodes//evil-host/share/x.mjs",
|
||||
"/extensions/ausboss-nodes/%2F%2Fevil-host/share/x.mjs",
|
||||
"/extensions/ausboss-nodes/%5C%5Cevil-host%5Cshare%5Cx.mjs",
|
||||
"/extensions/ausboss-nodes/../../evil-host/x.mjs",
|
||||
"/extensions/ausboss-nodes/%2e%2e/%2e%2e/evil-host/x.js",
|
||||
"/extensions/ausboss-nodes/C:/evil-host/x.mjs",
|
||||
]:
|
||||
response = await middleware(self.make_request("GET", path), handled)
|
||||
self.assertEqual(response.text, "from the handler", path)
|
||||
|
||||
asyncio.run(run())
|
||||
self.assertEqual(touched, [])
|
||||
|
||||
def test_without_a_tag_the_files_are_served_as_before(self):
|
||||
path = "/extensions/ausboss-nodes/shared/b.mjs"
|
||||
status, body, headers = self.fetch([path], token=None)[path]
|
||||
|
||||
@@ -182,6 +182,20 @@ class SplitTests(unittest.TestCase):
|
||||
def test_a_quote_can_end_a_sentence(self):
|
||||
self.assertEqual(moments('A voiceover says "this is how it ended." She walks away.'), ['A voiceover says "this is how it ended."', "She walks away"])
|
||||
|
||||
def test_an_at_after_a_quote_starts_a_new_moment(self):
|
||||
self.assertEqual(moments('@1 says "Hi." @1 waves and says "Bye."'), ['@1 says "Hi."', '@1 waves and says "Bye."'])
|
||||
self.assertEqual(moments('@1 says "Hi" and @2 says "Bye."'), ['@1 says "Hi"', '@2 says "Bye."'])
|
||||
self.assertEqual(moments('"Hi," she says. @1 laughs.'), ['"Hi," she says', "@1 laughs"])
|
||||
|
||||
def test_every_line_in_a_row_is_spoken(self):
|
||||
def said(script, jobs):
|
||||
return re.findall(r"<d>\[English\] (.*?)</d>", write_prompt(script, seconds=8.0, jobs=jobs))
|
||||
|
||||
self.assertEqual(said('@1 says "Hi!" @2 answers "Hello." @1 says "Bye."', {1: "person", 2: "person"}), ["Hi!", "Hello.", "Bye."])
|
||||
self.assertEqual(said('@1 says "Hi" and @2 says "Bye."', {1: "person", 2: "person"}), ["Hi.", "Bye."])
|
||||
self.assertEqual(said('"Hi," @1 says.', {1: "person"}), ["Hi."])
|
||||
self.assertEqual(said('@1 says "我有冰淇淋。" @1 says "好吗?" @1 says "但是,"', {1: "person"}), ["我有冰淇淋。", "好吗?", "但是。"])
|
||||
|
||||
def test_the_timeline_shows_plain_words_for_a_hold(self):
|
||||
self.assertEqual(script_report("3s: she jumps", 8)["moments"][0]["text"], "Holds still")
|
||||
|
||||
@@ -643,7 +657,7 @@ class VideoJobTests(unittest.TestCase):
|
||||
for job in ("outfit", "thing"):
|
||||
text = write_prompt("He wears @1 in place of his blue cap.", seconds=8.0, jobs={1: job}, video="change something")
|
||||
self.assertIn("<Subject 2> is the person in <Video 1>, kept as they are", text)
|
||||
self.assertIn("[video editing] <Subject 2> wears <Subject 1> in place of his blue cap. Everything else in <Video 1> is kept.", text)
|
||||
self.assertIn("[video editing + audio reuse] <Subject 2> wears <Subject 1> in place of his blue cap. Everything else in <Video 1> is kept.", text)
|
||||
self.assertIn("Retain the look of <Video 1>.", text)
|
||||
self.assertIn("<Subject 2> has the same words, mouth movements and moves as in <Video 1>.", text)
|
||||
self.assertIn("The original sound of <Video 1> continues throughout.", text)
|
||||
@@ -659,7 +673,7 @@ class VideoJobTests(unittest.TestCase):
|
||||
self.assertIn("put only this outfit on <Subject 2>", text)
|
||||
self.assertNotIn("<Subject 1> is the person in <Picture 1>", text)
|
||||
text = write_prompt("He wears the propeller hat instead of his cap.", seconds=8.0, jobs={1: "person"}, video="change something")
|
||||
self.assertIn("[video editing] <Subject 2> wears <Subject 1> instead of his cap.", text)
|
||||
self.assertIn("[video editing + audio reuse] <Subject 2> wears <Subject 1> instead of his cap.", text)
|
||||
|
||||
def test_change_something_keeps_a_person_picture_that_is_not_put_on(self):
|
||||
self.assertEqual(put_on_from_pictures("@1 stands next to him.", {1: "person"}), ("@1 stands next to him.", {1: "person"}))
|
||||
@@ -696,9 +710,8 @@ class VideoJobTests(unittest.TestCase):
|
||||
|
||||
def test_a_hat_with_no_person_picture_points_to_change_something(self):
|
||||
for job in ("outfit", "thing"):
|
||||
for video in ("swap the person", "copy its moves"):
|
||||
cast = Cast(number_pictures({1: job}, video), video)
|
||||
self.assertIn('pick the video job "change something"', cast.problem("He wears @1."))
|
||||
cast = Cast(number_pictures({1: job}, "swap the person"), "swap the person")
|
||||
self.assertIn('pick the video job "change something"', cast.problem("He wears @1."))
|
||||
|
||||
def test_copying_moves_keeps_the_performer_out(self):
|
||||
parts = parts_of("@1 dances on a rooftop.", jobs={1: "person"}, video="copy its moves")
|
||||
@@ -710,11 +723,62 @@ class VideoJobTests(unittest.TestCase):
|
||||
|
||||
def test_the_timeline_says_so_when_nothing_is_connected(self):
|
||||
self.assertIn("Connect a picture or a video", script_report("She runs.", 6, {})["warning"])
|
||||
self.assertIn("To copy the moves", script_report("She runs.", 6, {}, "copy its moves")["warning"])
|
||||
self.assertEqual(script_report("She runs.", 6, {}, "copy its moves")["warning"], "")
|
||||
|
||||
def test_copying_moves_with_nobody_to_give_them_to_says_so(self):
|
||||
def test_copying_moves_with_no_person_picture_keeps_the_person_in_the_video(self):
|
||||
cast = Cast(number_pictures({1: "place"}, "copy its moves"), "copy its moves")
|
||||
self.assertIn("To copy the moves", cast.problem("She dances."))
|
||||
self.assertEqual(cast.problem("She dances in @1."), "")
|
||||
parts = parts_of("She dances in @1.", jobs={1: "place"}, video="copy its moves")
|
||||
self.assertIn("<Subject 2> is the person in <Video 1>, with the same face, hair, body and clothes", parts["subject_definitions"])
|
||||
self.assertIn("the background of <Video 1> is not kept, and the place is <Subject 1>", parts["retention_analysis"])
|
||||
self.assertIn("<Subject 2> dances in <Subject 1>. <Subject 2> moves exactly as in <Video 1>", parts["detailed_description"])
|
||||
self.assertNotIn("performs the same moves as", parts["detailed_description"])
|
||||
|
||||
def test_restyling_an_empty_clip_names_no_person_and_drops_the_look(self):
|
||||
parts = parts_of("It looks real, like @1.", jobs={1: "style"}, video="change something")
|
||||
self.assertIn("<Video 1> supplies the place, the action, the timing, the camera movement and the sound.", parts["subject_definitions"])
|
||||
self.assertNotIn("the person in <Video 1>", parts["subject_definitions"])
|
||||
self.assertNotIn("the look", parts["retention_analysis"])
|
||||
self.assertNotIn("Retain the look of <Video 1>", parts["detailed_description"])
|
||||
|
||||
def test_an_edit_with_a_place_picture_takes_the_place_and_look_from_it(self):
|
||||
parts = parts_of("The hallway looks like @1.", jobs={1: "place"}, video="change something")
|
||||
self.assertIn("<Video 1> supplies the action, the timing, the camera movement and the sound.", parts["subject_definitions"])
|
||||
self.assertIn("the colours and the film grain of <Picture 1>", parts["detailed_description"])
|
||||
self.assertNotIn("is in the shot", parts["detailed_description"])
|
||||
|
||||
def test_an_edit_about_someone_still_keeps_them(self):
|
||||
parts = parts_of("She looks real, like @1.", jobs={1: "style"}, video="change something")
|
||||
self.assertIn("is the person in <Video 1>, kept as they are", parts["subject_definitions"])
|
||||
self.assertIn("the person", parts["subject_definitions"].splitlines()[0])
|
||||
|
||||
def test_copying_only_a_camera_move_names_no_person(self):
|
||||
parts = parts_of("The camera glides down @1.", jobs={1: "place"}, video="copy its moves")
|
||||
self.assertNotIn("the person in <Video 1>", parts["subject_definitions"])
|
||||
self.assertIn("apply only its camera movement", parts["retention_analysis"])
|
||||
self.assertNotIn("moves exactly as in", parts["detailed_description"])
|
||||
|
||||
def test_a_thing_that_does_the_action_takes_the_moves(self):
|
||||
parts = parts_of("@1 waves its finger.", jobs={1: "thing"}, video="copy its moves")
|
||||
self.assertIn("<Subject 1> is the character in <Picture 1>, with the same shape, colours and drawing style.", parts["subject_definitions"])
|
||||
self.assertIn("apply only the body motion, choreography and timing to <Subject 1>", parts["retention_analysis"])
|
||||
self.assertIn("<Subject 1> performs the same moves as <Subject 2>", parts["detailed_description"])
|
||||
|
||||
def test_a_thing_can_come_in_for_the_one_in_a_swap(self):
|
||||
parts = parts_of("@1 replaces the yellow hand.", jobs={1: "thing"}, video="swap the person")
|
||||
self.assertIn("<Subject 1> is the yellow hand in <Video 1>, selected for replacement.", parts["subject_definitions"])
|
||||
self.assertIn("has the shape, colours and drawing style of the character in <Picture 1>", parts["detailed_description"])
|
||||
self.assertEqual(script_report("", 6, {1: "thing"}, "swap the person")["warning"], "")
|
||||
|
||||
def test_a_held_thing_stays_a_prop(self):
|
||||
parts = parts_of("He holds @1 and dances.", jobs={1: "thing"}, video="copy its moves")
|
||||
self.assertIn("<Subject 1> is the object in <Picture 1>", parts["subject_definitions"])
|
||||
self.assertIn("<Subject 2> moves exactly as in <Video 1>", parts["detailed_description"])
|
||||
|
||||
def test_copying_moves_with_only_an_outfit_dresses_the_person_in_the_video(self):
|
||||
parts = parts_of("He wears @1.", jobs={1: "outfit"}, video="copy its moves")
|
||||
self.assertIn("the clothes change to <Subject 1>", parts["retention_analysis"])
|
||||
self.assertIn("keep its place unless the script names a new one", parts["retention_analysis"])
|
||||
|
||||
def test_a_swap_with_nobody_to_put_in_says_so(self):
|
||||
with self.assertRaisesRegex(ValueError, "new person"):
|
||||
@@ -755,6 +819,30 @@ class VideoJobTests(unittest.TestCase):
|
||||
self.assertEqual(add_looks("Swap him for @1.", {1: "person"}, lambda slot: said),
|
||||
"Swap him for @1.\n@1 looks like a man with curly brown hair, wearing dark-framed glasses")
|
||||
|
||||
def test_a_swap_reads_the_looks_sentence_on_the_same_line(self):
|
||||
from nodes._h3_cast_helpers import add_looks
|
||||
typed = "replace the girl on the car with @1 .@1 looks like a young woman with long dark hair"
|
||||
text = write_prompt(typed, seconds=8.0, jobs={1: "person"}, video="swap the person")
|
||||
self.assertIn("<Subject 1> is the girl on the car in <Video 1>", text)
|
||||
self.assertIn("this appearance: a young woman with long dark hair.", text)
|
||||
self.assertNotIn("looks like", text)
|
||||
# A typed sentence still wins over the model, wherever it sits on the line.
|
||||
self.assertNotIn("bald", add_looks(typed, {1: "person"}, lambda slot: "a bald man"))
|
||||
# A spoken line that happens to hold the words is left alone.
|
||||
said = 'Replace the guy with @1.\n@1 says: "He said @1 looks like trouble."'
|
||||
self.assertNotIn("this appearance", write_prompt(said, seconds=8.0, jobs={1: "person"}, video="swap the person"))
|
||||
# Only a new sentence is split off: inside a sentence the words are left where they are.
|
||||
from nodes._h3_cast_helpers import looks_on_own_lines
|
||||
self.assertEqual(looks_on_own_lines("Put @1 looks like a friend here."), "Put @1 looks like a friend here.")
|
||||
self.assertEqual(looks_on_own_lines("Swap him for @1! @1 looks like a pilot"), "Swap him for @1!\n@1 looks like a pilot")
|
||||
|
||||
def test_a_swap_reads_clothes_typed_on_their_own_line(self):
|
||||
swap = dict(seconds=8.0, jobs={1: "person"}, video="swap the person")
|
||||
text = write_prompt("Replace the man in the checked shirt with @1.\n@1 is wearing a black suit.", **swap)
|
||||
self.assertIn("this appearance: the same hair, wearing a black suit.", text)
|
||||
both = "Replace the guy with @1.\n@1 looks like a man with short dark hair.\n@1 wears a black suit."
|
||||
self.assertIn("this appearance: a man with short dark hair, wearing a black suit.", write_prompt(both, **swap))
|
||||
|
||||
def test_a_swap_gets_the_firm_wording_without_a_looks_like_line(self):
|
||||
text = write_prompt("Replace the guy with @1.", seconds=8.0, jobs={1: "person"}, video="swap the person")
|
||||
self.assertIn("<Subject 2> has the face, hair and clothes of the person in <Picture 1>", text)
|
||||
@@ -948,6 +1036,30 @@ class VideoJobTests(unittest.TestCase):
|
||||
self.assertIn("each word landing exactly on the lip movement of <Video 1>", parts["detailed_description"])
|
||||
self.assertIn("Quiet room tone", parts["overall_soundscape"])
|
||||
|
||||
def test_a_swap_keeps_every_line_the_new_person_says(self):
|
||||
script = (
|
||||
"Replace the young man in the black t-shirt with @1.\n"
|
||||
'@1 holds a phone up and says: "How do I get rich?" The phone answers: "Get a job." '
|
||||
'@1 looks up, confused, and says: "Bro, what are they talking about?"'
|
||||
)
|
||||
text = write_prompt(script, seconds=8.0, jobs={1: "person"}, video="swap the person")
|
||||
self.assertEqual(re.findall(r"<d>\[English\] (.*?)</d>", text),
|
||||
["How do I get rich?", "Get a job.", "Bro, what are they talking about?"])
|
||||
# With the clip's sound kept, the phone's words come from the clip and only the new person's own lines are written.
|
||||
text = write_prompt(script, seconds=8.0, jobs={1: "person"}, video="swap the person", clip_sound=True)
|
||||
self.assertEqual(re.findall(r"<d>\[English\] (.*?)</d>", text), ["How do I get rich?", "Bro, what are they talking about?"])
|
||||
|
||||
def test_a_looks_line_keeps_only_the_looks(self):
|
||||
script = (
|
||||
"Replace the young man with @1.\n"
|
||||
"@1 looks like a cartoon wooden log, no clothes. The trees behind @1 stay the same."
|
||||
)
|
||||
parts = parts_of(script, jobs={1: "person"}, video="swap the person")
|
||||
self.assertIn("this appearance: a cartoon wooden log, no clothes. The background", parts["subject_definitions"])
|
||||
self.assertNotIn("trees", parts["subject_definitions"])
|
||||
for name, part in parts.items():
|
||||
self.assertNotRegex(part, r"@\d", name)
|
||||
|
||||
def test_a_video_needs_words_unless_it_can_go_without(self):
|
||||
with self.assertRaises(ValueError):
|
||||
write_prompt("", jobs={1: "person"}, video="copy its moves")
|
||||
@@ -1521,5 +1633,110 @@ class ClipSoundTests(unittest.TestCase):
|
||||
self.assertEqual(moves[0], node.write('@1 dances.', 6, picture_1=PICTURE, video=frames, video_sound=sound, video_job="copy its moves")[0])
|
||||
|
||||
|
||||
class PromptBugBatchTests(unittest.TestCase):
|
||||
def test_a_swap_action_line_does_not_name_who_goes(self):
|
||||
for script in ("@1 dances.", "He dances."):
|
||||
text = write_prompt(script, seconds=6, jobs={1: "person"}, video="swap the person")
|
||||
self.assertIn("<Subject 1> is the original person in <Video 1>, selected for replacement.", text)
|
||||
self.assertNotIn("dances", text)
|
||||
|
||||
def test_two_people_need_a_named_speaker(self):
|
||||
jobs = {1: "person", 2: "person"}
|
||||
with self.assertRaisesRegex(ValueError, r"name who speaks: @2 says"):
|
||||
write_prompt('@1 and @2 talk. She says "hi". He says "hello".', seconds=6, jobs=jobs)
|
||||
parts = parts_of('@1 says "hi". @2 says "hello".', seconds=6, jobs=jobs)
|
||||
self.assertIn("<Subject 1> and <Subject 2> speak", parts["summary"])
|
||||
|
||||
def test_keep_going_uses_the_person_picture_as_the_main_person(self):
|
||||
parts = parts_of("He walks.", seconds=6, jobs={1: "person"}, video="keep going")
|
||||
text = join_sections(parts)
|
||||
self.assertNotIn("<Subject 2>", text)
|
||||
self.assertNotIn("main subject in <Video 1>", text)
|
||||
self.assertIn("<Subject 1> walks", parts["detailed_description"])
|
||||
placed = parts_of("He walks in @2.", seconds=6, jobs={1: "person", 2: "place"}, video="keep going")
|
||||
self.assertNotIn("same place", join_sections(placed))
|
||||
empty = parts_of("The room is empty.", seconds=6, jobs={}, video="keep going")
|
||||
self.assertNotIn("is in the shot", join_sections(empty))
|
||||
self.assertNotIn("main subject", join_sections(empty))
|
||||
|
||||
def test_an_outfit_with_no_person_goes_on_the_person_in_the_shot(self):
|
||||
parts = parts_of("He wears @1.", seconds=6, jobs={1: "outfit"})
|
||||
self.assertIn("put only this outfit on the person in the shot", parts["retention_analysis"])
|
||||
self.assertNotIn("on <Subject 1>", parts["retention_analysis"])
|
||||
|
||||
def test_in_the_shot_is_only_for_defined_people_and_things(self):
|
||||
for jobs in ({1: "style"}, {1: "end on it"}, {1: "outfit"}):
|
||||
self.assertNotIn("is in the shot", write_prompt("A calm lake.", seconds=6, jobs=jobs), jobs)
|
||||
self.assertIn("<Subject 1> is in the shot", write_prompt("A calm lake.", seconds=6, jobs={1: "thing"}))
|
||||
|
||||
def test_new_voice_keeps_the_clips_person(self):
|
||||
with self.assertRaisesRegex(ValueError, "swap the person"):
|
||||
write_prompt('She says "hi".', seconds=6, jobs={1: "person"}, video="new voice")
|
||||
text = write_prompt('She says "hi".', seconds=6, jobs={}, video="new voice")
|
||||
self.assertNotIn("faces the camera", text)
|
||||
|
||||
def test_he_or_she_binds_the_only_person_picture(self):
|
||||
parts = parts_of("She walks in @2.", seconds=6, jobs={1: "person", 2: "place"})
|
||||
self.assertIn("<Subject 1> walks in <Subject 2>", parts["detailed_description"])
|
||||
with self.assertRaisesRegex(ValueError, "no line names @1"):
|
||||
write_prompt("A dog sits in @2.", seconds=6, jobs={1: "person", 2: "place"})
|
||||
|
||||
def test_a_swap_with_a_thing_or_place_gets_its_own_advice(self):
|
||||
with self.assertRaisesRegex(ValueError, "He holds @1"):
|
||||
write_prompt("He holds @1.", seconds=6, jobs={1: "thing"}, video="swap the person")
|
||||
with self.assertRaisesRegex(ValueError, "keeps the place"):
|
||||
write_prompt("He dances.", seconds=6, jobs={1: "place"}, video="swap the person")
|
||||
|
||||
def test_the_task_tag_adds_audio_reuse_when_the_clips_sound_stays(self):
|
||||
edit = parts_of("She wears a red coat.", seconds=6, jobs={}, video="change something")["summary"]
|
||||
self.assertTrue(edit.startswith("[video editing + audio reuse]"))
|
||||
kept = parts_of("", seconds=6, jobs={1: "person"}, video="swap the person", clip_sound=True)["summary"]
|
||||
self.assertTrue(kept.startswith("[video editing + audio reuse]"))
|
||||
plain = parts_of("", seconds=6, jobs={1: "person"}, video="swap the person")["summary"]
|
||||
self.assertNotIn("audio reuse", plain)
|
||||
|
||||
def test_only_a_place_or_start_picture_sets_the_look(self):
|
||||
for job in ("outfit", "thing"):
|
||||
self.assertNotIn("film grain", write_prompt("He wears @1.", seconds=6, jobs={1: job}), job)
|
||||
self.assertIn("film grain of <Picture 1>", write_prompt("He sits in @1.", seconds=6, jobs={1: "place"}))
|
||||
|
||||
def test_change_something_puts_an_unnamed_outfit_on_the_video_person(self):
|
||||
parts = parts_of("Add a hat.", seconds=6, jobs={1: "outfit"}, video="change something")
|
||||
self.assertIn("<Subject 2> wears <Subject 1>.", parts["detailed_description"])
|
||||
|
||||
|
||||
|
||||
class SpeechAcrossCutTests(unittest.TestCase):
|
||||
CARRIED = 'She looks out the window. She says "I never thought this would happen..."\n\nA close-up of the road ahead.'
|
||||
|
||||
def test_a_trailing_line_carries_across_the_cut(self):
|
||||
first, second = shots(self.CARRIED, seconds=8)
|
||||
self.assertTrue(first.endswith("<scenetrans>"))
|
||||
self.assertNotIn("closes the lips", first)
|
||||
self.assertIn("<scenetrans> The line from the previous shot continues", second)
|
||||
self.assertNotIn("stays silent", second)
|
||||
self.assertNotIn("<Video 1>", second)
|
||||
self.assertEqual(write_prompt(self.CARRIED, seconds=8).count("I never thought"), 1)
|
||||
|
||||
def test_a_trailing_last_line_is_cut_off(self):
|
||||
(only,) = shots('She says "Wait, I was going to tell you that..."', seconds=4)
|
||||
self.assertTrue(only.endswith("<cutoff>"))
|
||||
self.assertNotIn("closes the lips", only)
|
||||
|
||||
def test_a_carried_line_is_not_warned_about(self):
|
||||
long = 'She says "this is a very long line that would never fit inside two short seconds of video"\n\nA close-up of the road.'
|
||||
self.assertEqual(script_warning(read_script(long), 4), "")
|
||||
self.assertIn("<scenetrans>", write_prompt(long, seconds=4))
|
||||
|
||||
def test_plain_lines_carry_nothing(self):
|
||||
text = write_prompt('She says "hello."\n\nHe waves back.', seconds=6)
|
||||
self.assertNotIn("<scenetrans>", text)
|
||||
self.assertNotIn("<cutoff>", text)
|
||||
|
||||
def test_four_seconds_is_allowed(self):
|
||||
seconds = AusBossH3PromptWriter.INPUT_TYPES()["optional"].get("seconds") or AusBossH3PromptWriter.INPUT_TYPES()["required"]["seconds"]
|
||||
self.assertEqual(seconds[1]["min"], 4)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -127,7 +127,7 @@ class ListingTests(FolderCase):
|
||||
|
||||
def test_no_folder_name_leads_out_of_the_root(self):
|
||||
for bad in ("..", "../private", "people/../../private", "/etc", "~/Pictures", "C:/Windows", "//server/share",
|
||||
"people\\..\\..\\private"):
|
||||
"people\\..\\..\\private", "people/C:/Windows", "people/\x00/x"):
|
||||
with self.assertRaises(ValueError, msg=bad) as refused:
|
||||
safe_folder(self.root, bad)
|
||||
self.assertEqual(str(refused.exception), OUTSIDE, bad)
|
||||
@@ -149,6 +149,11 @@ class ListingTests(FolderCase):
|
||||
self.assertEqual(place_inside(str(self.root) + "/", roots), ("input", ""))
|
||||
self.assertEqual(place_inside(str(roots["output"] / "set"), roots), ("output", "set"))
|
||||
self.assertEqual(place_inside(f'"{self.root / "people"}"', roots), ("input", "people"), "quotes from a copied path")
|
||||
# A "~" path is never expanded into a real one, and no node code does that.
|
||||
self.assertIsNone(place_inside("~/ComfyUI/input/people", roots))
|
||||
nodes_dir = Path(__file__).resolve().parents[1] / "nodes"
|
||||
for source in nodes_dir.glob("*.py"):
|
||||
self.assertNotIn("expanduser", source.read_text(encoding="utf-8"), source.name)
|
||||
|
||||
def test_a_whole_path_anywhere_else_is_not_a_place(self):
|
||||
roots = {"input": self.root, "output": Path(self.temp.name) / "output"}
|
||||
@@ -376,7 +381,13 @@ class RouteTests(FolderCase):
|
||||
("thumb", {"name": "loop/picture.png"})]:
|
||||
with self.subTest(route=route):
|
||||
response = self.request(route, **query)
|
||||
self.assertEqual(response.status, 400)
|
||||
# A looped link is a bad folder to the lists. The thumbnail
|
||||
# answers 400 or 404 depending on whether this Python's
|
||||
# resolve() raises on the loop; neither answer names the path.
|
||||
if route == "thumb":
|
||||
self.assertIn(response.status, (400, 404))
|
||||
else:
|
||||
self.assertEqual(response.status, 400)
|
||||
self.assertNotIn(str(self.root), json.loads(response.text)["error"])
|
||||
|
||||
def test_thumbnail_versions_change_with_subsecond_rewrites_and_size(self):
|
||||
|
||||
@@ -1,17 +1,20 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
import json
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import tempfile
|
||||
import unittest
|
||||
from unittest.mock import patch
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(ROOT))
|
||||
if "nodes" in sys.modules and not hasattr(sys.modules["nodes"], "__path__"):
|
||||
del sys.modules["nodes"]
|
||||
|
||||
from nodes._image_hold_helpers import GONE, HELD_FOLDER, hold_pictures, keep_pictures
|
||||
from nodes import _image_hold_helpers # noqa: E402
|
||||
from nodes._image_hold_helpers import GONE, HELD_FOLDER, hold_pictures, keep_pictures # noqa: E402
|
||||
|
||||
NOW = datetime(2026, 11, 10, 9, 30, 0)
|
||||
|
||||
@@ -99,12 +102,70 @@ class HoldTests(unittest.TestCase):
|
||||
|
||||
def test_a_folder_outside_the_output_folder_is_refused_at_the_click_too(self):
|
||||
token, _ = self.hold(count=1)
|
||||
manifest = self.temp / HELD_FOLDER / f"{token}.json"
|
||||
manifest.write_text(manifest.read_text().replace('"output_dir": ""', '"output_dir": "../elsewhere"'))
|
||||
_image_hold_helpers._RUNS[token]["output_dir"] = "../elsewhere"
|
||||
with self.assertRaises(ValueError):
|
||||
self.keep(token)
|
||||
self.assertFalse((Path(self.box.name) / "elsewhere").exists())
|
||||
|
||||
def test_a_run_planted_in_the_temp_folder_is_not_a_run(self):
|
||||
# ComfyUI's temp folder takes uploads, so a file there can be
|
||||
# anyone's. A planted list naming a server file must copy nothing.
|
||||
secret = Path(self.box.name) / "secret.txt"
|
||||
secret.write_text("server secret")
|
||||
held = self.temp / HELD_FOLDER
|
||||
held.mkdir(exist_ok=True)
|
||||
token = "a" * 16
|
||||
planted = dict(
|
||||
settings(extension="png/../../../secret.txt", output_dir="x/y"), count=1, kept={},
|
||||
)
|
||||
(held / f"{token}.json").write_text(json.dumps(planted))
|
||||
(held / f"{token}_000.png").mkdir() # an upload's subfolder makes the ".." walk work
|
||||
with self.assertRaises(LookupError):
|
||||
self.keep(token)
|
||||
self.assertEqual(list(self.output.rglob("*")), [])
|
||||
|
||||
def test_a_held_picture_is_named_only_from_the_token_number_and_format(self):
|
||||
token, previews = self.hold(count=1)
|
||||
for extension in ["png/../../x", "txt", "", "PNG"]:
|
||||
with self.assertRaises(ValueError):
|
||||
_image_hold_helpers._held_name(token, 0, extension)
|
||||
_image_hold_helpers._RUNS[token]["extension"] = "png/../../../secret.txt"
|
||||
with self.assertRaises(ValueError):
|
||||
self.keep(token)
|
||||
self.assertEqual(list(self.output.iterdir()), [])
|
||||
|
||||
def test_a_picture_swapped_for_a_link_is_not_copied(self):
|
||||
token, previews = self.hold(count=1)
|
||||
secret = Path(self.box.name) / "secret.txt"
|
||||
secret.write_text("server secret")
|
||||
picture = self.temp / HELD_FOLDER / previews[0]["filename"]
|
||||
picture.unlink()
|
||||
try:
|
||||
picture.symlink_to(secret)
|
||||
except (OSError, NotImplementedError):
|
||||
self.skipTest("no symlinks here")
|
||||
with self.assertRaises(LookupError):
|
||||
self.keep(token)
|
||||
self.assertEqual(list(self.output.iterdir()), [])
|
||||
|
||||
def test_counts_are_capped(self):
|
||||
with patch.object(_image_hold_helpers, "MAX_HELD_PICTURES", 4):
|
||||
with self.assertRaises(ValueError):
|
||||
self.hold(count=5)
|
||||
token, _ = self.hold(count=2)
|
||||
with self.assertRaises(ValueError):
|
||||
self.keep(token, [0] * 5)
|
||||
self.assertEqual(list(self.output.iterdir()), [])
|
||||
|
||||
def test_old_runs_are_forgotten_first(self):
|
||||
with patch.object(_image_hold_helpers, "MAX_HELD_RUNS", 2):
|
||||
oldest, _ = self.hold(count=1)
|
||||
self.hold(count=1)
|
||||
newest, _ = self.hold(count=1)
|
||||
with self.assertRaises(LookupError):
|
||||
self.keep(oldest)
|
||||
self.assertEqual(len(self.keep(newest)["saved"]), 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
@@ -1148,6 +1148,157 @@ class CanvasStitcherBboxTests(unittest.TestCase):
|
||||
|
||||
|
||||
|
||||
class ToneMatchInpaintTests(unittest.TestCase):
|
||||
"""color_match on a Crop For Inpaint crop: the drift is read inside the
|
||||
painted area, so it holds when the sampler used its own, wider noise
|
||||
mask and left the soft edge close to the original (issue #361)."""
|
||||
|
||||
def _case(self, repaint):
|
||||
generator = torch.Generator().manual_seed(4)
|
||||
rows = torch.linspace(0.0, 1.0, 320).view(1, 320, 1, 1)
|
||||
image = torch.tensor([0.72, 0.55, 0.45]).view(1, 1, 1, 3) - 0.15 * rows
|
||||
image = (image + 0.01 * torch.randn((1, 320, 320, 3), generator=generator)).clamp(0.0, 1.0)
|
||||
mask = torch.zeros((1, 320, 320))
|
||||
mask[:, 110:210, 110:210] = 1.0
|
||||
crop, sampling, stitcher = build_crop(image, mask, 2.0, 16, 16)
|
||||
# The model drifts where it paints, but the workflow's noise mask and
|
||||
# the model itself keep the soft edge on the original: the drift
|
||||
# fades out before the paste's feathered band begins.
|
||||
painted = inpaint_helpers.blur_mask(sampling, 3.0).unsqueeze(-1)
|
||||
patch = crop + painted * (repaint(crop) - crop)
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
ox, oy, ow, oh = stitcher["canvas_to_original"]
|
||||
inside = stitcher["blend"][0, oy : oy + oh, ox : ox + ow] > 0.999
|
||||
return image, patch, stitcher, inside
|
||||
|
||||
def _left_over(self, image, stitched, inside):
|
||||
from nodes._color_helpers import rgb_to_lab
|
||||
return float((rgb_to_lab(stitched) - rgb_to_lab(image))[0][inside].median(dim=0).values.norm())
|
||||
|
||||
def test_a_washed_out_repaint_comes_back_to_the_picture(self):
|
||||
image, patch, stitcher, inside = self._case(lambda crop: crop * 0.85 + 0.11)
|
||||
off = self._left_over(image, apply_stitch(stitcher, patch, color_match=0.0), inside)
|
||||
half = self._left_over(image, apply_stitch(stitcher, patch, color_match=0.5), inside)
|
||||
full = self._left_over(image, apply_stitch(stitcher, patch, color_match=1.0), inside)
|
||||
self.assertGreater(off, 3.0)
|
||||
self.assertLess(half, 0.65 * off)
|
||||
self.assertLess(full, 0.25 * off)
|
||||
|
||||
def test_the_band_alone_read_a_fraction_of_that_drift(self):
|
||||
# What the slider had to work with before: the regression in numbers.
|
||||
image, patch, stitcher, inside = self._case(lambda crop: crop * 0.85 + 0.11)
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
region = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
|
||||
alpha = stitcher["blend"][:, cy : cy + ch, cx : cx + cw]
|
||||
band = inpaint_helpers.estimate_tone_offset(patch, region, alpha)
|
||||
inner, held = inpaint_helpers.inner_tone_offset(patch, region, alpha)
|
||||
self.assertTrue(bool(held[0]))
|
||||
self.assertLess(float(band.norm()), 0.5 * float(inner.norm()))
|
||||
|
||||
def test_new_content_is_not_pulled_toward_the_old_colour(self):
|
||||
# A black jacket over skin is a change, not a drift: the reading
|
||||
# holds at zero, so the band is not asked and nothing is shifted.
|
||||
image, patch, stitcher, inside = self._case(lambda crop: torch.full_like(crop, 0.05))
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
region = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
|
||||
alpha = stitcher["blend"][:, cy : cy + ch, cx : cx + cw]
|
||||
inner, held = inpaint_helpers.inner_tone_offset(patch, region, alpha)
|
||||
self.assertTrue(bool(held[0]))
|
||||
self.assertEqual(float(inner.abs().max()), 0.0)
|
||||
stitched = apply_stitch(stitcher, patch, color_match=1.0)
|
||||
self.assertLess(float(stitched[0][inside].mean()), 0.12)
|
||||
|
||||
def test_new_content_painted_to_the_edge_is_left_as_painted(self):
|
||||
# A model with no noise mask repaints the soft edge too. The band
|
||||
# then reads the new colour as drift; Tone match must not act on it.
|
||||
def repaint(crop):
|
||||
return torch.tensor([0.85, 0.7, 0.1]).view(1, 1, 1, 3).expand_as(crop).clone()
|
||||
|
||||
image, _patch, stitcher, _inside = self._case(repaint)
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
crop = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
|
||||
patch = repaint(crop)
|
||||
plain = apply_stitch(stitcher, patch, color_match=0.0)
|
||||
matched = apply_stitch(stitcher, patch, color_match=1.0)
|
||||
self.assertLess(float((matched - plain).abs().max()), 1.0 / 255)
|
||||
|
||||
def _shirt_case(self, paint):
|
||||
"""Skin with a dark shirt on it. The mask covers the shirt with a
|
||||
thin margin, and the model paints skin there, so the painted area
|
||||
has almost no "before" of its own to be read against."""
|
||||
generator = torch.Generator().manual_seed(7)
|
||||
skin = torch.tensor([0.72, 0.55, 0.45]).view(1, 1, 1, 3)
|
||||
rows = torch.linspace(0.0, 1.0, 320).view(1, 320, 1, 1)
|
||||
clean = (skin - 0.1 * rows + 0.01 * torch.randn((1, 320, 320, 3), generator=generator)).clamp(0.0, 1.0)
|
||||
image = clean.clone()
|
||||
image[:, 104:216, 104:216] = 0.06
|
||||
mask = torch.zeros((1, 320, 320))
|
||||
mask[:, 100:220, 100:220] = 1.0
|
||||
crop, sampling, stitcher = build_crop(image, mask, 2.0, 16, 16)
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
ox, oy, ow, oh = stitcher["canvas_to_original"]
|
||||
wanted = clean[:, oy : oy + oh, ox : ox + ow][:, cy : cy + ch, cx : cx + cw]
|
||||
painted = inpaint_helpers.blur_mask(sampling, 3.0).unsqueeze(-1)
|
||||
patch = crop + painted * (paint(wanted) - crop)
|
||||
inside = stitcher["blend"][0, oy : oy + oh, ox : ox + ow] > 0.999
|
||||
return clean, patch, stitcher, inside
|
||||
|
||||
def test_new_skin_is_matched_to_the_skin_beside_it(self):
|
||||
# Skin where a shirt was, painted paler than the skin around it:
|
||||
# the area's own "before" is the shirt, so it is read against the
|
||||
# untouched skin just outside the paste.
|
||||
clean, patch, stitcher, inside = self._shirt_case(lambda skin: skin * 0.85 + 0.13)
|
||||
off = self._left_over(clean, apply_stitch(stitcher, patch, color_match=0.0), inside)
|
||||
full = self._left_over(clean, apply_stitch(stitcher, patch, color_match=1.0), inside)
|
||||
self.assertGreater(off, 3.0)
|
||||
self.assertLess(full, 0.35 * off)
|
||||
|
||||
def test_new_skin_that_already_matches_is_left_alone(self):
|
||||
clean, patch, stitcher, inside = self._shirt_case(lambda skin: skin)
|
||||
plain = apply_stitch(stitcher, patch, color_match=0.0)
|
||||
matched = apply_stitch(stitcher, patch, color_match=1.0)
|
||||
self.assertLess(float((matched - plain).abs().max()), 2.0 / 255)
|
||||
|
||||
def test_a_mixed_rim_is_not_read_as_drift(self):
|
||||
# Half the painted area turned into something else: too little of
|
||||
# the rim agrees, so nothing inside is trusted.
|
||||
def repaint(crop):
|
||||
out = crop.clone()
|
||||
out[:, :, : crop.shape[2] // 2] = 0.05
|
||||
return out
|
||||
image, patch, stitcher, inside = self._case(repaint)
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
region = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
|
||||
alpha = stitcher["blend"][:, cy : cy + ch, cx : cx + cw]
|
||||
offset, held = inpaint_helpers.inner_tone_offset(patch, region, alpha)
|
||||
self.assertLess(float(offset.norm()), 1.0)
|
||||
|
||||
def test_an_unchanged_crop_reads_no_drift(self):
|
||||
image, _patch, stitcher, _inside = self._case(lambda crop: crop)
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
crop = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
|
||||
stitched = apply_stitch(stitcher, crop, color_match=1.0)
|
||||
# Nothing to undo; what is left is the colour-space round trip.
|
||||
self.assertLess(float((stitched - image).abs().max()), 0.5 / 255)
|
||||
|
||||
def test_a_large_picture_is_read_at_the_size_the_model_painted(self):
|
||||
_image, patch, stitcher, _inside = self._case(lambda crop: crop * 0.85 + 0.11)
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
canvas = stitcher["canvas"][:, cy : cy + ch, cx : cx + cw]
|
||||
blend = stitcher["blend"][:, cy : cy + ch, cx : cx + cw]
|
||||
small, small_held = inpaint_helpers.inner_tone_offset(patch, canvas, blend)
|
||||
|
||||
def grow(image):
|
||||
return torch.nn.functional.interpolate(image.movedim(-1, 1), scale_factor=4, mode="bilinear").movedim(1, -1)
|
||||
|
||||
big, big_held = inpaint_helpers.inner_tone_offset(
|
||||
grow(patch), grow(canvas), grow(blend.unsqueeze(-1)).squeeze(-1), sampled=(cw, ch)
|
||||
)
|
||||
self.assertTrue(bool(small_held[0]) and bool(big_held[0]))
|
||||
# Pasted back four times larger, the reading is the same one.
|
||||
self.assertLess(float((big - small).norm()), 0.3)
|
||||
|
||||
|
||||
class ToneMatchTests(unittest.TestCase):
|
||||
"""color_match: the drift is read off the feathered band and undone."""
|
||||
|
||||
|
||||
@@ -108,6 +108,15 @@ class _FolderPaths:
|
||||
def get_folder_paths(self, kind: str) -> list[str]:
|
||||
return [str(self.lora_root)] if kind == "loras" else []
|
||||
|
||||
def get_filename_list(self, kind: str) -> list[str]:
|
||||
if kind != "loras":
|
||||
return []
|
||||
return [
|
||||
path.relative_to(self.lora_root).as_posix()
|
||||
for path in self.lora_root.rglob("*")
|
||||
if path.is_file() and path.suffix == ".safetensors"
|
||||
]
|
||||
|
||||
def get_user_directory(self) -> str:
|
||||
return str(self.user_root)
|
||||
|
||||
@@ -605,7 +614,15 @@ class ResolveLoraPathTests(unittest.TestCase):
|
||||
def test_path_outside_the_folders_is_refused(self):
|
||||
self.full_path = lambda kind, name: str(self.root / ".." / "other_drive" / "real.safetensors")
|
||||
with self.assertRaisesRegex(ValueError, "escapes the loras folders"):
|
||||
_lora_helpers.resolve_lora_path("../other_drive/real.safetensors")
|
||||
_lora_helpers.resolve_lora_path("plain.safetensors")
|
||||
|
||||
def test_a_name_off_the_list_never_reaches_the_disk(self):
|
||||
asked = []
|
||||
self.full_path = lambda kind, name: asked.append(name)
|
||||
for name in ("../other_drive/real.safetensors", str(self.outside / "real.safetensors")):
|
||||
with self.assertRaisesRegex(ValueError, "not found"):
|
||||
_lora_helpers.resolve_lora_path(name)
|
||||
self.assertEqual(asked, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -61,7 +61,7 @@ from nodes._media_helpers import ( # noqa: E402
|
||||
resolve_server_file,
|
||||
resolve_video_path,
|
||||
)
|
||||
from nodes._route_errors import safe_route_error # noqa: E402
|
||||
from nodes._route_errors import error_detail, safe_route_error # noqa: E402
|
||||
|
||||
NETWORK_REFUSAL = "Network source paths are not allowed."
|
||||
WHOLE_PATH_REFUSAL = "A whole path is not read"
|
||||
@@ -425,6 +425,12 @@ class RouteErrorTests(unittest.TestCase):
|
||||
with self.subTest(text=text):
|
||||
self.assertEqual(safe_route_error(ValueError(text)), text)
|
||||
|
||||
def test_a_node_error_keeps_its_words_unless_they_name_a_path(self):
|
||||
self.assertEqual(error_detail(ValueError("bad shape (3, 4)")), "ValueError: bad shape (3, 4)")
|
||||
for text in ("cannot open /home/me/models/x.pt", "Error in `C:\\Comfy\\models\\sam3.pt`", "~/loras/a.safetensors"):
|
||||
with self.subTest(text=text):
|
||||
self.assertEqual(error_detail(OSError(text)), "OSError")
|
||||
|
||||
def test_no_route_sends_a_file_by_its_path(self):
|
||||
for path in sorted((ROOT / "nodes").glob("*.py")):
|
||||
with self.subTest(module=path.name):
|
||||
|
||||
@@ -9,6 +9,13 @@ image and blend_mask with the output the 2.3.0 code gave.
|
||||
The legacy inpaint crop keeps its original geometry here; its expected
|
||||
mask is limited to the paste window. Crop sizing is tested separately.
|
||||
|
||||
A Crop For Inpaint crop with Tone match on is the one case no longer held to
|
||||
2.3.0: that release read the drift in the soft edge, which read a fraction of
|
||||
it when the workflow samples with a wider noise mask, so Tone match hardly
|
||||
did anything (issue #361). It is read inside the painted area now, and
|
||||
ToneMatchInpaintTests in test_inpaint_crop_helpers.py holds that. Its Tone
|
||||
match 0 stitch and its blend mask stay pinned.
|
||||
|
||||
Blend in with Tone match at 0 is pinned the same way, to the 2.4.0 code:
|
||||
the published Qwen Image 2.1 Rotate + Outpaint workflow stitches that way.
|
||||
|
||||
@@ -78,6 +85,11 @@ PINS = {
|
||||
}
|
||||
|
||||
|
||||
# Tone match on an inpaint crop is read inside the painted area since issue
|
||||
# #361, so these two no longer give the 2.3.0 picture. See the docstring.
|
||||
REREAD = {("crop for inpaint", 0.5), ("crop for inpaint", 1.0)}
|
||||
|
||||
|
||||
def picture(width: int, height: int, seed: int) -> Image.Image:
|
||||
"""Smooth colour structure plus fine grain, as 8-bit RGB."""
|
||||
y, x = np.mgrid[0:height, 0:width].astype(np.float32)
|
||||
@@ -207,7 +219,8 @@ class ClassicSeamPinTests(unittest.TestCase):
|
||||
with self.subTest(case=name, color_match=strength):
|
||||
stitcher, patch_image = build()
|
||||
image, blend_mask = stitch(stitcher, patch_image, strength)
|
||||
assert_released_image(image, reference_image(name, strength))
|
||||
if (name, strength) not in REREAD:
|
||||
assert_released_image(image, reference_image(name, strength))
|
||||
expected_mask = reference_mask(name)
|
||||
if name == "crop for inpaint":
|
||||
cx, cy, cw, ch = stitcher["crop_to_canvas"]
|
||||
|
||||
@@ -0,0 +1,79 @@
|
||||
import assert from "node:assert/strict";
|
||||
import test from "node:test";
|
||||
import { readFileSync } from "node:fs";
|
||||
import { COPY_EDITS_SETTING, copyPicture, transformPicture } from "../js/shared/transform_picture.mjs";
|
||||
import { mergeSettings } from "../js/shared/node_settings.mjs";
|
||||
|
||||
function render(values = {}, resize = null, includeEdits = false) {
|
||||
const calls = [];
|
||||
const context = new Proxy({}, { get: (target, name) => name in target ? target[name] : (...args) => calls.push([name, ...args]) });
|
||||
const canvas = { getContext: () => context };
|
||||
const result = transformPicture({
|
||||
sourceWidth: 100, sourceHeight: 60,
|
||||
values: { canvas_multiple: 1, ...values }, resize, includeEdits, fill: "#123456",
|
||||
drawSource: (ctx) => ctx.drawImage("source", -50, -30, 100, 60),
|
||||
createCanvas: () => canvas,
|
||||
});
|
||||
return { result, calls, context };
|
||||
}
|
||||
|
||||
test("default copy uses the original picture, ignoring crop, rotation, padding and resize", () => {
|
||||
const { result, calls } = render({ crop_x: 10, crop_width: 40, rotation_degrees: 90, pad_left: 80, pad_top: 20, canvas_multiple: 64 }, { megapixels: 1, steps: 8 });
|
||||
assert.deepEqual([result.width, result.height], [100, 60]);
|
||||
assert.deepEqual(calls, [
|
||||
["translate", 50, 30], ["drawImage", "source", -50, -30, 100, 60],
|
||||
]);
|
||||
});
|
||||
|
||||
test("the gear preference defaults off for existing browsers and accepts an explicit opt-in", () => {
|
||||
assert.equal(mergeSettings([COPY_EDITS_SETTING], {}).copy_edits, false);
|
||||
assert.equal(mergeSettings([COPY_EDITS_SETTING], { show_mask: true }).copy_edits, false);
|
||||
assert.equal(mergeSettings([COPY_EDITS_SETTING], { copy_edits: "true" }).copy_edits, false);
|
||||
assert.equal(mergeSettings([COPY_EDITS_SETTING], { copy_edits: true }).copy_edits, true);
|
||||
});
|
||||
|
||||
test("opting in includes crop, padding and alignment in the copied bounds and source offset", () => {
|
||||
const { result, calls } = render({ crop_x: 10, crop_y: 5, crop_width: 40, crop_height: 30, pad_left: 8, pad_top: 4, pad_right: 8, pad_bottom: 6, canvas_multiple: 8 }, null, true);
|
||||
assert.deepEqual([result.width, result.height], [56, 40]);
|
||||
assert.ok(calls.some((c) => JSON.stringify(c) === JSON.stringify(["rect", 8, 4, 40, 30])));
|
||||
assert.ok(calls.some((c) => JSON.stringify(c) === JSON.stringify(["translate", 48, 29])));
|
||||
});
|
||||
|
||||
test("opting in applies rotation and resize without copying a screen-sized preview", () => {
|
||||
const { result, calls } = render({ rotation_degrees: 90 }, { megapixels: 0.25, steps: 8 }, true);
|
||||
assert.deepEqual([result.width, result.height], [400, 664]);
|
||||
assert.ok(calls.some((c) => c[0] === "rotate" && c[1] === Math.PI / 2));
|
||||
assert.ok(calls.some((c) => c[0] === "scale" && c[1] === 400 / 60 && c[2] === 664 / 100));
|
||||
});
|
||||
|
||||
test("clipboard write starts before PNG encoding finishes and contains only the clean PNG", async () => {
|
||||
let finish, item, written = false;
|
||||
const canvas = { toBlob: (callback, type) => { assert.equal(type, "image/png"); finish = callback; } };
|
||||
class ClipboardItem { constructor(data) { this.data = data; } }
|
||||
const clipboard = { write: async (items) => { written = true; [item] = items; await item.data["image/png"]; } };
|
||||
const pending = copyPicture(canvas, { clipboard, ClipboardItem });
|
||||
assert.equal(written, true);
|
||||
assert.deepEqual(Object.keys(item.data), ["image/png"]);
|
||||
await Promise.resolve(); await Promise.resolve();
|
||||
const png = new Blob(["picture"], { type: "image/png" });
|
||||
finish(png);
|
||||
await pending;
|
||||
assert.equal(await item.data["image/png"], png);
|
||||
});
|
||||
|
||||
test("clipboard and encoding failures are reported instead of false success", async () => {
|
||||
assert.throws(() => copyPicture({}, { clipboard: null, ClipboardItem: null }), /Open image/);
|
||||
class ClipboardItem { constructor(data) { this.data = data; } }
|
||||
const canvas = { toBlob: (callback) => callback(null) };
|
||||
const clipboard = { write: async ([item]) => await item.data["image/png"] };
|
||||
await assert.rejects(copyPicture(canvas, { clipboard, ClipboardItem }), /could not make/);
|
||||
await assert.rejects(copyPicture({ toBlob: () => {} }, { clipboard: { write: async () => { throw new Error("Denied"); } }, ClipboardItem }), /Denied/);
|
||||
});
|
||||
|
||||
test("both image editing surfaces replace native canvas copying, video is unchanged", () => {
|
||||
const source = readFileSync(new URL("../js/shared/transform_editor.mjs", import.meta.url), "utf8");
|
||||
assert.match(source, /attachStageHandlers\(state, preview, state.panelAbort.signal\)/);
|
||||
assert.match(source, /attachStageHandlers\(state, canvas, abort.signal\)/);
|
||||
assert.match(source, /if \(state.kind === "image"\) \{\s*canvas.addEventListener\("contextmenu"/);
|
||||
assert.match(source, /copyPicture\(picture\)/);
|
||||
});
|
||||
Reference in New Issue
Block a user